The probability density of the ground state is time independent, so there is no motion in this sense. Yet the expectation value of the kinetic energy is non-zero, so there is movement in this sense. How are these notions of movement reconciled?
First off, classically, if we had a particle in a $1/r$ potential and released it from rest, it would indeed bob back and forth like a pendulum as you describe. But in quantum mechanics we can't say the electron is taking any specific path around the proton. As there is no specific path, we cannot fully reconcile these notions of movement with any classical preconceptions.
Let's discuss some various notions of motion in quantum mechanics, that may help you here.
The ground state of the hydrogen atom is
$$ \psi(r,\theta,\phi,t) = A\, \exp\left(-\frac{r}{a}-i\frac{E t}{\hbar}\right)$$
Where $A =\sqrt{\frac{1}{\pi a^3}},a=\frac{\hbar^2}{me^2},E=\frac{-m e^4}{8 h^2 \varepsilon_0^2}$
The radial momentum operator in this basis is:
$$\vec{p}_r = -i\hbar\hat{r}\frac{\partial}{\partial r}$$
where $\hat{r}$ is the radial unit vector (not an operator).
In calculating the expectation value of this:
$$\langle \psi|\vec{p}_r|\psi\rangle
= \int \psi^* (- i\hbar\hat{r}\frac{\partial}{\partial r} \psi) \sin(\theta) r^2 dr\,d\theta\,d\phi
= \int \psi^* (- i\hbar\hat{r}\frac{-1}{a} \psi) \sin(\theta) r^2 dr\,d\theta\,d\phi
$$
Due to symmetry, this will of course be zero. But the density term in the integral is
$$\psi^*(- i\hbar\hat{r}\frac{-1}{a} \psi) = i\hbar\frac{1}{a}(\psi^*\psi)\hat{r}$$
This might be what you want to interpret as 'motion', but since $(\psi^*\psi)\ge 0$ this is purely imaginary and doesn't have a directly physical interpretation as motion. As its imaginary, it's neither towards or away from the center.
Another notion of motion is the probability current:
$$ \vec{j} = \frac{\hbar}{2mi}\left(\psi^* \vec\nabla \psi - \psi \vec\nabla \psi^{*} \right)$$
This is related to conservation of probability by:
$$ \rho = \left|\psi\right|^2,\quad \frac{\partial \rho}{\partial t} + \vec\nabla \cdot \vec{j} = 0 $$
For the hydrogen ground state we have:
$$ \vec{j} = \frac{\hbar}{2mi}\hat{r}\left(\psi^* \left(\frac{\partial}{\partial r}\psi\right) - \psi \left(\frac{\partial}{\partial r}\psi^{*}\right) \right)$$
$$ = \frac{\hbar}{2mi}\hat{r}\left(\psi^* \left(\frac{-1}{a}\psi\right) - \psi \left(\frac{-1}{a}\psi^{*}\right) \right) = 0$$
There is no probability current at any point. So any sense in which there is motion at some location, the net current into/out of this point is still zero. Which takes me to the only remaining way I know to discuss "motion" here. We are writing the state in the position basis, let me make this more clear, and also use the cartesian basis for a bit:
$$ |\psi\rangle = \int \phi(x,y,z,t)|x,y,z\rangle $$
The state $|\phi\rangle$ is but one vector in the infinite vector space, that is the Hilbert space for the electron here. When we write $\phi(x,y,z,t)$ these are really time dependent components for each basis element $|x,y,x\rangle$ in the chosen basis for this vector space. The state $|1,0,0\rangle$ by itself is the closest interpretation of your idea of starting with the electron at say x=1,y=0,z=0 and dropping it to see how it moves.
We can start with this pure position state and watch how it evolves according to the Hamiltonian operator. Since this state is not an energy eigenstate, it will spread (evolve to a state that now needs to be written as a super-position of many of our $|x,y,x\rangle$ basis states). It however will not swing like a pendulum though the origin like you imagine. It will spread out in all directions (since by the uncertainty principle, a pure position state is completely spread in momentum space).
The magic of the ground state is that if we consider this special weighted super position of an enormous (infinite) number of position states individually spreading, they spread exactly such that the super position of states remains the same and the net current is zero at every point. You could view this a bit like equilibrium with the principle of detailed balance: the position states will evolve into each other, but the amount that is "leaving" a pure position state must be replaced with exactly the same amount "entering" that state from other position states in this super position.
So in a sense, there is movement (kinetic energy is non-zero, the time evolution operator (Hamiltonian) is constantly evolving pure position states at each point to spread out), but the "net movement" of the wavefunction is zero (probability current is zero) and the probability density is time independent.
Consider this section an extended comment:
Akrasia suggested another way of looking at motion here: momentum decomposition.
Basically we can also write the state in terms of momentum basis in Hilbert space.
$$ |\psi\rangle = \int \phi(k_x,k_y,k_z,t)|k_x,k_y,k_z\rangle $$
These basis states are spread over all space (uniformly so). So they can't tell us about motion in some region. But we can get a probability density on this space, giving a notion of motion for parts of the state. And for the hydrogen ground state, it will be built up as standing waves of opposing momentum basis states. Since these cover all space, the momentum of a plain wave state is not just in the radial direction. So in this sense, the "motion" is not just in the radial direction.
Here is a related stack exchange question:
Hydrogen wave function in momentum space
And here is a paper which claims to work out the hydrogen wave function in spherical momentum space.
They find the ground state to be:
$$\phi_{1,0,0} = \frac{1}{(p_r - i p_0)^2} \frac{1}{p_\theta^{1/2}}J_{1/2}(p_\theta)\delta(p_\phi)$$
Which means there is no contribution from basis elements with non-zero $p_\phi$ momentum, but there is for non-zero $p_\theta$. That is surprising to me, and I don't have time to read through the paper right now. So it would be best if someone else wrote up an answer covering this portion. If that paper works, then that seems to be a very nice notion of motion for which to answer the question here "Is there only radial motion in the Hydrogen ground state?".
