I know that we can very easily derive an expression for the time period of oscillations in an underground tunnel due to gravity. It does not depend on length or inclination of the tunnel, and is 5050 seconds.
But if that's true it should be true even for tunnels that are very small in length. What I can not understand is why we don't just start falling and oscillating.
Yes, it is the same for all lengths of tunnel. This is really no more surprising than that the period of a mass on an ideal spring is independent of its amplitude. A larger amplitude causes the mass to move more rapidly, covering a greater distance in the same time.
The formula derived by Naveen Balaji in Period of oscillation through a hole in the earth is independent of amplitude :
$$T=2\pi\sqrt{\frac{R^3}{GM}}$$
Naveen also shows in his derivation that the period is independent of the length of the tunnel through the Earth.
It is not clear what you are getting at when you ask "Why don't we just start falling and oscillating?" An obvious answer is : Because there are no tunnels available to fall through.
You could dig a very shallow tunnel close to the surface of the Earth, perhaps only a few km long. Then you would oscillate through that tunnel with a period of 42 minutes, the same as when it passes through the centre of the Earth, provided that the friction from the sides of the tunnel is negligible. You would still be attracted toward the centre of the Earth; this force causes your acceleration but it also leads to friction. At such a shallow angle of descent the normal reaction from the sides of the tunnel (hence also friction) is much greater than the accelerating force.
