Question edited:
If you dig a tunnel through a large asteroid like Ceres along its axis of rotation and jumped in then what would be the first thing to stop you from falling back and forth along the tunnel length ?
Possibilities include:
Dmitry Brant mentions gravitational waves and charged particles. John Rennie mentions drag from impure vacuum.
Perturbations from other planets could shift axis of rotation causing you to hit the side.