If you solve the equations for two bodies interacting via gravity (or indeed any inverse square law force) then the bound orbits are all elliptical - a circle is a special case of an ellipse with zero eccentricity. So any object in an elliptical orbit will remain in that orbit forever. No external forces are needed. To see how the orbit is calculated have a look at the question Kepler problem in time: how do two gravitationally attracted particles move?.
To try and make this answer a bit more interesting than just yes I'll mention a few other interesting points:
Elliptical orbits are only stable with an inverse square force, or possibly also a harmonic force though I'm not sure about this. There is a theorem called Bertrand's theorem that tells us this.
Real satellites don't orbit in an inverse square force, because you need to take into account the gravitational forces between the satellites. For artificial satellites orbiting the Earth this is entirely negligable, but if you look at the planets orbiting the Sun none of them travel in perfect ellipses because of this effect. For example the eccentricity of Earth's orbit around the Sun changes continually mostly due the the gravitational force between Earth and Jupiter.
Even in a system of just two bodies the orbit still isn't a perfect ellipse because the Newton inverse square force law is actually only an approximation and is modified by effects due to General Relativity. So even in a two body system orbits aren't perfectly elliptical. You may have heard that the precession of Mercury is due to this, though actually only a small part of the precession is due to General Relativity.
Finally, a two body system radiates gravitational waves, so if you wait long enough the orbits will decay and the two bodies will collide. However unless the system is an extreme one like a binary pulsar the timescales required for this are long compared to the age of the universe.