The answer depends on whether you neglect radiation reaction or not. The fact is that the moving charge creates a field on its own, and if it accelerates -- as it would be the case, in the presence of the other, stationary charge -- it can essentially feel the effect of its own field, creating what is usually called a "self-force", and experience a "radiation reaction" -- check chapter 11 of Griffiths' Introduction to Electrodynamics if you want to take a deeper look. This term creates a force which is proportional to the rate of change of the acceleration, and thus the equation of motion of the particle becomes a third-order differential equation in its position! I don't know of any analytical solution to that, in a general scenario.
The problem becomes much simpler, however, if you neglect radiation reaction -- you argue that the charge does not accelerate so quickly, so its interaction with itself is negligible compared with the force generated by the other particle, or something like that. In that case, you have simply a particle undergoing motion under what is called a central potential, and in the case of the field generated by a single stationary charge, the potential you get is in fact identical to the gravitational potential between two masses in Newton's theory of gravity, so the motion of the moving charge is pretty much the same as the motion of planets -- ellipses for bound states, hyperbolas for scattering states, and parabolae in the threshold between those two cases. A standard reference to study motion under a central potential as an undergraduate is chapter 8 of Marion and Thornton's Classical Dynamics of Particles and Systems, and I highly recommend taking a closer look at that (they explore lots of things about planetary motion in that chapter, but since the Coulomb potential is identical to the potential between masses interacting gravitationally, the same procedure applies). All the math you need to know in order to tackle that chapter is essentially undergraduate calculus, but you should also be familiar with Lagrangian mechanics -- which is also covered in previous chapters of Marion and Thornton.
