I've been having a go at programming an orbital simulator from scratch (without using any existing libraries) using the formula below to find the acceleration first for particle A and then for particle B towards each other.
$$F = \frac{m_Am_BG}{r^2}$$
$$a_{AB} = \frac{m_BG}{r^2}$$ $$a_{BA} = \frac{m_AG}{r^2}$$
The particles have masses: $m_A = 300\times10^{23}, m_B = 10\times10^{23}$
This produces what looks to be almost a correct orbit with the lighter particle B orbiting around particle A however particle A does not move in an ellipse rather a half ellipse before continuing upwards with another half ellipse and so on. This can be seen in the image below:

The time step does not seem to be an issue as I've adjusted that and it made no difference, it is currently set to $0.000001\mathrm{s}$. I'm not sure as to what might cause this.