Below is an image of an EM wave from wikipedia. The energy density of an EM wave is given by $u = \frac{1}{2} \left( \varepsilon_0 E^2 + \frac{B^2}{\mu_0} \right)$. I am wondering how there can be energy in an EM wave at the points of the oscillation where both $E$ and $B$ are zero.
The expression you have used is the energy density and applies at a point in space and time. The "energy in an EM wave" at any given time would have to be an integral over a volume.
At the point in question, the energy has been conserved, it has simply been moved to a different place. After all, a propagating EM wave does transfer energy in the direction $\vec{E}\times \vec{B}$.
