The main problem is radiation reaction. It's a fact that if I take a charge and jiggle it about, it will somehow "inject" energy into the electromagnetic field around it; we know this because we can detect this energy in the form of electromagnetic radiation. This means that when I grabbed the charge and jiggled it, I must have performed extra work on it (above the work needed to accelerate its mass) which then ends up as energy in the far field. This extra work can only have been performed against a force, which is electrical in nature since it depends on the charge. Since the only electric field present is that of the charge, we conclude that the charge must have interacted with its own field in some way.
However, it's also obvious that a point charge cannot interact with its own electric field in the same way that it does with the field of other charges. At $\mathbf r=0$, the electric field $$\mathbf E=q\frac{\mathbf r}{r^3}$$ is singular and ill-defined; it doesn't even have a direction. So the self-interaction needs to be something different.
Everything that follows is an attempt to patch together these two disparate facts. How do you account exactly for that radiated energy, and how can you formulate consistent laws that have energy conservation built in intrinsically? In the far field, it is easy to account for the radiated energy via an electromagnetic energy density proportional to $|\mathbf E|^2$, but if you take this at face value then it blows up if you have point charges. If you try to take this directly but remove the point charge self energies by hand, the resulting theory is clunky, hard to use, and its results are not always Lorentz invariant. If you don't ascribe an energy content to the field (which means you cannot be considering it as a dynamical variable, as done by e.g. Wheeler-Feynman), how do you account for the fact that one can transfer energy via electromagnetic radiation?
More fundamentally, though: do the electromagnetic fields at a given point depend on how they were made? Is it enough to say "the electric field at $\mathbf r$ is 5V/m pointing in the $\hat{\mathbf z}$ direction"? Or do we need to specify which point charges created it? The latter stands very much against the concept of field, and how we use it in practice to calculate and prove things, so we'd need to have a very close look at all of electrodynamics (i.e. where is the 'which-charge' information in Maxwell's equations?). If the former is true, however, with what justification can we just yank out certain specific charge-dependent components from the EM field energy?
So you see, these are tough questions, and they are nowhere near solved, so it's an interesting area (but also it's not clear whether it's even possible to solve these questions, so keep that in mind).
