$\bullet$ Is it fair to assume that the classical Hamiltonian or Lagrangian of a system (a particle or a field) is always a real-valued function?
$\bullet$ If not, can you provide counter-examples?
$\bullet$ The cases in which the Hamiltonian represents the total energy of a system, it must be real on physical grounds. Apart from that is there any other criterion which demands that a Lagrangian or Hamiltonian must be real?
Any physical lagrangian or hamiltonian is real. There may be models out there in which $\cal{L}$ and $\cal{H}$ are complex, but in those cases the "complexity" must be for mathematical convenience, and the quantities calculated in such a model will need to be converted to real quantities or simply not correspond to physical observables. One strong reason for this is that the integral of a complex function is, in general , complex as well. Therefore a complex lagrangian would give rise to a complex action, which doesn't make any sense. Even if the hamiltonian doesn't correspond to the energy, a complex-value $\cal{H}$ would result in a complex lagrangian, so the problem remains.
I am not sure a classical Lagrangian must be real. The reason is as follows. If we start with some real Lagrangian and add a complex total divergence, we obtain a complex Lagrangian, but this final Lagrangian will have the same equations of motion as the initial Lagrangian, so one can use this final Lagrangian, if it is more convenient for some reasons, and obtain correct results.
Example: let us note that the standard Dirac Lagrangian $\bar{\psi}(i\gamma^\mu\partial_\mu-m)\psi$ is generally complex (and you can consider this theory classical before second quantization), but you can add a complex total divergence and make this Lagrangian real (symmetric Lagrangian $\frac{i}{2} (\bar\psi \gamma^\mu \partial_\mu \psi - (\partial_\mu \bar\psi) \gamma^\mu \psi ) - m \bar\psi \psi$).
