What is the physical meaning of the generators which appear to be matrices?
Basically, they physically represent infinitesimally small operations, for example,
angular momentum as the generator of rotations,
linear momentum as the generator of translations,
electric charge being the generator of the U(1) symmetry group of electromagnetism,
the color charges of quarks are the generators of the SU(3) color symmetry in quantum chromodynamics,
They are very useful, due to their simplicity, in checking commutation relations, related to the Lie Algebra of any particular group.
The Pauli matrices, multiplied by $i$, are the basis for $su(2)$, i.e. the Lie algebra involved, not the group.
${\displaystyle {\begin{aligned}\sigma _{1}=\sigma _{x}&={\begin{pmatrix}0&1\\1&0\end{pmatrix}}\\\sigma _{2}=\sigma _{y}&={\begin{pmatrix}0&-i\\i&0\end{pmatrix}}\\\sigma _{3}=\sigma _{z}&={\begin{pmatrix}1&0\\0&-1\end{pmatrix}}\,.\end{aligned}}}$
However, as you probably know, they are incorporated in an exponential function when we need to express say, a rotation through an angle of 30 degrees.
Given a $SO(3; 1)$ representation, one may try to construct a representation of $SO(3; 1)$, the identity component of the Lorentz group, by using the exponential mapping. If $X$ is an element of $SO(3; 1)$ in the standard representation, then
${\displaystyle \Lambda =e^{iX}\equiv \sum _{n=0}^{\infty }{\frac {(iX)^{n}}{n!}}}$
is a Lorentz transformation by general properties of Lie algebras.
EDIT From a comment by WetSavannaAnimal aka Rod Vance
One thinks of Lie algebras of Lie groups as Lie algebras over real fields, so that $i$ is not thought of as a scalar in the field: the co-efficients of superpositions in the Lie Algebra are always real.