Is the notion of 'spin' for massless fields poorly worded or am I confused?

Question

My question is coming from the (perhaps not fully correct?) understanding that the different spins amount to different possible irreducible representations of SU(2): spin 0 being a singlet (acting on a 1 dimensional space), spin 1/2 a doublet (acting on a 2 dimensional space) etc. In this view, a spin 2 particle should be given realised by an irreducible representation where the space is 5 dimensional.

Now, if I understand correctly, this would be the case, if the particle had positive mass (eg from Wigner's classification https://en.wikipedia.org/wiki/Wigner%27s_classification)- that is, a spin 2 particle with positive mass would be realised by a 5 dimensional field. The notion of spin so far is arising from the fact that the 'stabiliser group' is $SU(2)$ for this massive case. Is this correct so far?

Now, the graviton would be massless. From Wigner's classification, the stabiliser group is now no longer $SU(2)$, but we still call it spin 2, even though it does not seem to relate to $SU(2)$ and it also does not mean that the corresponding particle should be a 5 dimensional field (for example this was discussed in this question What is the relation between the metric tensor and the graviton?).

So now to return to my titled question: based on what I wrote above, is the notion of spin for massless fields slightly misguiding (ie when we say "gravitons are would-be spin 2 particles" is it a little untrue, at least if spin is to be taken as labelling $SU(2)$ representations) or am I simply confused?

Answer 1

Firstly, the graviton is absolutely a spin-2 particle and not a 5-dimensional field (as the post you linked asked about). We do not need to discuss SU(2), we can just stick with good-ole spin-2 representation of SO(3).

The spin-2 representation of SO(3) can be written as $U(R) = R_i^aR_j^b$ which is a product of two rotation matrices (i.e. each copy acts on a vector). The corresponding spin-2 state can be written as \begin{equation} \Psi(x)^{ij} = \langle\mathbf{x}|\psi\rangle^{ij} =\sum_\lambda\Psi_\lambda(\mathbf{x},t)\epsilon^{ij}_\lambda \end{equation} where $\epsilon_\lambda^{ij}$ is the polarization tensor with helicity $\lambda$. They too transform under a product of rotation matrices. There is a short procedure in order to derive the polarization matrices that can be readily found online, but you will find that there are 5 matrices, $\lambda = \{\pm 2,\pm1 ,0\}$.

Now, to get to a 4-dimensional tensor representation of spin-2, we can promote the 3-dimensional matrices to a 4-dimensional one by placing $0$s in the new row and column since the time-components are non-dynamical. At the moment, we have 5-dynamical degrees of freedome. We can perform a Lorentz boost on all of the matrices of the form \begin{equation} \epsilon^{\mu\nu}\rightarrow \Lambda^\mu_\alpha\Lambda^\nu_\beta\epsilon^{\alpha\beta} \end{equation} which for the spin-2 matrices gives \begin{equation} \epsilon_{\pm 2}^{\mu\nu}\rightarrow \frac{1}{2} \begin{pmatrix} 0&0&0&0\\ 0&1&\pm i&0\\ 0&\pm i&-1&0\\ 0&0&0&0 \end{pmatrix} \end{equation} which is actually unchanged from their previously un-boosted matrix. Note that they are orthogonal $\epsilon^{ij}_{\pm 2}p_j = 0$. However, the $\pm 1$ and $0$ matrices change and pick up factors of $\propto p_z/m$ and $\propto E/m$ (for a boost in the z-direction).

Now, obviously in the massless limit the spin-2 modes are completely fine, but the $\pm 1$ and $0$ modes are not. In order to remove these modes we must project them out via $\epsilon^{ij}p_j = 0$.

To give a TLDR version: massive spin-2 particles carry 5 polarizations while massless spin-2 particles carry only 2 polarizations.

Now, the much more modern take on massless fields is to solely consider transformations under Wigner's little group and consider what is a particle, not a field (which is why I tried to never say field above). The beginning of section 2 of Scattering Amplitudes For All Masses and Spins gives a really great introduction and derivation of how particles are representations of Wigner's little group.