Why do we need (or prefer?) $SU(2)$ rather than $U(2)$ to perform rotation of 3d Pauli vectors?

Question

In order to rotate the usual 3d vectors (Written as Column vectors), We start with the idea that Rotation perserves lengths, which leads us to the group of $O(3)$. But Reflections also perserve length, which have determinant of negative one. This restricts us to $SO(3)$ which also perserve orientations unlike reflections and have $\det = +1$.

I am learning the very basics of spinors, group theory stuff for QM, particle physics etc. If we start with the similar intuition about rotation, this leads us to unitary matrices with unit complex number as a determinant.

Now here is my Question: Why did we choose $SU(2)$ (determinant = +1), even though when we look at the double sided rotation formula for Pauli Vectors, it seems like Unitary matrices with any unit complex number as determinant could be used (unlike the connection between $O(3)$ and $SO(3)$)?

https://www.youtube.com/watch?v=Xe8qjtqZJcc&list=PLJHszsWbB6hoOo_wMb0b6T44KM_ABZtBs&index=6

Proof given in the last part of the video makes it seem like this.

Answer 1

The TL;DR answer to your question (as I understand it) is that we choose $+1$ because it simple. Why would we do anything else?

Here is a practically-minded roadmap of how this group isomorphism arises in QM, which is beyond the scope of your question but might provide some additional insight as you study more.

---

Everything starts from the question of how we might implement rotations on generic Hilbert spaces. This requires us to define what we mean by applying a (spatial) rotation to e.g. a 2-component complex vector $\psi \in \mathbb C^2$.

In technical language, what we want is to find a representation $\rho$ which eats an element of $\mathrm{SO}(3)$ and spits out an operator on $\mathbb C^2$. That operator must be invertible of course - we should be able to mathematically undo a rotation - and the space of such operators is $\mathrm{GL}(\mathbb C^2)$, called the "general linear group on $\mathbb C^2$." Therefore, the representation we want to find is a map $\rho:\mathrm{SO}(3) \rightarrow \mathrm{GL}(\mathbb C^2)$

---

We have some additional requirements we would like to impose on $\rho$. Firstly, it needs to be compatible with the group structure of $\mathrm{SO}(3)$. For example - let $R_{\hat n, \theta}$ correspond to rotation about the axis $\hat n$ by an angle of $\theta$. Then $$R_{\hat y,\pi/2} R_{\hat x,\pi/2} R_{\hat z,\pi/2} = R_{\hat x,\pi/2}$$ we should therefore have that $$\rho\big(R_{\hat y,\pi/2} \big)\rho\big(R_{\hat x,\pi/2}\big)\rho\big( R_{\hat z,\pi/2}\big) = \rho\big(R_{\hat x,\pi/2}\big)$$ More generally, if $R_1$ and $R_2$ are elements of $\mathrm{SO}(3)$, then $$\rho\big(R_2 R_1) = \rho(R_2)\rho(R_1)$$ This compatibility requirement is what makes $\rho$ a representation of $\mathrm{SO}(3)$. We would also like $\rho$ to be faithful - meaning that if $R_1$ and $R_2$ are different rotations, then $\rho(R_1)$ and $\rho(R_2)$ are different operators - and that $\rho$ be irreducible, which essentially means that it mixes the whole Hilbert space together and that there are no subspaces of $\mathbb C^2$ which are left completely unchanged by rotations.

---

So how do we find such a map? You could try to just write one down and see what happens, but you will be hamstrung by the problem that you have too much freedom in $\rho$. There are basically no constraints in how you might write a function which eats a $3\times 3$ rotation matrix and spits out a $2\times 2$ complex matrix. Knowing whether the result will be invertible, much less whether it obeys the compatibility condition we specified above, is a tall order and it seems from this perspective like the task is hopeless.

We are saved by considering infinitesimal rotations. If we let $R \approx \mathbf 1 + \epsilon A$ for some tiny $\epsilon$, then the defining characteristics of $\mathrm{SO}(3)$ (namely that rotations are orthogonal and have determinant +1) imply that $A$ is antisymmetric with trace 0. The space of all such matrices is called $\mathfrak{so}(3)$, and is the Lie Algebra corresponding to the Lie Group $\mathrm{SO}(3)$.

This $\mathfrak{so}(3)$ is a vector space, which will prove to be our saving grace. The standard basis $A_x,A_y,A_z$ for $\mathfrak{so}(3)$ (called infinitesimal generators of $\mathrm{SO}(3)$) is chosen such that e.g. $R_{z,\theta} = e^{\theta A_z}$. The group structure of $\mathrm{SO}(3)$ translates into a structure for $\mathfrak{so}(3)$ as well, but now in the form of the following commutation relations among the infinitesimal generators:

$$[A_i,A_j] = \epsilon_{ijk} A_k$$

The intimacy of the relationship between $\mathrm{SO}(3)$ and $\mathfrak{so}(3)$ is such that any element of the former can be written as the matrix exponential of some element of the latter. That is, for all $R\in \mathrm{SO}(3)$, there exist some $A\in \mathfrak{so}(3)$ such that $R = e^A$.

---

We are then led to the following idea. Perhaps we can simply look for a linear map from $\mathfrak{so}(3)$ to the $2\times 2$ complex matrices which obey the same commutation relations. Then if we exponentiate those, we might end up with the representation we seek. Disregarding a few subtleties for the moment, that ends up being precisely what happens. The map in question is provided by

$$\matrix{A_x \\ A_y \\ A_z} \mapsto \matrix{\frac{i}{2} \sigma_x \\ \frac{i}{2}\sigma_y \\ \frac{i}{2}\sigma_z}$$

And by appropriate exponentiation, we find that the group of rotation matrices on $\mathbb C^2$ is $\mathrm{SU}(2)$.

---

There are some subtleties along the way which I've swept under the rug, chiefly the fact that this map is not a true representation, but rather a projective representation (which is actually all that we need for QM). We can write the (projective) representation mapping from $\mathrm{SO}(3)$ to $\mathrm{SU}(2)$ as follows:

$$\mathrm{SO}(3) \ni R = \exp\left[\sum_{n=1}^3 c_n A_n\right] \mapsto \exp\left[i\sum_{n=1}^3 c_n \frac{\sigma_n}{2}\right] \in \mathrm{SU}(2)$$

Your main point seems to be that we should be able to choose some constant phase $e^{i\lambda}$ and write instead

$$\mathrm{SO}(3) \ni R = \exp\left[\sum_{n=1}^3 c_n A_n\right] \mapsto \color{red}{\exp[i\lambda]}\exp\left[i\sum_{n=1}^3 c_n \frac{\sigma_n}{2}\right]$$

This is true, insofar as it also provides a projective representation of $\mathrm{SO}(3)$. But why would we do this?

Answer 2

TL;DR: One can extend the Lie group action $\rho:G\times su(2)\to su(2)$ in Ref. 1 given by $$ g\quad \mapsto\quad\rho(g)\sigma~:= ~g\sigma g^{\dagger}, \qquad g\in G,\qquad\sigma\in su(2), \tag{1} $$ from the Lie group $G=\color{red}{SU(2)}$ to the Lie group $G=\color{red}{U(2)}$, cf. OP's question.

In more details:

1. The Lie algebra $$\begin{align} su(2)~:=~&\{\sigma\in {\rm Mat}_{2\times 2}(\mathbb{C}) \mid \sigma^{\dagger}=\sigma \wedge {\rm tr}(\sigma)=0\} \cr ~=~& {\rm span}_{\mathbb{R}} \{\sigma_k \mid k=1,2,3\},\end{align}\tag{2}$$ is the linear span of Pauli matrices.

2. There is a bijective isometry from 3D Euclidean space $(\mathbb{R}^3,||\cdot||^2)$ to the space of $2\times2 $ traceless Hermitian matrices $(su(2),\det(\cdot))$, $$\mathbb{R}^{3} ~\cong ~ su(2), $$ $$\mathbb{R}^{3}~\ni~\vec{x}~=~(x^1,x^2,x^3) \quad\mapsto \quad\sigma~=~\sum_{k=1}^3x^k\sigma_k~\in~ su(2), $$ $$ ||\vec{x}||^2 ~=~\sum_{k=1}^3x^{k} x^{k} ~=~-\det(\sigma), \tag{3}$$ cf. e.g. my related Phys.SE answer here.

3. Since $\rho$ is a continuous map and $G$ is a connected set, the image $\rho(G)$ must again be a connected set. In fact, one may show that there is a surjective Lie group homomorphism$^1$
   $$\rho: G \quad\to\quad SO(su(2);\mathbb{R})~\cong~ SO(3;\mathbb{R}) ,\tag{4} $$

4. One issue is: The kernel ${\rm ker}(\rho)$ of the Lie group homomorphism $\rho$ will depend on OP's choice of $G$. For $G=\color{red}{U(2)}$ one gets ${\rm ker}(\rho)\cong U(1)$, while for $G=\color{red}{U(2)}$ one gets ${\rm ker}(\rho)\cong \mathbb{Z}_2$.

5. By the isomorphism theorem, we get $$ \color{red}{U(2)}/U(1)~\cong~\color{red}{SU(2)}/\mathbb{Z}_2~\cong~SO(3) .\tag{5}$$

References:

1. eigenchris, Spinors for Beginners 6: Pauli Vectors and Pauli Matrices, 2023 YouTube video.