The invariance follows readily from Grassman algebras. The answer below consists of a crash course on Grassman algebras followed by a simple calculation to prove that the singlet is preserved under $U\otimes U$. My other answer extracts the key fact from this post to provide an elementary argument.
TL;DR
The singlet $|01\rangle-|10\rangle$ is distinguished among the four Bell states by its relationship to a certain Grassmann algebra. Namely, it is the sole Bell state which is not annihilated by the canonical projection of the two-qubit Hilbert space onto the second exterior power of the single-qubit Hilbert space. For a qubit, the second exterior power is a one-dimensional subspace preserved by $U\otimes U$, so the only possible linear action of $U\otimes U$ on the subspace is multiplication by a scalar.
Crash course in Grassman algebras
Definition
Let $\mathcal{H}$ denote a vector space over field $K$. Let $T^n(\mathcal{H}):=\mathcal{H}^{\otimes n}$ denote the $n$th tensor power of $\mathcal{H}$ and $T(\mathcal{H}):=\bigoplus_{n=0}^\infty T^n(\mathcal{H})$ the tensor algebra over $\mathcal{H}$. Elements of $T(\mathcal{H})$ can be multiplied by the scalars in $K$, added to each other and multiplied by each other using the tensor product $\otimes$.
Let $I$ denote the two-sided ideal in $T(\mathcal{H})$ generated by all elements of the form $|\psi\rangle\otimes|\psi\rangle$. The Grassman or exterior algebra of $\mathcal{H}$ is the quotient $\bigwedge(\mathcal{H}):=T(\mathcal{H})/I$. Elements of $\bigwedge(\mathcal{H})$ can be multiplied by scalars in $K$ and can be added to each other. They can also be multiplied by each other using the exterior product
\begin{align}
|\psi\rangle\wedge|\phi\rangle := |\psi\rangle\otimes|\phi\rangle + I.\tag1
\end{align}
An easy, but important observation is that
\begin{align}
|\psi\rangle\wedge|\psi\rangle=0\tag2
\end{align}
for every $|\psi\rangle$. This follows from the fact that $|\psi\rangle\otimes|\psi\rangle\in I$ for every $|\psi\rangle$. Taking for $|\psi\rangle$ a superposition of two states, one can generalize $(2)$ to
\begin{align}
|\psi\rangle\wedge|\phi\rangle=-|\phi\rangle\wedge|\psi\rangle.\tag3
\end{align}
Graded structure
The set $\bigwedge^k(\mathcal{H})$ consisting of all elements of $\bigwedge(\mathcal{H})$ that are exterior products of exactly $k$ elements of $\mathcal{H}$ is a vector space called the $k$th exterior power of $\mathcal{H}$. It is a simple exercise to verify that $\dim\bigwedge^k(\mathcal{H})={\dim\mathcal{H} \choose k}$. The algebra $\bigwedge(\mathcal{H})$ is the direct sum of the subspaces $\bigwedge^k(\mathcal{H})$ for $k=0,1,\ldots$.
Canonical projection
The canonical projection $\pi:T(\mathcal{H})\to\bigwedge(\mathcal{H})$ defined by $\pi(|\psi\rangle)=|\psi\rangle+I$ preserves the graded structure of the tensor and exterior algebras, i.e. it sends $T^k(\mathcal{H})$ to $\bigwedge^k(\mathcal{H})$.
Induced maps
Every linear map $A:\mathcal{H}\to\mathcal{H}$ induces a unique map $\bigwedge^k(A):\bigwedge^k(\mathcal{H})\to\bigwedge^k(\mathcal{H})$ which sends $|\psi_1\rangle\wedge\ldots\wedge|\psi_k\rangle$ to $A|\psi_1\rangle\wedge\ldots\wedge A|\psi_k\rangle$. In the special case where $k=\dim\mathcal{H}$ the map acts on the one-dimensional space $\bigwedge^{\dim\mathcal{H}}(\mathcal{H})$ as scalar multiplication by the determinant$^1$ of $A$. In particular, if $U$ is a single-qubit operator, then $U\wedge U$ sends $|\psi\rangle$ to $\det(U)|\psi\rangle$.
End of crash course.
Invariance of singlet under $U\otimes U$
Let $\mathcal{H}:=\mathbb{C}^2$ be the single-qubit Hilbert space and consider the restriction $\pi_2$ of $\pi$ to the second grade of the tensor and exterior algebras. In other words, consider the map $\pi_2:\mathcal{H}^{\otimes 2}\to\bigwedge^2(\mathcal{H})$ given by
\begin{align}
\pi_2(|\psi\rangle\otimes|\phi\rangle) := |\psi\rangle\wedge|\phi\rangle.\tag4
\end{align}
Note that $\dim\bigwedge^2(\mathcal{H})=1$, so by rank-nullity theorem $\pi_2$ has three-dimensional kernel. Indeed, by equation $(2)$, we have $\pi_2(|00\rangle)=\pi_2(|11\rangle)=0$. Consequently, the Bell states $|00\rangle+|11\rangle$ and $|00\rangle-|11\rangle$ are in the kernel of $\pi_2$. In addition, using $(3)$ we find
\begin{align}
\pi_2(|01\rangle+|10\rangle)&=|0\rangle\wedge|1\rangle+|1\rangle\wedge|0\rangle=\tag5\\
&=|0\rangle\wedge|1\rangle-|0\rangle\wedge|1\rangle=0.\tag6
\end{align}
By contrast,
\begin{align}
\pi_2(|01\rangle-|10\rangle)&=|0\rangle\wedge|1\rangle-|1\rangle\wedge|0\rangle=\tag7\\
&=|0\rangle\wedge|1\rangle+|0\rangle\wedge|1\rangle=\tag8\\
&=2|0\rangle\wedge|1\rangle\ne 0.\tag9
\end{align}
Thus, the singlet $|01\rangle-|10\rangle$ spans the preimage of $\bigwedge^2(\mathcal{H})$ via the restricted canonical projection $\pi_2$.
Finally, from equation $(4)$, we have
\begin{align}
\pi_2\circ(U\otimes U) = (U\wedge U)\circ\pi_2\tag{10}
\end{align}
but $U\wedge U$ is the scalar multiplication by the determinant of $U$, so the action of $U\otimes U$ on $|\Psi^-\rangle$ is to multiply it by the determinant of $U$.
$^1$ This is the only non-trivial fact whose proof is omitted in the crash course. However, note that we don't actually need to know that $U\wedge U$ is multiplication by $\det U$. All we need is that it is multiplication by some scalar. And this does follow trivially from the fact that the second exterior power of a two-dimensional vector space is one-dimensional.