When we study non-gravitational fundamental interactions, we distinguish internal symmetries associated with only such interactions from the external symmetries of spacetime. For all fundamental interactions, there is a finite-dimensional Lie group characterizing that interaction's symmetries. In the case of gravity, the Lie derivative of Killing vector fields on the spacetime manifold defines the associated Lie algebra's structure constants; for the other interactions, there is a "space" that plays a role analogous to this manifold, but it's not spacetime itself. Instead, it's a space of legal values for a field over spacetime.
For example, electromagnetism's $U(1)$ symmetry (let's put electroweak unification aside for the moment) is the rotational invariance of $|\phi|$ for $\phi\in\Bbb C$ with $|\phi|:=\sqrt{\phi^\ast\phi}$, or equivalently for $\phi\in\Bbb R^2$ with $|\phi|:=\sqrt{\phi\cdot\phi}$. (I'm denoting the set of values $\phi$ can have at each point in spacetime, say $\Bbb R^4$, so as a function $\phi\in X^{\Bbb R^4}$ for $X=\Bbb C$ or, somewhat less helpfully in QFT, $X=\Bbb R^2$.) So if there is a space which is "curved" in this context by electromagnetism, it is not spacetime per se.
As for unification implications, Wikipedia notes
For ordinary Lie algebras, the gauge covariant derivative on the space symmetries... cannot be intertwined with the internal gauge symmetries... this is the content of the Coleman–Mandula theorem. However, a premise of this theorem is violated by the Lie superalgebras (which are not${}^\dagger$ Lie algebras!) thus offering hope that a single unified symmetry can describe both spatial and internal symmetries: this is the foundation of supersymmetry
${}^\dagger$ emphasis in the original.
