In Particle Physics, I've seen Scalar potentials which look like this $$ V = a \Phi^2 + b \Phi^4$$
$\Phi$ is scalar (a number).
What about vector potentials, and spinor potentials? How are they constructed? I know that when constructing expressions, a number of symmetries should be respected but I don't understand the whole story.
Potentials are always scalars, so there is no such thing as a vector or spinor potential. There are things called the "vector potential", but this is something else. You can learn the representation theory of the rotation group as follows:
The expression you give from particle physics is not constructed by symmetry, it is constructed from renormalizability. This is the reason that it looks the way it does, with quadratic and quartic pieces only.
The quick and useful way to construct things that are invariant is to understand Einstein summation convention and tensors. If you want to make something that is invariant, it should follow the Einstein convention for index contraction.
There are quantities $T_{abcd...}$ which are tensors, you contract the indices to make the appropriate indices left over. The rotation invariance is SU(2), so that the tensor indices range from 1 to 2 and take complex values. Lorentz invariance is two SU(2)s, so there are two kinds of indices both going from 1 to 2, and the values are complex. In particle physics, you need to understand bigger groups, at the very least SU(3) for the strong interaction, but the index contractions are always how you produce invariants.
The other thing you need are the invariant tensors, which are those tensors whose components are invariant under the group. For SU(N) you have $\delta_{ij}$ and $\epsilon_{i....z}$ where the number of indices is equal to N. You can use invariant tensors in appropriate contractions to make invariant objects.
