I saw this video which provides a clear, though simple, introduction to the argument. I think the main thing missing is a proof for:
which is presented as sort of a given. I saw a bunch of sites. Some say it's due to Cleibsch-Gordan coefficients, others begin with the Quadratic Stark Shift... maybe it's all connected, but I'm still confused.
There is a mathematical and a physical way to look at it. From the physical point of view, photons carry angular momentum $\pm 1$. Thus, due to the conservation of the total angular momentum, the absorption/emission of a photon is possible only between the states whose total angular momentum differs by $1$. Since only one electron participate sin an absorption/emission event, flipping its spin could account for this, but this does not necessarily happen - hence the rule for the orbital angular momentum.
Mathematically, this follows from calculating the matrix element of the atom-photon interaction between the relevant atomic orbitals (i.e., the matrix element that enters the Fermi golden rule). In the dipole approximation this interaction is proprotional to $\mathbf{r}$, i.e. we calculate $$ \int d\mathbf{r}\psi_{final}^*(\mathbf{r})\mathbf{r}\psi_{initial}(\mathbf{r}) $$ The transitions are possible only when this matrix element is not zero. This is where Cleibsch-Gordan coefficients or some other math may enter - depending on the specific situation one is dealing with.
Remark: here is my short derivation of the dipole approximation (in somewhat different context)
