In the NRQM'al theory of a charged particle, if we gauge transform the potentials using some arbitrary gauge function $\Lambda(\textbf{X},t)$ then Schrodinger's equation implies that the quantum state $|\psi\rangle$ changes to $$e^{ie\Lambda(\textbf{X},t)/\hbar c}|\psi\rangle$$ where $\textbf{X}$ is the position operator (so this is not a trivial numerical phase factor). I think if we want to establish that this is physically equivalent to $|\psi\rangle$ we need to show that the probability density is unchanged as well as expectation values. The former is trivial but not all expectation values are unchanged, specifically $\langle \textbf{P}\rangle$ will be different due to the fact that $\textbf{X}$ and $\textbf{P}$ do not commute. At this point, the usual explanation I've seen is that this expectation value is not observable so it's fine for it to change since it doesn't correspond to anything physical. Since all observable quantities supposedly don't change then the gauge transformed state is physically equivalent to the original one. My question is the following: are we appealing to experiment here (in telling us which quantities are observable and which are not) to get this conclusion? and is there a way to see that the two states are physically equivalent just from the theory?
In classical E&M the logic is more clear to me: the state of the system is described by its trajectory which from the Lorentz force law depends only on the electric and magnetic fields which are invariant under gauge transformations, so it seems that in classical E&M the fact that gauge equivalent potentials are physically equivalent just comes out of the theory.