Intuitively, why does Quantum Mechanics involve a sum over all possibilities?

Question

I understand that one can just mathematically derive the path integral from the Schrodinger equation. I'm looking for an intuitive explanation in contrast with classical mechanics.

Consider a probability distribution on the classical phase space. We can try to represent the time evolution of this probability distribution in a path-integral fashion. But the only paths we'd need to consider would be the classical paths.

For example, let's say a particle is 50% likely to occupy the state $(x, p)$ and remaining 50% for the state $(x', p')$. The time evolution of this state can be calculated as the 50-50 probabilistic superposition of the classical paths $(x(t), p(t)) $ and $(x'(t), p'(t)) $.

Contrast this with Quantum Probabilities, where the two key differences are :

1. We need to consider all possible paths.

2. The paths can interfere destructively.

Now, the key difference between quantum and classical mechanics is the non-commutativity of observables. I would like to intuitively understand how exactly is non-commutativity tied to these two new properties of Quantum evolution compared to classical mechanics:

1. All paths are considered

2. Paths can carry negative weights.

Answer 1

I suspect you might be ready to study the third autonomous complete formulation of QM, namely the phase-space formulation. In it, in contrast to classical mechanics addressing delta functions in phase space, you study "fuzzed-up" quasiprobability distributions in phase space, representing density matrices in phase space. There is a direct technical connection to the sums over all possibilities of the path integral.

They are not quite probability distributions, as two points in phase space sufficiently close (of order-ℏ close) are not exclusive of each other (thus violating Kolmogorov's third axiom of bonafide probabilities, on independence/exclusion of alternatives), since you know they are linked by the uncertainty principle, something built in the formulation.

In addition, or, rather, in connection to that, the quasibrobability distributions may, and often must, get negative in "small" phase-space regions (a highly technical term). If you watch the movies in the link of the previous paragraph, you see how these "clouds" of quasi probability evolve in time and influence each other nonlocally: Normally, all points of the distribution affect the evolution of all other points (with the exception of the free particle, the linear potential, and the oscillator).

In the classical limit, this nonlocality goes away and the fuzzy distributions morph to local δ-functions.

The noncommutativity involved is further built in the formulation and is solely carried by just one operator in the formulation, the star product, carrying the non-commutativity burden for absolutely everything! Ultimately, it is responsible for transitioning from δ-functions to Wigner function distributions.

Answer 2

Contrast this with Quantum Probabilities, where the two key differences are :

1. We need to consider all possible paths.
2. The paths can interfere destructively. how exactly is non-commutativity tied to these

I'd argue that's putting the cart before the horse. My intuition sequence would be:

• Quantum mechanics has a "non-classical" probability theory, resulting from the need for continuous state evolution, that's associated with complex amplitudes, hence your second key difference.
• This complex-amplitude theory requires pure state vectors in a complex Hilbert space, and more general mixed state matrices. Everything you can do with the former, you can do with a special case of the latter, and we can then obtain expectations of linear operators from the Hilbert space to itself.
• Since the expectations thereby obtained are those of suitable probability distributions, the operators are observables. But, as is so often the case with matrices, they don't always commute.
• If you divide a period of time evolution into many brief steps, you recover path integrals like $Z=\int\mathcal{D}\varphi\exp\frac{iS[\varphi]}{\hbar}$ rather than $Z=\int\mathcal{D}\varphi\exp\frac{iS[\varphi]}{\hbar}\delta(E[\varphi])$ with $E$ an Euler-Lagrange operator, which would limit us to paths satisfying $E[\varphi]=0$. Fortunately, large-$E[\varphi]$ paths don't contribute much because in QM (QFT) $E=\frac{\partial S}{\partial\varphi}$ ($E=\frac{\delta S}{\delta\varphi}$; see here for a notation explanation). Therefore, such a $\delta$ factor would be redundant.
• However, this has one downside: if there's a gauge symmetry, physically equivalent states/paths lead to an overcounting problem. Funnily enough, (different) $\delta$ factors can fix this.

We've gone from difference $2$ to commutators and difference $1$, and commutators aren't the crux of difference $1$ either because arbitrarily brief time slices make commutators negligible.