Why doesn't matter collapse, if I can view it as being made of bosons (atoms/molecules)?

Question

I have often heard it said that the Pauli exclusion principle is responsible for the existence/stability of bulk matter. Wikipedia makes such claims and quotes great physicists who made such claims. The idea, I think, is that since electrons are fermions, they cannot all fall into the lowest-energy orbitals, and this would dramatically change all chemistry and solid state physics (certainly true!). Maybe more generally, identical fermionic particles in any piece of matter cannot crowd into the same state, preventing a "collapse." I have heard the same idea used (perhaps more sketchily) to explain why solid matter cannot pass through itself.

The problem with these explanations is that physical systems can be viewed on different scales and described on different levels, and all descriptions should make the same prediction. In particular, I can view matter as made of atoms (bosons, generally) or molecules (bosons, generally), and this should not invalidate or completely change the predicted behavior. But bosons don't obey the exclusion principle, so naively it would seem that matter should "collapse" in some sense, and be able to pass through other solid matter. So what resolves this seeming contradiction?

---

Some thoughts and possible directions:

1. The first idea is wrong. While the Pauli exclusion principle (PEC) is surely behind electrons not falling into their lowest orbitals (and hence crucial for explaining chemistry), the other ways PEC is invoked might be inaccurate. (E.g., the explanation of matter not passing through other matter probably involves complicated interactions going on at the surface where two solid-state systems come into contact.)
2. The second idea is wrong. Perhaps my claim that "matter is made of atoms, which are bosons" is inaccurate. E.g., the atoms in metal do not have all of their electrons localized near the nucleus or in traditional orbitals, so maybe metal being described as a system of bosons (which should then fall into the same lowest-energy state) is wrong.
3. Temperature. Even an ideal system of bosons doesn't actually fall into its lowest energy state at finite temperature. Maybe typical temperatures are too high for any sort of condensation of its atoms, or related "bosonic phenomena."

Answer 1

TL;DR: you have successfully predicted the existence of Bose-Einstein condensates at low temperatures

Fermi-Dirac and Bose-Einstein statistics each describe distinct types of quantum system: the former describes those with half-integer spins, fermions, and the latter describes those with integer spin, bosons (hence the names for the types of particle). The statistics state that the average number of fermions $\tilde n_i$ in a given state $i$ at a given temperature $T$, total chemical potential $\mu$, and single-particle energy $\varepsilon_i$ is

$$\tilde n_i=\frac{1}{e^{(\varepsilon_i-\mu)/k_BT}+1},$$

while for bosons the number is

$$\tilde n_i=\frac{1}{e^{(\varepsilon_i-\mu)/k_BT}-1}.$$

The former Fermi-Dirac statistics end up giving a result around $\frac{1}{2}$ for $T\approx0$ (approximate because it can't ever really be zero), while for the latter Bose-Einstein statistics the number actually approaches infinity as $T\to0$.

In fact, this does mean that solid matter can start behaving strangely at low temperatures. This is why we consider Bose-Einstein condensates (which, given the eloquence of your question, I'm sure you've heard of) to be a completely separate state of matter. At that temperature, quantum effects begin to prevail across larger distances as bosonic atoms/molecules start violating classical mechanics in weird ways that aren't entirely relevant. BEC is characterized by the cooling of integer-spin atoms (often rubidium) until they begin behaving like bosons and "collapsing" into a weird new state of matter.

In other words, you are exactly correct! At low temperatures, bosonic atoms like rubidium do in fact stop behaving normally, and although the electron clouds remain to keep particles separated through electrostatic repulsion and Pauli exclusion, the physics of the nuclei begins to change very close to absolute zero.

Key phrase here: very close to absolute zero. As you should also know about BECs, they take an extreme amount of cooling to produce and, due to temperature/quantum fluctuations, also don't last very long. As temperature goes up, Fermi-Dirac and Bose-Einstein statistics get more and more similar until, at temperatures even a little greater than absolute zero, you can just say that you can ignore the difference between them. Pauli exclusion isn't really even why atoms and molecules don't pass through each other, although it is why it's always only two electrons to an orbital; electrostatic repulsion normally handles that, although in extreme cases like neutron stars that can begin to break down and Pauli exclusion is then what's keeping neutron stars from undergoing further collapse (the outward force Pauli exclusion causes is called electron degeneracy pressure, and when it fails, you have a black hole).

Answer 2

You may look at my answer elsewhere. Briefly, there are two definitions of bosons and fermions, one of them is based on the total spin and the other one is based on the symmetry properties of the wave functions. Composite particles are not exactly bosons or fermions under the second definition, and the commutation relations of the creation/annihilation operators of composite particles do not quite coincide with such relations for elementary particles, so, e.g., atoms with integer total spin will not behave like bosons at high density. @controlgroup discusses BECs, but BECs made of atoms typically have very low density compared to that of solid state.

Answer 3

I kinda don't like the existing answers, so here goes. Let us begin with some pedantic corrections.

controlgroup correctly points out that Bose-Einstein condensates exist. His expression for the limit of the Fermi-Dirac statistics is wrong, though. Let $\theta$ refer to the (symmetric) Heaviside step function, i.e. $$\tag1\theta(x)=\begin{cases}1&,x>0\\0&,x<0\\\frac12&,x=0\end {cases}$$ then, the Fermi-Dirac statistics tends to $$\tag2\lim_{T\to0}\tilde n_i=\theta(\mu-\varepsilon_i)$$ i.e. fully occupied ($=1$) below the chemical potential, and completely empty ($=0$) above. He said the function is half in that limit, but that is obviously wrong. That erroneous value would not even be renormalisable: it would have infinitely many particles and all sorts of other nonsense.

However, it will soon become clear that this is only one side of the story.

---

akhmeteli is also obviously trying to convey a physical fact, but that is expressing it in a rather awkward manner. His claim is also technically wrong; while, yes, composite particles are only approximately handled by coarse-graining their wavefunctions, in particular, treating the extended multi-position constituent wavefunction as just the point-like wavefunction of the centre of mass, but this point-like wavefunction actually has to be boson-symmetric, in direct contradiction to what he claims. It is necessarily so, to obtain the boson statistics.

---

1. The first idea is wrong.

No, the idea is approximately correct. First off, let's be clear with the terminology. Pauli's Exclusion Principle came before we realised that this is coming from fermion statistics, which in turn comes from skew-symmetry of the wavefunction when you exchange particle identifiers. This skew-symmetry causes "free particles" to have what is called degeneracy pressure.

When you actually attempt to quantify the repulsion of matter from penetrating matter, at otherwise zero pressure, you will realise that the contribution from degeneracy pressure and the contribution from the short-range effective repulsion from electrostatic potential, to be roughly the same magnitude. i.e. the kinetic energy increase from degeneracy pressure and the electrostatic potential energy increase from quasi-neutral electrostatic repulsion at short distances give rise to the actual reason why matter does not penetrate matter.

This is not exactly complicated; only the derivation might be. The standard argument as given in textbooks might not be the full picture, but it is pretty much good enough; I mean, you get about half the contribution, which is more than good enough when all you really care about is getting within an order of magnitude.

3. Temperature.

This is the only one that is unequivocally wrong. Bose-Einstein condensation happens, and so what you are saying there is entirely wrong.

2. The second idea is wrong.

It is also approximately correct. Between controlgroup's and akhmeteli's answers, lies a nuanced answer:

The bosonic symmetry of the simplified wavefunction, only encapsulating the centre-of-mass's degrees of freedom, is an approximation scheme that is wildly successful, but gives the wrong impression that matter would collapse. The correct picture that it captures is that, once you fix the non-centre-of-mass degrees of freedom to just signify the ground state, then, whenever this ground-state approximation still holds as good, then the Bose-Einstein condensation actually can happen. The quantum system can behave as if a macroscopically large number of the molecules have collapsed into one single ground state, exhibiting the collective quantum behaviour that is so counter-intuitive, contrary to classical expectations.

But the underlying reality is still made up of fermions. The fermionic repulsion is still there. This means that, for each additional bosonic composite particle that occupies the ground state, its internal degrees of freedom must occupy higher and higher energy states, e.g. wider and wider momentum pairs, if they are Cooper pairs. You originally had a filling up of states up to the Fermi surface, so this is really just a slight modification of that, even before the electron-phonon coupling, which is so important for creating the Cooper pairing in the first place, starts making the corrections near to the Fermi surface. There is no method at which the Bose-Einstein condensation would result in a collapse, if you consider this properly.

There is another nuance. In QFT, which is what you really need to do to consider interacting systems properly, you will also learn and realise that bosonic fields must have an interaction that has a repulsive core. Or else, the field would not be renormalisable, or that the field would not have a sensible physical interpretation. Sadly, I cannot remember which nice condensed matter field theory book it was that I got to read this fact, but seems to not be in Fetter & Walecka, even though F&W just straight up suddenly discussed the bosonic system with repulsive cores.

Anyway, with these nuances, you should be able to see that quantum theory is not in disagreement with structural stability of matter. At least, until some black hole physics intervenes. Last I heard, we still do not have any physical way to prevent the formation of a black hole singularity, but that is a problem for quantum gravity to solve. We should take the current results as good enough, at least for now.