What would be the proper distribution to model the number of particles in a state in canonical ensemble

Question

Suppose my system has $N$ particles, and I want to find a distribution for $n_i$, the number of particles in the $\epsilon_i$ energy state.

What I do know is the boltzmann probability, which tells me the probability of a single particle being in a certain energy state, and is denoted using $p_i$. I also know the expected number of particles in a system, given by $p_iN=\mu$.

According to some people that I asked, the distribution of particles would follow a binomial distribution, where the number of particles in the state would be a random variable. Hence, it would be given by :

$$p(n_i)={^{N}C_{n_i}}\space p_i^{n_i}(1-p_i)^{N-n_i}$$

Using this I can find the probability of each possible occupation number, through a binomial distribution.

Physically, this is equivalent to picking $N$ particles out of the system and counting the number of particles in $n_i$, and then repeating this experiment many many times and checking the frequency of each value of $n_i$. Because of fluctuations, we can be sure that $n_i$ keeps on changing every time I do this trial. It is basically like the distribution of heads in a $100$ coin tosses.

However, there is a second distribution that came to my mind. We know that the system can have $n_i$ particles, where $n_i$ ranges from $0$ to $N$. So, we consider $N+1$ different systems, labelled from $0$ to $N$, where the label denotes the number of particles in $\epsilon_i$ state of this system at a particular timestamp. Now what we do is, we check the probability of each of these systems of being the original system. Since these systems have a fixed number of $n_i$ each, we are indirectly getting the distribution of the number of particles $n_i$ of the original system.

So, what we check is, the probability of each of these systems, to give us a $p_i$ probability of picking up a particle at random, and finding it is in the $i$ state. In a sense, this is probability of probabilities.

How do we do that ?

Well, you know the expected number of particles in the energy state is given by $\mu$. So, we take each of these systems, and take out $N$ particles with replacement, and count the number of particles in that state, and repeat this experiment many many times. Whenever $\mu$ out of $N$ particles are in the desired state, we consider it a success. We check the frequency of success for each of these $N+1$ systems. This would give us a probability distribution of which system is how likely to give us a $p_i$ chance of picking up a random particle and finding it in a particular state. This distribution would be given by :

$$p(n_i)={^NC_\mu} \space\space(\frac{n_i}{N})^\mu \space\space (1-\frac{n_i}{N})^{N-\mu}$$

This distribution is not normalized, and looks a lot like the Binomial distribution. However, this takes into account, that for each system, the probability of getting a certain particle to be in a certain state, would be different. Moreover, the physical analogy also makes more sense in this case.

As discussed in my previous question, the boltzmann distribution is more like an estimator of the true probability of finding a particle in a certain state, as written by @Roger Vadim in this answer. So, instead of the distribution of the particles being a perfect boltzmann distribution, should it not be the distribution of different systems based on how likely the probability of getting a single particle in $i$ state in that system matches the boltzmann probability of the same.

In other words, shouldn't the second distribution be more accurate ?

I've checked that for large numbers, the two distributions produce more or less the same result, but I wanted to know which one is more accurate.

Answer 1

Counting how many times the system's actual $n_i$ value comes out to its expected value, as you are doing in the second part, is finding the distribution for the estimator of another distribution's parameter. This is where all the business about T, F and $\chi^2$ distributions comes from in statistics that is supposedly focused on Gaussian random variables.

For the distribution of $n_i$ which you are actually looking for, binomial is correct. I will take $i = 1$ without loss of generality and write it as \begin{equation} P(n_1) = \binom{N}{n_1} \left ( \frac{e^{-\beta E_1}}{Z_1} \right )^{n_1} \left ( 1 - \frac{e^{-\beta E_1}}{Z_1} \right )^{N - n_1}. \end{equation} To check this, we can go back to the brute force definition. What is the probability that there are $n_1$ particles in level $1$? It's the probability that the system is in a microstate with $n_1$ particles in level $1$, summed over all such microstates. This means we take the $N$ particle partition function \begin{equation} Z_N = Z_1^N = \sum_{n_1 + \dots + n_k = N} \binom{N}{n_1, \dots, n_k} e^{-\beta(n_1 E_1 + \dots + n_k E_k)} \end{equation} and perform a very similar sum where $n_1$ is held fixed. \begin{align} P(n_1) &= Z_N^{-1} \sum_{n_2 + \dots + n_k = N - n_1} \binom{N}{n_1, \dots, n_k} e^{-\beta(n_1 E_1 + \dots + n_k E_k)} \\ &= Z_N^{-1} e^{-\beta n_1 E_1} \binom{N}{n_1} \sum_{n_2 + \dots + n_k = N - n_1} \binom{N - n_1}{n_2, \dots, n_k} e^{-\beta(n_2 E_2 + \dots + n_k E_k)} \\ &= Z_N^{-1} e^{-\beta n_1 E_1} \binom{N}{n_1} \left ( Z_1 - e^{-\beta E_1} \right )^{N - n_1} \\ &= Z_1^{-N} e^{-\beta n_1 E_1} \binom{N}{n_1} \left ( Z_1 - e^{-\beta E_1} \right )^{N - n_1} \end{align} The claim now follows. To go from line 2 to 3, we had to remember that the level $E_1$ was no longer present, making the combinatorial sum equal to a power of a corrected $Z_1$.

Discussion of the alternative scenario

After sleeping on it, I think the experiment you describe is a little more puzzling than constructing an estimator. First, when I see

$${^NC_\mu} \space\space(\frac{n_i}{N})^\mu \space\space (1-\frac{n_i}{N})^{N-\mu}$$

that is clearly the probability for something to happen $\mu$ times. Yet you are interpreting it as the probability for a certain variable to be equal to $n_i$ by saying that it needs to be normalized by hand. This is a red flag. It is common for physics arguments to only tell us "what probabilities are proportional to" but ideas like the law of large numbers are mathematical, not physical.

So I will try to make an analogy with coins. It will actually be an analogy to the formula

$${^NC_\mu} \space\space(\frac{1}{N})^\mu \space\space (1-\frac{1}{N})^{N-\mu}$$

because I don't know how you get $n_i$ to appear in the way it did. One of the $N$ copies has $n_i$ particles in state $i$. Not $n_i$ of the $N$ copies.

Anyway, suppose you bought a coin where heads comes up with probability $p$ and tails with probability $1 - p$. After $N$ flips, $\mu = Np$ is the expected number of heads. Now suppose you go to the same store and buy the same coin again. With your two identical coins, you paint over the head of one and the tail of the other and put them in a hat.

You can reach into this hat and draw coins with replacement many times and build up a distribution. But there is one coin of each type so this will be the distribution of a fair coin. I.e. every draw from the hat is like a 50-50 flip, not a $p$ biased flip. So nothing you do with the hat will give information about the probability of heads coming up a certain number of times in the original problem.

Answer 2

Based on the chat, the comments and the answers in this question and a related one, I seem to have found the primary source of my confusion.

The first thing to remember here is that our system is dynamic. This means that at any time, due to collisions and other phenomena, the number of particles that occupy any energy level keeps changing, and thus, the energy of this system keeps fluctuating. This has to be kept in mind.

Before I can explain the confusion further, I need to talk about the source of this confusion that comes from a related but vastly different problem.

In the above problem I had a bag with a fixed number of marbles in it, and some of them were blue. I used sampling to find the approximate probability of picking up a random ball and finding it to be blue. Using this, I wanted to create a probability distribution for the number of blue marbles in the bag.

The correct way to deal with this problem is to remember that the number of blue marbles would be between $0$ and $N$ where $N$ is the total number of marbles. So, we create $N+1$ different systems labelled from $0$ to $N$, where the label denotes the number of blue marbles in the system. Using the methods discussed in the answers there, we find the required distribution. However, in short, what we do is, find the probability of each of these $N+1$ systems, of being our original system ( most similar ). That would indirectly tell us the probability of having different numbers of blue marbles in the original system i.e. our required probability distribution.

If you think about it carefully, we have just created an ensemble here. We have replaced our system with $N+1$ indentical copies, and tried to find out the probability of our original system being any one of these.

We tried to use this same logic and analogy for the number of particles in a certain energy level in our system. However, there were a few major flaws, that make the two cases similar, but not identical.

Let us note these flaws first :

1. In case of the marbles, the system is not dynamic. The number of blue marbles in the bag, although a random unknown variable, is fixed i.e. it doesn't change. You can think of it as being frozen in time. However, in case of our particles, as we have mentioned in the beginning, the system is dynamic, the number of particles in a state, keeps changing again and again.

This creates a problem. Suppose you do make an ensemble with $N+1$ identical states, just like the marble case. However, in the marble case, the original system was static, and hence, the $N+1$ systems were also constant ( distinguishable ). In case of the particles, since your original system is dynamic, your $N+1$ systems must also be dynamic. If they are dynamic, they are all identical to each other, as they are all changing constantly, and you'd get no answer about which one is more likely and so on, because you might not have $N+1$ different answers to choose from.

As you can see, because of this reason, you can't use the reasoning that you used in case of the marbles. Maybe if you freeze the entire system in time, along with the $N+1$ copies, then you can use it, but that would be worthless the second you 'unfreeze' this system.

2. The second difference is that the marbles in a bag, are not independent from each other. The probability that you pick up a blue marble, depends on how many marbles in the bag are actually blue ( a fixed number ). In our particles case, we have assumed that they don't interact. Hence, each particle can be thought of as independent from the rest.

Instead of finding an analogy between the marbles and the particles, it would be better to compare these particles to be individual dice. The probability that a single die would roll a six, is practically independent of how many of the $N$ other identical dies in the room, rolled a six.

Another thing that should be emphasized is that, the colour of a marble is an unique property and is fixed. A blue marble doesn't become green. However, the number on a die, is not fixed. A die can show a different number on each roll. Our particles are dynamic, just like these dice. So, the physical analogy of checking the energy of a single particle would not be picking up a marble from a bag of marbles, and checking the probability of it being blue. It would be more like, picking up a die at random from a bag full of dice, and checking what is the probability that it would 'roll' a six.

As you can see, the first analogy depends on the number of marbles that are blue. Moreover, remember, since the marbles have unique colour, they are different from one another, in a sense they are distinguishable. Our particles are like identical dice ( biased dice, since lower energy levels are more probable, but identical nonetheless ).

Hence, the probability of finding a particle in a certain state, has nothing to do with the actual number of particles in that state. It would have mattered if the particles had a fixed energy - in that case the probability of finding a particle in that particular energy state, would have depended on the number of particles in that state. Since, they are dynamic, it doesn't matter.

Since we have managed to show that the probability of a single particle of having a certain energy is independent of all the other particles, finding the probability that $n$ particles have the same energy, becomes a straight forward problem, related to binomial distribution. The probability that $n$ out of $N$ particles are in the $\epsilon_i$ state, is basically finding the probability that each of the $n$ particles are in that state, multiplied by the probability that the rest $N-n$ of them, are not in that state. Moreover, if the particles are dynamic and distinguishable, you also have to multiply the number of ways of choosing $n$ particles from $N$. This is nothing but the binomial distribution that @Connor Behan derived in a different answer.

P.S. note that I've used the word distinguishable in two different senses here. In the first sense, two different marbles or two different particles or dice, can be distinguished from one another. In this sense, the particles are distinguishable. However, even though two dice or particles are distinguishable, they have identical properties. In fact, all the dice and the particles have identical properties, they are all dynamic, and they don't have some unique energy or something. In this sense, the particles are identical. However, in case of the marbles, all the marbles don't have the same property. Not only are they distinguishable from each other, we know that blue marbles are different from green marbles and so on. So, they are distinguishable in the sense of their properties too. The particles are identical in this sense, because there is nothing call a blue particle or a green particle and so on - any one of them can be anything !