Derive Poisson distribution from probability per time of event

Question

Suppose we have a probability per time $\lambda$ that something (e.g. nuclear decay, random walk takes a step, etc.) happens. It is a known result that the probability that $n$ events happen in a time interval of length $T$, is given by the Poisson distribution $$P(n) = \frac{e^{-\lambda T} (\lambda T)^n}{n!} \, .$$ How do we prove this?

Answer 1

Probability distribution of time until next event

First we calculate the probability density that a time $t$ passes without any event happening. Divide $t$ into $N$ small intervals each of length $dt = t/N$. Defining $\lambda$ as the probability per time that the event occurs, the probability that no event occurs within any one short time interval is approximately $(1 - \lambda dt)$. Therefore, the probability that no event happens in any of the intervals, but then does happen within an interval of length $dt$ right at the end of all the intervals is $$\left( \prod_{i=1}^N \left( 1 - \lambda dt \right) \right) \lambda dt = \left( 1 - \frac{\lambda t}{N} \right)^N \lambda dt \stackrel{N\rightarrow \infty}{=} \lambda dt \exp \left( - \lambda t \right) \, .$$ In other words, given a starting time $0$, the probability density that no event has happened after a time $t$, but then happens right at $t$ is $\lambda \exp(-\lambda t)$.

Probability of multiple events

Now we ask the probability that we get $n$ events in the time interval $T$. Suppose the first event happens at $t_1$, the second even happens at $t_2$, etc. We therefore have a series of intervals $$\{[0, t_1], [t_1, t_2], \ldots [t_n, T] \} $$ with events happening at the end of each interval. The probability that our events occur in this way is $$P(t_1, t_2, \ldots t_n) = \lambda \exp(-\lambda t_1) \lambda \exp(-\lambda (t_2 - t_1)) \cdots \lambda \exp(-\lambda(T - t_n)) = \lambda^n \exp(-\lambda T) \, .$$ Of course, any arrangement of $\{t_1, t_2, \ldots t_n \}$ such that $t_1 < t_2 < \ldots < t_n$ counts as an $n$-event arrangement, so we have to add up the probabilities of all these possible arrangements, i.e. the probability of $n$ events is \begin{align} P(n \text{ events}) &= \int_0^T dt_1 \int_{t_1}^T dt_2 \cdots \int_{t_{n-1}}^T dt_n P(t_1, t_2, \ldots t_n) \\ &= \lambda^n \exp(-\lambda T) \int_0^T dt_1 \int_{t_1}^T dt_2 \cdots \int_{t_{n-1}}^T dt_n \, . \end{align} The multiple integral is the volume of a right simplex and has value $T^n/n!$, so the final result is $$P(n\text{ events}) = \frac{(\lambda T)^n \exp(-\lambda T)}{n!} $$ which is the Poisson distribution with mean $\lambda T$.

Related

• Direct evaluation of the simplex volume integral
• Explicit computation for $n\in\{0, 1, 2\}$

Answer 2

The Poisson distribution describes the probability of a certain number ($n$) of unlikely events ($p\ll 1$) happening given $N$ opportunities.

This is like doing a very unfair coin toss $N$ times, with the probability $p$ of the coin turning up heads. The number of heads would follow the binomial distribution:

$$P(n|p,N) = ~^{N}C_n~p^n (1-p)^{N-n} =\frac{N!}{(N-n)!~n!} p^n (1-p)^{N-n}$$

Now it remains to prove that when $N\rightarrow \infty$ and $p\rightarrow 0$ while $Np\rightarrow \lambda T$, that the above converges to the known result. In essence, I argue that when you make the number of opportunities tend to infinity, you go from a discrete to a continuous approach; but as long as you are careful with your infinities, the result should still be valid.

First, we find an approximation for $(1-p)^{N-n}$. Taking the log, we get

$$\log\left((1-p)^{N-n}\right) = (N-n)\log(1-p)\approx (N-n)\cdot (-p)$$

Since $N\gg n$, we obtain $(1-p)^{N-n}\approx e^{-Np}$

Next, we approximate the $~^N C_n$ term using Stirling's approximation that $\log N! \approx N\log N - N$ and noting that $n\ll N$

Then

$$\begin{align} \log\left(\frac{N!}{(N-n)!}\right) &= N\log N - N - (N-n)\log(N-n) + (N-n) \\ &=N\log N - (N-n)\log(N-n) - n\\ &= N \log N -(N-n)\left(\log(N)+\log\left(1-\frac{n}{N}\right)\right) - n\\ &\approx N\log N -(N-n)\left(\log(N)-\frac{n}{N}\right) - n\\ &\approx n\log N + n -n \log n - n\\ &=n\log(N-n)\end{align}$$

It follows that $\frac{N!}{(N-n)! n!} \approx \frac{N^n}{n!}$

Finally, we note that $pN = \lambda T$, and we get

$$P(n|N,p) = \frac{N^n p^n e^{-Np}}{n!}$$

this easily rearranges to

$$P(n) = \frac{(\lambda T)^n e^{-\lambda T}}{n!}$$

Which is the result we set out to prove.

I made use of this article to remind me of some of the steps in this.

Answer 3

Let $A^n_t$ be the event: exactly $n$ point events happened over a time interval $t$. Then, for small $\Delta t$

$$\begin{align}P(A^n_{t+\Delta t}) &= P( A^n_t \cap A^0_{\Delta t }) + P(A^{n-1}_t \cap A^1_{\Delta t }) \\ &= P(A^n_t) P (A^0_{\Delta t }) + P(A^{n-1}_t )P( A^1_{\Delta t })\\ \end{align} $$ where we've used independence of occurrence. Now, defining $p_n(t) \equiv P( A^n_t)$ and $\lambda = \lim_{\Delta t\to 0} p_1(\Delta t)/\Delta t$, and taking the limit for $\Delta t \to 0$ we get

$$ p_n(t+\delta t)=p_n(t)(1 - \lambda \, \delta t) + p_{n-1}(t) \lambda \, \delta t $$

which leads to the differential equations:

$$p'_n(t)= \begin{cases} -\lambda ( p_n(t) - p_{n-1}(t) ) & n>0\\ -\lambda p_n(t) & n=0 \end{cases}$$

with the initial conditions $$p_n(0)=\begin{cases}0 & n>0\\1 & n=0 \, .\end{cases}$$

The solution is given by the Poisson distribution:

$$p_n(t)=\frac{(\lambda t)^n \exp(-\lambda t)}{n!} \, .$$

Answer 4

As pointed out in Daniel Sank's answer, once we know that the inter-event times are independent $$ T\sim\mathrm{Exponential}(\Phi), $$ with flux $\Phi$ in events/$s$, the proof quickly follows.

Denoting $S_1=T_1$ as the time the first event occurs and \begin{equation} S_n=T_1+T_2+\cdots+T_n \end{equation} as the time the $n$th event occurs, the total number of events per time interval $\tau$ is $N(\tau)=\{n:S_n\leq\tau\}$. From first principles it follows that the probability distribution of $N(\tau)$ is given by \begin{equation} \tag{1} \mathsf P(N(\tau)=n)=\mathsf P(S_n\leq\tau)-\mathsf P(S_{n+1}\leq\tau). \end{equation} Since the sum of independent exponential random variables is gamma distributed, it follows that each term on the r.h.s. of $(1)$ are gamma cumulative distribution functions, i.e., \begin{equation} \label{eq:pmf_of_N_2} \mathsf P(N(\tau)=n)=\frac{\gamma(n,\Phi\tau)}{\Gamma(n)}-\frac{\gamma(n+1,\Phi\tau)}{\Gamma(n+1)}, \end{equation} where $\gamma(a,z)=\int_0^zt^{a-1}e^{-t}\,\mathrm dt$ is the lower incomplete gamma function and $\Gamma(a)=\int_0^\infty t^{a-1}e^{-t}\,\mathrm dt$ is the complete gamma function. Using the recurrence relation for the lower incomplete gamma function (DLMF 8.8.1) and the property $n!=n\Gamma(n)=\Gamma(n+1)$ we find \begin{equation} \frac{\gamma(n+1,\Phi\tau)}{\Gamma(n+1)}=\frac{\gamma(n,\Phi\tau)}{\Gamma(n)}-\frac{(\Phi\tau)^ne^{-\Phi\tau}}{n!}. \end{equation} Thus the number of events per time interval $\tau$ is given by the Poisson distribution \begin{equation} \label{eq:pmf_of_N_3} \mathsf P(N(\tau)=n)=\frac{(\Phi\tau)^ne^{-\Phi\tau}}{n!}. \end{equation}