Magical relationship between Exponential distribution and Poisson process

Question

Consider i.i.d. random variables $X_1,X_2,\ldots,X_n$ satisfying exponential distribution $\operatorname{Exp}(1)$. Let $Y=X_1+X_2+\ldots+X_n$. We know that the p.d.f. of $Y$ is the Gamma distribution $$ f_Y(t)=\frac{t^{n-1}}{(n-1)!}e^{-t}, (t\geq 0). $$ What surprised me is that this is exactly the p.m.f. of a Poisson distribution with parameter $t$. Define a Poisson process with rate $1$, $N_t$, then $\frac{t^{n-1}}{(n-1)!}e^{-t}$ equals $P(N_t=n-1)$. Since we can define Poisson process with i.i.d. exponentials, we actually can write $$ P\left(\sum_{i=1}^{n-1}X_i\leq t, \sum_{i=1}^n X_i>t\right)= P(N_t=n-1)=\frac{t^{n-1}}{(n-1)!}e^{-t}. $$

I believe there must exist a intuitive explanation of this relationship. But I couldn’t find it.

Answer 1

I will give an intuitive explanation, only needing the defining properties of the Poisson Process and the exponential distribution, without needing any calculations involving densities.
In short, the Poisson Process having independent and stationary increments and the exponential distribution being memoryless explains this connection:

More precisely, the Poisson Process is an $\mathbb N_0$-valued process with the property that for all $s,t>0$ we have that $N_{t+s}-N_t$ and $N_t$ are independent, or more generally that for $t_0<t_1<\dots<t_k$ that $$N_{t_k}-N_{t_{k-1}},\dots, N_{t_1}-N_{t_0}$$ are independent. In simpler terms, the knowledge that $x$ jumps occured in the time interval $[t_{i-1},t_i)$ does not help you predict the amount of jumps that occured in the time interval $[t_i,t_{i+1})$. Furthermore, the distribution of $N_{t+s}-N_t$, i.e. the jumps in the time interval $[t,t+s)$, is the same as the one of $N_s$, i.e. the jumps in the time interval $[0,s)$. Hence the distribution of the amount of jumps only depends on the length of the time interval.

The exponential distribution being memoryless means that if $T\sim\text{Exp}(\lambda)$ for all $x,y>0$ we have $$\mathbb P\bigg[T>x+y\big| T>y\bigg]=\mathbb P[T>x].$$ In simpler terms, if you know that $T>y$, than the probability that $T$ is greater than an additional amount of $x$ is exactly the same as the probability that $T$ was greater than $x$ in the first place.

Now we are ready to see the connection: For simplicity, let $T$ be the distribution of the first jump of the Poisson Process $N$. Then by definition $$\begin{align*}\mathbb P\bigg[T>t+s\big|T>t\bigg]&=\mathbb P\bigg[N_{t+s}=0\big| N_t=0\bigg] \\ &=\mathbb P\bigg[N_{t+s}-N_t=0\big| N_t=0\bigg] \\ (\text{independence of increments}) &= \mathbb P[N_{t+s}-N_t=0] \\ (\text{stationarity of increments})&= \mathbb P[N_s=0] = \mathbb P[T>s] \end{align*}$$ Thus we have shown that $T$ must be memoryless. Once we convince ourselves that the distribution of $T$ must also be continuous, we necessarily have that $T$ follows an exponential distribution, as they are the only memoryless distributions without atoms.

You may try to repeat this argument for not only the first waiting time $T$, as I have done above, but for more general interarrival times, i.e. the times between consecutive jumps. Then you will have shown that the waiting times of a homogeneous Poisson Process are i.i.d. exponentially distributed, as @geetha290krm already noted.