The Poisson, Exponential, and Gamma Distributions
These three types of random variable arise when waiting for “events” of a certain kind to happen. This could be waiting at a pond for some ducklings to appear, or looking out for shooting stars in space.
There are three assumptions about the process:
- the events or (“accidents”) must happen at a constant rate
- the events must be independent
- the probability of more than one happening at the same time is negligible
A Poisson distribution $N$ counts the number of events that occur within a fixed period of time:
| Notation: | $N \sim \text{Pois}(\lambda)$ |
| Type: | $ \text{Discrete} $ |
| PMF: | $\mathbb{P}(N = k) = \frac{\lambda^k e^{-\lambda}}{k!} $ |
| Support: | $k \in \{0,1,2,\dots\}$ |
| Mean: | $\mathbb{E}(N) = \lambda$ |
| Variance: | $\mathbb{V}\text{ar}(N) = \lambda$ |
An exponential distribution $X$ gives the time until the first event:
| Notation: | $X \sim \text{Exp}(\lambda)$ |
| Type: | $ \text{Continuous} $ |
| PDF: | $ f_X(x) = \lambda e^{- \lambda x}, \ x \ge 0 $ |
| CDF: | $ F_X(x) = 1- e^{- \lambda x}, \ x \ge 0 $ |
| Support: | $x \in [0,\infty) $ |
| Mean: | $\mathbb{E}(X) = \frac{1}{\lambda}$ |
| Variance: | $\mathbb{V}ar(X) = \frac{1}{\lambda ^2} $ |
The exponential is “memoryless”; that is, at any point, it doesn’t matter how long we’ve already been waiting so far – the outlook for waiting for the event to occur is still the same.
If $\alpha$ is a positive integer, the gamma distribution $Y$ gives the time until the $\alpha$-th event has occurred.
| Notation: | $Y \sim \text{Gamma}(\alpha,\beta)$ |
| Type: | $ \text{Continuous} $ |
| PDF: | $ f_Y(y) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} y^{\alpha-1} e^{- \beta y}, \ y≥0 $ |
| Support: | $y \in [0,\infty)$ |
| Mean: | $\mathbb{E}(Y) = \frac{\alpha}{\beta}$ |
| Variance: | $\mathbb{V}ar(Y) = \frac{\alpha}{\beta ^2} $ |
However, the gamma distribution is also defined for any $\alpha>0, \beta>0.$
The $\Gamma(\alpha)$ in the PDF is the “gamma function”.
If $\alpha$ is an integer, we have:
$$\Gamma(\alpha) = (\alpha-1)! = (\alpha-1)\times(\alpha-2)…3 \times 2 \times 1$$
Otherwise, we have:
$$\Gamma(\alpha)= \int_{0}^{\infty} t^{\alpha-1}e^{-t}dt$$
Sometimes an alternative parameterisation is used, where $\beta$ is replaced with the average time between accidents, $\theta = \frac{1}{\beta}$.
$\beta$ is called a “rate parameter” and $\theta$ is a “scale parameter”.