How Does The Sum of Two Random Variables Behave?
Suppose we have two discrete random variables, $X$ and $Y$, which can only take non-negative values. Suppose further that $X$ and $Y$ are independent.
Their behaviour is fully described by the following two sequences of numbers.
For $X$, we have the sequence of probabilities $p_0, p_1, \dots $, where $p_k = \mathbb{P}(X=k)$.
For $Y$, we have the sequence of probabilities $q_0, q_1, \dots $, where $q_k = \mathbb{P}(Y=k)$.
Now, consider the new random variable given by their sum, $X+Y$. We find the probability that $X+Y=n$. The following cases are possible:
$$ \begin{aligned} X &= 0 \quad \text{and} \quad Y = n \\ X &= 1\quad \text{and} \quad Y = n-1 \\ X &= 2 \quad \text{and} \quad Y = n-2 \\ & \quad \quad \quad \vdots \\ X &= n \quad \text{and} \quad Y = 0 \end{aligned} $$
Since $X$ and $Y$ are independent, we can then find the probability of each case by multiplying. Adding up all cases, we have
$$\mathbb{P}(X+Y=n) = p_0 q_n + p_1 q_{n-1} + p_2 q_{n-2} + \dots + p_n q_0 $$
We can write this with sigma (summation) notation as
$$ \mathbb{P}(X+Y=n) = \sum_{k=0}^n \ p_k q_{n-k} $$
We call this new sequence of probabilities, with terms $c_n = \sum_{k=0}^n \ p_k q_{n-k}$, the convolution of the sequences $p_0, p_1, \dots $ and $q_0, q_1, \dots $
Now, suppose instead that $X$ and $Y$ are continuous, though still independent.
Consider again the continuous random variable $X+Y$. Then if $X+Y=z$, we must have that $X=t$, $Y=z-t$ for some number $t$.
Since they are independent, the “relative likelihood” of this happening – in the sense of a PDF value – will be $f_X(t)f_Y(z-t)$, since we may obtain their joint PDF by multiplying their marginal ones.
Finally, we consider all values of $t$ by taking an integral, just as we took a sum in the discrete case.
Putting this together, we have:
$$ f_{X+Y}(z) = \int_{t=-\infty}^{t=\infty}f_X(t)f_Y(z-t) \ dt$$
This defines a new function of $z$, again called the “convolution” of the two marginal PDFs, $f_X$ and $f_Y$.
If, as above, $X$ and $Y$ have to be non-negative, and $z \gt 0$, then $t$ must be between $0$ and $z$. That is:
$$ f_{X+Y}(z) = \int_{t=0}^{t=z}f_X(t)f_Y(z-t) \ dt$$