How Do We Think About Random Variables Mathematically?
In formal terms, a random variable is a kind of function.
It takes as its input a member of a sample space; the set of all possible outcomes. The sample space is denoted by the Greek letter $ \Omega $.
For instance, suppose we plan to flip a coin three times. Then we have:
$$\Omega = \{ HHH, HHT, HTH, HTT, THH, THT, TTH, TTT \}$$
Where “HHH” means getting three heads in a row, and so on.
A random variable provides a numerical summary of each outcome $\omega$. For instance, suppose we define the random variable $X$ to be the number of heads obtained.
Then we have that $X(HTT)=1$.
When probabilities are defined, they are of events; that is, subsets of $\Omega$.
For instance, in this experiment, if the coin is fair, all 8 outcomes are equally likely. We would therefore want to define $ \mathbb{P}( \{HTT,THT,TTH \}) $ as equal to $\frac{3}{8}$.
When we talk about the probability that e.g. $X=1$, what we really mean is the probability of the subset of $\Omega$ consisting of elements that $X$ maps to 1. That is:
$$ \mathbb{P}(X=1) \stackrel{\text{def}}{=} \mathbb{P}( \{HTT,THT,TTH \}) = \frac{3}{8} $$
So, we need to ensure that the probability of all such events is defined, for any probabilities relating to $X$ that we might be interested in.
Making this fully precise leads us to the definition of a “measurable function”, though this requires studying some other measure theory basics to make precise. Here we will see that, for technical reasons, in more complex cases, we often don’t assign probabilities to all events, but only some of them.
Essentially, it is enough that $ \mathbb{P}(X \le a) $ is always assigned a probability, for all $a \in \mathbb{R}$. Otherwise, the CDF of $X$ will not be well-defined. This would be a problem as the CDF of $X$ is fundamental to its identity as a random variable.