What is the Variance of a Random Variable?
Suppose that $X$ depends on the outcome of an experiment. If the same experiment is repeated many times, the average value of $X$ will probably come out close to its expected value, $\mathbb{E}(X)$.
However, if the experiment is performed only once, $X$ may be quite different to its own expected value.
For instance, if $X$ is the number shown on a fair die roll, then $ \mathbb{E}(X)=3.5$, but it is not even possible to roll a $3.5$ on the die.
The variance is a “measure of spread”, telling us how “chaotic” a random variable is. Specifically, we compare it to its own mean, looking at the difference $X-\mathbb{E}(X)$.
There is no point looking at the expected value of $X-\mathbb{E}(X)$, since it will always be zero – this is because $X$ is sometimes more than its expected value and sometimes less than it.
Noting that the square of a number can never be negative, we instead define the variance as
$$ \mathbb{V}\text{ar}(X) = \mathbb{E} \left( (X-\mathbb{E}(X))^2 \right) $$
The smaller this is, the more tightly clustered around $\mathbb{E}(X)$ values of $X$ will tend to be.
Using the shorthand $\mu = \mathbb{E}(X)$, this becomes:
$$ \mathbb{V}\text{ar}(X) = \mathbb{E} \left( (X-\mu)^2 \right) $$
Expanding out, we get an equivalent formula that is usually easier to work with:
$$ \mathbb{V}\text{ar}(X) = \mathbb{E}(X^2) - \left( \mathbb{E}(X) \right)^2 = \mathbb{E}(X^2) - \mu^2 $$
If $X$ is discrete, we find $\mathbb{E}(X^2)$ much as in the discrete case:
$$ \mathbb{E}(X^2) = \sum_{\forall x} \ x^2 \ \mathbb{P}(X=x)$$
Where $\forall x$ means the sum is “for all $x$”, i.e. over all values $x$ which $X$ might take. Note that the probabilities are still the same as for $\mathbb{E}(X)$ and we have just replaced $x$ with $x^2$. We can in fact find the expectation of any function of $X$ this way.
The continuous case is covered on the next slide.
We must take care when moving constants outside the bracket in a variance:
$$ \mathbb{V}\text{ar}(aX+b) = a^2 \ \mathbb{V}\text{ar}(X)$$
Finally, the “standard deviation” is defined as the square-root of the variance, $\sqrt{\mathbb{V}\text{ar}(X)}.