What is a Conditional Variance?
The conditional variance of a random variable $Y$ expresses how its variance depends on another random variable.
If $X$ is a random variable that is related to $Y$, our impression of the spread of $Y$ might change once we know the value of $X$. The conditional variance records this updated variance in light of the information that $X=x$.
Recall the definition of the variance of a random variable $Y$:
$$ \mathbb{V}\text{ar}(Y) = \mathbb{E}(Y^2) - \left( \mathbb{E}(Y) \right)^2 $$
The conditional variance of $Y$ is similar, but with expectations taken conditional on $X=x$:
$$ \mathbb{V}\text{ar}(Y \mid X=x ) = \mathbb{E}(Y^2 \mid X=x) - \left( \mathbb{E}(Y \mid X=x) \right)^2 $$
For instance, let $Y$ be a fair die roll, and let $X=0$ for a low roll ($Y \in {1,2,3}$) and $X=1$ for a high roll ($Y \in {4,5,6}$).
Then for a low roll, we have:
$$ \mathbb{E}(Y \mid X=0) = 1 \times \frac{1}{3}+2 \times \frac{1}{3}+3 \times \frac{1}{3} = \frac{6}{3}=2$$
$$ \mathbb{E}(Y^2 \mid X=0) = 1^2 \times \frac{1}{3}+2^2 \times \frac{1}{3}+3^2 \times \frac{1}{3} = \frac{14}{3}$$
So, we calculate
$$ \mathbb{V}\text{ar}(Y \mid X=0 ) = \mathbb{E}(Y^2 \mid X=0) - \left( \mathbb{E}(Y \mid X=0) \right)^2 = \frac{14}{3}-2^2=\frac{14-12}{3} = \frac{2}{3} $$
Conditional variances are very important in econometrics; particularly the conditional variance of the error term when we know the value of another variable:
$$ \mathbb{V}\text{ar}(U_i \mid X_i=x_i) $$
What is the Law of Total Variance?
The Law of Total Variance splits the variance of a random variable $Y$ into two parts, each relating differently to a second random variable $X$.
The conditional variance $ \mathbb{V}\text{ar}(Y \mid X=x )$ is typically a function of the number $x$. If we replace $x$ with its random counterpart $X$, we therefore get a random variable that is a function of $X$:
$$ \mathbb{V}\text{ar}(Y \mid X)$$
As a random variable, it has an expectation:
$$ \mathbb{E}( \mathbb{V}\text{ar}(Y \mid X) ) $$
This is the expected variability of $Y$ once we fix $X$.
Meanwhile, recall that $\mathbb{E}(Y \mid X)$ is a random variable, obtained from replacing $x$ with $X$ in the conditional expectation $\mathbb{E}(Y \mid X =x) $ .
As a random variable, it has a variance:
$$ \mathbb{V}\text{ar} ( \mathbb{E}(Y \mid X))$$
This is the variation in the conditional expectation of $Y$ given $X$ as $X$ varies.
Then the law of total variance says that the overall variance of $Y$ is given by the sum of these two quantities:
$$ \mathbb{V}\text{ar}(Y) = \mathbb{E}( \mathbb{V}\text{ar}(Y \mid X) ) + \mathbb{V}\text{ar} ( \mathbb{E}(Y \mid X)) $$
In our example, $\mathbb{V}\text{ar}(Y \mid X)$ is $=\frac{2}{3}$ regardless of the value of $X$. However, we have seen previously that $\mathbb{E}(Y \mid X) = 2+3X$.
We calculate:
$$\mathbb{E}(X) = 0 \times \frac{1}{2} +1 \times \frac{1}{2} = \frac{1}{2}$$
$$\mathbb{E}(X^2) = 0 \times \frac{1}{2} +1^2 \times \frac{1}{2} = \frac{1}{2}$$
$$ \mathbb{V}\text{ar}(X) = \frac{1}{2}- \left(\frac{1}{2} \right) ^2 = \frac{1}{4}$$
Alternatively, we could have noted that $X$ is Bernoulli, or $X \sim \text{Bin}(1,1/2)$.
So, for a fair die roll $Y$,
$$ \mathbb{V}\text{ar}(Y) = \mathbb{E}( \frac{2}{3} ) + \mathbb{V}\text{ar} ( 2+3X)= \frac{2}{3}+9 \ \mathbb{V}\text{ar}(X) = \frac{2}{3} + \frac{9}{4} = \frac{8+27}{12}=\frac{35}{12} $$
In a real-world example, $X$ could indicate the membership in a particular group, such as industry of employment, and $Y$ could represent some outcome of interest, like wages.
Then the overall variation of wages could be explained by both the typical variation of wage within each demographic group, $\mathbb{E}( \mathbb{V}\text{ar}(Y \mid X) )$, and the variation between average wages as we compare different groups, $\mathbb{V}\text{ar} ( \mathbb{E}(Y \mid X))$.
That is, both expected wage differences within a particular industry, and differences in expected wages across industries, each contribute to the overall variation in wages.