How do we Apply the Law of Iterated Expectations?
The law of iterated expectations is often useful when we want to find the expected value of $Y$, and $Y$ depends on another random variable $X$. Often the problem becomes much simpler if we first think of $X$ as taking a fixed value. With some practice, it is possible to apply this method quite systematically.
We have the following three steps:
Step 1.
First fix a value of $X$; that is, let $X=x$ for some number $x$.
Then find the expected value of $Y$ when $X$ is equal to this number; $\mathbb{E}(Y \mid X=x)$.
Step 2.
Replace $x$ with its random version $X$ in the formula you just found, to get a random variable that is a function of $X$.
Step 3.
Take an expectation again.
Let’s illustrate with another example.
Suppose that we first pick a number $X$ uniformly from the interval $[10,20]$. That is, we have
$$X \sim \mathcal{U}[10,20]$$
Then, given $X=x$, we draw $Y$ from a normal distribution with mean $x$ and variance $1$:
$$ Y _ {\ \mid X=x } \sim \mathcal{N}(x,1) $$
The distribution of $Y$ is quite complicated; it’s like a normal distribution, but with a “random parameter” (later we will call these “compound distributions”).
However, finding its expectation using the above three steps is quite easy.
Step 1.
Given $X=x$, we have said that $ Y _ {\ \mid X=x} \sim \mathcal{N}(x,1) $.
Hence, we have that $\mathbb{E}(Y \mid X=x) = x$.
Step 2.
Replacing $x$ with its random counterpart $X$, we write:
$$\mathbb{E}(Y \mid X) = X$$
Step 3.
Finally, we have
$$\mathbb{E}(Y) = \mathbb{E}( \mathbb{E}(Y \mid X ) ) = 15 $$