Slide explaining estimator efficiency using target practice analogy and variance comparison of sample mean versus single observation

What is an Efficient Estimator?

The efficiency of an estimator relates to its variance. One estimator is more efficient than another if it has the lower variance.

Efficiency is another desirable property of estimators, in addition to unbiasedness.

Let $Y_1, Y_2, \dots , Y_n$ be a random sample from a population (that is, independent and identically distributed). Let $\mu=\mathbb{E}(Y_i)$ be their common mean, and $\sigma^2=\mathbb{V}\text{ar}(Y_i)$ their common variance.

Then it is true that the sample mean is an unbiased estimator of $\mu$:

$$\mathbb{E}(\bar{Y})=\mathbb{E} \left( \frac{Y_1+Y_2+\dots+Y_n}{n} \right) =\mu$$

However, the same is also true if we just look at the first observation in the sample, and consider that as an estimator of $\mu$:

$$\mathbb{E}(Y_1)=\mu$$

To see why using the sample mean is a superior method, we must compare their variances:

$$\mathbb{V}\text{ar}(\bar{Y})=\frac{\sigma^2}{n}, \quad \mathbb{V}\text{ar}(Y_1)=\sigma^2$$

For $n \gt 1$ the variance of the sample mean is smaller. This should make sense intuitively: the outcome of one fair die roll $X$ is not especially likely to be close to $\mathbb{E}(X)=3.5$, but the average of many die rolls will be.

Usually we use the concept of efficiency in this comparative way, to indicate that one estimator is more efficient than another. However, there are also limits on how small it is possible for the variance of an unbiased estimator to be.

Sometimes when we have a choice between different estimators there is a tradeoff between bias and variance (accuracy vs precision).

Background: