Statistics is all about working with data to estimate unknown quantities. This contrasts with probability, which is about precise calculations relating to random variables,

We often resort to statistics when the calculations required in the probability approach cannot be carried out – for instance, because we do not know the PMF of a random variable, but what to estimate its mean.

Probability theory can then be used to ensure our statistical methods are sound.

We can also use statistics to test whether a given claim about a random variable or population is true.