Sampling distribution
When a population parameter is estimated through a sample statistic (e.g., the
sample mean as an estimate of the population mean), we obtain different
results with different samples.
The different values for the sample mean obtained with different samples
constitute also a random variable. As such it has a mean or expected value and
a variance.
The distribution of the statistic (in this example the sample mean) is called
the sampling distribution of the statistic.
Central limit theorem
There is a very important result in statistics that states:
"The sampling distribution of the sample mean can be approximated a a normal
probability distribution when the sample size is large, regardless of the
distribution of the original random variable."
This says, that even when a variable follows a uniform distribution, or an
exponential distribution, the probability distribution of the sample mean
computed from samples comming from those distribution will be normal as the
sample size increases. This usually happens as the size of the sample is
greater than 30.
Why is this important?
The importance of this theorem is found at the time of making inferences about
the value of the sample mean. It means that for inference purposes we can use
the normal distribution if the sample size is large regardless of the
distribution of the original variable. If the original variable was normally
distributed, then the central limit theorem does not need to be applied.