Guides / Probability

The central limit theorem and standard errors: divide σ by √N, standardize with the SE, and round the sample size up

An average of N independent measurements is itself a random quantity. Its spread, the standard error, is σ/√N, and the central limit theorem says its distribution is close to normal once N is large, whatever the distribution of one measurement. Those two facts answer most questions about estimates from data: how likely an average is to land in a range, how wide a confidence interval is, and how many samples a target accuracy needs. The mistakes come from dividing σ by N instead of √N, from scaling the spread of a sum by N instead of √N, and from rounding a sample size down.

Why it works

Variances of independent quantities add, so the sum of N draws has variance Nσ² and standard deviation σ√N. The average is the sum divided by N, and dividing a random quantity by N divides its variance by N²: Var(x̄) = Nσ²/N² = σ²/N. Taking the square root gives σ/√N. The errors of the individual draws partly cancel, which is why the spread shrinks, and why it shrinks only with the square root: four times the data halves the error.

The central limit theorem adds the shape. However skewed or lumpy one measurement is, provided its variance is finite, the standardized average (x̄ − μ)/(σ/√N) approaches the standard normal as N grows. That is what lets one table, Φ, answer questions about averages of anything. The approximation is good sooner for symmetric distributions and later for skewed ones; for proportions, a common rule of thumb is Np ≥ 10 and N(1 − p) ≥ 10.

Example 1: the probability that an average falls short

A machine fills bottles with a mean of 500 ml and a standard deviation of 12 ml. A quality check averages 36 bottles. Approximate the probability that the average is below 497 ml.

1. SE = 12/√36 = 12/6 = 2The average of 36 bottles is six times less spread out than one bottle, not 36 times.
2. z = (497 − 500)/2 = −1.5Standardize with the standard error, because the quantity is an average.
3. P(x̄ < 497) ≈ Φ(−1.5)The central limit theorem makes x̄ approximately normal, even if single fills are not.
4. Φ(−1.5) = 1 − 0.9332 ≈ 0.0668About 6.7%. For a single bottle, z = −3/12 = −0.25 and the probability is about 0.401: one short bottle is common, a short average of 36 is not.

Example 2: the total of many independent pieces

Songs on a playlist have a mean length of 3.5 minutes and a standard deviation of 1.2 minutes. Approximate the probability that 48 songs chosen at random last more than 180 minutes in total.

1. E[S] = 48 · 3.5 = 168The means of the pieces add.
2. sd(S) = 1.2√48 ≈ 8.314The variances add, 48 · 1.44 = 69.12, so the standard deviation grows like √N.
3. z = (180 − 168)/8.314 ≈ 1.443Song lengths are continuous, so no continuity correction is needed.
4. P(S > 180) ≈ 0.07451 − Φ(1.443), about 7.4%. The same answer comes from the average: x̄ > 3.75 with SE = 1.2/√48 gives the same z.

Example 3: a confidence interval for a proportion

In a survey of 900 users, 270 say they would use a new feature. Find a 95% confidence interval for the proportion of all users who would.

1. p̂ = 270/900 = 0.3A proportion is the average of 0/1 answers.
2. SE = √(0.3 · 0.7/900) ≈ 0.01528One 0/1 answer has variance p(1 − p); with p unknown, use p̂ in its place.
3. 1.96 · 0.01528 ≈ 0.0299The margin of error. The normal approximation is safe: 270 yes and 630 no answers are both well above 10.
4. 0.3 ± 0.030 ≈ (0.270, 0.330)Between 27% and 33% of users, at 95% confidence. Halving the margin to 1.5 points would take four times the sample, 3600 users.

Example 4: how many samples a target accuracy needs

Measurements have a standard deviation of 15. How many are needed for a 95% confidence interval for the mean of half-width at most 2?

1. 1.96 · 15/√N ≤ 2The half-width of the interval is 1.96 SE.
2. √N ≥ 1.96 · 15/2 = 14.7Solve for √N first.
3. N ≥ 14.7² = 216.09Square both sides.
4. N = 217Round up, because N is a lower bound. Check: with 217 samples the half-width is about 1.996, inside the target.

Common mistakes, and the exact line they happen on

1. Dividing σ by N instead of √N

The standard error of Example 1.

1. σ = 12, N = 36Right so far.
2. SE = 12/36 = 1/3This is the mistake. The sum of 36 fills has variance 36σ², and dividing it by 36 divides that variance by 36², which leaves σ²/36: the standard deviation is divided by √36 = 6. With 1/3 the z-score is −9 and the probability is about 10⁻¹⁹, which would make a short average essentially impossible. The right standard error is 12/6 = 2, and the probability is about 0.067.

2. Scaling the spread of a sum by N

The standard deviation of the total in Example 2.

1. E[S] = 48 · 3.5 = 168Right so far: means do scale by N.
2. sd(S) = 48 · 1.2 = 57.6This is the mistake. Standard deviations don't add; variances do. 48 · 1.2 would be right only if the 48 lengths were perfectly correlated, all long or all short together. Here z = 12/57.6 ≈ 0.21 gives about 0.42 instead of 0.074. The right value is 1.2√48 ≈ 8.31.

3. Rounding the sample size down

The last step of Example 4.

1. N ≥ 216.09Right so far.
2. N = 216This is the mistake. Rounding to the nearest whole number gives a sample just too small: with 216 measurements the half-width is 1.96 · 15/√216 ≈ 2.0004, over the target. A sample size from an inequality always rounds up, to 217.

Practice

1. Measurements have σ = 20. Find the standard error of the average of 100 of them, and the number of measurements that would halve it.

   Show answer

   SE = 20/√100 = 2. Halving it takes four times the data, N = 400, for SE = 20/20 = 1.

2. A proportion is estimated from 1600 samples when the true p is 0.2. Find the standard error of p̂.

   Show answer

   SE = √(0.2 · 0.8/1600) = 0.4/40 = 0.01.

3. The sum of 64 independent values with mean 5 and standard deviation 2: find its mean and standard deviation, and approximate the probability that it exceeds 336.

   Show answer

   Mean 64 · 5 = 320, standard deviation 2√64 = 16. Then z = (336 − 320)/16 = 1 and P ≈ 1 − Φ(1) ≈ 0.159.

4. A sample of 144 values has mean 12.4 and standard deviation 3. Find a 95% confidence interval for the mean.

   Show answer

   SE = 3/12 = 0.25, and 12.4 ± 1.96 · 0.25 = 12.4 ± 0.49, so about (11.91, 12.89).

5. Two independent estimates have standard errors 3 and 4. Find the standard error of their difference.

   Show answer

   The variances add, even for a difference: √(3² + 4²) = √25 = 5, not 3 + 4 = 7 and not 4 − 3 = 1.