Guides / Probability

The normal distribution and z-scores: divide by σ, pick the right tail, and add the variances

Every normal distribution is a shifted and stretched copy of the standard normal, so one function, Φ, answers every question about all of them once a value is turned into a z-score: how many standard deviations it lies from the mean. The conventions are those of the practice set: Normal(μ, σ²) has mean μ and variance σ², Φ(z) is the probability below z for the standard normal, and the table values used are Φ(1) = 0.8413, Φ(1.5) = 0.9332, Φ(2) = 0.9772 and the cutoff z_0.95 = 1.6449. Densities and CDFs in general are in the continuous random variables guide; the normal is what averages tend to, which is the central limit theorem guide, and what a z test compares against, in the hypothesis tests guide. The mistakes come from dividing by the variance, from reading the wrong tail, and from subtracting variances for a difference.

Why it works

If X is Normal(μ, σ²), then (X − μ)/σ is Normal(0, 1): subtracting μ moves the bell to 0, and dividing by the standard deviation σ squeezes it to unit width. The event X ≤ x is the same event as Z ≤ (x − μ)/σ, so it has the same probability, Φ(z). That is the only fact needed for the first half of every problem. The other half is the shape of Φ: the standard normal is symmetric about 0, so the area below −z equals the area above z, which is why a table only needs positive z.

Sums work through variances because variances of independent variables add, and standard deviations don't. A difference is a sum with one term multiplied by −1, and (−1)² = 1, so its variance is also σ₁² + σ₂². The normal is special only in that the result is again normal, so the same z-score step finishes the job.

Example 1: three probabilities from one distribution

A battery's lifetime is Normal(500, 40²) in hours. Find P(X < 560), P(X > 460) and P(420 < X < 580).

1. z = (560 − 500)/40 = 1.5Divide by the standard deviation, 40.
2. P(X < 560) = Φ(1.5) = 0.9332A left tail is Φ itself.
3. z = (460 − 500)/40 = −1One standard deviation below the mean.
4. P(X > 460) = 1 − Φ(−1)A right tail is 1 minus Φ …
5. = Φ(1) = 0.8413… and by symmetry the area above −1 is the area below 1.
6. z = −2 and z = 2420 and 580 are two standard deviations either side.
7. Φ(2) − Φ(−2) = 0.95440.9772 − 0.0228: the area below the top end minus the area below the bottom end, the "95%" of the 68-95-99.7 rule.

Example 2: from a probability back to a value

The manufacturer wants to advertise the lifetime that only the best 5% of batteries exceed. Find it, and the lifetime below which the worst 5% fall.

1. P(X > x) = 0.05The top 5% is a right tail …
2. Φ(z) = 0.95, z = 1.6449… so 95% lies below. Invert Φ.
3. x = 500 + 1.6449 × 40Undo the standardization: x = μ + zσ.
4. x ≈ 565.8 hoursThe top-5% cutoff.
5. 500 − 65.8 = 434.2 hoursThe bottom 5% uses z = −1.6449, the mirror image. A one-sided 5% uses 1.6449; 1.96 is for 2.5% in each of two tails.

Example 3: the chance one model beats another

Over random seeds, model A's accuracy is Normal(0.80, 0.03²) and model B's is Normal(0.75, 0.04²), independently. On one run each, what is the chance that B scores higher?

1. D = B − AB scores higher exactly when D > 0.
2. mean 0.75 − 0.80 = −0.05Means subtract.
3. var 0.03² + 0.04² = 0.0025Variances add, even for a difference.
4. sd = 0.05The square root, √0.0025.
5. z = (0 − (−0.05))/0.05 = 1Standardize the value 0 under D's distribution.
6. P(D > 0) = 1 − Φ(1) = 0.1587B wins about one run in six, though its mean is 5 points lower. A simulation of 10⁶ pairs agrees to within 0.001.

Example 4: a count, with the continuity correction

A page converts each of 900 independent visitors with probability 0.1. Approximate the chance of at least 104 conversions.

1. mean np = 90The binomial's mean.
2. var np(1 − p) = 81, sd 9The binomial's variance.
3. P(X ≥ 104) ≈ P(Y > 103.5)The count 104 stands for the bar from 103.5 to 104.5, so the area starts half a unit below.
4. z = (103.5 − 90)/9 = 1.5Standardize.
5. 1 − Φ(1.5) = 0.0668The exact binomial value is 0.0690. Without the correction, z = 14/9 and the answer is 0.0599, about four times as far off.

Common mistakes, and the exact line they happen on

1. Dividing by the variance

Example 1, with the second parameter read as the thing to divide by.

1. X ~ Normal(500, 40²)Right so far: the variance is 1600 and the standard deviation 40.
2. z = 60/1600 = 0.0375This is the mistake. A z-score counts standard deviations, so it divides by σ = 40, not σ² = 1600. This gives P(X < 560) ≈ 0.515, as if 560 hours were barely above average, when it is the 93rd percentile.

2. Reading the wrong tail

A question about lasting longer than 560 hours.

1. z = 1.5, Φ(1.5) = 0.9332Right so far.
2. P(X > 560) = 0.9332This is the mistake. Φ is the area to the left of z. The area to the right is 1 − 0.9332 = 0.0668. A quick check: 560 is above the mean, so fewer than half the batteries last longer.

3. Subtracting variances for a difference

Example 3, with the minus sign carried into the variance.

1. mean of B − A = −0.05Right so far: means do subtract.
2. var = 0.04² − 0.03² = 0.0007This is the mistake. Var(B − A) = Var(B) + (−1)² Var(A). Subtracting gives a standard deviation of 0.0265 and P(B > A) ≈ 0.029, five times too small. Two noisy scores make their difference noisier, never less noisy.

Practice

1. X is Normal(70, 5²). Find P(X > 80).

   Show answer

   z = 2, so 1 − Φ(2) = 0.0228.

2. Scores are Normal(100, 15²). Find the score at the 97.5th percentile, using z_0.975 = 1.96.

   Show answer

   100 + 1.96 × 15 = 129.4.

3. X is Normal(10, 2²). What is the distribution of 3X − 4?

   Show answer

   Normal(26, 6²): the mean is 3 × 10 − 4, the standard deviation 3 × 2, so the variance is 36.

4. X₁ is Normal(5, 3²) and X₂ is Normal(2, 4²), independently. Find the distribution of X₁ − X₂ and P(X₁ < X₂), given Φ(0.6) = 0.7257.

   Show answer

   Normal(3, 5²), since 9 + 16 = 25. P(X₁ − X₂ < 0) = Φ(−0.6) = 0.2743.

5. What fraction of any normal distribution lies within 1.5 standard deviations of its mean?

   Show answer

   2Φ(1.5) − 1 = 0.8664, about 87%.