Guides / Probability

Expected value and variance of a discrete random variable: weight by the probabilities, then square before you average

A discrete random variable is a number that comes out of a random experiment: the count of heads, the number of free throws made, the number of empty buckets in a hash table. Its expected value is the long-run average, and its variance is how far it typically lands from that average, squared. Both are weighted sums over the table of values and probabilities. The mistakes come from squaring the mean instead of averaging the squares, from treating variance as if it were linear, and from forgetting how many ways a count can happen.

Short answer

Multiply each value by its probability and add: E[X] = Σ x · P(X = x). For the variance, average the squares the same way and subtract the square of the mean: Var(X) = E[X²] − (E[X])², and the standard deviation is its square root. Expectation is linear, E[aX + b] = aE[X] + b and E[X + Y] = E[X] + E[Y] for any X and Y. Variance is not: Var(aX + b) = a²Var(X), and variances add only for independent variables. For a binomial count, P(X = k) = C(n, k) p^k(1 − p)^{n − k}, with mean np and variance np(1 − p).

Why it works

Imagine running the experiment many times. A value with probability 0.3 turns up in about 30% of the runs, so in the average of all the results it carries a weight of 0.3. That's all E[X] is: the average of the values, weighted by how often each one happens. The same reasoning gives E[g(X)] = Σ g(x) P(X = x) for any function g: apply g to each value and keep the same weights.

Variance is the expected squared distance from the mean, E[(X − μ)²]. Expanding the square gives the shortcut E[X²] − μ². Because it's built from squares, multiplying X by a multiplies the variance by a², and adding a constant moves every value and the mean together, which changes nothing. The gap between E[X²] and (E[X])² is exactly the variance, so it can't be negative, and it's zero only when X never varies.

Example 1: mean, variance and standard deviation from a table

X takes the values 1, 2 and 4 with probabilities 0.5, 0.3 and 0.2. Find E[X], Var(X) and the standard deviation.

0.5 + 0.3 + 0.2 = 1Check that the probabilities add to 1 before using the table.
E[X] = 1(0.5) + 2(0.3) + 4(0.2) = 1.90.5 + 0.6 + 0.8: each value times its probability.
E[X²] = 1(0.5) + 4(0.3) + 16(0.2) = 4.9Square the values, not the probabilities, and keep the same weights.
Var(X) = 4.9 − 1.9² = 4.9 − 3.61 = 1.29The mean of the squares minus the square of the mean.
E[X] = 1.9, Var(X) = 1.29, SD ≈ 1.136The standard deviation is √1.29, back in the units of X.

Example 2: rescaling a variable

X is the roll of a fair die. A game scores Y = 3X − 2 points. Find the mean and variance of Y.

E[X] = 1 + 2 + ⋯ + 66 = 3.5Each face has probability 1/6.
Var(X) = 916 − 3.5² = 3512E[X²] = (1 + 4 + 9 + 16 + 25 + 36)/6 = 91/6, and 3.5² = 49/4.
E[Y] = 3(3.5) − 2 = 8.5Expectation is linear: scale and shift the mean.
Var(Y) = 3² · 3512 = 1054 = 26.25The −2 shifts every score and the mean together, so it drops out. The 3 triples every distance from the mean, which multiplies every squared distance by 9.

Example 3: a binomial count

A player makes each free throw with probability 0.7, independently. In 5 attempts, what is the probability of exactly 3 makes, and what are the mean and variance of the number made?

X ~ Binomial(5, 0.7)A fixed number of independent tries with the same success probability.
0.7³ · 0.3² = 0.343 · 0.09 = 0.03087The probability of one particular pattern, such as make, make, make, miss, miss.
C(5, 3) = 5!3! 2! = 10The number of patterns with exactly 3 makes: choose which 3 of the 5 attempts they are.
P(X = 3) = 10 · 0.03087 = 0.3087The 10 patterns are disjoint and equally likely, so their probabilities add.
E[X] = 5(0.7) = 3.5, Var(X) = 5(0.7)(0.3) = 1.05X is a sum of 5 independent indicators, each with mean p and variance p(1 − p), so the means add and, because they're independent, so do the variances.

Example 4: linearity when the pieces depend on each other

10 keys are hashed into 5 buckets, each key independently and uniformly. What is the expected number of empty buckets?

N = I₁ + I₂ + ⋯ + I₅I_j is 1 if bucket j is empty and 0 if not. Counting empty buckets is adding a 1 for each one.
P(bucket j empty) = (4/5)¹⁰ ≈ 0.1074Each of the 10 keys misses bucket j with probability 4/5, independently.
E[N] = 5 · (4/5)¹⁰ ≈ 0.537The expectation of an indicator is its probability, and the expectations add. The I_j are not independent (all 5 buckets can't be empty at once), and that doesn't matter: linearity of expectation never needs independence. Only adding variances does.

Common mistakes, and the exact line they happen on

1. Squaring the mean instead of averaging the squares

The table from Example 1.

E[X] = 1.9Right so far.
E[X²] = 1.9² = 3.61, so Var(X) = 3.61 − 3.61 = 0This is the mistake. An expectation doesn't pass through a square. A variance of 0 would mean X is always 1.9, a value it never takes. Square each value first, then weight: E[X²] = 4.9, and Var(X) = 1.29.

2. Scaling the variance by a instead of a²

The score from Example 2.

E[3X − 2] = 3E[X] − 2 = 8.5Right so far: expectation is linear.
Var(3X − 2) = 3 · 3512 − 2 = 6.75This is the mistake. Variance is measured in squared units, and it isn't linear. The shift drops out entirely, and the 3 comes out squared: Var(3X − 2) = 9 · 35/12 = 26.25. Writing 3 · 35/12 = 8.75 without the −2 is the same mistake in a quieter form. It's the standard deviation, not the variance, that triples.

3. Dropping the binomial coefficient

The free throws from Example 3.

3 makes and 2 misses: 0.7³ · 0.3²Right so far, for one specific order of makes and misses.
P(X = 3) = 0.03087This is the mistake. "Exactly 3 makes" can happen in C(5, 3) = 10 different orders, each with that probability, so P(X = 3) = 10 · 0.03087 = 0.3087. A quick check catches it: without the coefficients, the six probabilities P(X = 0) through P(X = 5) don't add to 1.

Practice

X takes the values −1, 0 and 2 with probabilities 0.2, 0.5 and 0.3. Find E[X] and Var(X).
Show answer
E[X] = −0.2 + 0 + 0.6 = 0.4, E[X²] = 0.2 + 1.2 = 1.4, so Var(X) = 1.4 − 0.16 = 1.24.
Each attempt succeeds with probability 1/4, independently, and you keep trying until one does. What is the expected number of attempts, and the probability that the first two both fail?
Show answer
This is geometric with p = 1/4, so E[X] = 1/p = 4. Two failures in a row: (3/4)² = 9/16.
A Poisson count has mean 2. Find P(X = 1) and P(X ≥ 1).
Show answer
P(X = 1) = 2e⁻² ≈ 0.2707, and P(X ≥ 1) = 1 − e⁻² ≈ 0.8647, through the complement.
X and Y are independent with Var(X) = 4 and Var(Y) = 9. Find the variance and standard deviation of X − Y.
Show answer
Var(X − Y) = 4 + (−1)² · 9 = 13, so the standard deviation is √13 ≈ 3.606. Subtracting Y still adds its variance.
Each of 20 independent examples is mislabeled with probability 0.1. Find the mean and variance of the number mislabeled, and the probability that none are.
Show answer
Binomial(20, 0.1): mean 2, variance 20(0.1)(0.9) = 1.8, and P(X = 0) = 0.9²⁰ ≈ 0.1216.