Guides / Probability

Chi-square tests: compare counts with expected counts, divide each squared gap by E, and get the degrees of freedom right

A chi-square test asks whether counts in categories are close to the counts a model predicts: a fair spinner, a 9:3:3:1 genetic ratio, or two variables in a table that have nothing to do with each other. For each cell, square the gap between the observed count O and the expected count E, divide by E, and add up the cells. A large total means the counts are too far from the model to blame on chance, judged against a chi-square distribution with the right degrees of freedom. Most wrong answers come from testing percentages instead of counts, using the wrong degrees of freedom, or reading a large p-value as proof that the model is right.

Why it works

A count in one cell varies from sample to sample, and if the model is right its spread is about √E. So (O − E)/√E is roughly a standard normal, and its square is the cell's term in the sum. Adding squared standard normals gives a chi-square distribution, whose mean is its number of degrees of freedom.

The terms are not all free. In a goodness-of-fit test the counts must add up to n, so once k − 1 cells are known the last one is fixed, and that costs one degree of freedom. In a table, the expected counts are built to match every row total and every column total, which leaves (r − 1)(c − 1) cells free. Only large values of χ² count against the model, so the test is one-sided.

Example 1: is the spinner fair?

A spinner with four equal sectors is spun 200 times and lands on A, B, C and D 68, 40, 50 and 42 times. Test fairness at the 5% level.

1. E = 200 × 1/4 = 50The same expected count for each of the four sectors.
2. O − E: 18, −10, 0, −8The gaps add to 0, as they always do.
3. (324 + 100 + 0 + 64)/50Square each gap; every E is 50, so one division does it.
4. χ² = 488/50 = 9.76The statistic.
5. df = 4 − 1 = 3Four categories, one constraint: the counts add to 200.
6. 9.76 > 7.815: rejectp ≈ 0.021. Sector A comes up more often than a fair spinner would allow.

Example 2: a 9:3:3:1 ratio

A cross is predicted to give four types of offspring in the ratio 9:3:3:1. Of 160 offspring, 101, 24, 26 and 9 are of each type. Do the data fit?

1. 160/16 = 10The ratio has 16 parts, each worth 10 offspring.
2. E = 90, 30, 30, 109, 3, 3 and 1 parts. They add to 160.
3. 121/90 + 36/30The first two terms: gaps of 11 and −6.
4. + 16/30 + 1/10The last two: gaps of −4 and −1.
5. χ² = 143/45 ≈ 3.181.344 + 1.2 + 0.533 + 0.1.
6. 3.18 < 7.815With 3 df, p ≈ 0.37: do not reject. The data are consistent with 9:3:3:1, which is not the same as proving it (Mistake 3).

Example 3: a test of independence

100 students from a morning and an afternoon section get an A, a B, or a C or below.

1. Morning: 24, 28, 8A, B, C or below. The row total is 60.
2. Afternoon: 6, 22, 12Row total 40. Column totals: 30, 50, 20.
3. E = 60 × 30/100 = 18Row total times column total over N, for the morning A cell.
4. E: 18, 30, 12 / 12, 20, 8The same rule for every cell. Each row and column of E has the observed totals.
5. 36/18 + 4/30 + 16/12The morning row: 2 + 0.133 + 1.333.
6. + 36/12 + 4/20 + 16/8The afternoon row: 3 + 0.2 + 2.
7. χ² = 26/3 ≈ 8.67The statistic.
8. df = 1 × 2 = 2(2 − 1)(3 − 1). Since 8.67 > 5.991, reject independence: p = e^−8.67/2 ≈ 0.013. The grade mix differs between the sections.

Example 4: a 2 × 2 table with the shortcut

Of 50 patients on a treatment, 30 improve; of 50 on a placebo, 18 improve. Is improvement related to the treatment?

1. a, b = 30, 20Treatment: improved, not improved.
2. c, d = 18, 32Placebo: improved, not improved. N = 100.
3. ad − bc = 960 − 360The cross products: 30 · 32 and 20 · 18.
4. = 600This gap is what drives the statistic.
5. 50 · 50 · 48 · 52The four margins: two row totals and two column totals. The product is 6,240,000.
6. χ² = 100 · 600²/6,240,000The shortcut N(ad − bc)² over the product of the margins, the same as the sum over four cells.
7. χ² ≈ 5.77 > 3.841With 1 df, p ≈ 0.016: reject independence. Every expected count (24 or 26) is well above 5.

Common mistakes, and the exact line they happen on

1. Running the test on percentages

The spinner of Example 1, written as 34%, 20%, 25% and 21%.

1. E = 25% eachRight so far, as a proportion.
2. (81 + 25 + 0 + 16)/25 = 4.88This is the mistake. The statistic grows with the sample size, so it must use counts. Percentages pretend there were 100 spins instead of 200, which halves χ² from 9.76 to 4.88, under the 7.815 cutoff. The same data now look fair.

2. Using rc − 1 degrees of freedom for a table

The 2 × 3 table of Example 3.

1. χ² ≈ 8.67Right so far.
2. df = 6 − 1 = 5This is the mistake. The expected counts were made to match both row totals and all three column totals, not just N. That leaves (2 − 1)(3 − 1) = 2 free cells. With 5 df the cutoff would be 11.07 and the test would wrongly keep independence (p ≈ 0.12 instead of 0.013).

3. Reading a large p-value as proof the model fits

The offspring counts of Example 2, tested against a different ratio.

1. 9:3:3:1: χ² ≈ 3.18p ≈ 0.37, so do not reject. Right so far.
2. So the ratio is 9:3:3:1This is the mistake. Test the same counts against 10:3:2:1, with E = 100, 30, 20, 10: χ² = 3.11 and p ≈ 0.37 again. Two different models both pass. Not rejecting means the data can't tell them apart, not that one of them is true.

Practice

1. A goodness-of-fit test has 5 categories and no estimated parameters. How many degrees of freedom?

   Show answer

   4.

2. A coin lands heads 60 times in 100 flips. Find χ² for fairness and decide at the 5% level.

   Show answer

   100/50 + 100/50 = 4, above 3.841, so reject (p ≈ 0.046).

3. In a table with N = 200, a cell's row total is 40 and its column total is 25. What is its expected count under independence?

   Show answer

   40 × 25/200 = 5.

4. How many degrees of freedom does a test of independence on a 3 × 4 table have?

   Show answer

   (3 − 1)(4 − 1) = 6.

5. A die rolled 60 times shows 1 to 6 with counts 8, 9, 12, 11, 6 and 14. Is it fair at the 5% level?

   Show answer

   E = 10 each, so χ² = (4 + 1 + 4 + 1 + 16 + 16)/10 = 4.2. With 5 df that is under 11.07 (p ≈ 0.52): do not reject.