Guides / Linear algebra

Quadratic forms and positive definite matrices: halve the cross term, test with eigenvalues or pivots, and read the axes off the eigenvectors

A quadratic form is a sum of squares and cross products, such as 2x² − 6xy + 5y², and every one can be written as x^TAx with a symmetric matrix A. The question that matters is its sign. A form that is positive for every nonzero x is positive definite, and that is what makes a critical point a minimum, a loss bowl-shaped with one lowest point, and a covariance matrix valid. The usual mistakes are a cross term put into the matrix at full size, a positive diagonal taken as proof, and an ellipse drawn with its axes swapped.

Why it works

Multiplying out x^TAx for a 2 × 2 gives a₁₁x² + (a₁₂ + a₂₁)xy + a₂₂y². The cross term collects both off-diagonal entries, so in a symmetric matrix each one is half its coefficient. A symmetric matrix has perpendicular unit eigenvectors, and in coordinates along them the cross terms disappear: x^TAx = λ₁u₁² + λ₂u₂² + …, where u_i is the component of x along the i-th eigenvector. A weighted sum of squares is positive for every nonzero vector exactly when every weight is positive, which is the eigenvalue test. The same sum shows the ellipse: along eigenvector i the curve λ_iu_i² = k is reached at u_i = √(k/λ_i), so a small eigenvalue lets the curve go far.

Completing the square does the same job without eigenvalues. It writes the form as pivots times squares, d₁(x + …)² + d₂(y + …)², and those pivots are the diagonal of D in A = LDL^T. The leading principal minors are products of the first pivots, d₁, d₁d₂, …, so they are all positive exactly when the pivots are.

Example 1: the matrix of a form

Write q(x, y) = 2x² − 6xy + 5y² as x^TAx with A symmetric, and check it at (1, 1).

1. a₁₁ = 2, a₂₂ = 5The coefficients of the squares go on the diagonal.
2. a₁₂ = a₂₁ = −6/2 = −3The cross term −6xy is shared between the two off-diagonal entries.
3. A = 2−3−35Symmetric, as required.
4. x^TAx = 1 at (1, 1)At (1, 1) it is the sum of all four entries, 2 − 3 − 3 + 5, which matches q(1, 1) = 2 − 6 + 5 = 1.

Example 2: three tests on the same matrix

Decide whether the matrix A of Example 1 is positive definite by its leading minors, by completing the square, and by its eigenvalues.

1. a₁₁ = 2, det A = 1Both leading principal minors are positive (det A = 10 − 9), so A is positive definite.
2. q = 2(x − 3y/2)² + y²/2Take out 2 from the x terms: 2(x² − 3xy) = 2(x − 3y/2)² − 9y²/2, and 5 − 9/2 = 1/2.
3. pivots 2 and 1/2Both positive, and their product is det A = 1. As A = LDL^T, L = 10−3/21 and D = diag(2, 1/2).
4. λ² − 7λ + 1 = 0The characteristic equation: trace 7, determinant 1.
5. λ ≈ 6.854 and 0.146Exactly λ = (7 ± 3√5)/2. Both positive, so all three tests agree. The small eigenvalue says the form grows slowly in one direction, which is why it can hide a near-miss: change the 5 to 4.5 and the determinant, the second pivot and the small eigenvalue all become 0.

Example 3: leading minors of a 3 × 3

Test A = 211121112 and B = 120210003.

1. minors of A: 2, 3, 4The determinants of the top-left 1 × 1, 2 × 2 and 3 × 3 blocks; the middle one is 4 − 1.
2. A is positive definiteAll three minors are positive. Its eigenvalues are 4, 1 and 1.
3. B: 1, 1 − 4 = −3The second minor is already negative, so B is not positive definite, and the test can stop.
4. B is indefiniteq(1, −1, 0) = −2 is negative and q(0, 0, 1) = 3 is positive. A negative second minor with a positive first one means the eigenvalues have mixed signs; here they are 3, 3 and −1.

Example 4: the axes of an ellipse

Find the axes, the semi-axis lengths and the area of the ellipse 3x² + 2xy + 3y² = 8.

1. A = 3113Halve the cross coefficient 2.
2. A(1, 1) = 4 (1, 1)So (1, 1) is an eigenvector, with λ = 4.
3. A(1, −1) = 2 (1, −1)And (1, −1) is one with λ = 2, perpendicular to the first.
4. semi-axes √2 and 2Along eigenvector i the ellipse reaches √(k/λ_i): √(8/4) along (1, 1) and √(8/2) along (1, −1).
5. long axis along (1, −1)The smaller eigenvalue gives the longer axis, which ends at (√2, −√2). Check: 3 · 2 − 2 · 2 + 3 · 2 = 8.
6. area = π · √2 · 2 = 2√2 ππ times the product of the semi-axes, which is also πk/√(det A). On the unit circle the same form runs from 2 to 4.

Common mistakes, and the exact line they happen on

1. The cross coefficient left unhalved

The matrix of Example 1.

1. a₁₁ = 2, a₂₂ = 5Right so far.
2. A = 2−6−65This is the mistake. This matrix gives 2x² − 12xy + 5y²: at (1, 1) it is −5, not 1. Its determinant is −26, so it would also call a positive definite form indefinite. Check any matrix by plugging in one point.

2. A positive diagonal taken as proof

The form x² + 6xy + 2y².

1. A = 1332Right so far.
2. A is positive definiteThis is the mistake. Both diagonal entries are positive, but a positive diagonal is necessary, not enough: the cross term can outweigh it. Here det A = 2 − 9 = −7, and q(1, −1) = 1 − 6 + 2 = −3, so A is indefinite.

3. The long axis put along the largest eigenvalue

The ellipse of Example 4.

1. λ = 4 and λ = 2Right so far: along (1, 1) and (1, −1).
2. long axis along (1, 1)This is the mistake. A large eigenvalue makes the form grow fast in that direction, so the ellipse reaches its level sooner: only √(8/4) = √2 along (1, 1). The long axis, of semi-length 2, is along (1, −1), the eigenvector of the smaller eigenvalue.

Practice

1. Write the matrix of x² + 4xy + y² and classify it.

   Show answer

   A = 1221, with eigenvalues 3 and −1: indefinite.

2. Is 4223 positive definite?

   Show answer

   Yes: the leading minors are 4 and 12 − 4 = 8, both positive.

3. For which c is 1cc4 positive definite?

   Show answer

   4 − c² > 0, so −2 < c < 2.

4. Find the largest value of 6x² + 4xy + 3y² on the unit circle, and where it occurs.

   Show answer

   The matrix has trace 9 and determinant 14, so eigenvalues 7 and 2. The maximum is 7, at ±(2, 1)/√5.

5. Find the Cholesky factor L, lower triangular with positive diagonal, of 4225.

   Show answer

   L = 2012: l₁₁ = √4, l₂₁ = 2/2, l₂₂ = √(5 − 1).