Practice / Linear algebra

Quadratic forms and positive definite matrices

Ten problems on quadratic forms and positive definite matrices: writing a form as xᵀAx with a symmetric matrix, classifying definite, semidefinite and indefinite matrices by their eigenvalues, the leading principal minors of a 3×3 matrix, completing the square as LDLᵀ, a Cholesky factor by hand, the range of a parameter that keeps a matrix positive definite, the largest and smallest values on the unit circle, the axes of an ellipse, why AᵀA is positive semidefinite and when it is definite, and the Hessian of a ridge regression loss, with worked solutions and the mistakes of the unhalved cross term, a positive diagonal taken as proof, leading minors for a semidefinite test, the long axis along the largest eigenvalue and the extremes read off the diagonal.

Before you start

A quadratic form is a sum of squares and cross products, such as $3x^2 + 4xy + 2y^2$ and every one of them can be written as $x^\top Ax$ with a symmetric matrix $A$ The question that matters is its sign: a form that is positive for every nonzero $x$ is positive definite, and that single property is what makes a critical point a minimum, a loss convex with a unique minimizer, a covariance matrix valid and a Cholesky factor exist. These ten problems build the matrix of a form, test definiteness three ways (eigenvalues, leading principal minors and completing the square), factor a matrix by Cholesky, find the extremes of a form on the unit circle and the axes of an ellipse, and end with why ridge regression always has exactly one solution.

A quadratic form is $q(x) = x^\top Ax = \sum_{i,j} A_{ij}x_ix_j$ with $A$ symmetric. The coefficient of $x_i^2$ is $A_{ii}$ and the coefficient of $x_ix_j$ for $i \ne j$ is $2A_{ij}$ so the off-diagonal entries are half the cross coefficients.
$A$ is positive definite (PD) if $x^\top Ax > 0$ for every $x \ne 0$ positive semidefinite (PSD) if $x^\top Ax \ge 0$ for every $x$ negative (semi)definite if $-A$ is, and indefinite if $x^\top Ax$ takes both signs.
Eigenvalue test. A symmetric $A$ is $Q\Lambda Q^\top$ with $Q$ orthogonal, so with $y = Q^\top x$ $x^\top Ax = \sum_i\lambda_iy_i^2$ $A$ is PD exactly when every $\lambda_i > 0$ PSD when every $\lambda_i \ge 0$ and indefinite when there are eigenvalues of both signs.
Sylvester's criterion. $A$ is PD exactly when its leading principal minors (the determinants of the top-left $1\times1$ $2\times2$ …, $n\times n$ blocks) are all positive. For semidefiniteness the leading minors are not enough: every principal minor must be $\ge 0$
Cholesky. $A$ is PD exactly when $A = LL^\top$ with $L$ lower-triangular and its diagonal positive. Equivalently $A = LDL^\top$ with $L$ unit lower-triangular and the pivots of elimination on the diagonal of $D$ all positive.
Rayleigh quotient. For symmetric $A$ the largest and smallest values of $x^\top Ax$ over unit vectors $x$ are $\lambda_{\max}$ and $\lambda_{\min}$ reached at their eigenvectors. For PD $A$ the set $x^\top Ax = c$ is an ellipse (ellipsoid) with axes along the eigenvectors and semi-axis lengths $\sqrt{c/\lambda_i}$
For any matrix $M$ $v^\top M^\top Mv = \|Mv\|^2$

Builds on: Eigenvalues and eigenvectors by hand, Gaussian elimination, inverses and determinants by hand

Problems

·
Write $q(x, y) = 3x^2 + 4xy + 2y^2$ as $x^\top Ax$ with $A$ symmetric, and check it at $(1, -1)$ Then write the matrix of $x^2 + 2y^2 + 3z^2 + 2xy - 4yz$
·
Classify each matrix as positive definite, positive semidefinite or indefinite from its eigenvalues: $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ $B = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ $C = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$
··
Use the leading principal minors to show that $A = \begin{pmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{pmatrix}$ is positive definite, and confirm with its eigenvalues.
··
Complete the square in $q(x, y) = 3x^2 + 4xy + 2y^2$ and write the result as $A = LDL^\top$
··
Find the Cholesky factor $L$ of $A = \begin{pmatrix} 4 & 2 & -2 \\ 2 & 10 & 2 \\ -2 & 2 & 6 \end{pmatrix}$ and use it to get $\det A$
··
For which real $k$ is $A = \begin{pmatrix} 2 & 1 & 0 \\ 1 & 2 & k \\ 0 & k & 2 \end{pmatrix}$ positive definite?
··
Find the largest and smallest values of $q(x, y) = 5x^2 + 4xy + 2y^2$ on the unit circle $x^2 + y^2 = 1$ and where they occur.
··
Find the axes, the semi-axis lengths and the area of the ellipse $5x^2 - 4xy + 5y^2 = 21$
···
Show that $M^\top M$ is positive semidefinite for any matrix $M$ and positive definite exactly when the columns of $M$ are independent. Test $M_1 = \begin{pmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{pmatrix}$ and $M_2 = \begin{pmatrix} 1 & 2 \\ 1 & 2 \\ 1 & 2 \end{pmatrix}$
···
Ridge regression minimizes $L(w) = \|Xw - y\|^2 + \lambda\|w\|^2$ Find the gradient and Hessian of $L$ With $X = M_2$ from Problem 9, $y = (1, 2, 3)$ and $\lambda = \tfrac12$ show that the Hessian is positive definite and find the unique minimizer.

Answers

$A = \begin{pmatrix} 3 & 2 \\ 2 & 2 \end{pmatrix}$ $\begin{pmatrix} 1 & 1 & 0 \\ 1 & 2 & -2 \\ 0 & -2 & 3 \end{pmatrix}$
$A$ is positive definite, $B$ is indefinite, $C$ is positive semidefinite (not definite)
$\Delta_1, \Delta_2, \Delta_3 = 2, 3, 4$ the eigenvalues are $2 - \sqrt2 \approx 0.586$ $2$ and $2 + \sqrt2 \approx 3.414$ all positive
$q = 3\big(x + \tfrac23y\big)^2 + \tfrac23y^2$ $A = LDL^\top$ with $L = \begin{pmatrix} 1 & 0 \\ \tfrac23 & 1 \end{pmatrix}$ $D = \operatorname{diag}(3, \tfrac23)$ positive definite
$L = \begin{pmatrix} 2 & 0 & 0 \\ 1 & 3 & 0 \\ -1 & 1 & 2 \end{pmatrix}$ $\det A = (2\cdot3\cdot2)^2 = 144$
$-\sqrt3 < k < \sqrt3$
Maximum $6$ at $\pm\tfrac{1}{\sqrt5}(2, 1)$ minimum $1$ at $\pm\tfrac{1}{\sqrt5}(1, -2)$
Major semi-axis $\sqrt7$ along $(1, 1)$ minor semi-axis $\sqrt3$ along $(1, -1)$ area $\pi\sqrt7\sqrt3 = \pi\sqrt{21} \approx 14.40$
$M^\top M$ is always PSD; $M_1^\top M_1$ is PD, $M_2^\top M_2$ is only PSD, with $v^\top M_2^\top M_2v = 0$ at $v = (2, -1)$
$\nabla^2L = 2(X^\top X + \lambda I)$ PD with eigenvalues $1$ and $31$ $w^* = \big(\tfrac{12}{31}, \tfrac{24}{31}\big)$

Worked solutions

Problem 1

Write $q(x, y) = 3x^2 + 4xy + 2y^2$ as $x^\top Ax$ with $A$ symmetric, and check it at $(1, -1)$ Then write the matrix of $x^2 + 2y^2 + 3z^2 + 2xy - 4yz$

$A_{11} = 3$ and $A_{22} = 2$ The squares go on the diagonal.
$2A_{12} = 4$ so $A_{12} = A_{21} = 2$ The cross term $4xy$ is $A_{12}xy + A_{21}yx$ split evenly between the two off-diagonal entries.
$A = \begin{pmatrix} 3 & 2 \\ 2 & 2 \end{pmatrix}$ Symmetric, as a form's matrix should be.
At $(1, -1)$ $q = 3 - 4 + 2 = 1$ and $x^\top Ax = (1, -1)\begin{pmatrix} 1 \\ 0 \end{pmatrix} = 1$ $Ax = (3 - 2, 2 - 2) = (1, 0)$ Using $4$ off the diagonal instead gives $-3$ (Mistake 1).
$A = \begin{pmatrix} 3 & 2 \\ 2 & 2 \end{pmatrix}$ $\begin{pmatrix} 1 & 1 & 0 \\ 1 & 2 & -2 \\ 0 & -2 & 3 \end{pmatrix}$ In the second form, $2xy$ gives $A_{12} = 1$ $-4yz$ gives $A_{23} = -2$ and there is no $xz$ term, so $A_{13} = 0$

Problem 2

Classify each matrix as positive definite, positive semidefinite or indefinite from its eigenvalues: $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ $B = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ $C = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$

$A$ $\lambda^2 - 4\lambda + 3 = 0$ so $\lambda = 1, 3$ Trace $4$ determinant $3$
$B$ $\lambda^2 - 2\lambda - 3 = 0$ so $\lambda = 3, -1$ Trace $2$ determinant $-3$ A negative determinant of a $2\times2$ always means one eigenvalue of each sign.
$C$ $\lambda^2 - 5\lambda = 0$ so $\lambda = 0, 5$ Determinant $0$ the rows are proportional.
$q_B(1, -1) = 1 - 4 + 1 = -2$ and $q_C(2, -1) = 4 - 8 + 4 = 0$ A witness for each: $(1, -1)$ is $B$ 's eigenvector for $-1$ and $(2, -1)$ spans $C$ 's null space.
$A$ is positive definite, $B$ is indefinite, $C$ is positive semidefinite (not definite) $B$ has positive diagonal entries and is still not PD (Mistake 2).

Problem 3

Use the leading principal minors to show that $A = \begin{pmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{pmatrix}$ is positive definite, and confirm with its eigenvalues.

$\Delta_1 = 2$ The top-left entry.
$\Delta_2 = 2\cdot2 - (-1)(-1) = 3$ The top-left $2\times2$ block.
$\Delta_3 = 2(4 - 1) - (-1)(-2 - 0) + 0 = 6 - 2 = 4$ Expand along the first row.
All three are positive, so $A$ is PD by Sylvester's criterionThis is the second-difference matrix behind a discretized $-u''$
$\Delta_1, \Delta_2, \Delta_3 = 2, 3, 4$ the eigenvalues are $2 - \sqrt2 \approx 0.586$ $2$ and $2 + \sqrt2 \approx 3.414$ all positiveTheir product is $4 = \det A$ and their sum is $6 = \operatorname{tr}A$

Problem 4

Complete the square in $q(x, y) = 3x^2 + 4xy + 2y^2$ and write the result as $A = LDL^\top$

$3x^2 + 4xy = 3\big(x^2 + \tfrac43xy\big) = 3\big(x + \tfrac23y\big)^2 - \tfrac43y^2$ Factor out the $x^2$ coefficient, then half of $\tfrac43$ is $\tfrac23$ and $3\cdot\tfrac49 = \tfrac43$
$q = 3\big(x + \tfrac23y\big)^2 + \tfrac23y^2$ $2 - \tfrac43 = \tfrac23$
Both coefficients are positive, so $q > 0$ unless $y = 0$ and $x + \tfrac23y = 0$ that is unless $x = y = 0$ A sum of positive multiples of squares of independent combinations.
$L = \begin{pmatrix} 1 & 0 \\ \tfrac23 & 1 \end{pmatrix}$ $D = \operatorname{diag}\big(3, \tfrac23\big)$ The rows of $L^\top$ are the combinations being squared; the entries of $D$ are the pivots of elimination on $A$
$q = 3\big(x + \tfrac23y\big)^2 + \tfrac23y^2$ $A = LDL^\top$ with $L = \begin{pmatrix} 1 & 0 \\ \tfrac23 & 1 \end{pmatrix}$ $D = \operatorname{diag}(3, \tfrac23)$ positive definiteCheck: $\det A = 3\cdot\tfrac23 = 2 = 6 - 4$

Problem 5

Find the Cholesky factor $L$ of $A = \begin{pmatrix} 4 & 2 & -2 \\ 2 & 10 & 2 \\ -2 & 2 & 6 \end{pmatrix}$ and use it to get $\det A$

$l_{11} = \sqrt4 = 2$ $l_{21} = \tfrac22 = 1$ $l_{31} = \tfrac{-2}{2} = -1$ The first column of $A$ is $l_{11}$ times the first column of $L$
$l_{22} = \sqrt{10 - l_{21}^2} = \sqrt9 = 3$ Subtract what the first column already supplies, then take the root.
$l_{32} = \dfrac{2 - l_{31}l_{21}}{l_{22}} = \dfrac{2 - (-1)(1)}{3} = 1$ From $A_{32} = l_{31}l_{21} + l_{32}l_{22}$
$l_{33} = \sqrt{6 - l_{31}^2 - l_{32}^2} = \sqrt{6 - 1 - 1} = 2$ Every square root was of a positive number, which is the proof that $A$ is PD.
$L = \begin{pmatrix} 2 & 0 & 0 \\ 1 & 3 & 0 \\ -1 & 1 & 2 \end{pmatrix}$ $\det A = (2\cdot3\cdot2)^2 = 144$ $\det A = \det L\,\det L^\top$ Sampling $\mathcal{N}(0, A)$ is $L\varepsilon$ with $\varepsilon \sim \mathcal{N}(0, I)$

Problem 6

For which real $k$ is $A = \begin{pmatrix} 2 & 1 & 0 \\ 1 & 2 & k \\ 0 & k & 2 \end{pmatrix}$ positive definite?

$\Delta_1 = 2 > 0$ and $\Delta_2 = 4 - 1 = 3 > 0$ These do not involve $k$
$\Delta_3 = 2(4 - k^2) - 1(2 - 0) + 0 = 6 - 2k^2$ Expand along the first row.
$6 - 2k^2 > 0 \iff k^2 < 3$ Sylvester's criterion needs all three minors positive.
At $k = \sqrt3$ the determinant is $0$ and the smallest eigenvalue is $0$ PSD but not PDThe boundary of the range.
$-\sqrt3 < k < \sqrt3$ Check $k = 2$ $(1, -2, 2)$ gives $x^\top Ax = 2 - 4 + 8 + 8 - 16 = -2 < 0$

Problem 7

Find the largest and smallest values of $q(x, y) = 5x^2 + 4xy + 2y^2$ on the unit circle $x^2 + y^2 = 1$ and where they occur.

$A = \begin{pmatrix} 5 & 2 \\ 2 & 2 \end{pmatrix}$ Half of $4$ off the diagonal.
$\lambda^2 - 7\lambda + 6 = 0$ so $\lambda = 6, 1$ Trace $7$ determinant $10 - 4 = 6$
$\lambda = 6$ $(A - 6I)v = 0$ with rows $(-1, 2)$ so $v = \tfrac{1}{\sqrt5}(2, 1)$ Check: $q = \tfrac15(20 + 8 + 2) = 6$
$\lambda = 1$ rows $(4, 2)$ so $v = \tfrac{1}{\sqrt5}(1, -2)$ Check: $q = \tfrac15(5 - 8 + 8) = 1$
Maximum $6$ at $\pm\tfrac{1}{\sqrt5}(2, 1)$ minimum $1$ at $\pm\tfrac{1}{\sqrt5}(1, -2)$ Every other unit vector gives a value in between, for example $q(1, 0) = 5$ The smallest diagonal entry, $2$ is not the minimum (Mistake 5).

Problem 8

Find the axes, the semi-axis lengths and the area of the ellipse $5x^2 - 4xy + 5y^2 = 21$

$A = \begin{pmatrix} 5 & -2 \\ -2 & 5 \end{pmatrix}$ has $\lambda = 3$ for $(1, 1)/\sqrt2$ and $\lambda = 7$ for $(1, -1)/\sqrt2$ $A(1, 1) = (3, 3)$ and $A(1, -1) = (7, -7)$
In the rotated coordinates $y = Q^\top x$ the ellipse is $3y_1^2 + 7y_2^2 = 21$ The cross term disappears along the eigenvectors.
Semi-axes $\sqrt{21/3} = \sqrt7 \approx 2.646$ along $(1, 1)$ and $\sqrt{21/7} = \sqrt3 \approx 1.732$ along $(1, -1)$ Divide by the eigenvalue: the smaller eigenvalue gives the longer axis.
Check the end of the long axis, $\sqrt7\cdot\tfrac{1}{\sqrt2}(1, 1)$ where $x = y = \sqrt{3.5}$ $(5 - 4 + 5)\cdot3.5 = 21$ It lies on the ellipse.
Major semi-axis $\sqrt7$ along $(1, 1)$ minor semi-axis $\sqrt3$ along $(1, -1)$ area $\pi\sqrt7\sqrt3 = \pi\sqrt{21} \approx 14.40$ Putting the long axis along the eigenvector of $7$ gives a point where the form is $49$ (Mistake 4).

Problem 9

Show that $M^\top M$ is positive semidefinite for any matrix $M$ and positive definite exactly when the columns of $M$ are independent. Test $M_1 = \begin{pmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{pmatrix}$ and $M_2 = \begin{pmatrix} 1 & 2 \\ 1 & 2 \\ 1 & 2 \end{pmatrix}$

$v^\top M^\top Mv = (Mv)^\top(Mv) = \|Mv\|^2 \ge 0$ A squared length is never negative, so $M^\top M$ is PSD.
$\|Mv\|^2 = 0 \iff Mv = 0$ and $Mv$ is the combination of the columns with coefficients $v$ So $v^\top M^\top Mv > 0$ for every $v \ne 0$ exactly when no nonzero combination of the columns is $0$
$M_1^\top M_1 = \begin{pmatrix} 3 & 6 \\ 6 & 14 \end{pmatrix}$ with minors $3$ and $42 - 36 = 6$ Both positive: PD, as the columns $(1, 1, 1)$ and $(1, 2, 3)$ are independent.
$M_2^\top M_2 = \begin{pmatrix} 3 & 6 \\ 6 & 12 \end{pmatrix}$ with determinant $0$ and eigenvalues $0$ and $15$ The second column is twice the first, and $v = (2, -1)$ gives $M_2v = 0$
$M^\top M$ is always PSD; $M_1^\top M_1$ is PD, $M_2^\top M_2$ is only PSD, with $v^\top M_2^\top M_2v = 0$ at $v = (2, -1)$ This is why the normal equations have a unique solution exactly when the features are not collinear.

Problem 10

Ridge regression minimizes $L(w) = \|Xw - y\|^2 + \lambda\|w\|^2$ Find the gradient and Hessian of $L$ With $X = M_2$ from Problem 9, $y = (1, 2, 3)$ and $\lambda = \tfrac12$ show that the Hessian is positive definite and find the unique minimizer.

$\nabla L = 2X^\top(Xw - y) + 2\lambda w$ and $\nabla^2L = 2(X^\top X + \lambda I)$ $L = w^\top(X^\top X + \lambda I)w - 2y^\top Xw + y^\top y$ a quadratic form plus a linear term.
$v^\top(X^\top X + \lambda I)v = \|Xv\|^2 + \lambda\|v\|^2 > 0$ for $v \ne 0$ Whatever $X$ is, when $\lambda > 0$ So $L$ is strictly convex with one minimizer.
$X^\top X + \tfrac12I = \begin{pmatrix} 3.5 & 6 \\ 6 & 12.5 \end{pmatrix}$ with eigenvalues $0.5$ and $15.5$ and $X^\top y = (6, 12)$ The eigenvalues of $X^\top X$ $0$ and $15$ each move up by $\lambda$
$\det = 43.75 - 36 = 7.75$ so $w_1 = \dfrac{12.5\cdot6 - 6\cdot12}{7.75} = \dfrac{12}{31}$ and $w_2 = \dfrac{3.5\cdot12 - 6\cdot6}{7.75} = \dfrac{24}{31}$ Cramer's rule on $(X^\top X + \lambda I)w = X^\top y$
$\nabla^2L = 2(X^\top X + \lambda I)$ PD with eigenvalues $1$ and $31$ $w^* = \big(\tfrac{12}{31}, \tfrac{24}{31}\big)$ Without $\lambda$ the Hessian is singular and every $w$ with $w_1 + 2w_2 = 2$ is a minimizer. Ridge picks one along $(1, 2)$ with $w_1 + 2w_2 = \tfrac{60}{31} \approx 1.935$ shrunk slightly below $2$

Where this goes wrong

1. The cross coefficient left unhalved

The coefficient of $xy$ is copied straight into the matrix.

$q = 3x^2 + 4xy + 2y^2$ with $3$ and $2$ on the diagonalRight so far: Problem 1.
“The $xy$ coefficient goes off the diagonal: $A = \begin{pmatrix} 3 & 4 \\ 4 & 2 \end{pmatrix}$ ”Both off-diagonal entries multiply $xy$ so together they give $8xy$
The form is indefinite, since $\det A = 6 - 16 = -10 < 0$ That matrix is the form $3x^2 + 8xy + 2y^2$ At $(1, -1)$ it gives $-3$ while $q(1, -1) = 1$ With $A_{12} = 2$ the determinant is $2 > 0$ and $q$ is positive definite (Problem 4).

2. A positive diagonal taken as proof

A diagonal matrix with positive entries is positive definite, and the shortcut gets applied to every matrix.

$B = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ has $B_{11} = B_{22} = 1 > 0$ Right so far: Problem 2. A positive diagonal is necessary, since $e_i^\top Be_i = B_{ii}$
“Every diagonal entry is positive, so every $x^\top Bx$ is positive.”The cross term $4xy$ can outweigh the squares.
$B$ is positive definite $q_B(1, -1) = 1 - 4 + 1 = -2$ and the eigenvalues are $3$ and $-1$ A positive diagonal is not sufficient; a $2\times2$ also needs $\det > 0$ and here $\det B = -3$

3. Leading minors used to test semidefiniteness

Sylvester's criterion is remembered as “check the leading minors”, with $> 0$ relaxed to $\ge 0$

$A = \begin{pmatrix} 0 & 0 \\ 0 & -1 \end{pmatrix}$ has $\Delta_1 = 0$ and $\Delta_2 = 0$ Right so far: both leading minors are $\ge 0$
“All leading minors are non-negative, so $A$ is PSD.”For semidefiniteness every principal minor must be checked, including the bottom-right entry $-1$
$A$ is positive semidefinite $(0, 1)^\top A(0, 1) = -1 < 0$ $A$ is negative semidefinite. The $1\times1$ principal minor $A_{22} = -1$ is the one the leading minors skip.

4. The long axis along the largest eigenvalue

The biggest eigenvalue is thought of as the biggest direction, as it is for a covariance matrix.

$5x^2 - 4xy + 5y^2 = 21$ has eigenvalues $3$ along $(1, 1)$ and $7$ along $(1, -1)$ Right so far: Problem 8.
“The major semi-axis $\sqrt7$ points along the eigenvector of $7$ ”Along an eigenvector the form is $\lambda r^2$ so a large $\lambda$ reaches $21$ at a small $r$
The ellipse reaches $\sqrt7\cdot\tfrac{1}{\sqrt2}(1, -1)$ There $x = -y = \sqrt{3.5}$ and the form is $(5 + 4 + 5)\cdot3.5 = 49$ not $21$ The semi-axis along an eigenvector is $\sqrt{c/\lambda}$ $\sqrt{21/7} = \sqrt3$ along $(1, -1)$ and $\sqrt7$ along $(1, 1)$ For a Gaussian the ellipse is $x^\top\Sigma^{-1}x = c$ whose eigenvalues are $1/\lambda$ which is why the covariance's largest eigenvalue does give the long axis.

5. The extremes read off the diagonal

On the unit circle the values at $(1, 0)$ and $(0, 1)$ are the diagonal entries, and these are taken as the extremes.

$q = 5x^2 + 4xy + 2y^2$ has $q(1, 0) = 5$ and $q(0, 1) = 2$ Right so far: Problem 7.
“The coordinate axes give the largest and smallest values.”Only when the cross term is $0$ so that the axes are the eigenvectors.
On the unit circle, $2 \le q \le 5$ The eigenvalues are $6$ and $1$ at $\tfrac{1}{\sqrt5}(2, 1)$ and $\tfrac{1}{\sqrt5}(1, -2)$ The diagonal entries always lie between $\lambda_{\min}$ and $\lambda_{\max}$ so they never show the extremes unless the matrix is diagonal.

Print this set: quadratic-forms-and-positive-definite-matrices.pdf (problems, answers, and worked solutions on separate pages).

Read the guide: Quadratic forms and positive definite matrices: halve the cross term, test with eigenvalues or pivots, and read the axes off the eigenvectors