Eigenvalues and eigenvectors by hand

Worked solutions

Problem 1

1. $A - \lambda I = \begin{bmatrix}2-\lambda&1\\1&2-\lambda\end{bmatrix}$, so $\det(A - \lambda I) = (2-\lambda)^2 - 1$.Subtract $\lambda$ from each diagonal entry, then $ad - bc$.
2. $(2 - \lambda)^2 = 1$, so $2 - \lambda = \pm1$ and ${\lambda = 1}$ or ${\lambda = 3}$.The eigenvalues are the roots of the characteristic polynomial; this one is a square minus $1$, so take square roots rather than expanding.
3. $\lambda = 3$: $A - 3I = \begin{bmatrix}-1&1\\1&-1\end{bmatrix}$, and with $v = (x, y)^\top$, $(A - 3I)v = 0$ says $-x + y = 0$, so $v = (1, 1)^\top$.The eigenvectors for $\lambda = 3$ are the null space of $A - 3I$. Its two rows are multiples of each other, as they must be when the determinant is $0$, so one equation remains; choose $x = 1$.
4. $\lambda = 1$: $A - I = \begin{bmatrix}1&1\\1&1\end{bmatrix}$, and $(A - I)v = 0$ says $x + y = 0$, so $v = (1, -1)^\top$.Null space of $A - I$; the equation $x + y = 0$ fixes $v$ up to scale.
5. $\lambda = 3$ with $v = (1,1)^\top$; $\lambda = 1$ with $v = (1,-1)^\top$Check: $A(1,1)^\top = (3,3)^\top$ and $A(1,-1)^\top = (1,-1)^\top$.

Problem 2

1. $\det(A - \lambda I) = (4-\lambda)(3-\lambda) - 1\cdot2$.$ad - bc$ for $A - \lambda I = \begin{bmatrix}4-\lambda&1\\2&3-\lambda\end{bmatrix}$.
2. $(4-\lambda)(3-\lambda) - 2 = \lambda^2 - 7\lambda + 12 - 2 = \lambda^2 - 7\lambda + 10$.Expand the product and collect terms.
3. $\lambda^2 - 7\lambda + 10 = (\lambda - 5)(\lambda - 2)$.Two numbers with product $10$ and sum $7$.
4. $\lambda = 5$: $A - 5I = \begin{bmatrix}-1&1\\2&-2\end{bmatrix}$ gives $-x + y = 0$ for $v = (x, y)^\top$, so $v = (1, 1)^\top$.Null space of $A - 5I$; the second row is $-2$ times the first, so it adds nothing.
5. $\lambda = 2$: $A - 2I = \begin{bmatrix}2&1\\2&1\end{bmatrix}$ gives $2x + y = 0$, so $v = (1, -2)^\top$.Null space of $A - 2I$, with $x = 1$.
6. $\lambda^2 - 7\lambda + 10 = (\lambda-5)(\lambda-2)$; $\lambda = 5$, $v = (1,1)^\top$; $\lambda = 2$, $v = (1,-2)^\top$Check: $A(1,1)^\top = (5,5)^\top$ and $A(1,-2)^\top = (2,-4)^\top$. The eigenvectors are not perpendicular; $A$ is not symmetric, and Problem 8 shows why that matters.

Problem 3

1. $\det(R - \lambda I) = \det\begin{bmatrix}-\lambda&-1\\1&-\lambda\end{bmatrix} = \lambda^2 + 1$.$ad - bc = (-\lambda)(-\lambda) - (-1)(1)$.
2. $\lambda^2 + 1 = 0$ has no real root; $\lambda = \pm i$.$\lambda^2 \ge 0$ for every real $\lambda$, so $\lambda^2 + 1 \ge 1$; the two roots are complex.
3. No real eigenvector.A real eigenvector $v$ would make $Rv = \lambda v$ with both sides real, so $\lambda$ would be real; there is no real eigenvalue. The eigenvectors are complex: $\lambda = i$ has $v = (1, -i)^\top$, since $R(1,-i)^\top = (i, 1)^\top = i(1,-i)^\top$.
4. $R(x, y)^\top = (-y, x)^\top$, which is $v = (x, y)^\top$ turned $90°$ anticlockwise.$v \cdot Rv = -xy + yx = 0$ and $\|Rv\| = \|v\|$, so for every nonzero real $v$, $Rv$ is a nonzero vector perpendicular to $v$, off the line through $v$. An eigenvector is exactly a vector that stays on its own line.
5. $\lambda^2 + 1 = 0$, $\lambda = \pm i$; no real eigenvector: a rotation by $90°$ sends every real direction somewhere elseSteps 2 to 4.

Problem 4

1. $\det(A - \lambda I) = (1-\lambda)^2 - 1\cdot 0 = (\lambda - 1)^2$.$A - \lambda I$ is upper triangular, so its determinant is the product of the diagonal entries.
2. $\lambda = 1$, a double root.The only root, counted twice: its algebraic multiplicity, the number of times it is a root, is $2$.
3. $A - I = \begin{bmatrix}0&1\\0&0\end{bmatrix}$ has one nonzero row, so $\operatorname{rank}(A - I) = 1$.The rank decides how many independent eigenvectors $\lambda = 1$ has: the null space of a $2\times2$ matrix has dimension $2 - \operatorname{rank}$.
4. For $v = (x, y)^\top$, $(A - I)v = 0$ says $y = 0$, so every eigenvector is $(x, 0)^\top$, a multiple of $(1, 0)^\top$.The null space has dimension $2 - 1 = 1$: the geometric multiplicity, the number of independent eigenvectors, is $1$.
5. Not diagonalisable.$A = PDP^{-1}$ needs two independent eigenvectors for the columns of an invertible $P$, and there is only one direction. Directly: $D$ would have to be $\operatorname{diag}(1,1) = I$, and then $PIP^{-1} = I \neq A$.
6. $(\lambda - 1)^2$, $\lambda = 1$ (double); $A - I = \begin{bmatrix}0&1\\0&0\end{bmatrix}$ has rank $1$, so the only eigenvectors are multiples of $(1,0)^\top$; not diagonalisableSteps 1 to 5.

Problem 5

1. Let $A = \begin{bmatrix}a&b\\c&d\end{bmatrix}$. Then $\det(A - \lambda I) = (a - \lambda)(d - \lambda) - bc$.$ad - bc$ for $A - \lambda I$.
2. $(a - \lambda)(d - \lambda) - bc = \lambda^2 - (a + d)\lambda + (ad - bc)$.Expand and collect powers of $\lambda$.
3. $\det(A - \lambda I) = (\lambda - \lambda_1)(\lambda - \lambda_2) = \lambda^2 - (\lambda_1 + \lambda_2)\lambda + \lambda_1\lambda_2$.$\lambda_1$ and $\lambda_2$ are the two roots, complex or repeated if need be, and a quadratic whose $\lambda^2$ coefficient is $1$ is the product of $(\lambda - \text{root})$ over its roots.
4. $a + d = \lambda_1 + \lambda_2$ and $ad - bc = \lambda_1\lambda_2$.Steps 2 and 3 are the same polynomial, so their coefficients of $\lambda$ and their constant terms agree.
5. $\operatorname{tr}A = \lambda_1+\lambda_2$, $\det A = \lambda_1\lambda_2$; Problem 2: $7 = 5+2$, $10 = 5\cdot2$For Problem 2, $\operatorname{tr}A = 4 + 3 = 7$ and $\det A = 4\cdot3 - 1\cdot2 = 10$. This is a quick check on any eigenvalue computation.

Problem 6

Write $t = 2 - \lambda$, so the diagonal entries of $A - \lambda I$ are all $t$ and the off-diagonal entries are those of $A$. Each eigenvector is written $v = (x, y, z)^\top$.

1. $\det(A - \lambda I) = t\det\begin{bmatrix}t&1\\1&t\end{bmatrix} - 1\cdot\det\begin{bmatrix}1&1\\0&t\end{bmatrix} + 0$.Cofactor expansion along the first row, $(t, 1, 0)$: each entry times the determinant left after deleting its row and column, with signs $+, -, +$.
2. $t(t^2 - 1) - t = t^3 - 2t = t(t^2 - 2)$.The two $2\times2$ determinants are $t^2 - 1$ and ${t - 0}$.
3. $\det(A - \lambda I) = (2 - \lambda)\big((2 - \lambda)^2 - 2\big)$.Substitute $t = 2 - \lambda$ back.
4. $\lambda = 2$, or $2 - \lambda = \pm\sqrt2$, that is $\lambda = 2 \mp \sqrt2$.Set each factor to $0$. Check with the trace: $2 + (2 - \sqrt2) + (2 + \sqrt2) = 6 = \operatorname{tr}A$.
5. $\lambda = 2$ ($t = 0$): the rows of $A - 2I$ give $y = 0$, $x + z = 0$, $y = 0$, so $v = (1, 0, -1)^\top$.The null space of $A - 2I$, whose rows are $(0,1,0)$, $(1,0,1)$, $(0,1,0)$.
6. $\lambda = 2 + \sqrt2$ ($t = -\sqrt2$): the rows give $-\sqrt2\,x + y = 0$, $x - \sqrt2\,y + z = 0$, $y - \sqrt2\,z = 0$.The null space of $A - (2 + \sqrt2)I$, whose diagonal entries are $-\sqrt2$.
7. $y = \sqrt2\,x$ and $y = \sqrt2\,z$, so $z = x$; with $x = 1$, $v = (1, \sqrt2, 1)^\top$.The first and third equations; the middle one then holds, $1 - 2 + 1 = 0$, as it must because the determinant is $0$.
8. $\lambda = 2 - \sqrt2$ ($t = \sqrt2$): the same equations with $\sqrt2$ in place of $-\sqrt2$ give $y = -\sqrt2\,x = -\sqrt2\,z$, so $v = (1, -\sqrt2, 1)^\top$.Only the sign of $t$ changed, so only the sign of $y$ changes.
9. $\lambda = 2$ with $(1,0,-1)^\top$; $\lambda = 2\pm\sqrt2$ with $(1, \pm\sqrt2, 1)^\top$Check: $A(1,\sqrt2,1)^\top = (2+\sqrt2,\ 2+2\sqrt2,\ 2+\sqrt2)^\top = (2+\sqrt2)(1,\sqrt2,1)^\top$. $A$ is symmetric, and the three eigenvectors are mutually perpendicular (Problem 8).

Problem 7

1. $P = \begin{bmatrix}1&1\\1&-2\end{bmatrix}$, $D = \operatorname{diag}(5, 2)$.The columns of $P$ are the eigenvectors from Problem 2, and $D$ lists their eigenvalues in the same order. Then $AP = PD$ column by column, which is $Av = \lambda v$ twice.
2. $\det P = 1\cdot(-2) - 1\cdot1 = -3$, so $P^{-1} = -\tfrac13\begin{bmatrix}-2&-1\\-1&1\end{bmatrix} = \begin{bmatrix}2/3&1/3\\1/3&-1/3\end{bmatrix}$.The $2\times2$ inverse: swap the diagonal, negate the off-diagonal, divide by the determinant. $\det P \neq 0$ because the two eigenvectors are independent.
3. $PD = \begin{bmatrix}5&2\\5&-4\end{bmatrix}$ and $PDP^{-1} = \begin{bmatrix}4&1\\2&3\end{bmatrix} = A$.Check the diagonalisation by multiplying it out: the first row of $PDP^{-1}$ is $(10/3 + 2/3,\ 5/3 - 2/3) = (4, 1)$.
4. $A^5 = PDP^{-1}\,PDP^{-1}\cdots PDP^{-1} = PD^5P^{-1}$.Each inner $P^{-1}P$ is $I$, so the five copies collapse. Powers of a diagonal matrix are powers of its entries: $D^5 = \operatorname{diag}(3125, 32)$.
5. $PD^5 = \begin{bmatrix}3125&32\\3125&-64\end{bmatrix}$.Multiplying by a diagonal matrix on the right scales the columns of $P$ by $5^5$ and $2^5$.
6. $PD^5P^{-1} = \tfrac13\begin{bmatrix}6250+32&3125-32\\6250-64&3125+64\end{bmatrix} = \begin{bmatrix}2094&1031\\2062&1063\end{bmatrix}$.Row times column with $P^{-1} = \tfrac13\begin{bmatrix}2&1\\1&-1\end{bmatrix}$.
7. $P = \begin{bmatrix}1&1\\1&-2\end{bmatrix}$, $D = \operatorname{diag}(5,2)$, $P^{-1} = -\tfrac13\begin{bmatrix}-2&-1\\-1&1\end{bmatrix}$; $A^5 = PD^5P^{-1} = \begin{bmatrix}2094&1031\\2062&1063\end{bmatrix}$Check: $(1,1)^\top$ is an eigenvector with eigenvalue $5$, so $A^5(1,1)^\top = 3125(1,1)^\top$, and both rows do sum to $3125$.

Problem 8

1. $(1,1)^\top \cdot (1,-1)^\top = 1 - 1 = 0$.Orthogonal means a zero dot product.
2. $Q = \tfrac1{\sqrt2}\begin{bmatrix}1&1\\1&-1\end{bmatrix}$, $\Lambda = \operatorname{diag}(3, 1)$.Divide each eigenvector by its length $\sqrt2$ so the columns are unit vectors; they are still eigenvectors. $\Lambda$ lists the eigenvalues in column order.
3. $Q^\top Q = \tfrac12\begin{bmatrix}2&0\\0&2\end{bmatrix} = I$.The columns are orthogonal unit vectors, so $Q$ is orthogonal and $Q^{-1} = Q^\top$.
4. $Q\Lambda Q^\top = \tfrac12\begin{bmatrix}3&1\\3&-1\end{bmatrix}\begin{bmatrix}1&1\\1&-1\end{bmatrix} = \tfrac12\begin{bmatrix}4&2\\2&4\end{bmatrix} = A$.$A = Q\Lambda Q^{-1}$ as in Problem 7, with $Q^{-1}$ replaced by $Q^\top$; multiplying it out confirms it.
5. For symmetric $A$ with $Av_1 = \lambda_1v_1$, $Av_2 = \lambda_2v_2$: $\lambda_1v_1^\top v_2 = (Av_1)^\top v_2 = v_1^\top A^\top v_2 = v_1^\top Av_2 = \lambda_2v_1^\top v_2$.Move $A$ across the dot product: $(Av_1)^\top = v_1^\top A^\top$, and $A^\top = A$ is where symmetry is used.
6. $(\lambda_1 - \lambda_2)\,v_1^\top v_2 = 0$, so $v_1^\top v_2 = 0$ when $\lambda_1 \neq \lambda_2$.Subtract the two ends of step 5; a product is $0$ only if a factor is. A repeated eigenvalue of a symmetric matrix has as many independent eigenvectors as its multiplicity (the spectral theorem), and they can be chosen orthogonal, so $A = Q\Lambda Q^\top$ holds for every real symmetric matrix.
7. $Q = \tfrac1{\sqrt2}\begin{bmatrix}1&1\\1&-1\end{bmatrix}$, $\Lambda = \operatorname{diag}(3,1)$, $A = Q\Lambda Q^\top$; for symmetric $A$, $\lambda_1 v_1^\top v_2 = (Av_1)^\top v_2 = v_1^\top Av_2 = \lambda_2 v_1^\top v_2$, so distinct eigenvalues force $v_1^\top v_2 = 0$Steps 1 to 6.

Problem 9

1. $x_0 = \tfrac12(1,1)^\top + \tfrac12(1,-1)^\top$.Write $x_0$ in the eigenvectors from Problem 1, where $A$ acts by scaling; the coefficients solve $c_1 + c_2 = 1$, $c_1 - c_2 = 0$.
2. $A^k(1,1)^\top = 3^k(1,1)^\top$ and $A^k(1,-1)^\top = 1^k(1,-1)^\top = (1,-1)^\top$.Each application of $A$ multiplies an eigenvector by its eigenvalue, so $k$ applications multiply it by the $k$-th power.
3. $A^kx_0 = \tfrac{3^k}{2}(1,1)^\top + \tfrac12(1,-1)^\top$.$A^k$ is linear, so apply step 2 to each term of step 1. Check: $k = 1$ gives $(2, 1)^\top = Ax_0$, and $k = 2$ gives $(5, 4)^\top$.
4. $A^kx_0 = \tfrac{3^k}{2}\big((1,1)^\top + 3^{-k}(1,-1)^\top\big)$.Factor out the growing term; the positive scale factor does not change the direction.
5. $\tan\theta_k = 3^{-k}$, where $\theta_k$ is the angle between $A^kx_0$ and $(1,1)^\top$.In step 4 the two eigenvectors are perpendicular and have the same length $\sqrt2$, so the ratio of their coefficients, $3^{-k}$, is the tangent of that angle, and it goes to $0$.
6. $A^kx_0 = \tfrac{3^k}{2}(1,1)^\top + \tfrac12(1,-1)^\top$; the direction approaches $(1,1)/\sqrt2$, the error shrinking like $(1/3)^k$ — the ratio of the two eigenvaluesThis is power iteration: repeated multiplication (with a rescaling each step to keep the length finite, since it grows like $3^k$) finds the eigenvector of the largest eigenvalue, at a rate set by $\lambda_2/\lambda_1 = 1/3$.

Problem 10

1. $A^\top A = \begin{bmatrix}3&4\\0&5\end{bmatrix}\begin{bmatrix}3&0\\4&5\end{bmatrix} = \begin{bmatrix}9+16&20\\20&25\end{bmatrix} = \begin{bmatrix}25&20\\20&25\end{bmatrix}$.Row times column; the result is symmetric, as $A^\top A$ always is.
2. $\det(A^\top A - \lambda I) = (25 - \lambda)^2 - 400$, so $25 - \lambda = \pm20$ and $\lambda = 45$ or $5$.The same shape as Problem 1: a square minus a constant. The eigenvectors are $(1,1)^\top$ and $(1,-1)^\top$ again.
3. $\sigma_1 = \sqrt{45} = 3\sqrt5$ and $\sigma_2 = \sqrt5$.The singular values are the square roots of the eigenvalues of $A^\top A$: if $A = U\Sigma V^\top$ with $U$, $V$ orthogonal and $\Sigma = \operatorname{diag}(\sigma_1, \sigma_2)$, then $A^\top A = V\Sigma^2V^\top$, an orthogonal diagonalisation as in Problem 8 with $\Lambda = \Sigma^2$. The eigenvalues are never negative, since $v^\top A^\top Av = \|Av\|^2 \ge 0$.
4. $\sigma_1\sigma_2 = 3\sqrt5\cdot\sqrt5 = 15$, and $\det A = 3\cdot5 - 0\cdot4 = 15$.$\det(A^\top A) = (\det A)^2$ is the product of its eigenvalues $\sigma_1^2\sigma_2^2$ (Problem 5), so $\sigma_1\sigma_2 = |\det A|$; here $45\cdot5 = 225 = 15^2$.
5. $A^\top A = \begin{bmatrix}25&20\\20&25\end{bmatrix}$ with eigenvalues $45$ and $5$; singular values $3\sqrt5$ and $\sqrt5$; their product $15 = |\det A|$The eigenvalues of $A$ itself are $3$ and $5$, its diagonal entries since it is triangular: same product, but not the singular values. They coincide, up to sign, only for special matrices such as symmetric ones.