Matrix exponentials and linear systems of ODEs

Worked solutions

Problem 1

1. $\lambda^2 - 7\lambda + 10 = 0$, so $\lambda_1 = 5$, $\lambda_2 = 2$The trace is $7$ and the determinant $12 - 2 = 10$.
2. $v_1 = (1, 1)$, $v_2 = (1, -2)$$(A - 5I)v = 0$ reads $-x + y = 0$; $(A - 2I)v = 0$ reads $2x + y = 0$.
3. $c_1(1, 1) + c_2(1, -2) = (2, -1)$ gives $c_1 = 1$, $c_2 = 1$Subtracting the second equation from the first: $3c_2 = 3$.
4. $x(t) = e^{5t}(1, 1) + e^{2t}(1, -2)$Each eigenvector component grows at its own rate $e^{\lambda t}$.
5. $x(t) = \big(e^{5t} + e^{2t},\ e^{5t} - 2e^{2t}\big)$Check: $x(0) = (2, -1)$ and $x'(0) = (5 + 2, 5 - 4) = (7, 1) = A\,(2, -1)$. For large $t$ the $e^{5t}$ term dominates and $x$ points along $(1, 1)$.

Problem 2

1. $V = \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix}$, $V^{-1} = \tfrac13\begin{pmatrix} 2 & 1 \\ 1 & -1 \end{pmatrix}$The eigenvectors of Problem 1 as columns; $\det V = -3$.
2. $e^{\Lambda t}V^{-1} = \tfrac13\begin{pmatrix} 2e^{5t} & e^{5t} \\ e^{2t} & -e^{2t} \end{pmatrix}$A diagonal matrix on the left scales the rows.
3. $e^{At} = \tfrac13\begin{pmatrix} 2e^{5t} + e^{2t} & e^{5t} - e^{2t} \\ 2e^{5t} - 2e^{2t} & e^{5t} + 2e^{2t} \end{pmatrix}$Multiply by $V$ on the left.
4. At $t = 0$ it is $I$, and its derivative there is $\tfrac13\begin{pmatrix} 12 & 3 \\ 6 & 9 \end{pmatrix} = A$The two properties that define $e^{At}$.
5. $e^{At}$ as in step 3; $e^{At}(2, -1) = (e^{5t} + e^{2t},\ e^{5t} - 2e^{2t})$The first entry is $\tfrac13(4e^{5t} + 2e^{2t} - e^{5t} + e^{2t})$, matching Problem 1. Writing $V^{-1}e^{\Lambda t}V$ instead gives the transpose of this matrix (Mistake 3).

Problem 3

1. $\lambda^2 - 4\lambda + 4 = (\lambda - 2)^2$Trace $4$, determinant $3 + 1 = 4$.
2. $N = A - 2I = \begin{pmatrix} 1 & 1 \\ -1 & -1 \end{pmatrix}$, and $Nv = 0$ only for multiples of $(1, -1)$$N$ has rank $1$, so there is one eigenvector and $V$ cannot be built.
3. $N^2 = 0$, so $e^{Nt} = I + Nt$The series stops after the linear term.
4. $e^{At} = e^{2t}(I + Nt) = e^{2t}\begin{pmatrix} 1 + t & t \\ -t & 1 - t \end{pmatrix}$$2I$ commutes with every matrix, so $e^{(2I + N)t} = e^{2t}e^{Nt}$.
5. $x(t) = e^{2t}(1 + t,\ -t)$The first column of $e^{At}$. Check: $x'(0) = (2 + 1, -1) = (3, -1) = A\,(1, 0)$. The factor $t$ next to $e^{2t}$ is what a repeated eigenvalue adds, as $te^{rt}$ does for a repeated characteristic root (Mistake 4).

Problem 4

1. $J^2 = -I$, so $J^3 = -J$ and $J^4 = I$The powers cycle with period $4$, like the powers of $i$.
2. $e^{Jt} = \big(1 - \tfrac{t^2}{2!} + \cdots\big)I + \big(t - \tfrac{t^3}{3!} + \cdots\big)J$Group the even and the odd powers.
3. $e^{Jt} = \begin{pmatrix} \cos t & -\sin t \\ \sin t & \cos t \end{pmatrix}$The two series are $\cos t$ and $\sin t$: rotation by the angle $t$.
4. $B = -I + 2J$, so $e^{Bt} = e^{-t}e^{2Jt}$$-I$ commutes with $J$. The eigenvalues of $B$ are $-1 \pm 2i$.
5. $x(t) = e^{-t}(\cos 2t,\ \sin 2t)$The first column of $e^{-t}$ times the rotation by $2t$.
6. $x(\pi/2) = (-e^{-\pi/2}, 0) \approx (-0.208, 0)$ and $|x(t)| = e^{-t}$A spiral that turns at angular speed $2$, the imaginary part of the eigenvalues, and shrinks at rate $1$, the negative of the real part. By $t = \tfrac\pi2$ it has turned through the angle $\pi$, halfway around.

Problem 5

1. $A_1$: $\tau = -6$, $\Delta = 8$, $\tau^2 - 4\Delta = 4 > 0$Real eigenvalues $\tfrac{-6 \pm 2}{2} = -2, -4$: a stable node.
2. $A_2$: $\tau = 0$, $\Delta = -1 + 5 = 4$Eigenvalues $\pm 2i$: a center. Orbits are closed curves that neither shrink nor grow.
3. $A_3$: $\Delta = -6 < 0$Eigenvalues $3$ and $-2$, of opposite signs: a saddle, unstable.
4. $A_4$: $\tau = 4$, $\Delta = 13$, $\tau^2 - 4\Delta = -36 < 0$Eigenvalues $2 \pm 3i$: an unstable spiral.
5. $A_1$ stable node, $A_2$ center, $A_3$ saddle, $A_4$ unstable spiralOnly $A_1$ sends every solution to $0$. $A_4$ has $\Delta > 0$ like $A_1$, so the determinant alone cannot decide stability; and a matrix with a negative diagonal can still be a saddle (Mistake 5).

Problem 6

1. $A = \begin{pmatrix} 0 & 1 \\ -2 & -3 \end{pmatrix}$, $x(0) = (1, 0)$The first row says $y' = y'$; the second is the equation solved for $y''$.
2. $\det(A - \lambda I) = \lambda^2 + 3\lambda + 2$, so $\lambda = -1, -2$The characteristic polynomial of $A$ is the characteristic equation of the ODE.
3. $v_1 = (1, -1)$, $v_2 = (1, -2)$For this companion matrix the eigenvector for $\lambda$ is $(1, \lambda)$: $y = e^{\lambda t}$ has $y' = \lambda e^{\lambda t}$.
4. $c_1 + c_2 = 1$, $-c_1 - 2c_2 = 0$, so $c_1 = 2$, $c_2 = -1$Match $x(0) = (1, 0)$.
5. $y(t) = 2e^{-t} - e^{-2t}$The first entry of $2e^{-t}(1, -1) - e^{-2t}(1, -2)$. Check: $y(0) = 1$ and $y'(0) = -2 + 2 = 0$. Both eigenvalues are negative and real, so $y$ decays without oscillating: the system is overdamped.

Problem 7

1. Problem 2: $9\det e^{At} = (2e^{5t} + e^{2t})(e^{5t} + 2e^{2t}) - 2(e^{5t} - e^{2t})^2$The factor $\tfrac13$ squared comes out as $\tfrac19$.
2. $= (2e^{10t} + 5e^{7t} + 2e^{4t}) - (2e^{10t} - 4e^{7t} + 2e^{4t}) = 9e^{7t}$So $\det e^{At} = e^{7t}$, and $\operatorname{tr}A = 7$.
3. Problem 3: $\det e^{At} = e^{4t}\big((1 + t)(1 - t) + t^2\big) = e^{4t}$Pulling $e^{2t}$ out of both rows gives $e^{4t}$; the bracket is $1$. $\operatorname{tr}A = 4$.
4. In general $\det e^{At} = \prod_i e^{\lambda_it} = e^{t\sum_i\lambda_i}$The eigenvalues of $e^{At}$ are $e^{\lambda_it}$, and the determinant is their product.
5. $\det e^{At} = e^{7t}$ and $e^{4t}$, never $0$; the inverse is $e^{-At}$$At$ and $-At$ commute, so $e^{At}e^{-At} = e^{0} = I$. Running the system backward in time undoes it.

Problem 8

1. $e^A = I + A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$, $e^B = I + B = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix}$$A^2 = B^2 = 0$, so each series stops.
2. $e^Ae^B = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}$, $e^Be^A = \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix}$Already the two orders disagree.
3. $(A + B)^2 = I$, so $e^{A + B} = (\cosh 1)I + (\sinh 1)(A + B)$Even powers are $I$ and odd powers are $A + B$, like $e^{J}$ in Problem 4 without the signs.
4. $e^{A + B} = \begin{pmatrix} \cosh 1 & \sinh 1 \\ \sinh 1 & \cosh 1 \end{pmatrix} \approx \begin{pmatrix} 1.543 & 1.175 \\ 1.175 & 1.543 \end{pmatrix}$Symmetric, unlike either product.
5. $e^{A + B}$ equals neither $e^Ae^B$ nor $e^Be^A$, because $AB = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} \ne \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} = BA$The step $e^{a}e^{b} = e^{a + b}$ rearranges $a$s past $b$s, which matrices allow only when they commute. All three matrices have determinant $1 = e^{\operatorname{tr}(A + B)}$, so the determinant cannot catch the error (Mistake 2).

Problem 9

1. The eigenvalues of $Q$ are $0$ and $-4$Each row of $Q$ sums to $0$, so $Q\mathbf 1 = 0$; the trace gives the other, $-4$.
2. $P(t) = \tfrac14\begin{pmatrix} 3 + e^{-4t} & 1 - e^{-4t} \\ 3 - 3e^{-4t} & 1 + 3e^{-4t} \end{pmatrix}$The two-state formula of the Markov chains set with $\lambda_2^n$ replaced by $e^{-4t}$; at $t = 0$ it is $I$, and its derivative there is $Q$.
3. $\Pr(D \text{ at } t = \tfrac12 \mid U) = \tfrac14(1 - e^{-2}) \approx 0.216$Twelve hours is $t = \tfrac12$ day, so $e^{-4t} = e^{-2} \approx 0.135$.
4. As $t \to \infty$, both rows tend to $(\tfrac34, \tfrac14)$The stationary distribution solves $\pi Q = 0$: $-\pi_U + 3\pi_D = 0$.
5. $P(t)$ as in step 2; down at $12$ hours with probability $\approx 0.216$; up $\tfrac34$ of the timeThe mean up time is $1$ day and the mean repair $\tfrac13$ day, and $1/(1 + \tfrac13) = \tfrac34$. The other entry of the top row at $t = \tfrac12$ is $\approx 0.784$, and each row sums to $1$, which an entrywise exponential does not (Mistake 1).

Problem 10

1. $H$ has eigenvalues $4$ and $2$ with eigenvectors $(1, 1)$ and $(1, -1)$, and $w(0) = (1, 1) + (1, -1)$$H$ is symmetric, so its eigenvectors are orthogonal.
2. $w(t) = e^{-4t}(1, 1) + e^{-2t}(1, -1)$$e^{-Ht}$ multiplies each eigenvector component by $e^{-\lambda t}$.
3. $L(w(t)) = \tfrac12\big(4\cdot2e^{-8t} + 2\cdot2e^{-4t}\big) = 4e^{-8t} + 2e^{-4t}$Orthogonal components add in $w^\top Hw$; each $(1, \pm1)$ has squared length $2$. At $t = 0$, $L = 6$.
4. With $u = e^{-4t}$: $4u^2 + 2u = 0.01$ gives $u \approx 0.00495$ and $t = -\tfrac14\ln u \approx 1.33$The positive root of the quadratic. The slow direction, eigenvalue $2$, sets the time.
5. Gradient descent multiplies the components by $1 - 0.1\cdot4 = 0.6$ and $1 - 0.1\cdot2 = 0.8$ per step$(I - \eta H)$ has the same eigenvectors as $H$.
6. After $10$ steps: $0.6^{10} \approx 0.00605$ and $0.8^{10} \approx 0.107$, against $e^{-4} \approx 0.0183$ and $e^{-2} \approx 0.135$Each step is the first two terms of $e^{-\eta H}$; with $\eta = 0.1$ that is close for the slow direction and rough for the fast one.
7. $w(t) = e^{-4t}(1, 1) + e^{-2t}(1, -1)$, $L = 4e^{-8t} + 2e^{-4t}$, below $0.01$ from $t \approx 1.33$; gradient descent converges exactly for $0 < \eta < \tfrac12$Convergence needs $|1 - \eta\lambda| < 1$ for both eigenvalues, so $\eta < 2/\lambda_{\max} = \tfrac12$. The flow converges for every $t$; only the discrete steps can overshoot.