Guides / Derivatives

Lagrange multipliers: ∇f = λ∇g, then let the constraint fix λ and compare every candidate

To find the largest or smallest value of f(x, y) when the point has to stay on a curve g(x, y) = c, you can't just set the gradient to zero: the best point on the curve is usually not a flat point of f. Lagrange's method replaces ∇f = 0 with ∇f = λ∇g. The mistakes come from canceling a variable that could be zero, from forgetting the constraint once λ appears, and from calling a lone candidate a maximum on a curve that runs off to infinity.

Why it works

Walk along the constraint curve and watch f. At the highest or lowest point of the walk, f is momentarily not changing, so the direction you are walking is perpendicular to ∇f. The walking direction is tangent to the curve, and the gradient of g is perpendicular to the curve (it is perpendicular to every level curve of g). So ∇f and ∇g are both perpendicular to the same tangent line, which in the plane means they are parallel: ∇f = λ∇g for some number λ.

Put another way: at the best point, the level curve of f just touches the constraint curve without crossing it. If it crossed, moving along the constraint would carry you to higher values on one side and lower values on the other, and the point wouldn't be the best.

Example 1: a line on a circle

Find the maximum and minimum of f(x, y) = 3x + 4y on the circle x² + y² = 25.

1. 3 = 2λx, 4 = 2λy, x² + y² = 25∇f = (3, 4) and ∇g = (2x, 2y). λ can't be 0, since 3 ≠ 0, so dividing by λ is safe here.
2. x = 32λ, y = 2λ ⇒ 94λ² + 4λ² = 25Stationarity gives x and y in terms of λ; the constraint then fixes λ.
3. 254λ² = 25 ⇒ λ = ±12Both signs are solutions, and each gives a candidate.
4. (3, 4): f = 25 (max); (−3, −4): f = −25 (min)The circle is closed and bounded, so comparing the two candidates settles it. The maximum is in the direction of (3, 4), where the circle touches the highest level line 3x + 4y = 25.

Example 2: the closest point on a line

Find the point of the line 2x + y = 5 closest to the origin.

1. minimize f = x² + y² subject to 2x + y = 5Minimize the squared distance: it has the same minimizer as the distance, without a square root to differentiate.
2. 2x = 2λ, 2y = λ ⇒ x = λ, y = λ2∇f = (2x, 2y) and ∇g = (2, 1).
3. 2λ + λ2 = 5 ⇒ λ = 2Put x and y into the constraint.
4. (2, 1), f = 5, distance √5This is a minimum because moving along the line in either direction takes you farther from the origin without bound. The same fact means there is no maximum.

Example 3: solving by cases

Find the maximum and minimum of f(x, y) = x²y on the circle x² + y² = 3.

1. 2xy = 2λx, x² = 2λy, x² + y² = 3∇f = (2xy, x²) and ∇g = (2x, 2y).
2. 2x(y − λ) = 0 ⇒ x = 0 or y = λFactor, don't cancel x. Both branches have to be followed.
3. x = 0: y = ±√3, f = 0The second equation becomes 0 = 2λy, so λ = 0. Two candidates.
4. y = λ: x² = 2y² ⇒ 3y² = 3 ⇒ y = ±1, x = ±√2The second equation becomes x² = 2y², and the constraint gives y² = 1. Four candidates.
5. max 2 at (±√2, 1); min −2 at (±√2, −1)f = x²y = 2y on this branch. Compared with the f = 0 candidates, these are the extremes. The points (0, ±√3) aren't wasted: (0, √3) is a local minimum along the circle and (0, −√3) a local maximum.

Example 4: what λ measures

In Example 2, move the line to 2x + y = c. How does the minimum change?

1. x = 2c5, y = c5, f*(c) = c²5The same steps with c in place of 5: λ = 2c/5.
2. df*dc = 2c5 = 2 = λ at c = 5The multiplier is the rate at which the optimal value changes as the constraint is relaxed: moving the line to c = 5.1 raises the minimum by about 2 × 0.1 = 0.2. In economics λ is called a shadow price.

Common mistakes, and the exact line they happen on

1. Canceling a variable that can be zero

Find the extremes of f(x, y) = x² + 3y on the unit circle x² + y² = 1.

1. 2x = 2λx, 3 = 2λy, x² + y² = 1Right so far.
2. λ = 1 ⇒ y = 32 ⇒ x² = −54, no solutionThis is the mistake. Canceling x from 2x = 2λx assumes x ≠ 0, and here that branch is the only one with real points. With x = 0, y = ±1, and f = 3 at (0, 1) is the maximum, f = −3 at (0, −1) the minimum. The circle is closed and bounded, so a maximum and minimum must exist: a method that finds no candidates has thrown some away.

2. Setting ∇f = 0 and forgetting the constraint

The problem from Example 2.

1. minimize x² + y² subject to 2x + y = 5Right so far.
2. ∇f = (2x, 2y) = 0 ⇒ (0, 0), minimum 0This is the mistake. (0, 0) is the unconstrained minimum, and it isn't on the line: 2 · 0 + 0 = 0, not 5. Any answer has to satisfy g = c. The constrained minimum is 5, at (2, 1).

3. Calling the only candidate a maximum

Maximize x² + y² on the line 2x + y = 5.

1. The only candidate is (2, 1), with f = 5Right so far: the equations are those of Example 2.
2. So the maximum is 5This is the mistake. Lagrange's equations find points where f is flat along the constraint, and they don't say which kind. The line is unbounded, and (10, −15) is on it with f = 325. There is no maximum, and (2, 1) is the minimum.

Practice

1. Find the maximum and minimum of x + y on x² + y² = 18.

   Show answer

   1 = 2λx and 1 = 2λy give x = y, so 2x² = 18 and x = ±3. Maximum 6 at (3, 3), minimum −6 at (−3, −3).

2. Minimize x² + y² + z² subject to x + y + z = 6.

   Show answer

   2x = 2y = 2z = λ, so x = y = z = 2, λ = 4, and the minimum is 12.

3. Find the maximum and minimum of xy on the ellipse x² + 4y² = 8.

   Show answer

   y = 2λx and x = 8λy give x = 16λ²x. x = 0 forces y = 0, which is not on the ellipse, so λ = ±¼ and y = ±x/2. Then 2x² = 8: maximum 2 at (2, 1) and (−2, −1), minimum −2 at (2, −1) and (−2, 1).

4. Minimize x² + 2y² subject to x + y = 3.

   Show answer

   2x = λ and 4y = λ give x = 2y, so 3y = 3: the point (2, 1), with λ = 4 and minimum 6. The line is unbounded, but f grows without bound along it in both directions, so this candidate is the minimum.

5. Use λ from Problem 2 to estimate the minimum when the constraint is x + y + z = 6.1, and compare with the exact value.

   Show answer

   12 + 4 × 0.1 = 12.4. Exactly, f* = c²/3 = 6.1²/3 ≈ 12.4033.