The setup
Lagrange multipliers handle optimization with equality constraints. We want to minimize an objective while a constraint must hold exactly.
- Each constraint gets its own multiplier variable.
- We form a combined function called the Lagrangian.
The key condition
At a constrained optimum the gradient of the objective is parallel to the gradient of the constraint. The multiplier is exactly the scaling factor between them.
- Geometrically, the objective stops decreasing along the constraint surface.
- Setting the Lagrangian's gradient to zero recovers both the optimum and the multiplier.
Why it helps
It converts a constrained problem into an unconstrained one in more variables, which we can attack with standard calculus. The multiplier also has meaning: it measures how much the optimum would change if the constraint loosened.
This idea generalizes to inequality constraints through the KKT conditions.
Key idea
Lagrange multipliers fold equality constraints into a single Lagrangian whose stationary points align the objective and constraint gradients, turning a constrained problem into an unconstrained one.