Curves from a linear model
Polynomial regression models a curved relationship by expanding one feature x into x, x squared, x cubed, and so on, then fitting ordinary least squares. The model is still linear in the weights, so the same fitting math applies.
Why it stays linear
The word linear in linear regression refers to the parameters, not the features. Because each power becomes its own column, the solver treats them as ordinary features.
The degree tradeoff
- A low degree underfits and misses real curvature.
- A high degree overfits, wiggling to chase noise and exploding outside the training range.
- Choose the degree with cross validation rather than by eye.
Practical cautions
- Always standardize the feature before raising powers, since x cubed can dwarf x in scale.
- Polynomials extrapolate badly, so predictions far outside the training range are unreliable.
- Regularization can tame a high degree fit by shrinking large coefficients.
Key idea
Polynomial regression adds powers of a feature so a straight line solver can fit curves. It stays linear in the weights, but high degrees overfit and extrapolate poorly, so pick the degree by cross validation.