Polynomial and Interaction Features
Linear models fit straight relationships, but reality is often curved or combined. Polynomial and interaction features let a linear model express nonlinearity by feeding it richer inputs.
Two kinds of derived features
- Polynomial features add powers of a feature, such as its square or cube, letting the model bend a straight line into a curve.
- Interaction features multiply two features together, capturing effects that appear only when both are present.
A house price might depend on area squared and on the product of bedrooms and bathrooms, neither visible to a plain linear fit.
Costs and controls
- The number of terms explodes combinatorially as degree and feature count rise.
- High degree polynomials overfit wildly, chasing noise.
- New terms vary in scale, so scaling and regularization become important.
Use a modest degree, prefer interactions guided by domain knowledge, and lean on regularization to tame the expanded space.
Key idea
Polynomial and interaction features inject powers and products so linear models capture curves and joint effects, but the term count explodes and demands regularization.