What it is
Linear regression predicts a continuous target as a weighted sum of input features plus a bias. The model is simply a line in one dimension, or a flat plane in many dimensions.
The objective
We choose weights that minimize the mean squared error, the average of the squared gaps between predictions and true values.
- Squaring punishes large mistakes far more than small ones.
- The error surface is a smooth bowl with a single lowest point.
- Because the bowl is convex, there is exactly one best answer.
How we fit it
Two common routes reach the same place.
- The normal equation solves for the best weights in one closed form step using matrix algebra.
- Gradient descent walks downhill on the error surface, useful when there are too many features for matrix inversion.
What to watch
- Linear regression assumes a roughly linear relationship. Curved patterns need transformed features.
- It is sensitive to outliers because squared error magnifies them.
- Highly correlated features make the weights unstable.
Key idea
Linear regression fits a line by minimizing squared error, a convex problem with one unique best solution.