Measuring regression error
For numeric predictions we measure how far each guess lands from the truth. Two classic choices are the mean absolute error and the mean squared error.
The two metrics
- Mean absolute error averages the absolute size of each error. An error of ten contributes ten.
- Mean squared error averages the squared errors. An error of ten contributes one hundred.
- Root mean squared error takes the square root of MSE to return to the original units.
How outliers change things
Because MSE squares the errors, large mistakes dominate the total.
- MSE is sensitive to outliers and pushes the model to avoid big misses.
- MAE treats all errors proportionally, so it is robust to outliers.
- If a few extreme errors should be punished hard, prefer MSE. If outliers are noise to be ignored, prefer MAE.
Key idea
MSE squares errors so big misses dominate and outliers matter, while MAE weights every error in proportion and stays robust. Choose based on how much you want extreme errors to count.