Forecasting Evaluation Metrics
To judge a forecast you compare predictions to actual values with an error metric. Each metric emphasizes different mistakes.
Common choices
- Mean absolute error averages the size of the errors. It is easy to read and treats all errors equally.
- Root mean squared error squares errors before averaging, so it punishes large misses harder. It shares the units of the series.
- Mean absolute percentage error expresses error as a percent of the actual value, which aids comparison across series of different scales.
Watch the pitfalls
- Percentage error blows up when actual values are near zero.
- Squared metrics are sensitive to a few big outliers.
- Always compare against a naive baseline, such as carrying the last value forward, so you know the model adds value.
Scaled metrics
The mean absolute scaled error divides your error by the error of a naive forecast. A value below one means you beat the baseline, which makes results comparable across very different series.
Key idea
Pick a metric that matches your goal, beware near zero values and outliers, and always benchmark against a naive baseline.