The signature
A model overfits when it fits training data well but generalizes poorly. The tell is a large gap between low training error and high validation error.
- Training metric keeps improving while validation stalls or worsens.
- The model captures noise specific to the training set.
- It is too flexible for the amount of data available.
Confirming and fixing
Plot both curves against training progress. A widening gap confirms overfitting. The cure is to reduce variance.
- Add regularization such as weight decay or dropout.
- Gather more data or augment what you have.
- Reduce model capacity or stop training earlier.
Curves at a glance
The gap, not the absolute score, signals overfitting.
Key idea
Overfitting shows as a wide gap between low training error and high validation error, meaning the model memorized noise; reduce variance with regularization, more data, or less capacity.