Early Stopping

Early Stopping

Early stopping is a simple and powerful technique to combat overfitting. Instead of training for a fixed number of epochs, you watch the validation performance and stop when it stops improving.

The motivation comes from the validation curve. Validation error typically falls, reaches a low point, then begins to rise as the model overfits. The best model lives at that low point. Early stopping aims to halt training right there.

In practice it works like this:

• After each epoch, measure error on a held out validation set
• Track the best validation score seen so far and save those parameters
• If validation does not improve for a set number of epochs, stop training

That waiting window is called patience. A patience of, say, five epochs means you tolerate five epochs without improvement before stopping, which avoids quitting over a brief random dip. When training ends, you restore the best saved parameters, not the final ones.

Early stopping is appealing because it requires no change to the model itself and acts as a form of regularization, limiting how far the model can drift into memorizing noise. It also saves compute by not training longer than helpful. Its main cost is the need for a trustworthy validation set.

Key idea

Early stopping halts training when validation error stops improving, restoring the best parameters to prevent overfitting with no model changes.