What prediction drift is
Prediction drift is a change in the distribution of the model's outputs over time. If a fraud model that flagged two percent of transactions suddenly flags fifteen percent, the output distribution has shifted even before any labels confirm whether it is right.
Why monitor outputs
- Output drift is a fast, label free signal that something changed downstream of the inputs.
- It can catch failures that input monitoring misses, such as a broken feature defaulting to a constant.
- It directly reflects what downstream systems and users experience.
Combining the signals
Input drift and prediction drift together localize problems. Inputs stable but predictions shifted suggests a serving or feature bug. Inputs shifted and predictions shifted suggests genuine environment change that may call for retraining.
The ground truth gap
Prediction drift does not tell you the model is wrong, only that its behavior changed. Confirming harm still needs delayed labels and accuracy tracking, so prediction drift serves as the leading indicator that triggers deeper checks.
Key idea
Prediction drift monitors the model's output distribution for shifts, giving a fast label free leading indicator that, combined with input drift, localizes the cause.