Closing the loop
A feedback loop captures what actually happened after a prediction so it can be compared to what the model said. That outcome feeds both monitoring and the next round of training. Without it the model flies blind.
Sources of feedback
- Explicit, users rate, correct, or confirm a result.
- Implicit, behavior like clicks, purchases, or dwell time hints at quality.
- Delayed truth, a fraud chargeback or a loan default confirms a label later.
Watch for bias
A loop can poison itself. If the model only shows certain items, you only learn outcomes for those, and the data narrows over time. This feedback bias can spiral, so log enough exploration or counterfactual data to keep coverage broad.
Practical needs
- A stable id to join outcomes back to the original prediction.
- A label store that records timing and source of each outcome.
Key idea
Feedback loops capture real outcomes joined to predictions to power monitoring and retraining, but must guard against self reinforcing bias by preserving exploration.