The bias in click logs
Users click top items partly because they appear high, not only because they are best. This position bias poisons training data: a great item shown at position ten may get fewer clicks than a mediocre one at position one, teaching the model the wrong lesson.
The examination assumption
A common model says a click happens only if the user examines a position and the item is relevant.
- Examination probability depends mainly on position and falls fast with rank.
- Relevance depends on the user and item.
- The observed click probability is the product of the two.
Correcting it
- Inverse propensity weighting divides each example's loss by its examination probability, so deep position clicks count more.
- The examination propensities come from result randomization experiments or are learned jointly.
A practical architecture
- Add a small position tower during training that absorbs the position effect.
- At serving time drop that tower so ranking reflects relevance alone.
Why care
- Without correction the model just relearns the old ranking and cannot improve.
Key idea
Position bias makes high ranked items look better than they are, so recommenders separate examination from relevance and use inverse propensity weighting or a position tower to learn true relevance.