Accuracy is not enough
A ranker that maximizes predicted relevance per item can return a monotonous list, ten near identical items. Good recommendations also need diversity, variety across the list, and serendipity, relevant items the user would not have found alone.
Defining the goals
- Diversity measures how dissimilar items in the list are to each other.
- Serendipity rewards items that are both relevant and unexpected, beyond the obvious.
- Novelty rewards items the user has not seen before.
Promoting diversity
- Maximal marginal relevance picks each next item to balance relevance against similarity to items already chosen.
- Determinantal point processes model a whole set so it covers more ground.
The trade off
- Pushing diversity too hard lowers immediate click rates.
- Pushing it too little causes filter bubbles and boredom that hurt long term engagement.
Most systems tune a knob and judge it on long horizon metrics, not a single click.
Key idea
Beyond accuracy, recommenders tune diversity and serendipity so lists stay varied and surprising, trading a little immediate click rate for healthier long term engagement.