Beyond binary relevance
Real relevance is graded. A result can be perfect, useful, or irrelevant. NDCG, normalized discounted cumulative gain, handles graded relevance and position together.
Building it up
- Gain is the relevance grade of each item, often higher for stronger relevance
- Discounted means a position discount divides each gain by a function of its rank, usually the logarithm, so lower positions count less
- Cumulative sums these discounted gains down the list to get DCG
- Normalized divides DCG by the ideal DCG, the score of the best possible ordering
The result lands between 0 and 1, where 1 is the perfect ranking. Normalization makes scores comparable across queries with different numbers of relevant items.
Why the log discount
Users scan from the top and rarely reach far down. The logarithmic discount models diminishing attention, so an excellent item at rank 2 helps far more than at rank 20.
Trade offs
NDCG is the standard for web search and recommendation because it captures both grade and order. The cost is that it needs graded labels and the choice of gain and discount functions affects the number.
Key idea
NDCG combines graded relevance with a position discount, then normalizes by the ideal order. It is the go to metric when both how relevant and how high an item is matter.