← Lessons

quiz vs the machine

Gold1440

Machine Learning

Proxy Metric Pitfalls

When the measurable stand in drifts away from the goal you really want.

5 min read · core · beat Gold to climb

Why we use proxies

The true goal is often slow, expensive, or impossible to measure directly. Long term user happiness takes months, so teams optimize a proxy like clicks, watch time, or session length that is fast to measure.

Where proxies break

A proxy is a model of the goal, and like any model it can be wrong.

  • Clicks reward clickbait headlines that disappoint after the click
  • Watch time rewards autoplay loops that users later regret
  • Engagement can reward outrage and addiction rather than value

The proxy and the goal agree in the easy regime, then diverge exactly where optimization pushes hardest.

Defenses

  • Pair the proxy with counter metrics, like satisfaction surveys or long term retention
  • Periodically revalidate that the proxy still correlates with the goal
  • Prefer proxies closer to the true outcome even if noisier

The core risk

Optimization pressure finds and exploits any gap between proxy and goal. The harder you push, the more the proxy can mislead, which is the seed of metric gaming.

Key idea

Proxies stand in for goals that are hard to measure, but optimization pressure widens the gap between them. Guard proxies with counter metrics and revalidate them often.

Check yourself

Answer to earn rating on the learn ladder.

1. Why do teams optimize proxy metrics instead of the true goal?

2. When does a proxy most dangerously diverge from the goal?