A real tension
An unconstrained model is trained to maximize accuracy on its data. Adding a fairness constraint narrows the space of allowed models, so the best fair model can be no more accurate, and usually slightly less, than the best unconstrained one. This is the fairness accuracy tradeoff.
Why it happens
- When group base rates differ, an accuracy maximizing model exploits that difference. Removing the difference removes some predictive signal.
- Some of that signal may be legitimate, and some may be a proxy for the protected attribute. Constraints cannot always tell them apart.
It is not always large
The tradeoff is often small in practice. A model can frequently gain substantial fairness for a tiny accuracy drop, because much of the unfairness came from spurious rather than predictive signal.
How to reason about it
- Plot accuracy against a fairness metric to see the frontier.
- Decide how much accuracy you will spend for how much fairness, openly.
- Remember that measured accuracy on biased data is not the same as real world value.
Key idea
Constraining a model for fairness usually costs some accuracy because it removes exploitable signal, but the cost is often small and the frontier between the two should be examined and chosen deliberately.