Many fairnesses
There is no one definition of fairness. Instead there is a family of mathematical criteria, each capturing a different intuition about what equal treatment means. Choosing among them is a values decision, not a purely technical one.
Three broad families
- Independence: predictions should be independent of the protected attribute. Demographic parity lives here.
- Separation: errors should be balanced across groups, conditioned on the true outcome. Equalized odds and equal opportunity live here.
- Sufficiency: among people who get the same score, the true outcome rate should be equal across groups. Calibration lives here.
The hard truth
Famous impossibility results show that, except in trivial cases, you cannot satisfy independence, separation, and sufficiency all at once when base rates differ across groups. Picking one means giving up another.
How to choose
- Ask what harm you most want to prevent.
- Consider who bears the cost of each error type.
- Document the chosen definition and its tradeoffs openly.
Key idea
Fairness is a family of conflicting criteria grouped into independence, separation, and sufficiency, and impossibility results mean you must consciously choose which one matters most.