Documenting a model honestly
A model card is a short structured document that accompanies a released model, describing what it is, how it was evaluated, and where it should not be used. It makes safety and limitations legible to users and auditors.
Typical sections
- Intended use and out of scope uses.
- Training data sources at a high level and known gaps.
- Evaluation results, including disaggregated metrics across groups.
- Limitations and risks, such as known failure modes and bias findings.
- Ethical and safety considerations and recommended safeguards.
Why it matters
- It sets honest expectations so deployers do not misuse the model.
- It creates accountability and a record for audits and incident response.
- Disaggregated evaluation in the card exposes uneven performance that an aggregate score would hide.
Pitfalls
- A card can be vague or omit inconvenient findings, becoming marketing.
- It is a snapshot, so it must be updated as the model or its use changes.
Key idea
A model card is structured transparency documentation covering intended use, data, disaggregated evaluation, and limitations, creating accountability provided it is honest and kept up to date.