When the model makes things up
A hallucination is a confident statement the model presents as fact that is actually false or unsupported. Because models predict plausible text rather than verify truth, they can fabricate citations, numbers, or details that sound right but are not.
Why it happens
- The model optimizes for plausible continuation, not factual accuracy.
- It has no built in source to check claims against.
- Pressure to always answer discourages saying it does not know.
Grounding strategies
- Provide sources through retrieval and instruct the model to use only them.
- Ask for citations so claims can be traced to supplied text.
- Allow abstention, telling the model to say when the answer is not in the context.
- Verify critical outputs with tools or a second check.
A realistic view
Grounding reduces hallucination but does not eliminate it, since a model can still misread or overreach from provided text. Treat confident tone as no guarantee of truth, and reserve human review for high stakes claims where a mistake is costly.
Key idea
Hallucinations are confident but unsupported claims that arise because models predict plausible text, and grounding with sources, citations, and abstention reduces but never fully removes them.