CORE GUIDE
Hallucination as unsupported generation
A hallucination is generated content that sounds usable but lacks adequate support from the system's evidence or constraints; the engineering fix is evidence and verification, not asking the model to feel more certain.
Mental model
Treat hallucination as an evidence-boundary failure: the model can continue a plausible sequence even when the application has not supplied enough authoritative information to justify the claim.
Why it matters
Calling hallucination a personality flaw encourages weak mitigations such as 'be careful' prompts. Production systems need to know which claims require evidence, where that evidence should come from, what uncertainty looks like, and which outputs can be verified automatically or escalated. A fluent answer is not a provenance record.
01
Bind consequential claims to observable evidence
Define the claims the product is allowed to make and the sources or tools that can support them. Retrieve authoritative evidence, preserve citations or trace IDs through generation, and verify outputs against deterministic constraints where possible. When evidence is absent or conflicting, the runtime should abstain, qualify or escalate instead of rewarding completion at any cost.
02
Example: invented account status
A support assistant is asked whether a refund already settled. The language model knows common refund workflows but has no current transaction state. A hallucination-prone design lets it infer 'completed' from the conversation. A bounded design calls the payment system, cites the observed status, and returns 'cannot confirm' if the tool result is unavailable.
Common failure modes
- Treating a confident tone as evidence that a claim is supported.
- Adding stronger wording to the prompt without improving source or tool access.
- Measuring only average answer quality while ignoring unsupported high-impact claims.
Engineering heuristics
- Define evidence requirements by claim type and consequence.
- Preserve provenance from retrieval/tool result through the final answer.
- Evaluate abstention and unsupported-claim rates, not only helpfulness.
Takeaways
- 01Fluency is not evidence.
- 02Unsupported claims are a system-boundary problem.
- 03Retrieval, tools, abstention and verification are stronger controls than confidence language.
Reading evidence
This records actions you actually took; it does not claim mastery, proficiency, or certification.
Used in
This Concept is reused across these canonical learning paths.