What you'll learn
Quick Answer
A hallucination is a fluent, confident answer that is factually wrong or unsupported by the context you gave. Language models generate the most probable next tokens and have no built-in check that the result is true. Recent research argues that common training and evaluation setups reward guessing over admitting uncertainty. You cannot eliminate hallucinations, but grounding answers in retrieved text, allowing 'I don't know', and asking for citations all reduce them.
What a hallucination is
Anthropic's documentation defines it as text that is factually incorrect or inconsistent with the given context. There are two flavours: the answer contradicts a document you supplied, or it contradicts the world.
The classic example is a citation. Ask a model for a reference on a niche topic and you often get a plausible author, a plausible title, a journal name, and a year, all invented. Nothing in that output was looked up in a database of real papers. The model has learned the shape of a citation and produced something that fits the shape.
It helps to stop calling this a bug in the ordinary sense. The model is doing exactly what it was trained to do: continue the text with likely tokens. When the likely continuation happens to be false, that is a hallucination. The process that produces a correct answer and the process that produces confident nonsense are the same process, running on a question where the training data was thin, contradictory, or absent.
Why it happens
At core, the model predicts the next token from patterns in its training data. There is no separate stage that checks the claim against a source of truth.
A 2025 OpenAI paper, Why Language Models Hallucinate (Kalai and colleagues), sharpens this. It frames hallucinations as statistical errors that come partly from how models are scored. Most benchmarks award points for a correct answer and zero for both a wrong answer and an honest 'I don't know'. Under that scoring, a confident guess beats an abstention on average, so the training signal rewards guessing. The paper also argues that generation is harder than classification: a model might correctly judge whether a given statement is true, yet still produce false statements when asked to write an answer from scratch.
Layer the ordinary causes on top: knowledge cutoffs that leave recent events invisible, sparse training data on specialist topics, and ambiguous prompts where the model silently picks one interpretation and commits to it as if it were the only one.
Grounding in retrieved text
The strongest single lever is to stop asking the model to recall and start asking it to read. Retrieve the relevant source material, put it in the prompt, and instruct the model to answer only from it.
Hugging Face's RAG documentation notes that conditioning generation on fetched passages often makes answers more factual. Anthropic lists the matching prompt technique under external knowledge restriction: explicitly tell the model to use only the information in the provided documents and not its general knowledge.
The gotcha is that grounding only helps when retrieval actually found the answer. If the passages you retrieved do not contain it, a model told to 'use only the context' can still fabricate, unless you also tell it what to do when the answer is absent. Always pair the restriction with an explicit escape hatch: if the documents do not contain the answer, say that they do not and stop. Without that sentence, 'answer from the context' quietly becomes 'answer anyway'.
Prompt habits that help
Anthropic's guide on reducing hallucinations lists several concrete techniques worth building into your prompts:
- Allow 'I don't know'. Explicitly give the model permission to admit uncertainty. Anthropic calls this a simple technique that can drastically reduce false information.
- Quote first, then answer. For long documents, ask the model to pull word-for-word quotes relevant to the question before it does anything else, then reason only from those quotes.
- Verify with citations. After a draft, have the model find a supporting quote for every claim and retract any claim it cannot back up.
- Chain-of-thought. Ask it to reason step by step before the final answer, which tends to expose a shaky assumption before it becomes a confident conclusion.
Before and after: Summarise this contract's termination clause becomes Quote the exact termination clause, then summarise it; if there is no such clause, say so. The second version has nowhere to invent.
Verification and limits
Two more checks from the same guide. Best-of-N: run the same prompt several times and compare the outputs; disagreement between runs is a strong signal that the answer is not reliable. Iterative refinement: feed the answer back into a fresh prompt and ask the model to verify or expand each statement, which catches inconsistencies the first pass introduced.
Lowering the sampling temperature reduces randomness and repetition, but it does not add a truth check, so treat it as a minor help rather than a fix. A low-temperature model states a wrong answer just as fluently, only more predictably.
Anthropic is explicit that these techniques significantly reduce hallucinations but do not eliminate them, and that critical information should always be validated independently. For anything high-stakes, medical, legal, financial, or code that moves money, a human review or a deterministic check has to sit between the model's output and any real-world action. The model is a drafting tool, not the final authority.