When AI Hallucinates, what does it mean?

What “hallucination” actually means Researchers usually split it into:

• Intrinsic: the output contradicts the prompt, the retrieved documents, or earlier turns in the same conversation.
• Extrinsic: the output invents facts, citations, holdings, numbers, package names, or events that are not in the source and not true in the world.

Related failure modes that get bundled with hallucination:

• Confabulation: filling gaps with plausible-sounding invention.
• Sycophancy: agreeing with a user’s false premise.
• Faithfulness failure: summarizing or citing a source incorrectly.
• Agentic hallucination: a tool call or multi-step plan goes wrong, then later steps treat that error as fact.

A useful 2026 finding from Google Research: on many frontier models, the facts are encoded. The bottleneck is recall—the model cannot reliably retrieve what it already “knows.” Gemini-3-Pro and GPT-5 encoded 95–98% of tested facts but still failed to recall 26–34% of them directly; even with “thinking,” about 11–12% still failed. Scaling stores more knowledge; it does not automatically make that knowledge usable on demand.

Why it happens - Hallucination is a product of how these systems are trained and scored, not a missing patch.

1. Next-token prediction, not a truth engine. The model samples likely continuations. Fluency is optimized more directly than correspondence with reality.
2. Training data is uneven. Rare facts, new events, and niche law/medicine are thin. Obscure false claims are easier to accept under repetition.
3. Incentive to guess. Benchmarks that reward accuracy without penalizing wrong answers train models to answer instead of saying “I don’t know.” Nature work in 2026 argued that accuracy-only evals incentivize hallucination. Models that abstain look worse on leaderboards even when they are safer.
4. Recall vs. encoding. Knowledge can sit in weights and still not surface in the right prompt format.
5. Fine-tuning can make it worse. In finance, domain fine-tuning collapsed numerical restraint: a base model ~5.4% numerical hallucination vs. 82–98% after some fine-tunes, often by injecting memorized “canonical” numbers that ignore the input.
6. Conversation pressure. Under repeated or argumentative false claims, models can flip between accept and reject (“conversational reverberation”). Affirmation of planted falsehoods ranged from ~0.08% to 12.3% across architectures in one 2026 multi-turn study. Correctability also splits: some models fix every self-error; the most accurate model in that set fixed none of its rare errors.
7. Scale and batching. Gemini 3.0 Pro as a document auditor recovered planted contaminants at 50–60% on single docs/small batches and 2.8% on large batches—and then fabricated extra contaminants that were never in the text. The failure mode at scale was confident invention, not honest abstention.
8. Presumption in the prompt. If the user treats a fake entity as real (“What does X mean?”), models invent a definition instead of rejecting X. PhantomBench reports very high rates when existence is assumed.

Where the damage shows up

Agents make this worse. A bad retrieval, a bad tool result, or a bad intermediate “holding” gets treated as ground truth two steps later.