Hallucination.
When an AI model generates text that is factually incorrect, fabricated, or unsupported by its context, stated with confident fluency as if it were true.
Hallucination is the most significant reliability problem facing language models. Because LLMs generate text by predicting likely next tokens rather than retrieving facts from a database, they can produce plausible-sounding text that is entirely fabricated. Hallucinated content often includes fake citations, invented statistics, nonexistent people, and false historical claims.
The causes of hallucination are multi-fold: the model may have seen conflicting information during training, may be extrapolating beyond what it reliably knows, or may be optimizing for fluency over accuracy. Models trained heavily on human preference (RLHF) sometimes learn that confident-sounding answers receive higher ratings even when the model is uncertain.
Mitigation strategies include using Retrieval-Augmented Generation to ground responses in verified sources, prompting models to express uncertainty when they lack confidence, and using fact-checking pipelines that verify key claims. Anthropic's Constitutional AI and techniques like RLHF have reduced (but not eliminated) hallucination rates in frontier models.