AI Hallucination

What Is AI Hallucination?

An AI hallucination is confident output from a model that is factually wrong or entirely invented: a function that does not exist in the library, a parameter the API never accepted, a package nobody published, a citation to a paper never written. The output is fluent and plausible, which is exactly what makes it dangerous. The problem is measurable at scale: a 2024 Stanford study found LLMs hallucinated on 58 percent (ChatGPT) to 88 percent (Llama 2) of verifiable questions about random federal court cases [1].

Key Takeaways

• In software work, hallucinations concentrate on interfaces: invented methods, wrong argument names, deprecated signatures presented as current, and imports of packages that do not exist.
• Hallucination is a structural property of how language models generate text, predicting plausible continuations, so it can be reduced sharply but not engineered to zero.
• The severity ladder runs from annoying (code that fails to compile) to dangerous (code that runs but does the wrong thing) to exploitable (installing a hallucinated package name an attacker registered, the attack known as slopsquatting).
• Defenses are systemic: grounding in real context, tool access so the model can check instead of recall, compilers and tests as verification, and review by someone who can spot plausible-but-wrong.
• Fluency and accuracy are uncorrelated. The confident tone of a hallucinated answer carries no information about its truth.

How It Works

A language model generates text by predicting what plausibly comes next given its training data and the current context, and it has no built-in mechanism that distinguishes remembering from inventing. When you ask for a function from a library it saw thousands of times, prediction and fact usually coincide. When you ask about a niche library, a recent version, or your private codebase, the model interpolates: it produces the method that would exist if the library were designed the way similar libraries are. That is why hallucinated code is so convincing. client.messages.list(page_size=50) looks exactly like real SDK code, whether or not the SDK ever shipped that method.

Code hallucinations cluster in predictable places. Version boundaries, where the model's training data mixes several API generations and it confidently serves you the 2023 signature. Cross-language contamination, where a Python idiom bleeds into the JavaScript SDK. Plausible completions of your own codebase, inventing a utils.formatCurrency helper because most codebases like yours have one. And dependency suggestions, where a model recommends installing a package whose name it composed from fragments of real ones, the raw material for supply-chain attacks. A 2024 study of 576,000 generated code samples found at least 5.2 percent of package recommendations from commercial LLMs, and 21.7 percent from open-source models, pointed to hallucinated packages that do not exist [2]. The same researchers catalogued 205,474 unique hallucinated package names across 16 popular code-generation models, a ready-made attack surface for anyone willing to register them [3].

Mitigation stacks in layers. Grounding puts real documentation, type definitions, and source files into the context window so the model reads rather than recalls; retrieval-augmented generation automates this. Agentic tooling helps further, since an AI coding agent that can grep the codebase and run the compiler catches its own inventions in the loop. Then verification: type checkers, tests, and dependency allowlists convert silent hallucinations into loud failures. Human review handles the residue, the code that compiles and runs and is still wrong.

Example

An engineer asks a coding agent to add retry logic to a service using a popular HTTP client library. The agent writes clean code calling client.request(retries=3, backoff="exponential"). The parameters look idiomatic, the code reads well in review, and neither parameter has ever existed in that library; retries live in a separate adapter class. In a plain chat workflow, that snippet gets pasted, fails at runtime in a code path only exercised under network failure, and ships broken retry behavior that nobody notices until an outage. In an agentic workflow with guardrails, the type checker flags the unknown parameters immediately, the agent reads the library's actual source, finds the adapter pattern, and rewrites. Same hallucination in both cases; the difference is whether the system converts it into a visible failure within seconds or a production incident weeks later.

What People Get Wrong

The persistent misconception is that hallucination is a bug that better models will fix, so teams postpone building defenses and wait for the next release. Model generations have cut hallucination rates substantially, and the failures that remain are more dangerous, not less, because higher overall accuracy earns more trust and less scrutiny. A tool that is wrong a third of the time gets checked; a tool that is wrong two percent of the time gets believed, and that two percent lands in production. Treat hallucination like network failure: a permanent property of the medium you architect around, with grounding, verification, and review, rather than a defect you wait out.

FAQ

Why do LLM hallucinations happen even when the model has seen the correct information? Because the model stores statistical patterns, not a lookup table. Correct and incorrect continuations compete at generation time, and context, phrasing, or ambiguity can tip the balance toward a plausible invention even when the truth was in the training data. This is also why the same question phrased two ways can produce one right and one wrong answer.

Are AI hallucinations getting worse or better? Better by rate, in code especially, where 2026-era models with tool access hallucinate APIs far less than their predecessors because they can check documentation and run code instead of recalling. The risk profile has shifted rather than vanished: fewer failures, each individually more trusted, plus new surfaces like fabricated tool outputs in agent pipelines.

Is a hallucination the same as AI slop? No. A hallucination is a specific false assertion, one invented function, one fake citation. AI slop is a volume-and-process problem: low-quality generated content shipped without review. Slop frequently contains hallucinations, but a carefully reviewed system can still emit an occasional hallucination, and slop can be technically accurate while being worthless.

Sources

1. Dahl et al., Stanford. "Large Legal Fictions: LLMs hallucinated on 58% (ChatGPT) to 88% (Llama 2) of verifiable questions about random federal court cases." https://arxiv.org/abs/2401.01301. Accessed August 2026.
2. Spracklen et al., USENIX Security. "Study of 576,000 code samples: at least 5.2% of commercial-LLM and 21.7% of open-source-model package recommendations were hallucinated." https://arxiv.org/abs/2406.10279. Accessed August 2026.
3. Spracklen et al., USENIX Security. "205,474 unique hallucinated package names catalogued across 16 code-generation LLMs." https://arxiv.org/abs/2406.10279. Accessed August 2026.