Why Models Invent Things

The word "hallucination" suggests a malfunction. It is more useful to understand fabrication as the normal operation of the system, visible in the cases where plausible and true come apart.

Why it happens

A model generates the most plausible continuation. Where the training data contained a clear, consistent answer, plausible and true coincide. Where it did not, the model still generates something — it has no option to produce nothing.

Asked for a citation, it generates text shaped like a citation. Plausible author, plausible journal, plausible year, plausible title. Every component is drawn from patterns in real citations. The combination may not exist.

Asked for a specific figure, it generates a number in the right range and format. Right shape, wrong value.

Asked about something obscure, it generalises from similar things. A small company gets attributes typical of companies like it.

Asked a question with a false premise, it answers the question. "What year did X publish Y" produces a year, whether or not X published Y.

The failure is most likely exactly where verification is hardest: specific, obscure, checkable facts.

Where it concentrates

Citations and references. The worst category. Academic references, case law, standards, page numbers, DOIs.

Numbers. Statistics, prices, dates, measurements, version numbers.

Names. People, small organisations, product variants.

Quotations. Attributed statements are a reliable source of invention.

Anything after training. Recent events, current versions, present role holders.

Detail beyond the summary level. The general description is often right; the specifics fail.

What reduces it

Retrieval. A tool that searches and works from real documents fabricates far less. This is the single biggest improvement available, and it is why assistants with search are more reliable on facts.

It does not eliminate the problem. Models still misread sources, conflate two documents, and attribute a claim to the wrong one. Check that the cited source says what the answer claims. Frequently it does not.

Providing the material. Paste the document and ask questions about it. This converts recall into reading comprehension, which is what the technology is genuinely good at.

Asking for uncertainty explicitly. "Tell me what you are confident about and what you are not" produces more useful hedging than the default, which is confident by design.

Asking the same thing twice, separately. Inconsistent answers indicate the model is generating rather than recalling. Consistency is weak evidence of correctness and inconsistency is strong evidence of fabrication.

Requesting checkable form. Steps, sources, reasoning you can follow, rather than a conclusion.

Narrowing the question. Broad questions invite fluent summary. Specific questions about material you supplied are answerable.

What does not reduce it

Telling it not to make things up. The model does not know it is doing so. Instructions of this kind change tone, not accuracy.

Asking whether it is sure. It generates a plausible response to that question too. Repeated challenging often produces retraction of correct answers as readily as incorrect ones.

Using a larger or newer model. Rates fall with capability; they do not reach zero, and stronger models fabricate more convincingly.

Higher effort or reasoning modes. These help on multi-step problems. They do not supply facts that were never learned.

A working verification habit

For anything you will act on or publish:

Every number, name, date and citation gets checked against a source you found yourself.

Every quotation gets located in the original.

Any claim that would be embarrassing if wrong gets a second source.

Anything after the model's training cutoff is assumed unreliable unless retrieved.

This sounds laborious. In practice it takes a fraction of the time the drafting saved, and it is the difference between a tool that speeds you up and one that eventually humiliates you.

The trap in the middle

The dangerous zone is not obvious nonsense, which people catch. It is output that is 90 percent correct with a plausible error embedded.

A summary of a report where one figure is wrong. A list of nine real tools and one that does not exist. A legal explanation that is broadly right with an inverted exception.

This is why spot-checking fails. Checking three items out of twenty and finding them correct tells you very little, because the errors are sparse by nature. Either check everything that matters, or treat the whole output as unverified.

The domains where this costs most

Fabrication rates are not uniform. Certain fields combine high stakes with output that looks authoritative.

Law. Case citations are formulaic, which makes them easy to generate and hard to distinguish. There are now documented instances of filed court documents containing invented cases, with professional consequences for those who filed them.

Medicine. Dosages, interactions, guideline numbers. Plausible and wrong is the worst possible failure here.

Academic writing. Fabricated references are the most common way AI-assisted work is detected, because checking them is trivial for a marker.

Finance. Figures, filing references, regulatory requirements.

Technical documentation. Functions that do not exist, parameters that were never available, configuration options invented by analogy with real ones. Developers encounter this constantly, and the compiler catches it, which is why the field tolerates it.

The pattern across all five: the output format is highly regular, which makes generation easy, and the content is specific, which makes errors consequential. Treat regular-format specific content as the highest-risk category.