Skip to content

What Is an AI Hallucination? Why ChatGPT Makes Things Up and How to Prevent It

An AI hallucination is when a model confidently states something false or invented. Why language models hallucinate, real examples with real consequences, how to spot them, and the techniques that reduce them.

5 min read

Why AI Hallucinates — cover image from TensorCore.si

Short answer

An AI hallucination is output from an AI model that sounds confident and plausible but is false, invented or not supported by its sources, such as a fake citation, a wrong statistic or a non-existent product feature. It happens because language models generate the most likely next words rather than looking facts up, and because training and testing have often rewarded guessing over saying "I don't know". Hallucinations can be reduced with retrieval (RAG), citations, better prompts and verification, but not eliminated completely.

What a hallucination looks like

Common forms include:

  • Invented sources: citations to papers, court cases or URLs that do not exist.
  • Wrong facts: incorrect dates, numbers, specifications or quotes.
  • Fabricated details: a plausible biography of a person the model knows little about.
  • Unfaithful summaries: a summary that adds claims not in the original document.
  • Made-up code: calls to functions or libraries that do not exist.
  • Image errors: extra fingers or impossible text in generated images are the visual equivalent.

The danger is that hallucinations look exactly like correct answers. The model uses the same confident tone either way.

Why language models hallucinate

They predict text, they do not look it up

A large language model is trained to predict the next token based on patterns in its training data. When it has seen a fact many times, the most likely continuation is usually correct. When a fact is rare, ambiguous or absent, the model still produces the most plausible-sounding continuation, which may be wrong. There is no built-in database it checks against.

Training and testing reward guessing

In 2025, OpenAI researchers argued that hallucinations persist partly because of how models are trained and evaluated. Most benchmarks score answers simply as right or wrong, so a model that guesses gets some points while a model that says "I don't know" gets none. Like a student guessing on a multiple-choice test, models learn that a confident guess scores better than admitting uncertainty. Their proposed fix is to penalise confident errors more than abstentions.

Other causes

  • Knowledge cut-off: the model knows nothing after its training data ends unless connected to search.
  • Compression: billions of facts are squeezed into a fixed number of parameters, so rare details blur.
  • Misleading prompts: a question that assumes something false ("Why did Einstein win two Nobel Prizes?") invites the model to play along.
  • Sampling randomness: higher temperature makes less likely, and sometimes wrong, tokens more probable. See our LLM sampling simulator.
  • Long or noisy context: important information buried in a huge prompt can be missed (see context window).

Real-world consequences

  • Fake legal citations: in 2023 a US federal judge sanctioned two lawyers and their firm $5,000 after they filed a brief citing cases invented by ChatGPT (Mata v. Avianca). Courts in several countries have since seen similar incidents.
  • Customer service: in 2024 a Canadian tribunal ordered Air Canada to honour a refund policy its website chatbot had invented, rejecting the airline's argument that the chatbot was responsible for its own statements.
  • Search answers: AI-generated search summaries have repeated jokes and satire as fact.

The lesson is consistent: organisations are responsible for what their AI tells people.

How to reduce hallucinations

For everyday users

  • Ask for sources and check them. Open the links; do not trust a citation you have not seen.
  • Use search-enabled or document-grounded modes for facts, prices, news and anything recent.
  • Give the model the material. Paste the document and ask it to answer only from that text.
  • Allow uncertainty. Add "If you are not sure, say so" to your prompt.
  • Ask narrow questions instead of broad ones that invite padding.
  • Verify numbers, names, dates and quotes before you publish or act on them.

For developers

  • Retrieval-augmented generation: RAG fetches relevant passages, often with embeddings, and the model answers from them with citations.
  • Lower temperature for factual tasks.
  • Structured output and validation: check generated JSON, code and calculations automatically.
  • Tools: let the model call a calculator, code interpreter, database or search API instead of guessing.
  • Self-checking: have a second pass verify each claim against the sources.
  • Fine-tuning for refusal: training the model to abstain when unsure (see fine-tuning).
  • Evaluation: measure hallucination rates on your own test questions before deploying.

Do bigger or reasoning models hallucinate less?

Generally, newer and larger models hallucinate less on common knowledge, and models that can search the web make fewer factual errors on current events. But no model is hallucination-free. Some reasoning models have scored worse on certain factual benchmarks than their predecessors, because longer chains of reasoning create more chances to introduce an unsupported claim. Treat every model as a fast, helpful assistant whose work still needs checking.

Is hallucination always bad?

In creative work, the same ability to generate plausible new material is the point: story ideas, brainstorming, fictional worlds and design concepts. The problem is only when invented content is presented, or taken, as fact.

Frequently asked questions

Why is it called a hallucination?

The term borrows from psychology, where a hallucination is a perception of something not present. Some researchers prefer "confabulation", meaning confidently filling gaps with invented details, which describes the behaviour more precisely.

Can AI hallucinations be completely eliminated?

Not with current language model designs. Because they generate text from probabilities, there is always some chance of a plausible but wrong output. Grounding, tools and verification can make them rare in specific applications.

How often do chatbots hallucinate?

It depends heavily on the model, the task and whether it has access to sources. Rates are low for well-known facts and summarising a provided document, and much higher for obscure facts, citations and questions about little-known people.

Does RAG stop hallucinations?

It reduces them significantly by giving the model the right information, but the model can still misread or misquote a source. Good RAG systems show citations so users can check.

Sources and further reading

Think you understand it now? Prove it.

Solve real Tensor Core math in the TensorCore.si Challenge and earn a certificate with your name.

Take the challenge

Leave a Reply

Your email address will not be published. Required fields are marked *