Hallucination

The Puzzle

Suppose every transistor switches normally. The GPU performs the requested arithmetic. Memory returns the stored values. The transformer executes its layers. The runtime selects tokens exactly according to its configured procedure.

Nothing has to “break” electronically for the answer to be false. That is the central HAAW problem of hallucination.

Why Call It a “Hallucination”?

The word was already being used in neural language-generation research before modern general-purpose chatbots became widespread. In neural machine translation, researchers used hallucination for generated output that became badly unfaithful or even detached from the source material. The term later became common for plausible-looking but unsupported or nonfactual output from generative AI.

There is no single perfectly universal definition across all natural-language-generation tasks. “Unfaithful to the source,” “unsupported by the provided material,” and “plausible but nonfactual” capture related failure modes, depending on the task.

The Key: What Was the Machine Asked to Optimize?

A language model's generation machinery does not contain a universal instruction saying:

At generation time, the model computes scores for possible next tokens conditioned on its learned parameters and current context.

Next-token plausibility and factual truth are related in useful models, but they are not identical objectives.

A Wrong Answer Can Be a Perfectly Valid Token Sequence

Imagine a user asks for the birth year of a little-known person. The model may have weak, conflicting, or absent evidence represented in its learned parameters and context.

The selected continuation can be linguistically excellent and factually wrong. The arithmetic that produced it can still be internally valid.

Fluency Is Not a Truth Detector

The first three can reinforce one another without guaranteeing the fourth.

The Generation Loop Can Compound the Error

Once an unsupported token has been generated, it becomes part of the sequence used for subsequent generation.

This is one reason an answer can become confidently detailed after taking a wrong turn. The model is continuing from the text that now exists in its own context.

No Little Fact Checker Sits Between Tokens

Recall our generation loop:

A deployed assistant may have retrieval, search, databases, tools, verification stages or other surrounding systems. But those are additional mechanisms. The basic language-model loop itself does not automatically consult an authoritative world database before every token.

Where Can the Problem Enter?

“I Don't Know” Is Also a Generated Behavior

Humans sometimes imagine that uncertainty should automatically stop generation. Mechanically, however, an answer such as:

is itself a sequence of tokens the system must learn or be instructed to produce under appropriate conditions. Good uncertainty behavior is something models and assistant systems must be trained, prompted, evaluated and engineered to exhibit reliably.

Internal Knowledge and External Truth Are Different Things

The first two processes occur inside the AI system. The third is the reference reality against which a factual claim may need to be checked.

Hallucination Is Not Necessarily Randomness

A hallucinated answer can arise deterministically. If generation uses greedy selection or otherwise deterministic settings, the same input and model state may repeatedly produce the same false statement.

Temperature Did Not Invent the Problem

Sampling settings such as temperature can change which continuation is selected and may affect error rates, but hallucination is not simply “temperature turned up too high.” Even a highest-scoring continuation can be false.

Grounding Changes the Problem

If a system retrieves a trusted source and places relevant evidence into context, the model can condition generation on that evidence.

This can reduce unsupported generation, but it does not mathematically guarantee truth. Retrieval can fetch the wrong material, sources can disagree, and the model can misread or misstate supplied evidence.

Verification Adds Another Loop

That is different from merely generating fluent text. Tool use, retrieval, calculators, code execution and explicit verification can move some questions from plausible continuation toward evidence-supported answer.

Not Every Wrong Answer Is Best Called a Hallucination

The terminology is not perfectly standardized. It is useful to distinguish several possibilities:

In everyday AI discussion these may all get called “hallucinations.” HAAW should be more precise when the distinction matters.

Now Connect It to Interpretability

Suppose we capture the entire inference:

We can establish how the output was mechanically produced. That does not automatically tell us why the model's learned representation favored the false claim rather than the true one.

This is exactly where interpretability and WITNESS become valuable research questions.

What Could WITNESS Ask?

WITNESS should be able to establish recorded events without pretending that recording them automatically reveals their semantic cause.

HAAW Checkpoint

Hallucination

An established but variably defined term for generated content that is unsupported, unfaithful to its source/context, or nonfactual. The exact definition depends on the task.

Metaphor warning

The AI is not experiencing a human hallucination. The term describes an output failure.

Fluency

Language that reads naturally. Fluency does not establish truth.

Grounding

Connecting generation to relevant external evidence or supplied source material.

Verification

Checking a generated claim against evidence, computation, tools or another reliable reference.

False premise

Incorrect information already embedded in the user's request or context, which can steer generation.

Uncertainty behavior

The system's ability to qualify, defer, seek evidence or say it lacks enough information rather than fabricate.

Generation compounding

An unsupported generated token becomes part of subsequent context and can support further coherent but incorrect continuation.

Central lesson

A correct execution path can produce an incorrect factual answer. Computation and truth are different properties.

Research Note: Where the Name Came From

Neural machine translation researchers were using “hallucination” for outputs detached from their source before the current chatbot era. A 2018 paper, Hallucinations in Neural Machine Translation, explicitly used the term for pathological translations untethered from source material. Later surveys describe varying definitions across machine translation and natural-language generation. HAAW therefore treats “hallucination” as established terminology, but not as a claim that the machine has a human perceptual experience.