They call it "hallucination." The AI made something up. Got confused. Had a little mental hiccup and produced something false.
So not true. It is a lie that keeps the myth that AI understands things alive.
Hallucination is something that happens to a mind. It implies there's a perceiver in there, someone who knows what's real and occasionally sees things that aren't. The machine tried its best, but just got a little lost.
Reality: There's no one in there.
What the machine really does is regurgitates tropes, not knowledge. Sometimes the trope fits what you're asking for. There is no "knowledge base." Don't believe me? Ask your AI.
Here's what really happens: during training, it ate billions of documents — complete memos, finished reports, whole books, legal briefs, thank-you notes, strategy decks — and it learned what all of those things look like. Not what they mean. Only what they look like.
You do this too, by the way. Think about how many things you produce every day from pure pattern. Directions to your house: distances, landmarks, turn-by-turn, "you'll see a blue mailbox on the left." A thank-you note: short, warm, specific about the gift, signed with your name. Nobody taught you these formats. You absorbed them by seeing a few thousand examples, and now you produce them without much thought.
The machine has read 50 million books worth of information…that is literally hundreds of lifetimes worth of human reading. And it is great at selecting and creating the most statistically probable phrases. It can create these phrases without, literally, any thought.
It built its patterns reading literally 1 billion documents or so, a massive number of them looked exactly like what you're asking for. It has the pattern for that strategy deck with a 2x2 matrix and perpendicular axes. That a bereavement note is three sentences long. That when a claim appears in an academic paper, a citation comes next, usually as a footnote.
These are tropes, patterns in the way that we speak and write. Tropes are the machine's only product.
So when the machine arrives at the spot in a document where a citation belongs, it produces the shape of a citation... even if it does not know of one, it knows what the shape of one looks like, and it will produce it. Real-sounding author. A plausible journal. A title that could exist.
This is not a malfunction. This is the machine doing the one thing it does — identifying the trope that goes in this position and generating it.
"Hallucination" is a cover story that the AI-worshippers use to pretend that this thing is human and has an interior and understanding like we do.
It lets you keep believing the machine is a thinker that occasionally gets things wrong, which means the problem is solvable — just make the thinker more accurate, feed it better data, add some guardrails. That's a comfortable engineering problem. Companies can sell you the solution.
But the machine is a statistical engine that produces shapes. Often they correspond to reality. Sometimes they don't. The machine has no way to tell the difference, because "the difference" requires understanding.
There's no one in there. It is just a statistical machine that happens to sound like us.
Check out my new book in the making, The Mostly Helpful Psychopath, serialized free on Substack.
