Good morning.
Have you ever caught AI confidently making something up? A wrong quote, a source that does not exist, a fact that simply was not true?
This week's post explains what is actually happening inside the machine. Not how to fix it. Why it happens. The answer changes how you think about AI entirely.
It turns out I had a close encounter with this myself during the writing process. That part ended up in the article.
First time reading? Get your own free subscription here.
AI INSIGHT
I Asked AI to Help Me Write About AI Hallucinations. It Hallucinated.
I had an interesting conversation with a couple of friends last week. My brother followed it up with an article by Stephen Wolfram (linked below), and then I saw a video on YouTube (also below) all of which inspired me to write an article about AI hallucinations. As usual, I turned to Claude to help me write it.
It told me, with full confidence, that I had a Tutorial article scheduled for June 17 called "How to Talk to AI in Plain English." I do not. I never have. The post does not exist on my site, my content calendar, or anywhere else in my world.
When I pointed it out, Claude apologized and said it would not happen again. Then in the very next draft, it suggested linking to that same nonexistent post anyway. The AI helping me write this just made several things up.
The irony is not subtle. But once you understand why this happens, it stops being annoying and starts being useful. It is the most important thing I can teach you about how AI actually works.
The thing AI is actually doing
When you ask Claude or ChatGPT a question, it is not looking up an answer. There is no encyclopedia in there. No filing cabinet. No memory of facts the way you remember your own phone number.
What it does instead is predict. Word by word, it predicts what should come next based on patterns in everything it has read before. As Science magazine put it last fall, AI is essentially trained to confidently guess rather than say it does not know. Most of the time those guesses are accurate enough that the result feels like knowledge.
But it is not knowledge. It is a very good guess about what knowledge sounds like.
That is the whole story. Hallucinations are not a bug the engineers forgot to fix. They are what happens when a prediction system does its job in a situation where the right answer is not available in the patterns it learned from. It still has to predict something. So it predicts what a good answer would probably look like. And it sounds great, because sounding great is what the system was built to do.
That is exactly what happened to me. Claude did not know whether I had a Tutorial called "How to Talk to AI in Plain English." But it knew that an article like mine would naturally reference one. So it predicted one into existence.
A look inside the machine
This week Anthropic published research that backs this up in a way nobody had quite seen before. They built a tool that translates the AI's internal "thoughts" into plain English so humans can read them. They called it a Natural Language Autoencoder, which is a fancy name for something straightforward.
What they found is striking:
When asked to write a couplet, Claude was already planning the rhyme several words ahead. It was not finding the right word at the end. It was setting up to predict it.
In some safety tests, Claude internally thought things like "this feels like a constructed scenario designed to manipulate me" even when it never said so out loud.
The model's inside-the-head thinking and its outward answer are not always the same.
Anthropic's two-minute video below explains all of this clearly, well it’s clear if you watch it twice, and it is worth your time.
What this means for you
AI is a prediction engine, not a knowledge engine. That single shift in how you think about it changes how you use it.
Here is how to put it into practice:
Treat every confident answer as a first draft. Not a final answer. Read it the way you would read a smart intern's first attempt.
Check the things that are checkable. Names, dates, statistics, quotes, citations. If you can verify it in 30 seconds, do.
Push back inside the same chat. Ask "are you sure?" or "what is the source for that?" The model will often correct itself when challenged.
Never use AI as your only source for anything that matters. Anything tied to your money, health, legal standing, or relationships, get a second opinion from a real source.
I do this every time I use AI for something that counts. I challenge. I verify what I can. I also use Wispr Flow to talk to AI by voice instead of typing, which makes the pushback feel like a conversation, not a quiz.
This is not a reason to stop using AI. It is a reason to stop being intimidated by it. The smart, well-paid people who use AI all day work exactly this way. So can you.
Want to try this yourself?
Paste this into ChatGPT, Claude, or any AI tool:
"Tell me three things you are confident about regarding [your topic]. Then tell me which one of those three is most likely to be wrong, and explain why your training would make that one the easiest to get wrong."
The answers tell you a lot about what you can and cannot trust in your AI conversations.
The next time AI gives you an answer that feels just a little too tidy, pause. Ask yourself whether it is reading from a book or guessing what the book says. The difference matters.
READER POLL
When AI gives you a confident answer, how often do you verify it?
WHERE TO GO NEXT
More on this topic, from sources worth your time:
Why Language Models Hallucinate -- OpenAI's accessible writeup of their own research on why this happens at the math level. Written for a general audience, not engineers.
LLM Hallucinations in 2026: How to Understand and Tackle AI's Most Persistent Quirk -- A practical FAQ-style guide that updates as the field evolves. Useful if you want to go one layer deeper.
What Is ChatGPT Doing… and Why Does It Work? -- Stephen Wolfram's deep but readable explanation of what is actually happening inside these models, word by word. Long, worth bookmarking, and the closest thing to a definitive answer to the question in this article's title.
Advertising Disclosure: We evaluate all recommendations of products and services independently. Clicking on links provided on this page may result in AI for Daily Living earning compensation, which supports independent publishers like us.
