You click the link to check a citation, and the page does not exist. You search the journal's archive, then Google Scholar, then the author's faculty page. Nothing. The article your source quoted so confidently, complete with volume number, page range, and a lovely little pull quote, was never written by anyone (oh, that sinking feeling!).
That is an AI hallucination, and if you teach, edit, research, or simply ask a chatbot questions, you have probably met one already, whether you noticed it or not.
After almost two decades of publishing Wandering Educators, I read a LOT of submissions. I have written about how to spot AI writing by its tells and how to make AI writing sound human, and those two pieces are about style. This one is about something far more serious: whether what the words say is actually true. A clunky sentence wastes a reader's time. An invented fact can cost a student a grade, a lawyer a career, and a publication its readers' trust.
I read each submission carefully, checking facts, quotes, and citations. Too often, I see a source that sounds just a bit off, a topic that is definitely not that well-researched in real life, a photo that is in a location that definitely doesn't exist in real life. This is a teachable moment!
So let's talk about what AI hallucinations are, why generative AI makes things up, what it looks like in the wild, and (most importantly!) six practical ways to verify anything AI tells you before you use it, share it, or grade it.
Nine questions to ask before you trust it.
What an AI hallucination actually is
An AI hallucination is a response from an AI tool that sounds confident and plausible but is factually incorrect, unsupported, or entirely invented. It might be a book that was never published, a court case that never happened, a statistic nobody measured, or a quote that no one ever said.
The tricky part is the confidence. A large language model (the technology behind ChatGPT, Gemini, Claude, Copilot, and their cousins) delivers a fabricated citation in exactly the same calm, polished tone it uses for the boiling point of water. There is no wobble in its voice, no "um," no raised eyebrow (I wish there were!). The wrong answer and the right answer look identical on the screen.
Why the word hallucination is a little misleading
"Hallucination" suggests the AI is seeing things, as if it briefly lost touch with a reality it normally perceives. That framing gives these systems far too much credit. AI models do not perceive anything at all. They generate text by predicting which words are likely to come next, based on patterns in enormous amounts of training data.
When those predictions happen to match reality, we call the answer correct. When they don't, we call it a hallucination. The machine does EXACTLY the same thing in both cases! Some researchers prefer words like "confabulation" or "fabrication" for this reason, and I find it useful to keep that in mind (it keeps me appropriately skeptical).
Why AI models make things up instead of saying they do not know
This is the question I hear most from teachers and parents, and there is now a clear, well-documented answer.
In September 2025, OpenAI researchers published a paper called "Why Language Models Hallucinate." Their central finding is that standard training and testing reward guessing over admitting uncertainty. Think of a multiple-choice exam where a blank answer scores zero. A student who guesses on every question will, on average, score higher than one who leaves blanks, even though guessing produces more wrong answers. AI models are graded on similar scoreboards, so they learn to guess.
The researchers also point out that some facts simply cannot be learned from patterns. Spelling and grammar follow rules that repeat millions of times in the training data. A specific person's birthday, or the title of an obscure dissertation, appears rarely or not at all. In the paper's own example, a widely used chatbot was asked for the dissertation title of one of the authors, Adam Tauman Kalai. It confidently produced three different titles, and all three were wrong. Asked for his birthday, it offered three different dates, also all wrong (imagine a student doing that on an oral exam!).
So when you ask an AI tool something it cannot reliably know, it does what it was trained to do. It produces the most plausible-sounding answer available, with full confidence. Every single time!
What AI hallucinations look like in real life
Abstract explanations only go so far. Here are the AI hallucinations examples I think every educator should know, because each one shows a different way invented content slips past smart, busy people.
The fabricated citation, which is the one that catches educators
Fabricated citations (also called hallucinated citations or fake citations) are the most common hallucination in academic work, and the most dangerous for students. Ask an AI tool for sources on almost any topic and it will happily produce a tidy reference list: real-sounding authors, plausible journal titles, correct citation formatting, sometimes even a DOI.
Some of those references will be real. Some will be real authors attached to articles they never wrote. Some will be entirely invented. The formatting is perfect either way, which is exactly why a quick glance fails (and why so many busy people get caught!). A reference list that looks impeccable proves only that the AI knows what a reference list looks like.
The same thing happens outside academia. In May 2025, the Chicago Sun-Times published a summer reading list in a special section, and readers quickly discovered that most of the books did not exist. According to Axios, at least ten of the fifteen titles were fake, including Tidewater Dreams, attributed to Isabel Allende (a real author, a nonexistent novel). The freelance writer who produced the list told 404 Media that he had used AI and had not checked it. A SUMMER READING LIST. Of all things! As a lifelong reader, this one hurt my heart.
Perfectly formatted. This one is not fictional.
The confident wrong fact
This is the everyday version: a wrong date, a misattributed invention, a city placed in the wrong country, a policy that changed two years ago, a number that is off by a factor of ten. These errors are harder to catch than invented citations because they are surrounded by correct information. Sneaky! If nine sentences in a paragraph are accurate, the tenth inherits their credibility.
Confident wrong facts are especially common with numbers, dates, recent events, and anything specific to a small place or a single person, which are exactly the details that make writing feel authoritative.
The invented quote and the misattributed one
Ask an AI tool for a quote from a historical figure on a particular theme, and it will often deliver something beautifully on-topic. That is your first warning sign (real people rarely say exactly what we need them to say, in exactly the words we would choose!).
Sometimes the quote is entirely invented. More often, it is a real quote attributed to the wrong person, or a paraphrase that has been dressed up in quotation marks, or a line from a movie credited to the historical figure it was about. We ran into this head-on when we researched family travel quotes for this site, and the number of famous lines that famous people never said was genuinely astonishing. Quotation-collection sites copy each other, the errors spread, and AI tools trained on those sites repeat them with total confidence.
The real case that went to court
Two cases have become the standard teaching examples, and both are well documented.
The lawyers and the six fake cases. In Mata v. Avianca, a personal injury suit against an airline in federal court in New York, attorneys filed a brief citing court decisions that did not exist, including one called Varghese v. China Southern Airlines. ChatGPT had generated them. When one lawyer asked the chatbot whether Varghese was a real case, it assured him that it was. He later told the court he had been unaware that its content could be false. In June 2023, U.S. District Judge P. Kevin Castel fined the two attorneys and their firm $5,000 and ordered them to send copies of the ruling to their client and to each real judge whose name had been attached to the fake opinions.
That case was supposed to be the cautionary tale that ended the problem. It wasn't, not even close! A database maintained by researcher Damien Charlotin tracks court filings that included AI-fabricated material, filed by attorneys and by people representing themselves. When I checked on October 20, 2026, it listed 2,202 cases worldwide, and it had been updated the day before. By the time you read this, that number will be higher..
The airline chatbot that invented a policy. In 2022, Jake Moffatt used Air Canada's website chatbot to ask about bereavement fares after his grandmother died. The chatbot told him he could book now and apply for the reduced fare within 90 days. That was not Air Canada's actual policy, and the airline refused the refund. Air Canada then argued that it could not be held liable for what its chatbot said. In February 2024, British Columbia's Civil Resolution Tribunal rejected that argument, noting that the chatbot was part of Air Canada's own website, and ordered the airline to pay him roughly $812 Canadian in damages, interest, and fees.
The dollar amounts in both cases are small, and the principle behind them is large: if you publish, submit, or rely on AI output, you own what it says.
Why hallucinations are getting harder to spot, not easier
It is natural to assume that newer, more powerful AI models hallucinate less. Sometimes they do! But the evidence is more complicated than the marketing, and for educators the practical problem is getting harder.
When OpenAI released its o3 and o4-mini reasoning models in April 2025, its own testing found they hallucinated more on one benchmark of questions about people than the earlier o1 model. TechCrunch reported rates of 33 percent for o3 and 48 percent for o4-mini, against 16 percent for o1, and OpenAI acknowledged that more research was needed to understand why. Those figures come from one internal test and do not describe every use, but they are a useful corrective to the idea that the problem is solved.
Then there are AI search tools, which pull from live web pages and attach links to their answers. That sounds like the cure, because now there is a source to check! In practice, a source can be real and still not say what the AI claims it says (uff da). In a March 2025 study from Columbia University's Tow Center for Digital Journalism, researchers tested eight AI search tools on 1,600 queries and found they failed to retrieve the correct information more than 60 percent of the time. ChatGPT Search gave 134 incorrect answers and signaled uncertainty in only 15 of them.
Put those together and you can see the shift. Older hallucinations were often clumsy and easy to catch. Today's are more fluent, wrapped in real-looking links, and embedded in answers that are mostly right. The better AI writing gets, the more our verification habits matter.
Six ways to verify what AI tells you
This is the heart of it. None of these methods requires technical knowledge, special software, or a subscription. They require a few minutes and a healthy dose of skepticism, which educators have in abundance (we have heard every excuse for missing homework ever invented!).
1. Ask it for the source, then go look at the source
Whenever an AI tool gives you a fact you plan to use, ask where it came from. Then, and this is the step almost everyone skips, actually open the source and read the relevant passage yourself.
You are checking three separate things. Does the source exist? Is it the source it claims to be (right author, right publication, right year)? And does it actually say what the AI says it says? A real article about a related topic, linked beside a claim it never makes, is one of the most common failures in AI search tools, and it is easy to miss if you only check that the link loads.
If the AI cannot give you a source, or gives you one that does not hold up, treat the claim as unverified. It may still be true! It simply has not earned a place in your work yet.
2. Check the citation in Google Scholar or the library database
For academic sources, search the exact article title (in quotation marks) in Google Scholar. Real published research almost always appears, usually with a list of other works that cite it. If an article supposedly published in a peer-reviewed journal is completely invisible to Google Scholar, that is a red flag the size of a barn door!
Better still, go to a real database instead. Your school or public library likely provides free access to databases like JSTOR, ERIC, or ProQuest, and a librarian can help you confirm a citation in minutes (librarians are SUPERHEROES at this, and they love to help!). Check the journal's own website, too. If the volume and issue exist but the article is not in the table of contents, you have your answer.
For books, search the title in WorldCat or a library catalog. For court cases, search a legal database or the court's own records. For statistics, find the original report from the agency or organization that produced it, rather than a website that repeats it.
Search, search, search.
3. Watch for the too-perfect detail
Real research is messy. Real quotes are a little awkward. Real statistics come with caveats, odd sample sizes, and footnotes. Hallucinations tend to be suspiciously tidy.
Be wary of the quote that perfectly captures your thesis, the study that proves exactly what you hoped, the statistic that is a beautifully round number with no methodology attached, and the source that seems tailor-made for your argument. As an editor, the more I want a fact to be true, the harder I look at it, and that instinct serves just as well with AI.
4. Re-ask the question cold, in a new session
Start a brand new conversation (so the AI has no memory of your earlier exchange) and ask the same factual question again, worded a little differently. Then ask a different AI tool. It takes two minutes, TOPS.
Inconsistent answers are a strong signal of a hallucination. Consistent answers are reassuring, although AI tools can repeat the same widely circulated error, so they still deserve a check. Remember the dissertation example from OpenAI's own research: three attempts, three different titles. If you get a different date, author, or figure every time you ask, the model is guessing, and you should go find a human-verified source.
It also helps to ask the AI directly, "How confident are you in this, and what would I need to check?" It will not always know (that is the whole problem), but it sometimes flags the parts it is least sure of, which tells you where to start digging.
5. Use AI for the shape of an answer and a human source for the facts
This is the single most useful habit for working with these tools. AI is genuinely helpful for structure, such as brainstorming what questions to ask, outlining a topic, suggesting search terms, explaining a concept in simpler language, or showing you what a field generally looks like before you dive in.
Where it is weakest is the specific, checkable fact. So let the AI help you build the scaffolding, and then fill in every name, number, date, quote, and citation from a source a human being wrote, published, and stood behind. Think of AI as a well-read friend at a dinner party who has a fascinating opinion about everything, and who you would never cite in a paper without checking first (we all know that friend, lol).
6. Know which questions AI is worst at
Some kinds of questions produce hallucinations far more often than others. When you are working with any of these, raise your verification bar.
Citations and references of any kind, for all the reasons above.
Exact quotes, especially from historical figures, authors, and public figures.
Recent events and changing information, including prices, policies, laws, leadership, opening dates, and anything that might have changed since the model was trained.
Specific numbers, including statistics, percentages, measurements, dates, and dollar amounts.
Obscure or local facts that rarely appear online, such as small-town history, a little-known person's biography, or the details of a single school, museum, or business.
Travel details, which deserve their own mention on this site! AI tools will cheerfully recommend a restaurant that closed last year, give you a museum's opening hours from three seasons ago, or describe a ferry route that no longer runs. I wrote about this in our guide to AI travel planning, and the rule is the same: confirm every specific with the official website before you build a day around it.
Tip: Keep a running list of the hallucinations you catch. Within a few weeks, you will see your own patterns, and you will know exactly where your favorite AI tool tends to go wrong.
Teaching verification to students
Students are going to use AI. Many already do, every day. The most useful thing we can give them is AI literacy that includes the reflex to check, which will serve them long after any particular tool has been replaced by the next one. (For a broader look at what that literacy includes, our guide to AI literacy for educators and families covers the whole landscape.)
Families have a role here, too. Our article on what parents and kids should know about chatbots is a good starting point for conversations at home, and teachers looking for classroom-ready tools will find ideas in our roundup of AI tools in the digital classroom.
Nobody argues with a citation they checked themselves.
The assignment that teaches this better than a lecture
Telling students that AI makes things up is far less effective than letting them catch it in the act. One simple assignment does this beautifully.
Have students ask an AI tool for five sources on a topic you are studying, then verify every one using Google Scholar, the library database, or the publisher's website. For each source, they record whether it exists, whether the details are accurate, and whether it actually supports the claim the AI attached to it. Then have them share results as a class.
The conversation that follows is the lesson. Students who find an invented source are usually surprised, sometimes a little indignant (how DARE it!), and from then on, much more careful. Students whose sources all check out learn something equally valuable: verification takes time, and it is still worth doing. Older students can go further by comparing two different AI tools, or by tracing a fabricated citation back to the real research it may have been loosely modeled on.
I love experiential learning. When I've given my students this assignment, responses range from despair (students who are slow to use AI, for ethical or access reasons) to embarrassment (students who are frequent AI users). Critical thinking (and the tools behind it) is one of the best gifts that educators can give their students.
What to do when a student turns in a hallucinated source
First, take a breath. A fabricated citation tells you that AI was used and its output went unchecked. Whether that was deliberate dishonesty is a separate question, and treating every case as a cheating case misses the teaching opportunity. Plenty of adults (including, famously, some practicing attorneys!) have been fooled in exactly the same way.
Start with a conversation. Ask the student to walk you through how they found the source and to show you where they read it. Most students will quickly realize what happened. Then follow your school's academic integrity policy, which increasingly includes specific guidance on AI use in higher education and K-12 settings alike. If your policy is silent on AI, that is worth raising with colleagues, because students deserve clear expectations.
Finally, make verification part of the grade going forward. When students know that every citation will be checked, and that a verified bibliography earns credit, the behavior changes quickly. Hooray!
A verification checklist to keep beside you
Print this, pin it beside your computer, or share it with your students. Before you use anything an AI tool told you, run down the list.
- Did I open every source myself, rather than trusting that the link exists?
- Does each source actually say what the AI claimed it says?
- Did I confirm every academic citation in Google Scholar, a library database, or the journal's own website?
- Did I verify every direct quote against the original speech, book, or document?
- Did I trace every statistic back to the organization that produced it?
- Did I double-check any date, price, policy or opening hours against an official, current source?
- Did I re-ask any key fact in a fresh session, or in a second AI tool, and get the same answer?
- Did I take a second look at any detail that seemed too perfect for my argument?
- Did I use AI for structure and ideas, and human sources for the facts?
- Am I comfortable putting my own name behind every claim in this piece?
Ten for ten? Wonderful! You have done what the lawyers in Mata v. Avianca did not, and your readers (and your students) will thank you.
Frequently asked questions about AI hallucinations
What are AI hallucinations?
AI hallucinations are responses from generative AI tools that sound confident and plausible but are false, unsupported, or invented. Common examples include fake citations, wrong facts, invented quotes, and nonexistent books or court cases.
Why do AI models hallucinate?
Large language models generate text by predicting likely words based on patterns in their training data, and they do not check facts as they go. OpenAI's September 2025 research also found that common training and evaluation methods reward confident guessing over admitting uncertainty, so models learn to guess.
What is an example of an AI hallucination?
In Mata v. Avianca, attorneys submitted a court brief citing six cases that ChatGPT had invented, and were fined $5,000 in June 2023. In 2025, the Chicago Sun-Times published a summer reading list in which most of the recommended books did not exist.
How do you stop AI from hallucinating?
You cannot stop it entirely, because hallucination is a byproduct of how these AI models work. You can reduce the risk by asking for sources, using tools that search the live web, re-asking questions in a fresh session, and verifying every specific fact against a reliable human-written source before you use it.
How can you tell if AI is making something up?
Open and read the sources it cites, search citations in Google Scholar or a library database, be suspicious of details that seem too perfect, and re-ask the question in a new conversation. Inconsistent answers are a strong warning sign.
Do AI hallucinations go away as models improve?
Not reliably. Some newer models hallucinate less on some tasks, but OpenAI's own testing in 2025 found two of its newer reasoning models hallucinated more than an earlier model on one benchmark. As AI writing becomes more fluent, errors also become harder to notice, so verification matters more over time.
Can ChatGPT make up sources?
Yes. ChatGPT and other AI tools can produce fabricated citations with realistic authors, journal names, and page numbers, and they can also link real sources to claims those sources do not make. Always open and confirm every source yourself.
Is it safe to use AI for research?
AI can be a helpful starting point for research, especially for brainstorming, outlining, and finding search terms. It should never be the final source for a fact. Treat everything it tells you as a lead to verify, and cite the original human-written source, never the AI.
The best defense against AI hallucinations turns out to be the oldest skill in education, the one librarians and teachers have championed for generations: checking your sources (and then checking them again!).
Go check a citation today. Pick one you have been quoting for years and have never personally opened. I will wait. (Honestly, you might be surprised, and that is rather the point!)
Have you caught an AI tool making something up? What tipped you off?
More from Wandering Educators on AI, writing and research
AI Literacy for Educators and Families
Top Signs Your Writing Was Generated by AI (and How to Fix It)
How to Make AI Writing Sound Human (Without Losing Your Voice)
AI Chatbots: What Parents and Kids Should Know
Read This: Prompting Originality: The A.I. Handbook for Humans
Sources referenced in this article
- Walters, W.H. and Wilder, E.I., "Fabrication and errors in the bibliographic citations generated by ChatGPT," Scientific Reports 13, 14045, September 7, 2023.
- Mata v. Avianca, Inc., No. 22-cv-01461 (S.D.N.Y.), sanctions order June 22, 2023, Judge P. Kevin Castel.
- Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal.
- Charlotin, D., "AI Hallucination Cases" database, 2,202 cases as of October 19, 2026.
- OpenAI, "o3 and o4-mini System Card," April 2025, PersonQA hallucination evaluation.
Jessie Voigts is the founder and publisher of Wandering Educators. She has a PhD in International Education, has lived and worked in Japan and London, and traveled around the world. She is constantly looking for ways to increase intercultural understanding, and is passionate about study abroad and international education.