Short answer: ChatGPT, Claude, Gemini and other chatbots don’t “lie” in the human sense; they generate plausible-sounding text that is sometimes false (so-called hallucinations). They are most often wrong about rare facts, recent events, quotes, links and numbers, and they say it in a confident tone. Research shows that even tools with search can give wrong sources. So treat AI as a fast assistant for finding leads, not as the final source: ask for sources, open them, check the date and confirm important things with a second, independent source.
This article complements our guides to zero-click searches and meta-prompting. There we wrote about how AI answers for you and how to write prompts; here we focus on when not to believe it and how to check quickly. This applies to all tools, including Claude.
Why AI makes things up: “hallucinations” in plain English
According to OpenAI, hallucinations are confidently stated answers that sound plausible but are false. A model learns to predict the next word from a huge amount of text that has no “true” and “false” labels. Spelling mistakes disappear with scale because they follow patterns, but rare facts, like a specific person’s birthday, cannot be predicted from patterns, so the model guesses them. The authors give an example in which a popular chatbot, asked for the title of one of the study authors’ PhD dissertation, gave three different wrong answers, and for his birthday three different wrong dates.
The second cause is how models are evaluated. OpenAI argues that standard tests reward guessing rather than admitting uncertainty. In the SimpleQA test, the older o4-mini model abstained in 1% of cases, was correct 24% of the time and wrong as much as 75% of the time, while the newer gpt-5-thinking-mini abstained in 52%, was correct in 22% and wrong in 26%. Guessing yields slightly more hits at the cost of many more errors.
What research says: how often AI gets it wrong
| Study | What was tested | Result |
|---|---|---|
| EBU/BBC (published 2025) | Over 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity; 22 public service media, 18 countries, 14 languages | 45% of answers with a significant issue, 31% with serious sourcing problems, 20% with major factual errors; Gemini performed worst (76% with a significant issue) |
| Tow Center / Columbia Journalism Review (March 2025) | 8 AI search engines, matching quotes to sources | Over 60% of answers without correct information; from 37% errors (Perplexity) to 94% (Grok-3 Search); answers often confident despite being wrong |
| Demos, “Election hallucinations” (2026, via Demagog) | Questions about elections put to chatbots | Factual errors: Replika 56.4%, ChatGPT 46.2%, Grok 9%; nearly 47% of answers without sources or links |
| AI Security Institute (2025, via Demagog) | Comparison of GPT-4o, Claude and Mistral with Google Search for acquiring knowledge | Chatbots improved knowledge by 0.13 points (7-point scale), Google by 0.08; working with AI was 6–10% faster; 87% of participants considered chatbots reliable |
| Reuters Institute, Digital News Report 2025 | Whether and what people use chatbots for | 7% had used a chatbot for news in the past week (15% of under-25s); to check suspicious information, 9% would turn to a chatbot, 14% to social media, 38% to a trusted news source |
When AI is most often wrong: 10 situations
| # | Situation | Why it’s risky | What to do |
|---|---|---|---|
| 1 | Rare facts: dates, people, titles | No pattern to predict from (OpenAI) | Check in the primary source or an encyclopedia |
| 2 | Recent events | A model’s knowledge has a cutoff date. Demagog described how ChatGPT (GPT-3.5) claimed to be up to date although its data ended in 2021, and wrongly stated that Benedict XVI was alive | Use a mode with search and check the source’s date |
| 3 | Quotes and links | Tow Center: many tools give wrong citations; Gemini and Grok fabricated links more often than they gave correct ones | Open every link and find the quote in the text |
| 4 | Case law and legislation | A model can invent non-existent cases (the Mata v. Avianca case) | Check in the official database of laws or judgments |
| 5 | Numbers and statistics | The model “rounds” and mixes data from different years | Find the table in the source (national statistics office, Eurostat, the report) |
| 6 | Elections and politics | Demos: wrong dates, constituencies and candidates; it’s easy to confuse fact with opinion (Demagog) | Use official election commission websites and fact-checkers |
| 7 | Health, medicines, dosage | A mistake can cost your health; the model doesn’t know your history | Check the leaflet and consult a doctor or pharmacist |
| 8 | Local and niche topics | Few texts in the training data, so more guessing | Check the website of the authority or company, or local media |
| 9 | Questions with a false premise | The model tends to agree with the user (research on sycophancy) | Ask the question neutrally, without suggesting the answer |
| 10 | Absurd or joking sources online | In 2024, Google’s AI Overviews gave, among other things, advice about glue on pizza, which journalists linked to forum jokes and satire | Check whether the source is serious and not a meme |
Case study: four cases that cost money
1. Lawyers and non-existent cases (Mata v. Avianca, 2023)
Lawyers filed a brief in a New York court citing non-existent cases generated by ChatGPT. When the other side raised the problem, they continued to claim the cases were real. In a ruling of June 22, 2023, the court fined the two lawyers and their firm $5,000, pointing in the documents to obvious irregularities, such as mismatched judges’ names and sentences cut off mid-way. According to reports, ChatGPT didn’t confirm the cases existed; it just generated them. Lesson: asking the same model “is this real?” is not verification.
2. An airline chatbot (Moffatt v. Air Canada, 2024)
Air Canada’s chatbot suggested to a passenger that he could buy a regular ticket and apply for a bereavement fare refund within 90 days, which contradicted the airline’s policy. A tribunal in British Columbia ruled that the airline is responsible for information on its website, including from a chatbot, and awarded a partial refund of CAD 650.88 plus interest and fees. The tribunal also noted that customers cannot be expected to verify information across different parts of a website. Lesson: for important terms (refunds, contracts), save the answer and confirm it in the terms and conditions.
3. An AUD 440,000 report (Deloitte, 2025)
A report for the Australian government contained fabricated footnotes, including a made-up quote from a federal court judgment and references to non-existent academic papers. The corrected version of October 3, 2025 states that a generative AI system (Azure OpenAI) was used, and Deloitte agreed to refund part of its fee. Lesson: even professionals don’t check every footnote, and it’s footnotes that fail most often.
4. Glue on pizza (Google AI Overviews, 2024)
In May 2024, AI Overviews displayed advice such as adding “non-toxic” glue to pizza sauce and eating one small rock a day. According to The Register, this was influenced by Reddit jokes and a satirical article from The Onion, among others. Google admitted that some answers were odd or inaccurate, and announced better detection of satire and less reliance on user-generated content, e.g. from forums. Lesson: AI can repeat an internet joke with a completely straight face.
Model flattery: when AI tells you what you want to hear
In a 2023 study, Anthropic described sycophancy: a model tailors its answer to the user’s beliefs rather than to the truth. The authors link this to training on human ratings, because answers that agree with the user’s views are more likely to win approval, and five leading assistants tested showed sycophancy across four tasks. The study doesn’t give a single figure for how often it happens.
- Don’t suggest the answer: instead of “is it true that X is harmful?” ask “what does research say about the effect of X on health, and how strong is the evidence?”.
- Ask for counterarguments: “give me the strongest arguments against my thesis”. This complements what we described in our article on echo chambers.
- Don’t push with your tone: if the model answered correctly and you write “are you sure?”, a changed answer is not proof that the original was wrong.
How to check AI answers: 9 steps
This is our own compilation, based on the research and cases described above. It requires no specialist tools, and the first check takes a few minutes.
Assess the stakes of the question
Before you start, decide: does a mistake cost something (health, legal issues, money, publication)? The higher the stakes, the more verification. A brainstorm needs less than a quote in an article.
Ask for sources, but don’t take them on trust
A list of sources can give a false sense of credibility. Demagog described a test in which ChatGPT gave non-existent sources, and the Tow Center showed that tools with search also give wrong citations.
Open every link
Click to see whether the page exists at all (a 404 error is a red flag), whether it’s the right organisation and whether the text is on topic. Fabricated links were most common in Gemini and Grok in the Tow Center study.
Find the specific sentence in the source
Copy the key claim from the AI’s answer and search for it in the source text (Ctrl+F). Chatbots may cite pages that don’t support their claim.
Check the date
See when the source was created and whether the matter has been updated. A model’s knowledge has a cutoff date, and Demagog showed that a model can insist it is up to date even though its data is older.
Confirm with a second, independent source
Search for the same fact on Google, a trusted website or a database. A second chatbot is not an independent source, because it can repeat the same mistake. We describe the techniques in our guide to step-by-step fact-checking.
Check numbers and quotes in the original
Take percentages, dates and legal wording from the table, the legal act or the report, not from a summary. Our advice: if the model gives a number, ask for a link to the table and find it in that table.
Test the model with a question you know the answer to
First ask a question from your own field or about something verifiable. If it gets it wrong, treat the rest of the conversation with more caution.
For health and law, go to a human
In matters that affect life decisions, AI can help you prepare questions, but consult the decision with a doctor, lawyer or public authority. We do not give medical or legal advice.
10 prompts that reduce the risk of fabrication
A good prompt won’t eliminate hallucinations, but it helps you catch them. These are our suggestions (ready to copy and adapt). You’ll find more on building prompts in our guide to meta-prompting.
| # | Prompt | Why |
|---|---|---|
| 1 | If you’re not sure, say “I don’t know” instead of guessing. | Limits the guessing that OpenAI writes about |
| 2 | Separate confirmed facts from assumptions and mark the level of certainty. | Shows what needs checking |
| 3 | Give sources with working links and publication dates. | Makes checking easier, but you still have to open the links |
| 4 | Quote verbatim the passage of the source on which you base this claim. | Lets you search for the quote in the text |
| 5 | What is your knowledge cutoff date, and can you check this online? | Reminds about recency |
| 6 | Give the strongest arguments against this thesis. | Limits flattery and confirmation bias |
| 7 | Which parts of your answer are most likely to be wrong? | Points to places to verify |
| 8 | Ask me clarifying questions before you answer. | Less guessing of context |
| 9 | Suggest 5 phrases and operators to search for this on Google and in source databases. | AI as a search aid, not a judge (see our 15 Google operators) |
| 10 | List exactly what I need to check manually, and where. | Gives you a checklist |
When AI really helps with searching
The AI Security Institute study (via Demagog) found that chatbots can be as effective as a search engine, even slightly faster, but still require verification. They work well for tasks where a mistake is easy to spot or cheap:
- Brainstorming keywords: synonyms, technical names and foreign-language equivalents of phrases you’ll type into Google. See 15 Google operators.
- Summarising a text you pasted yourself: the risk of error is lower because the model works on your material, though it may still leave something out.
- Explaining concepts and planning research: ask for a map of the topic and a list of questions to check.
- Drafting arguments and counterarguments: for further verification.
- Translating and reading foreign-language sources: as a first pass; check the important passages.
A simple rule (ours): AI for finding leads, search engines and sources for confirming. We’ve also written about how you increasingly get an answer without visiting a website, which makes it all the more important to check where it comes from (zero-click searches).
Frequently asked questions (FAQ)
Not in the human sense, because it has no intention to deceive. It generates text that sounds plausible but may be false. Such errors are called hallucinations.
According to OpenAI, they are confidently stated answers that look plausible but are false. They arise partly because the model predicts the next words, and rare facts cannot be predicted from patterns.
Yes. Every current AI assistant, including Claude, ChatGPT, Gemini and Perplexity, can give wrong information, invent a source or use outdated data. So important information needs to be verified regardless of the tool.
It helps with recency but doesn’t guarantee correctness. In March 2025, the Tow Center showed that AI search engines failed to give correct information in over 60% of the cases tested, and results varied widely between tools.
Open the link, check the domain, date and author, then find the sentence the AI refers to in the text. If the page doesn’t exist or doesn’t support the claim, the answer is unreliable.
You can use it as an extra signal, but it’s not independent verification, because models can repeat the same mistakes. It’s better to check with a search engine and the primary source.
Models generate fluent text with no built-in “I’m not sure” signal. The Tow Center noted that many tools answer in a confident tone despite errors, and OpenAI points out that tests reward guessing.
Not without checking. In Mata v. Avianca, ChatGPT generated non-existent cases, and Deloitte’s report for the Australian government contained fabricated footnotes.
It can help with planning: suggest phrases to search for, point to types of sources and explain concepts. The judgement of what is true should rest on primary sources and fact-checkers.
Ask neutrally, ask for a confidence level, sources and a date, and for counterarguments. Don’t suggest the answer, because models tend to agree with the user (sycophancy).
It can help you understand concepts and prepare questions, but it doesn’t replace a doctor or lawyer. Wrong dosage or invented case law can have serious consequences.
In Moffatt v. Air Canada, the tribunal ruled that the airline is responsible for information given by its chatbot. However, that is a Canadian ruling, and laws in your country may differ, so in a dispute consult a lawyer.
There’s no single answer. Results depend on the tool, the version and the topic, and studies go out of date quickly. Choose a tool that shows its sources, and always check them.
Yes. Demagog, a Polish fact-checker, described chatbot errors, including on political matters and when citing sources. For any local context, it’s important to check in local, official sources.
Get the “How to Search?” guide
Practical methods for searching and verifying information, step by step, with dozens of examples.
See the guide → ☕Support us on Buy Me a Coffee
We create these materials in our spare time. Every bit of support helps us write more, and more often.
Support → ▶️Watch us on YouTube
The same topics in a shorter video format, with concrete examples on screen.
Subscribe →More from our blog
Sources: OpenAI: Why Language Models Hallucinate; arXiv: Why Language Models Hallucinate; Journalism.co.uk: EBU/BBC report; Nieman Lab: Tow Center study; Demagog: how chatbots get political topics wrong; Demagog: how chatbots work and what to watch for with sources; Press Gazette: Digital News Report 2025; Seyfarth: the Mata v. Avianca case; Manatt: Moffatt v. Air Canada; Fast Company: Deloitte’s report for the Australian government; The Register: Google AI Overviews; Anthropic: Towards Understanding Sycophancy in Language Models. Study results refer to specific tool versions and dates, and models change quickly. This text is not medical or legal advice.