Your AI Is Lying to You With Confidence — 4 Ways to Catch It
AI tools sometimes make up facts that sound completely real. Here are 4 simple ways to spot these hallucinations before they cause problems.
Last week, I asked ChatGPT for a list of books by a specific author. It gave me five titles with publication dates, summaries, and even ISBN numbers. Three of those books don't exist.
The scary part? It sounded absolutely certain.
This is called a hallucination — when an AI generates information that sounds real but is completely made up. It happens more often than you'd think, and it can get you in trouble if you use that fake info in a report, email, or presentation.
Here are four simple ways I've learned to catch AI lies before they catch me.
1. Ask the AI to Show Its Sources
This is the easiest first step. After getting any factual answer, follow up with a simple prompt:
prompt What sources did you use for this information? Can you provide links or references?
Here's what happens:
- If the AI gives you real, working links — great, verify them
- If it gives you broken links or says "I don't have access to sources" — that's a red flag
- If it confidently lists sources that don't exist — you've caught a hallucination
I tested this recently when ChatGPT cited a "2023 Harvard study" about productivity. I searched for it. No such study exists. The AI invented an academic paper to sound more credible.
2. Use the "Opposite Question" Trick
This one sounds silly but works surprisingly well. Ask the AI to argue the opposite of what it just told you.
prompt You just said X. Now tell me: what are the strongest arguments that X is actually wrong?
Why does this work? If the AI confidently gives you equally strong arguments for the opposite position, it probably doesn't "know" the answer — it's just generating plausible-sounding text for whatever you ask.
I used this when an AI told me a specific diet was "scientifically proven" to work. When I asked for arguments against it, the same AI gave me equally confident reasons why it doesn't work. Neither answer was based on real knowledge — both were just pattern-matching.
3. Ask for Specific Details, Then Verify One
Hallucinations often hide in vague statements. Push for specifics, then check just one.
prompt Give me specific names, dates, or numbers for this claim.
Once you get details, pick the easiest one to verify. Google the name. Check if that company exists. Look up that statistic.
For example, I asked an AI about a historical event. It gave me a date, a location, and three people's names. A quick search showed the date was right, the location was wrong, and one of the people was entirely fictional.
The rule: if one specific detail is wrong, assume others might be too.
4. Cross-Check With a Different AI
This is my favorite method for important information. Ask the same question to two different AI tools.
Try ChatGPT and then ask Google's Gemini (free at gemini.google.com) or Microsoft Copilot (free in Bing). If they give you different answers about basic facts, someone is hallucinating.
prompt [Paste the exact same question into a different AI tool]
Last month, I asked ChatGPT when a specific law was passed. It said 2019. I asked Gemini the same question. It said 2021. The actual answer? 2020. Both AIs were confidently wrong, but their disagreement told me to dig deeper.
Why Do AI Tools Hallucinate?
Here's the simple explanation: AI doesn't "know" things the way you do. It predicts what words should come next based on patterns it learned from text. Sometimes those predictions create sentences that sound factual but aren't.
It's like someone who's very good at sounding smart in conversation but sometimes makes things up to fill gaps in their knowledge.
The One-Minute Rule
Here's my personal system: for any AI-generated fact I'm going to use publicly — in an email to my boss, a social media post, a school assignment — I spend at least one minute verifying it.
One Google search. One check against another source. That's it.
It takes 60 seconds and has saved me from sharing made-up statistics, citing non-existent studies, and referencing books that were never written.
The Takeaway
AI tools are incredibly useful, but they're also confident liars. The four checks — ask for sources, try the opposite question, verify one specific detail, and cross-check with another AI — take less than five minutes combined.
Trust AI for brainstorming and drafts. Verify AI for facts. That's the difference between using AI well and letting it embarrass you.
Did you like this?
Every Monday I send one email with a prompt or a tool I tried. No spam, unsubscribe with one click.
Subscribe to the newsletter