The Answer Looked Perfect
Imagine asking an AI assistant:
“Please summarize the five main requirements in this policy.”
The answer arrives immediately.
It is clearly organized. It uses professional language. It includes dates, names, and numbered points.
There is only one problem:
One requirement was invented.
Nothing in the tone warns you. There is no nervous pause, uncertain expression, or change in voice. The incorrect statement appears with exactly the same confidence as the correct ones.
This is one of the most important lessons for anyone using generative AI:
AI can be wrong without sounding unsure.
Why Confidence Comes So Easily
A generative AI system is designed to produce a useful continuation of your request.
It predicts words and ideas that fit the context. It does not automatically stop and ask, “Is every sentence factually verified?”
When the system has strong patterns to work from, the answer may be excellent.
When information is incomplete, ambiguous, outdated, or absent, the system may still generate a response that sounds complete.
The smoothness comes from language generation.
The truthfulness must come from evidence.
Confidence Is Not a Built-In Lie Detector
People often expect uncertainty to sound uncertain.
A colleague might say:
- “I think this is correct.”
- “I am not completely sure.”
- “We should verify the number.”
- “I may be remembering this incorrectly.”
AI does not always give these social signals reliably.
It may use cautious wording when the answer is correct.
It may use decisive wording when the answer is wrong.
It may even provide a detailed explanation for a source, quotation, or policy that does not exist.
This does not mean the AI is intentionally trying to deceive you.
It means the system is generating a plausible response, not personally experiencing confidence or doubt the way a human does.
Coffee Break
Think of a GPS system that temporarily loses its map data.
Instead of saying “I do not know,” imagine it confidently drawing a road through a lake.
The problem is not the friendly voice.
The problem is that the route was not checked against reality.
The more polished the presentation, the more disciplined your verification should become.
Four Warning Signs
You do not need to be a technical expert to notice risk. Look for these four signs.
1. Precise details without a visible source
Be careful when an answer includes exact dates, percentages, regulations, quotations, or document titles but provides no reliable source.
Precision can make an answer look trustworthy. It can also make a fabricated detail more dangerous.
2. A source that cannot be found
AI may generate a realistic-looking article title, author, court case, standard, or website reference.
Do not accept a citation simply because it looks academic or official.
Open it. Search for it. Confirm that it exists and says what the AI claims.
3. The answer changes when asked again
Ask the same factual question in a different way.
If key facts change, the answer is not stable enough to trust without further checking.
Variation is useful for brainstorming. It is a warning sign for factual claims.
4. The answer fills gaps you never supplied
AI often tries to be helpful by completing missing context.
It may assume your country, industry, software version, policy date, audience, or business objective.
An answer based on a hidden assumption can be well written and still be wrong for your situation.
Use the S.A.F.E. Check
Before relying on an important AI answer, use four steps.
S — Source
Ask:
“What reliable sources support this answer?”
Then inspect the original sources—not only the AI summary.
A — Assumptions
Ask:
“What assumptions did you make?”
Check whether the answer assumed a location, date, role, policy, audience, or missing fact.
F — Facts
Separate factual claims from suggestions.
Dates, names, calculations, quotations, legal requirements, medical guidance, and financial figures deserve direct verification.
E — Effect
Ask:
“What happens if this answer is wrong?”
The higher the consequence, the stronger the review must be.
Match Trust to the Task
Not every AI output needs the same level of checking.
Low-risk work
Examples:
- Brainstorming headlines
- Rewriting a friendly email
- Organizing meeting notes
- Creating a first draft
- Generating questions for discussion
A quick human review may be enough.
Medium-risk work
Examples:
- Summarizing an internal report
- Comparing suppliers
- Drafting a project plan
- Preparing training content
- Interpreting operational data
Check important claims against the original material.
High-risk work
Examples:
- Legal decisions
- Medical decisions
- Financial commitments
- Safety instructions
- Regulatory compliance
- Public claims that may damage trust
Use authoritative sources and qualified human review. AI should assist the process, not become the final authority.
PichaiTech Insight
The right question is not “Can I trust AI?”
The better question is:
“How much verification does this task require?”
Trust should not be all or nothing.
It should be proportional to the risk, the available evidence, and the consequences of error.
Try It Today
Give AI a short document you already understand.
Ask:
“Summarize this document and list every factual claim that should be verified.”
Then ask:
“For each claim, show the exact sentence or section in the source document that supports it. If the source does not support a claim, label it unsupported.”
Compare the answer with the original document.
This simple exercise teaches two valuable habits:
- Ask AI to reveal its evidence.
- Check the evidence yourself.
Key Takeaways
- AI can sound equally confident when it is right or wrong.
- Professional wording is not proof of accuracy.
- Exact details and citations must be verified.
- Hidden assumptions can make a useful answer unsuitable for your situation.
- Use the S.A.F.E. check: Source, Assumptions, Facts, Effect.
- Match the strength of verification to the consequence of error.
- The human user remains responsible for the final decision.
Tomorrow's Question
If AI can be confidently wrong, how should we ask questions that produce better, safer answers?
That is where prompt quality begins.
Previous Issue
Issue #002 — Can AI Really Think?
`/en/ai-skills/can-ai-really-think`
Next Issue
Issue #004 — The Anatomy of a Good AI Prompt
Status: Coming Soon
Related Links
- AI Academy
- Books
- Videos
- Issue #002 — Can AI Really Think?
FAQ
Questions professionals often ask
Why does AI sound confident when it is wrong?
Generative AI is optimized to produce coherent, plausible language. Fluent wording does not automatically mean every factual claim has been verified.
What is an AI hallucination?
It is an AI-generated statement that appears plausible but is unsupported, fabricated, or inconsistent with reliable evidence.
How can I verify an AI answer?
Check the original sources, identify assumptions, verify factual claims, and consider the consequences if the answer is wrong.
Should I stop using AI because it makes mistakes?
No. Use AI for suitable tasks and apply stronger human review when accuracy and consequences matter.
What is the S.A.F.E. method?
S.A.F.E. stands for Source, Assumptions, Facts, and Effect—a four-step check for important AI answers.