ChatGPT can be confidently wrong: how to catch it before you decide
The short version: an AI writes in the same confident tone whether it knows the answer cold or is improvising, so tone tells you nothing. The three things that do tell you something: it never hedges on a question that deserves an “it depends”; it changes its answer when you push back; and it gets specific — a number, a date, a rule — in an area where specifics are hard to come by. When any of those show up on a decision that costs money or is hard to undo, check it against a second AI.
Ask ChatGPT, Gemini, or any AI something it doesn’t actually know for sure, and it will almost never tell you that. It will give you an answer. In the exact same confident tone it would use to tell you what 2+2 is.
That’s the underlying problem, not an occasional glitch: an AI doesn’t know what it doesn’t know. There’s no internal alarm that goes off when it’s improvising. And when the question actually matters — a contract, a job change, a big purchase — that unqualified confidence can get expensive.
Why an AI never says “I don’t know”
An AI generates the most probable answer based on what it’s learned, not the truest one. Most of the time those two things line up. But when they don’t, there’s no visible signal telling them apart: the same confident tone covers what it knows cold and what it’s making up.
It helps to know where the tone comes from. These systems are trained to produce answers people rate as good, and people rate confident, complete, well-structured answers as good. Hedging reads as unhelpful. So the writing style you’re reading was selected for how satisfying it is, not for how certain the model is underneath. Fluency is the product of the training; certainty was never measured.
That’s why “just ask the AI” isn’t a bad idea — it’s an incomplete one. It’s missing the follow-up question: and if it’s wrong, how would I even know?
Where it goes wrong most often
Not everything is equally risky. The failures cluster:
- Anything with a number attached. Deadlines, percentages, thresholds, fees. These are exactly the details that feel most authoritative and are easiest to get subtly wrong — the right rule with last year’s figure, say.
- Anything recent. A model’s knowledge has an end date, and it rarely volunteers it. Ask about a rule that changed six months ago and you may get a fluent description of the version that no longer applies.
- Anything local. Rules that differ by country, region or sector get flattened into the version that appeared most in the training data — usually the American one, whatever your question was.
- Anything about you. Your savings, your contract, your family situation. It can’t know these, so it fills them in. The answer is then perfectly reasoned from an invented starting point.
Three signs you should double-check
You don’t need to be an expert to spot them. With practice, these patterns show up again and again:
- It never hedges on something that deserves it. A genuinely expert human answer to a messy question almost always includes an “it depends, and here’s on what”. An answer with no conditions attached to a question that obviously has conditions is a flag — not proof of anything, but a reason to look twice.
- It flips when you push back. Ask the same thing a different way, or just say “are you sure?” If it reverses itself with the exact same confidence as the first answer, neither version was calibrated. Note what this tells you: it’s evidence about the confidence, not about which answer was right. A model that caves to mild pressure was never expressing certainty in the first place.
- It gives you a long answer to something simple. Sometimes length is padding standing in for certainty it doesn’t have. Not always — but a paragraph where a sentence would do is worth a second look.
There’s a fourth sign that’s tempting and doesn’t work: asking it how sure it is. It will produce a number, and that number is generated the same way the answer was. A confident “I’m 95% sure” is not a measurement. It’s more text.
The trick that actually works: cross-checking
The most reliable signal isn’t inside a single answer — it’s in comparing two. Ask the same question to two different AIs (ChatGPT and Gemini, say) and see what happens:
- If they agree, you have a real reason to trust it. Not a guarantee — they can share the same blind spot — but the odds are on your side.
- If they disagree, you just found the exact point where you should think twice — before deciding, not after.
The disagreement is worth more than it looks. It doesn’t just warn you; it points at what to check. Two answers that differ on a deadline send you to look up one fact. Two answers that differ because each assumed something different about your situation tell you which detail you forgot to mention. Either way you go from “I have an answer” to “I know what would make this answer wrong”, which is a much better place to decide from. There’s more on how to read that in how to compare two AI answers.
It’s the same principle we already use outside of AI: a panel rather than a single examiner, several experts rather than one. It isn’t about asking for another opinion just in case — it’s about the ruling being backed by several rather than depending on which one you happened to get. With a single AI you lose that backing without noticing, because its answer sounds complete even when it isn’t.
One caveat worth stating plainly: cross-checking reduces your risk, it doesn’t remove it. Two AIs trained on the same wrong internet page will confidently agree with each other. What comparison buys you is that a lone mistake stops being invisible — and lone mistakes are the common case.
When this actually matters
You don’t need to cross-check everything. For “what’s the capital of Portugal,” it doesn’t matter. But before a decision that costs money, time, or is hard to undo — negotiating a salary, choosing between two offers, deciding whether a big purchase is worth it — that extra minute of comparing is cheap next to the cost of being confidently wrong.
And there’s a category where the answer isn’t “cross-check”, it’s “ask a person”: anything medical, any legal matter with real consequences, anything involving a crisis. There, several AIs agreeing is still not the thing you need. The useful role for an AI is helping you turn a vague worry into precise questions before you walk into the appointment.
That’s exactly the idea behind The Judge: instead of convening several AIs yourself and comparing their answers on your own, it does it for you — gathers the best ones, makes them deliberate, and gives you a verdict with its confidence level, warning you when even they can’t agree.