05 Lesson 5 of 15
Confidence is a writing style
The most useful thing you can unlearn: taking a confident tone as a sign that an answer is right.
It sounded sure, so it was probably right. Vague, hedging answers feel less trustworthy. A direct, tidy, specific answer feels like it came from someone who knows. You have read human writing all your life, and this rule of thumb mostly works.
With people, it mostly does work. Someone who writes confidently about something they do not understand usually gives themselves away. And writing confidently takes effort, which people tend to save for things they know. Neither is true here. Fluency is free. The same process writes a right answer and a wrong answer in equally clean prose, and at no point does it check which one it is writing.
So the usual signs tell you nothing:
Cues that feel like knowledge, but are not
A confident opening. Specific numbers. Structure, headings, a numbered list. Technical vocabulary. Named sources. Length. A willingness to answer immediately. Saying “in fact” or “research shows”.
Cues that carry a little information
Whether you can check the claim, and it checks out. Whether two fresh runs agree. Whether the material came from you. Whether this is the kind of thing it is reliably good at. Whether it changes its answer as soon as you push back, which suggests it was not anchored to anything.
The sycophancy problem
That last one needs its own note. Push back on a right answer and these systems will often give in and agree with you. Push back on a wrong one and they will often give in too. Agreeing after you disagree is not confirmation. It is one of the easiest ways to talk yourself into a wrong answer, with the tool seeming to back you up.
One research finding is worth remembering here. Researchers have added explanations to an AI's suggestions, hoping people would think more critically. It often does not stop people relying on it too much. Explanations are generated text too. A smooth explanation of a wrong answer is as easy to produce as the wrong answer, and it feels much more convincing.
The diagram holds twenty-four answers to everyday questions. First, line them up by how sure each one sounds. Then reveal which ones were right.
Read this diagram as text
Twenty-four answers, each scored for how confident it sounds and marked right or wrong. Split them into three groups of eight by confidence. Of the eight that sound most sure, five are right. Of the middle eight, five are right. Of the eight that sound least sure, five are right. Tone tells you nothing here about which answers to trust. The line under the buttons says the same after each step.
Take the confidence out
The same answer, twice. Switch between the confident version and a flat one. The claims do not change; only the wording does. Notice which one you would be more inclined to believe.
One analogy. Think of it as a house style, not a giveaway. A newspaper's confident tone tells you about the newspaper, not about any one sentence in it.
Where the analogy breaks: a newspaper has editors and a reputation to protect, so its confidence is loosely tied to accuracy. Here there is no tie at all.
Take an answer a chatbot gave you recently and mark, honestly, which parts you actually checked. For most people, the first time they try this, the answer is none of it. That is the useful thing to notice.