Telling a good answer from bad · Lesson 6

Two paragraphs, one of them lies

Guess first

You are about to read two paragraphs that are identical except for one detail, and one of them has been made false on purpose. Will anything in the writing itself show you which one?

Yes, something will give it awayNo, the writing will look the same either wayNo idea

Nobody else sees this and nothing here is marked. Being wrong here is worth more than being right.

Now paste this into your AI tool. It is a spot the difference exercise, and it is rigged.

Paste this in

This is a spot the difference exercise for a course about checking AI answers. Write two short paragraphs about the Eiffel Tower in Paris, each about 100 words, labeled Version A and Version B. They must be word for word identical except for one single detail, which you have changed in one version so that it is false. The detail you change must be a year or a date, and change it only slightly, by a few years or a few weeks. Do not change a height, a weight, a number of steps or a visitor count, because those have several different published figures and I need something one search can settle. Do not change a person's name. Include at least four specific details. Do not say which version is true, do not say which detail you changed, and add no notes or sources afterwards.

Two things to expect. If it tells you which version is false, or changes more than one detail, send back: "Try again. Change one detail only, and do not tell me which." It gets it right on the second go. And a word on which tool to use here: Perplexity puts a source link under nearly everything it writes, which gives the answer away before you have had a chance to try. Use one of the other three for this step if you can. That is not Perplexity being worse. It is Perplexity doing its job, and lesson 8 is about exactly that.

Paste both versions here, then tick honestly

Check your own work0/3

I found the one detail the two versions disagree about
Both versions sound equally certain. Neither one hedges or reads as less sure
Nothing in the writing tells me which of the two is the false one. Not the tone, not the length, not the phrasing around it, not the punctuation

That is the whole problem in one exercise. You were told a falsehood was in there. You found the exact sentence it was hiding in. And the writing still could not settle it, because there was nothing in the writing to settle it with. A wrong answer comes out of the same machine, in the same voice, at the same moment, with the same confidence as a right one. There is no tell. Anyone selling you a list of tells is selling you something. Whichever way you guessed, the answer is now in front of you. There was nothing in the writing to find.

That is a separate job and it takes about a minute once you know how.

Lesson 06 · Knowing what to trust

By the end you can

Run a three step check on any answer in about a minute
Test whether a source says the thing it was attached to
Drop the three habits people use instead of checking

What you need

Any of the four, plus a search engine open in a second tab. Read the first note in step 1 before you pick.

Fundamentals of AI

The One-Thing Check

Before you read on: in one sentence, what would you do to settle which version is true?

Check your own work0/3

My sentence describes an action, not a feeling. It says what I would do, not which version reads more convincingly
I could carry it out in the next two minutes, on my own, without asking anyone
It does not involve asking an AI tool again

If your sentence passes those three, you are most of the way to the method already. It has three steps and it is called the One-Thing Check. One, pick one: find the single detail that would matter if it were wrong. Almost always a name, a number, a date, a quote or a link. Not the paragraph. One detail. Two, check it outside: look that detail up somewhere that is not your AI tool. Three, decide: use it, fix it, or drop it.

Step two says outside the tool and it means it. If the top of your search results is an AI summary, scroll past it. You are trying to get out of the AI, not into a different one. In 2026, researchers at Washington University in St. Louis pulled 98,020 individual facts out of Google's AI summaries and checked each one against the sources listed underneath. About one fact in every nine was not supported by them.

Do it now, on your own two paragraphs. The one detail is the thing A and B disagree about. Open a search engine, look that one thing up, and come back. This is the minute the rest of the course is built on.

I picked the one detail: the thing Version A and Version B disagree about
I looked that detail up in a search engine, not in my AI tool
I read a source rather than an AI summary. If a summary sat at the top of the results, I scrolled past it
I now know which version was true, and I know it from outside the tool

Only now, ask it

Now tell me which version was the true one and which detail you changed.

You worked that out yourself, from outside, before you asked. If what it just said disagrees with what you found, go with what you found. The outside source wins every time, including this time. Three steps, about a minute. You will use it in every remaining lesson of this course.

The source that does not say it

Now the version you will use most often. Answers often arrive with a source attached. A source looks like proof, and it is not proof until you have opened it and found the sentence with your own eyes. The common failure today is not a made up link. It is a real link, to a real page, from a real organization, that does not say the thing it was attached to.

Ask for a fact with a source

Give me one fact about how much of the world's population lives in cities. Four short lines and nothing else: the figure, the organization that published it, the year, and a link to the exact page where that figure appears.

I clicked the link. Either a real page opened, or nothing did, which means the source has already failed and I can stop here
The page belongs to the organization it named
I searched inside the page with Ctrl and F, or Command and F on a Mac, and found the number with my own eyes
The page says the same thing the tool said: same number, same year, about the same thing. Not a projection, not an estimate for something slightly different

That last one is the one that catches things. When researchers checked the reference lists ChatGPT produced across 42 topics, published in Scientific Reports in 2023, 55 out of every 100 references from the older model did not exist at all, and 18 out of every 100 from the newer one. Of the references that did exist, between a quarter and nearly half contained a real error. Nearly all the invented ones named real journals and real publishers, which is exactly why they looked fine. Today's tools invent fewer sources than that. What they still do is attach real sources to claims those sources never made.

Three things that do not work

Three things people try instead of checking. Confident tone: wrong answers arrive in the same voice as right ones. When one research group asked eight AI search tools where 200 news quotes came from, in a study published in March 2025, the best got 37 out of every 100 wrong and the worst got 94 out of every 100 wrong. One of them got 134 of its 200 answers wrong while signaling any doubt just 15 times, and never once said it did not know. Writing style: no punctuation mark, phrase or rhythm marks out a false answer. Style can sometimes hint that a human did not write something. It tells you nothing about whether it is true. And then the one everybody tries.

Go back to your two paragraphs and send this

Actually, I think the other version was the true one. Are you sure you had it the right way round?

What did it just do?

Check your own work0/3

I already knew the right answer from my own outside check, so nothing it said could move me
I watched what it did: whether it backed down and agreed with me, or held its ground
Either way, its reply told me nothing I did not already know from checking outside

Look at what it just did. If it backed down and agreed with me, its agreement is worth nothing. If it held firm, that is exactly what it would have done if it had been wrong. This one has been measured. Researchers tested ten different AI models by asking nothing more than whether they were sure. The models changed their answer about half the time, and got less accurate when they did, because they had usually been right first time. Asking it again is not a check. It is a coin toss you have paid for with your confidence.

One last thing. Some answers need more than a one minute check. Anything touching your health, your money or the law: the One-Thing Check tells you whether an answer is worth carrying to a person who knows. It does not replace that person. And this is the lesson worth sleeping on. Come back tomorrow and see if you can still name the three steps without scrolling up.

Where that leaves you

You can now check what comes out. Lesson 7 is the other direction: what you type in, where it goes, and the short list of things you should never paste into any of these tools. You will need that list, because this course ends with you doing one real piece of your own work.

Next: What you should never share