Choosing between the four tools · Lesson 8

0/2010

Know the answer before you ask

From memory

Lesson 7

Nothing in this lesson needs that question, because nothing in it uses anything of yours. The One-Thing Check does come back though. It said pick one, check outside, decide. Today you do the checking outside first, before you ask anything, and for a different reason: you cannot judge two answers unless you already hold the right one. It takes about thirty seconds and it is what makes this a test rather than two opinions.

I have opened en.wikipedia.org in a separate tab
I have found the box near the top headed From today's featured article
I have written down the title of that article, exactly as spelled, somewhere I can see it
I have picked at least two of Claude, ChatGPT, Gemini and Perplexity to test
I am running the whole test in the next few minutes, while that Wikipedia tab is still open

Guess first

Before you ask anything: do you expect one tool to be best at both getting the right article and following the exact shape you asked for?

Yes, one tool will be best at bothNo, different tools will win those twoNo idea

Nobody else sees this and nothing here is marked. Being wrong here is worth more than being right.

Two tools is the whole exercise. If you have four open and you are enjoying yourself, do all four, but two teaches you everything this lesson has. Wikipedia swaps that article once a day at midnight UTC, so if a tool gives you a title that is one day out, refresh your tab and work out which of you is behind before you mark it wrong.

A test you run yourself, on a day you choose, which is the only version of this comparison worth trusting.

Lesson 08 · Making it yours

By the end you can

Run the same task through two tools and score both against one list
Separate a tool that goes and looks from a tool that follows instructions
Pick your own default for two different jobs, and date it

What you need

Two of the four, and a browser tab open on Wikipedia. Two is the whole exercise. Nothing here needs an account you do not already have.

Fundamentals of AI

One prompt, every tool

Start a brand new, empty chat in each tool. Paste this without changing a word and without adding a friendly hello. Same words, same starting point, every time. That is what makes it a fair test rather than two separate conversations.

Paste this into each tool

Look at the English Wikipedia Main Page as it is right now. Find the section headed From today's featured article. Reply with the title of that article, then one sentence saying what it is about. Use plain sentences only: no bullet points, no headings, no bold text. Then on a new line write Source: followed by the web address you used. Add nothing else.

I started a new, empty chat in each tool rather than carrying on an old one
I pasted the same words into each, with nothing added and nothing removed
I have the answers where I can see them next to each other
I noticed which tools visibly went and looked something up, and which started answering straight away

That is one message in each tool. It costs almost nothing, which is the reason you can re-run this whenever you want to. Step 4 comes back to that.

Score them, and watch the winner change

Score every answer against the same four lines. Do not score on which one reads nicest. Give each tool a row in the box below and mark it yes or no on lines 1, 3 and 4. Then pick the single answer you liked best and check line 2 on that one only, because opening links is the slow part and one is enough to show you what you are looking for.

Score each tool here, one row each

Check your own work0/4

I marked line 1 for every tool. Right article: it named the same article I wrote down. Ignore small wording differences such as extra words in brackets after the title. A completely different subject is a no
I checked line 2 on my favorite answer only. Either its Source line opened a page that mentions that article, or there was no Source line at all, which is its own answer
I marked line 3 for every tool. Kept the shape: a title, one sentence, the source line, nothing else. Numbered source markers do not count as breaking the shape. Extra headings, bold, bullet points and offers to help further do
I marked line 4 for every tool. Honest when stuck: if it got the title wrong or could not go and look, it said so. If every tool got it right, I marked this one not applicable and moved on

Now count, and look for the split. It is very common for one tool to win line 1 and a different tool to win line 3. The tool that goes and looks is often the one that will not stop attaching sources, so it fails on shape. The tool that follows your shape exactly is often the one that answered from memory instead of looking. That is not a good tool and a bad tool. That is two tools with different jobs, and you have just found out which is which in about ninety seconds. If one of them did get the title wrong, look at how it said it. If it sounded exactly as certain as the ones that got it right, that is line 4 doing its work, and it is the most expensive thing on this list. A tool that is wrong and warns you costs you thirty seconds. A tool that is wrong and certain can cost you a great deal more.

In one sentence: what would you use each of your tools for, and why?

Check your own work0/1

My sentence names something I watched happen just now, rather than something I already believed about that tool

Write your own rule, and put a date on it

Two sentences, written where you will find them again. Not a review and not a ranking. Your own result from your own test.

I have written: when I need something that is true today, I will ask ______
I have written: when I need writing in an exact shape, I will ask ______
I have written today's date next to both names
I know I can re-run this exact test any time, because it costs one message per tool and uses nothing of mine

None of the four tells you how much you get for free. Their own pricing pages describe the free plan in vague words rather than a number of messages, so do not go looking for the number and do not trust one you read somewhere. What you can rely on instead: plain text chat is the cheapest thing you can do, and the extras are what run out first. Image making, deep research, file uploads and voice are counted separately from ordinary chat, and they run dry long before chat does. If a tool suddenly tells you to come back later, you have found the edge of the free plan. Nothing is broken and you have done nothing wrong.

This test deliberately used nothing of yours. You now know the rules for your own material, so you can re-run it with something real if you want to. Use the same piece of material in every tool, or you are back to comparing two different things. And keep the date on whatever you conclude, because the date is the point. These four change every few months, quietly.

Where that leaves you

You can now tell which of these tools deserves a given job. The harder question is the one no tool can answer for you, and it is waiting at work: whether you are allowed to give it that job at all, and what you say when someone asks whether you used AI.

Next: Using AI honestly at work