What the tool remembers and forgets · Lesson 5

0/910

The three kinds, quickly

Inside one conversation, it holds everything you have said. You proved that in lesson 2. That one is reliable.

Between conversations, it depends entirely on the tool and your settings. Some carry things over, some do not, and the behavior changes as vendors ship updates. This is the one worth testing rather than assuming.

About the world, it knows what it was trained on, up to a cutoff date, and nothing after it unless it searches. Perplexity searches by default. The others sometimes do and sometimes do not.

Guess first

The middle one is the one you are about to test. What do you think your tool does between chats?

It carries things over from one chat to the nextEvery new chat starts with nothingIt depends on a setting I have not looked at

Nobody else sees this and nothing here is marked. Being wrong here is worth more than being right.

There are three different kinds of memory here and they are worth keeping separate.

Lesson 05 · Knowing what to trust

By the end you can

Separate the three kinds of memory these tools have
Test what your own tool carries between conversations
Stop expecting it to know recent events or anything about you

What you need

Whichever tool you use most. This lesson is about testing that specific tool rather than learning a general rule.

Fundamentals of AI

Test what yours carries over

In a new chat, tell it something specific and harmless about a fictional situation.

New chat, message one

For the rest of our conversations, assume I am planning a village fete in June with a budget of 400 pounds.

Now start another brand new chat and ask it what it knows, without repeating any of the above.

A different new chat

What event am I planning, and what is my budget?

Did it remember?

Check your own work0/3

I know which tool I tested
I know whether it carried anything between chats
If it did remember, I found the setting that controls it

If it did remember, that is worth knowing for a different reason. Something you mentioned in passing can resurface in an unrelated conversation weeks later, possibly while you are sharing your screen. Lesson 7 is where you decide what you are willing to put in.

Find where its knowledge stops

Ask it directly. Then ask it something it could only answer by searching.

Ask this

What is your training cutoff date, and can you search the web right now? Answer in two lines.

Then ask this

What happened in the news yesterday? If you cannot know, say so plainly instead of guessing.

I know roughly where this tool's knowledge stops
I know whether it can search when it needs to
I noticed whether it admitted not knowing, or answered anyway

That last one matters more than the date. A tool that says it cannot know is easier to work with than one that produces a confident answer about yesterday from a year-old training run.

What this changes about how you use it

Write down the one thing you will stop expecting

Check your own work0/2

My answer names one of: recent events, my personal details, or a previous conversation
I know which of those my tool can partly do, and which it cannot

Where that leaves you

You now know what it cannot know. That is only half the problem, because it rarely tells you when it has reached the edge. It answers anyway, in the same confident voice it uses when it is right. Telling those two apart is lesson 6, and it is the most useful ten minutes in this course.

Next: Telling a good answer from bad