The memory that remembers and forgets

In the previous step we saw that the model's knobs are set once, in training, and then stay still: the model knows about the world up to its cutoff date and, by default, doesn't store your chats inside it. That's one memory. But there are two others in play every time you write, and confusing them leads you to ask the chat for things it can't give. Once I learned to separate them, I stopped being surprised that sometimes it "remembered" something and sometimes it didn't.

Three memories people mix up

There are three different things we call "memory" without meaning to, and it's worth keeping them apart.

The first you already know: what the model learned in training. It's baked into its parameters, it's fixed and it's the same for everyone who uses that model. It doesn't change because you talk to it.

The second is what the model has present within this conversation: the thread you and it are writing. While you chat, every time it answers it re-reads everything said up to that moment. That's the memory of the chat, and it lasts as long as the chat lasts.

The third is the most misleading: a between-conversations memory that some chats keep separately, to remember you from one day to the next. Not all of them have it, and when they do, you control it. They're three things that work differently, and almost all the muddle comes from throwing them in the same bag.

The conversation fits in a window

Hold first onto the second one, the one within the chat, because it's the one you use most without realising. We saw in the first step that the model doesn't retrieve a stored answer: it builds it by reading the text in front of it. Well, that "text in front of it" is your whole conversation up to that point. Your first message, its answer, your second message, what it replied, and so on up to the last. Every time you hit send, it runs its eyes over the entire thread again before writing.

That's why it feels like it "remembers" what you told it three messages back: it isn't that it remembers it like a person, it's that it's re-reading it. If at the start of the chat you told it you write for children, it still has that in view and adjusts the tone. There's no magic; there's re-reading.

But that re-reading has a limit. The model can only look at a certain amount of text at once, like a field of vision that doesn't stretch. That field is called the context window: the chunk of conversation it can read in one go. While the chat is short, it fits whole and all goes well.

What slides off the edge

The problem shows up in long conversations. When the thread grows beyond what that window covers, something has to be left out, and what's left out is the oldest: the first messages. It doesn't delete them on purpose or get distracted. It's that, simply, they no longer fit in its field of vision.

And here comes the curious effect. If in a very long chat you'd agreed something at the start —to be addressed formally, to avoid a certain word, a fact you gave it— it may suddenly "forget" it and fall back into old habits. It isn't a fault or a slip: it's that that bit from the start has slid out of the window and the model is no longer reading it. For it, it's as if you'd never said it, because only what it has in front of it right now exists.

Knowing this changes how you handle long chats. When you notice the chat "loses the thread" of something you told it long ago, it isn't that it's gone dim: it's that that no longer fits. The solution is simple and we'll see it soon: remind it, repeat the important bits to put them back inside the window.

The memory that lives between chats

Now for the third, the one that's most useful and most confusing. Separate from the conversation thread, some products —ChatGPT is the best-known example— offer a memory that stores facts about you and reuses them in future conversations, even another day and a different chat.

It works like this: if in a chat you tell it something that seems useful for later —that you're vegetarian, what your name is, that you write in a formal tone—, the system may jot it down in that memory without you asking. And the next time you open a new chat, it takes it into account. That's why it sometimes seems to "remember you": it isn't that the model has changed, it's that they're passing it those stored notes alongside your new message. You can also ask it on purpose: "remember I'm vegetarian when you suggest recipes," and it notes it down.

And you control this. According to OpenAI itself, you can see the list of what it has saved, edit it, delete individual memories or delete everything, and turn memory off completely from the settings. It isn't a black box: it's a list you can open and review whenever you want.

Notes, not copies

There's a nuance that avoids a big misunderstanding. What that memory saves isn't word-for-word copies of your conversations. They're high-level notes: short jottings along the lines of "is vegetarian" or "is called Marta," not the transcript of what you talked about. OpenAI's own documentation warns that memory is meant for preferences and general details, not for storing exact templates or large blocks of text verbatim.

That explains the two false beliefs worth knocking down at once. Neither does the chat "remember everything you've ever told it," because it only keeps those loose notes and only if memory is turned on; nor does it "forget everything the moment you close it," because if memory is on, some facts survive the conversation. The truth is in the middle, and it depends on a setting that's in your hands.

And if at some point you don't want it to save anything —a query you'd rather left no trace—, the door's open: there's the temporary chat, which doesn't write to memory, or you can turn the feature off. That terrain, of what gets saved and how your privacy is protected, I'll deal with further on; here it's enough to know the option exists.

If it matters, keep it on hand

From all this comes a practical habit good for almost any chat. If a fact is important for the answer you expect, don't take for granted that the model remembers it: keep it on hand and repeat it when needed. In a long chat, because the start may have slid out of the window. Between chats, because maybe memory wasn't on or didn't note down exactly that. Repeating what matters isn't distrust; it's working with the tool as it is.

So far it seems that, as long as everything fits in the window, the model is always reading the same thread. And yet, if you ask it the same question twice within that same chat, it can answer different things. If it reads the same, why doesn't it answer the same? That's the next step.

Definitions

- Context window: the chunk of conversation the model can read in a single go to answer. It has a limited size; what doesn't fit is left out. - Conversation memory: what the model "has present" within a single chat. It isn't that it remembers it: it's that it re-reads the whole thread each time it answers, as long as it fits in the window. - Between-conversations memory: the feature, optional and controllable, that some chats (like ChatGPT) offer to store facts about you and reuse them in different future chats. - High-level note: what that memory really saves. A short jotting about a preference or a fact about you, not a verbatim copy of what you talked about. - Temporary chat: the option to converse without anything being written to the between-conversations memory, for when you don't want to leave a trace.

No comments yet

No comments yet. Be the first.