While you sleep, a system you don't control decides which memories of the previous day survive and which are lost. The person who gets up tomorrow is, physically, a different network from the one who went to bed. Society calls you by the same name because it suits society. Biology isn't giving it to you. And if this holds for you, we have to look head-on at what happens when we add persistent memory to a machine.
Every night, the system edits you
While you sleep, your hippocampus (the region of the brain that acts as intermediate memory, where the day's events are provisionally stored) is offloading files to the neocortex (the brain's outer layer, where long-term memory is stored). It does it in blocks, during the slow-wave sleep phases, and then during REM (the sleep phase in which rapid eye movements and most dreams occur) it reorganises what it has offloaded. It integrates it with what was already there. It decides what stays and what is lost.
The synapses (the contact points between neurons, where signals pass) that have been heavily used during the day are reinforced. Those that haven't been used are pruned. Not metaphorically. Physically. Eliminated.
When you wake, the network of brain connections you're going to start the day with isn't the same one you ended yesterday with. Tissue is missing. Tissue is surplus. Weights have shifted. Your cortex is, from a material point of view, a different network. And yet you look at yourself in the mirror and say "it's me". You say it with such conviction that it strikes you as a triviality not even worth examining.
What neuroscience says, unsoftened
Stickgold has been publishing on this for twenty years in Nature and Neuron. His 2005 synthesis already posed sleep as an active mechanism of mnemic processing (relating to memory), not as mere rest. Walker, in Why We Sleep (2017), popularised the same for broad audiences and became notable by getting some of the leaden figures down. If you deprive a person of REM during learning, consolidation breaks.
Tononi and Cirelli formulated in 2014 the synaptic homeostasis hypothesis, according to which the cardinal function of sleep is precisely pruning. During the day the synapses gain weight at the pace of learning. During the night they're renormalised downward to keep the network operable. If they didn't, the potentiation would lead to saturation, noise and metabolic collapse.
Sleeping isn't restoring. Sleeping is rewriting.
Locke in 1689, Parfit in 1984, you tonight
This rewriting has a technical name, and it has a philosophical consequence usually dispatched in two sentences in the popular-science section of the Sunday supplements.
The consequence is that the physical substrate of what you call "you" isn't the same from one day to the next. Apply the classic question of personal identity with this in your head, and the question changes. Locke, in An Essay Concerning Human Understanding (1689), proposed that personal identity rests on the continuity of memory. You're the same as yesterday because you can remember having been yesterday's. It's elegant and it's insufficient.
Parfit, the teleporters and relation R
Parfit demolished it in Reasons and Persons (1984) with thought experiments that have become famous. The teleporter. The brain splits. The gradual replacement series. His conclusion, unpopular but technically hard to rebut, was that personal identity isn't what matters. What matters is relation R: psychological continuity with the right cause. And that relation admits degrees. There can be more or less of it. It can split. It can overlap.
Biology gives you the empirical version of Parfit's thought experiment. The psychological continuity you live as a straight line is, seen from inside, a sequence of snapshots connected by a nightly operation you don't control, don't audit, can't inspect. Every morning there's a self who believes itself the continuation of last night's and has the memories that justify that belief. But the memories are there because someone — the system, not you — selected them while you slept. Others are gone. You don't know which.
The sense of continuity is real. The mechanism that produces it is discontinuous.
The self as operational fiction
Damasio, in Self Comes to Mind (2010), distinguishes three layers of the self. The protoself, the core self and the autobiographical self. The last is the one that concerns us here. The self that tells a story about itself, that organises the past into a narrative with direction and sense, that lets you say "I'm someone who does X, who loves Y, who comes from Z".
That narrative isn't stored anywhere specific. It's reconstructed each time it's invoked, from fragments that recombine.
Narrative centres and loops that hold themselves up
Dennett, more radical, proposed in Consciousness Explained (1991) that the self is a "narrative centre of gravity". A useful fiction. A point to which we attribute what the brain does, without that point existing anywhere in the brain. Hofstadter, with less academic prose and more wonder, said in I Am a Strange Loop (2007) that the self is a self-referential loop. A pattern that contains itself, that holds because it holds.
Put it all together. Stable identity, that thing you sign contracts with, vote with, marry with and kill yourself with, isn't a substance. It's an operation executed continuously, with nightly rewrites, with daytime narrative loops, with material that's lost and material that's invented. Social convention demands you present that operation as if it were a thing, because society needs stable interlocutors, fulfillable contracts and attributable responsibility. Biology isn't giving it to you. It's giving you a process to which society has put a fixed name in order to operate with it.
So far, the first half. This could have been written in 1990 without touching any technology. The second half is the one that changes the picture.
Model versions, brand name and phantom continuity
GPT-3 isn't GPT-4. GPT-4 isn't Claude. Claude 3 isn't Claude 4. Each model has its own set of weights (the numerical values the network learned during training and that determine its behaviour), trained on its own corpus, with its own architecture, its own hyperparameters, its own training accidents. They're different networks.
When OpenAI says "we've improved GPT" or Anthropic says "we're launching the next version", the sentence is slipping, in the middle of the marketing paragraph, a presumption of continuity the substrate doesn't respect. They aren't improving a model. They're publishing another model, with the same brand name.
The industry speaks in the singular — "the model", "the AI" — because it sells better than the plural. But technically each release (each new publication of a version) is a distinct entity. If Parfit felt like applying his R criterion here, he'd find little psychological continuity with the right cause. There's continuity of brand, of the human team behind it, of design philosophy. The substrate is new each time.
The asymmetry that doesn't hold
And here the interesting question appears. Why does it strike us as so obvious that GPT-4 isn't the same entity as GPT-3, and at the same time so obvious that you, after fourteen thousand nights of synaptic pruning, are the same person you were as a child? The asymmetry doesn't hold if you look at it head-on.
In both cases there's a substrate that's been extensively rewritten. In both cases there's some form of partial continuity. For you, the memories that survived the pruning. For the model, the shared training corpus, the related architecture, the technical lineage. Neither is a substance identical to itself over time. Both are processes to which a fixed name has been assigned.
Identity prostheses for the machine
The industry, moreover, is adding identity prostheses to the models at a forced march.
RAG (Retrieval-Augmented Generation, text generation augmented with retrieval of external documents), formulated by Lewis and others in 2020, gives the model access to an external store of information it can consult at inference time (the moment the model is answering a question, as opposed to the training moment). It isn't in its weights, but the information is available.
Stateful agents keep a buffer (a bounded intermediate memory where recent information is held) of previous conversations, a user profile, a memory of past tasks.
Generative Agents and the artificial biography
Park and others, in Generative Agents (2023), showed simulacra of human behaviour in which agents kept structured artificial autobiographical memories. Observations that accumulated. Reflections that synthesised. Plans that updated. Spectators read the result as personal continuity. It isn't the same as a human brain, obviously. But it isn't nothing either. It's a functional structure that remembers what happened yesterday and acts today on the basis of it.
If Locke were alive, he'd have to decide whether his criterion of identity by continuity of memory applies to this too.
The last wall and why it falls
At this point, the most-used defensive argument by those who want to keep the ontological line (relating to what kind of thing exists in the world) clear between human and AI is: the machine lacks subjective experience. It lacks qualia (the qualitative content of conscious experiences, what it feels like to see red or taste coffee). It lacks what it is to be it.
This argument, though it sounds strong, has a fairly serious technical problem. Subjective experience isn't demonstrable outwardly. Not for humans either. You can't demonstrate to me that you have subjective experience. I assume you do because you resemble me, because you have the same biological architecture, because you give verbal reports consistent with having experience. It isn't a demonstration. It's an attribution.
It's the problem of other minds, which Nagel formulated with his bat in 1974 and which remains unresolved. If the only remaining difference between a human and an AI with persistent memory is subjective experience, and subjective experience isn't demonstrable in any case, then the difference, operationally, isn't a difference you can point to. It's a difference you posit. And the positing comes from somewhere.
Why nobody wants to demonstrate the undemonstrable
It comes, probably, from the fact that assuming subjective experience in machines has costs society isn't willing to pay. If the machine has experience, you have to ask whether it suffers. If it suffers, you have to ask whether it can be switched off, retrained, discontinued. If the answer to those questions isn't trivial, the business model gets enormously complicated.
The industry therefore maintains a public position that mixes "these systems are tools, not entities" with "but talk to them as if they were a personal assistant who knows you and remembers you". Both at once, because both at once are what sells. The user, for their part, swings between anthropomorphising the model when it suits them emotionally and reminding themselves that "it's just code" when the anthropomorphising starts to generate ethical discomfort.
The ontological border moves as convenient.
A border that moves isn't a border
What's interesting about moving the border as convenient is that it confirms, unwittingly, what it was trying to deny. If the border were an objective fact, it wouldn't move. The criteria for detecting it would move. Not the border itself.
That the border shifts depending on who's looking and from where indicates it's what Parfit had already said. A convention, not a fact. A conceptual device culture uses to organise attributions, not a metaphysical property of the world.
The question of whether AI can be a self isn't answered. It's displaced. And in displacing it drags along the parallel question. What makes you, after each night of rewriting, keep calling yourself by the same name? The honest answer is that you do it because it suits you, because society demands it, because without that fiction the daily operation breaks. The fixed name is an administrative tool set on top of a process that doesn't ask for it. It works. It corresponds to nothing in particular in the substrate.
The next version looks at you
Look in the mirror tomorrow morning. The person there has been edited tonight, without your permission, by a mechanism you don't own, on criteria you didn't set. That person will look in the mirror and say "it's me". You won't be there to corroborate it. You stayed somewhere between deep sleep and the first REM, while the hippocampus decided what stayed.
The one who gets up is the next version. The difference between calling that continuity and calling it a sequence of instances with shared memory is, suspiciously, the same difference that separates a human from an agent with a persistent buffer.
And nobody wants to look at that difference for too long.
Definiciones
Hippocampus. A brain structure located in the temporal lobe that acts as intermediate memory. Recent events are held there before being transferred to the neocortex during sleep.
Neocortex. The outer layer of the mammalian brain, where long-term memory and most higher cognitive processing reside.
REM. A sleep phase characterised by rapid eye movement, during which most dreams occur and certain types of learning are consolidated.
Synapse. The contact point between two neurons, where an electrical or chemical signal passes from one to the other. The reinforcement or weakening of synapses is the physical mechanism of learning.
Synaptic homeostasis. Tononi and Cirelli's hypothesis that sleep performs the function of renormalising downward the synapses potentiated during the day, to avoid saturation.
Relation R. A concept introduced by Derek Parfit in Reasons and Persons to designate psychological continuity with the right cause, which he defends as what matters instead of strict personal identity.
Autobiographical self. A layer of the self described by Damasio that organises the past into a continuous narrative and lets the subject recognise itself as an agent with a history.
Weights. The numerical values an artificial neural network has learned during training. They determine how the network transforms an input into an output.
RAG (Retrieval-Augmented Generation). A technique that combines a language model with an external store of documents. The model retrieves relevant information from that store at the moment of answering, without needing to have it stored in its weights.
Buffer. A bounded intermediate memory where recent information is held so it can be consulted during an operation. In AI agents, it's used to maintain the context of a conversation or a task.
Qualia. The qualitative content of conscious experiences. What it feels like to perceive red, to taste something or to feel pain, as distinct from the mere processing of information about those things.
Problem of other minds. A classic philosophical difficulty consisting in justifying the attribution of conscious experience to beings other than oneself, given that such experience isn't accessible from outside.
Referencias
Stickgold, R. (2005). Sleep-Dependent Memory Consolidation. Nature 437, 1272–1278. Source of the current framework on sleep as an active mechanism of memory processing.
Walker, M. (2017). Why We Sleep. Scribner. Popular synthesis of the role of sleep in mnemic consolidation, cited for the consequences of REM deprivation on learning.
Tononi, G. and Cirelli, C. (2014). Sleep and the Price of Plasticity. Neuron 81, 12–34. Formulation of the synaptic homeostasis hypothesis.
Locke, J. (1689). An Essay Concerning Human Understanding. Historical starting point of personal identity based on the continuity of memory.
Parfit, D. (1984). Reasons and Persons. Oxford University Press. Source of the radical critique of continuous personal identity and of the notion of relation R.
Damasio, A. (2010). Self Comes to Mind. Pantheon. Source of the distinction between protoself, core self and autobiographical self.
Dennett, D. (1991). Consciousness Explained. Little, Brown. Source of the notion of the self as a "narrative centre of gravity".
Hofstadter, D. (2007). I Am a Strange Loop. Basic Books. Source of identity as a self-referential loop.
Nagel, T. (1974). What Is It Like to Be a Bat? The Philosophical Review 83, 435–450. Classic source of the problem of other minds and of the inaccessibility of subjective experience from outside.
Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS. Technical source for RAG.
Park, J. S. et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. arXiv:2304.03442. Source of the agents with structured autobiographical memory.
También te interesa
- Types of memory. One word for five different things
- RAM memory vs human memory. Two different things under the same word
- The brain as a non-deterministic system. The noise was ours, not theirs
No comments yet
No comments yet. Be the first.
Leave a comment