Jonathan Haidt salvaged a Hindu proverb: the mind is an elephant with its rider. The rider is reason, the elephant is emotion, and the moment things get serious it's the elephant that decides where to go. Antonio Damasio showed it with patients whose arithmetic was intact and whose lives were in ruins. Contemporary AI, the kind sold as a substitute for human judgement, is only the rider. It lacks exactly the part that in a human did the deciding.
There's a Hindu proverb Jonathan Haidt salvaged in 2006, in The Happiness Hypothesis, and it's still the most useful metaphor for understanding how a human decides. The mind is an elephant with a rider. The rider is reason: it reasons, plans, justifies. The elephant is emotion: it weighs several tons, it's been walking for millions of years and it has its own idea of where it wants to go.
Under normal conditions the rider believes he's driving. He sets the pace, picks the route. The elephant, docile, obeys. The illusion works most of the day and that's what lets civilisation advance.
And then something happens. A car appears out of nowhere on the left, a voice says a sentence that touches a specific spot, someone makes a gesture that resembles another from twenty years ago. The elephant plants its feet, turns, or bolts, and the rider, clinging to its neck as best he can, goes along behind. When he gets his breath back, he writes the report. He explains why the decision the elephant just made was the correct, rational, well-considered one. The rider doesn't rule. The rider narrates. That's the first asymmetry of the pairing and almost nobody signs it out loud, because signing it means admitting that the version you have of your own decisions is, in part, fiction worked up after the fact.
The patient who decided with his head intact
Antonio Damasio described in 1994, in Descartes' Error, a series of patients neurology had spent decades not knowing exactly where to place. They were people with lesions in the ventromedial prefrontal cortex (the region of the brain, just behind the forehead and above the eye sockets, that connects deliberative calculation with the body's signals), usually from an excised tumour or a trauma. The intelligence test came out intact. Memory, intact. Language, intact. The capacity to reason about an abstract problem, intact. And their lives, in ruins.
They made appalling decisions, one after another. They invested in obvious losers; they married people anyone would have ruled out at first glance; they lost jobs through absurd choices. When asked to explain, they gave a rational, well-built account. They had reasoned. What they lacked was the other half.
Damasio and Antoine Bechara designed the Iowa Gambling Task (an experimental test with four decks of cards, rewards and punishments) to show the lesion operationally. A decade later, in The Somatic Marker Hypothesis. A Neural Theory of Economic Decision (2005), Bechara and Damasio gave the hypothesis its mature form applied to economic decision-making: what breaks in these patients isn't just a card game, it's the capacity to weigh contracts, offers and purchases. Four decks on a table. Two are a trap: high reward per card, but occasional punishments that ruin you in the long run. Two are safe: low reward, small punishments, positive balance. Healthy subjects start avoiding the bad decks before they can explain why. If you put sensors on their hands, the palms sweat as the hand nears the trap deck before consciousness has understood anything. The body learns first and warns first. Patients with the lesion never develop that sweat. They follow the calculation, pick the bad deck, lose, recalculate, pick the bad deck again. Intact calculation without the bodily lever is a machine that doesn't converge.
Damasio's conclusion was uncomfortable for a philosophical tradition that had spent two and a half millennia setting reason against emotion as its great enemy to be tamed. Emotion is not noise on top of reasoning. It's the signal that makes reasoning choose. Without a somatic marker (the bodily trace — tension, sweat, unease, attraction — that prior experience has tied to an option and that the organism fires before conscious deliberation) there's no sensible decision. There's calculation, which is something else, and which in humans isn't enough.
System 1 does the work, system 2 signs the paper
Daniel Kahneman published Thinking, Fast and Slow in 2011, after forty years of work with Amos Tversky. The scheme is well known and it's worth not trivialising it. System 1 (fast, automatic, almost always emotional, operating by pattern recognition, never tiring) processes most of your life without asking permission. System 2 (slow, deliberative, costly, requiring attention, tiring quickly) steps in when system 1 finds no pattern or contradicts itself.
The part popular culture kept is the one about biases. The one that matters here is another. Almost every decision you make in a day is made by system 1, and system 2 doesn't review them.
When you think you're deliberating, you're rationalising a decision system 1 closed half a second ago. The illusion of control is persistent because system 2 is the one writing the narration of your life; you'll never hear it say it wasn't in charge.
It's the same asymmetry as Haidt's, in another vocabulary. The elephant decides. The rider signs. If you want to see the rider really at work, watch an abstract problem with no emotional charge: a Sudoku, an arithmetic calculation, a literal translation. There system 2 does its thing. The moment body, biography or consequence enters, system 1 takes control and system 2 writes the reason.
Fear arrives before you know it has
Joseph LeDoux, in The Emotional Brain (1996), mapped the two routes of fear in the brain. A short one, subcortical, from the thalamus straight to the amygdala (the almond-shaped brain nucleus that fires defensive reactions), in milliseconds. A long one, cortical, from the thalamus to the sensory cortex and from there to the amygdala, much slower. The short route gets your body moving when something stirs in the grass before you know what it is. The long route confirms or denies that it's a snake.
This adds an operational nuance to the pairing. The elephant isn't only stronger than the rider. The elephant is also faster. When you think you're deciding to react, you're already reacting. Consciousness arrives late, does its part, writes the sensible version. The machinery of emotion is wired below and runs on timescales deliberative calculation can't touch.
In extreme conditions, that speed difference is what keeps you alive. In everyday conditions, it's what makes you fight someone over a sentence you haven't finished processing.
The part probabilistic calculation doesn't touch
There's a point where the computational model breaks and it's worth marking it well. When you decide whether to take a fixed- or variable-rate mortgage, there's a probability distribution. There's data. There's calculation. Out comes something that looks like a rational decision, even if underneath the elephant made it by another route and the rider is touching it up. The system works reasonably.
When you decide whether to jump into the water to pull someone out, there's no useful probability distribution. There's no time. There's no data. There's fear, there's impulse, there's a bodily lever pulling one way or the other and a body already acting before the rider has written anything. The one who goes in to pull them out doesn't compute expectations. The one who doesn't go in doesn't either. Both are being moved by the elephant; one by an elephant heading for the water, the other by an elephant moving away. The rider, in both cases, writes a coherent version afterward. What he doesn't write is the decision.
The same goes for rage. For desire. For grief. For loyalty. For the moral dilemmas where the mathematically calculated answer is manifestly repugnant and most healthy people do the opposite. The situations where the decision really matters are precisely those where probabilistic calculation doesn't apply, not because it's technically hard, but because there's nothing for it to apply to. The elephant decides in territory the rider doesn't know how to set foot in.
AI is only the rider
So far, neuroscience and psychology. Now the turn. Look at a contemporary AI. Look at any of them: a language model, a recommendation system, an automatic candidate evaluator, a conversational assistant. What's inside is calculation. A loss function (the equation the system tries to minimise during training, measuring how far the predicted answer is from the correct one), gradients, probability distributions over tokens (the minimal units of text the model manipulates, usually fragments of words) or over classes.
It's the rider. It's only the rider. It's a huge rider, in some cases astonishingly competent, with a capacity to process variables the human rider will never have. But it's only that.
It has no elephant. It has no body that sweats at the palms before knowing why. It has no biography that left somatic markers calibrated over thirty years. It has no subcortical route firing in milliseconds. It has no stake: it has nothing riding on what it decides, there's no consequence the system has paid or will pay. It has the apparatus for writing the report. It doesn't have the apparatus for making the decision the report is going to justify.
This isn't a technical defect fixed with more parameters. It's structural. Stuart Russell, in Human Compatible (2019), criticised the idea that it's enough to define the utility function better for the optimiser to converge on sensible decisions. It doesn't converge, because the utility functions a human really uses aren't functions: they're bodily levers with a biography, calibrated by evolution and experience, that weigh things calculation doesn't know how to weigh. Kate Crawford, in Atlas of AI (2021), tackled the other side: AI is a system without a body, and that's why the decisions it makes about specific bodies tend to carry an abstract bias its designers are slow to see.
What gets outsourced when you delegate
When a human decides about another human, part of the process is calculation and part is elephant. The elephant part includes the unease of signing a layoff for a person with small children, the filter that rejects a CV that meets every requirement but reeks of something off, the pause before denying an aid that in the cold seems deniable and in the warm seems criminal. It's what averts formally correct, materially repugnant decisions. It doesn't always avert them. But it averts them often enough for societies to work.
When that decision is delegated to a system that has only a rider, what's lost isn't precision. What's lost is the brake.
The system computes the optimal option according to the declared utility function and executes it. If the utility function was badly specified, it executes an atrocity with perfect consistency. The elephant, in humans, is what breaks the chain at that point: the human signs, looks at the result and something turns inside, and the next time does it differently. Without an elephant that feedback doesn't exist. The consistency is total. So is the error.
The market argument replies that you can put constraints into the loss function to emulate what the elephant does. And it's true you can put in constraints. What you can't do is calibrate them the way a hundred million years of evolution and thirty of biography in a specific body calibrated them. Each artificial constraint is a poor approximation to a lever that in humans was natural. Each patch reproduces badly what it was meant to replace, and at the same time introduces the patch's own problems.
The casting we've been describing wrong for centuries
Western culture has spent two thousand five hundred years telling a story where reason was the noble subject and emotion the animal to be tamed. Plato had already written it that way. Descartes closed it in a sentence. Kant solidified it. And the neuroscience of the last three decades has been showing, with clinical data, with functional imaging, with behavioural experiments, that the casting was exactly the reverse. Emotion decides and reason narrates. The elephant chooses and the rider writes the report.
It would have been an uncomfortable discovery even if AI didn't exist. It's more so now, because it turns out the technology we're building to make decisions is precisely the other half on its own. Plenty of rider and no elephant. Plenty of calculation and no lever. Plenty of consistency and no pause.
When someone tells you an AI is going to make better decisions than a human because it isn't contaminated by emotion, you now know what to answer. It lacks exactly the part that in humans did the deciding. What it has in surplus is the part that wrote the report. So ask it to decide something important, and watch what happens when all it has is the pencil.
Definiciones
Somatic marker. A bodily trace — sweat, muscle tension, unease, attraction — that prior experience has tied to an option and that the organism fires before conscious deliberation. Central concept of Damasio's hypothesis: without that bodily signal, calculation doesn't converge into a sensible decision.
Ventromedial prefrontal cortex. A region of the brain located just behind the forehead and above the eye sockets, connecting abstract reasoning with emotional and bodily signals. Its lesion leaves intelligence intact and ruins the capacity to decide in real life.
Iowa Gambling Task. An experimental test designed by Bechara and Damasio in 1994 with four decks of cards, two safe and two trap, to measure whether the subject develops the anticipatory bodily response that makes them avoid the bad option before being able to reason it out.
System 1 and system 2. Kahneman's scheme for describing two coexisting modes of thinking: 1 is fast, automatic, emotional and cheap; 2 is slow, deliberative, costly and tires. The 1 makes almost all the decisions; the 2 writes the justification.
Loss function. The equation a machine-learning system tries to minimise during training, measuring how far the predicted output is from the desired one. It's the system's only "compass" and that's why a badly defined function produces behaviours that are consistent and disastrous at once.
Tokens. The minimal units into which a language model breaks the text it receives and produces. They aren't whole words but fragments of words; the model assigns probabilities over the next token, not over the next idea.
Amygdala. An almond-shaped brain nucleus, lodged in the temporal lobe, in charge of processing the emotional response — especially fear — before the cortex has identified the stimulus.
Referencias
Bechara, A., Damasio, A. R., Damasio, H. and Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15. Original study introducing the Iowa Gambling Task and showing the dissociation between intact reasoning and ruined decision.
Damasio, A. (1994). Descartes' Error: Emotion, Reason, and the Human Brain. Putnam. Founding work of the somatic marker hypothesis, cited to describe the clinical picture of orbitofrontal patients.
LeDoux, J. (1996). The Emotional Brain: The Mysterious Underpinnings of Emotional Life. Simon & Schuster. Source for the description of the two routes of fear, the fast subcortical one and the slow cortical one.
Haidt, J. (2006). The Happiness Hypothesis: Finding Modern Truth in Ancient Wisdom. Basic Books. Origin of the contemporary use of the Hindu proverb of the elephant and the rider.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. Reference for the formulation of system 1 and system 2 and for the idea that rational deliberation is usually after-the-fact rationalisation.
Bechara, A. and Damasio, A. R. (2005). The Somatic Marker Hypothesis: A Neural Theory of Economic Decision. Games and Economic Behavior 52. Available at https://web.stanford.edu/~jlmcc/papers/BecharaEtAl05_TiCS.pdf. Late formulation of the somatic marker hypothesis applied to economic decision.
Russell, S. (2019). Human Compatible: Artificial Intelligence and the Problem of Control. Viking. Critique of the view of AI as a pure optimiser of a well-defined utility function, cited when discussing why patches to the loss function don't replace the elephant.
Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. Analysis of AI as a system without a body and of the political consequences of delegating to it decisions that fall on specific bodies.
Para profundizar
Haidt, J. (2012). The Righteous Mind. Why Good People Are Divided by Politics and Religion. Pantheon. Application of the elephant-and-rider metaphor to the moral and political terrain, where the asymmetry becomes especially visible.
Sapolsky, R. (2017). Behave. The Biology of Humans at Our Best and Worst. Penguin Press. A reading of human behaviour at several biological scales (seconds, days, years, evolution) that grounds in data the weight of the "elephant" over the "rider".
También te interesa
- What an emotion is, and why nobody wants to say it out loud
- States of mind. No test measures the same person twice
- The brain as a non-deterministic system. The noise was ours, not theirs
No comments yet
No comments yet. Be the first.
Leave a comment