cover
MORAL SCIENCES FOR THE MODERN WORLD · Book 1 of 25
THE HUMAN MIND

THE PSYCHOLOGY OF HUMAN MISJUDGMENT

Why smart people make terrible decisions

“Being clever is not the same as being careful.”
★ Ages 7+ ★
Portrait of Siri
Siri
Portrait of Charlie
Charlie
Portrait of Warren
Warren
CHAPTER 1

The Bridge That Learned to Dance

It was the first new bridge across that river in more than a hundred years.
It was the first new bridge across that river in more than a hundred years.
1A BRIDGE THAT PASSED EVERY TEST
Within minutes the deck began to swing sideways. People stopped and grabbed the rail.
Within minutes the deck began to swing sideways. People stopped and grabbed the rail.
Nobody planned it. Step by step, a whole crowd began walking in time.
Nobody planned it. Step by step, a whole crowd began walking in time.
Siri watched from the bank and wrote one word in her notebook. Why?
Siri watched from the bank and wrote one word in her notebook. Why?
It is brand new. It cannot be wrong.
Two days after it opened, the most famous new bridge in the country was shut.
Two days after it opened, the most famous new bridge in the country was shut.
SIRI’S NOTEBOOK

My first idea: somebody was careless. Cheap steel, or a sum done wrong. That is what I expect when something new goes wrong on day one.

Her first idea was the ordinary one. Something must have been built too weak.
Her first idea was the ordinary one. Something must have been built too weak.
But the tests had all passed, with room to spare. On paper the bridge was very strong.
But the tests had all passed, with room to spare. On paper the bridge was very strong.
So the engineers tested it again — this time with two thousand volunteers walking across.
So the engineers tested it again — this time with two thousand volunteers walking across.
Watch their feet, not the steel.
The bridge was never close to breaking. It was swaying, sideways, by a few centimetres.
The bridge was never close to breaking. It was swaying, sideways, by a few centimetres.
WORTH REMEMBERING

The bridge was strong enough the whole time. Nothing about it was weak. The problem was a question nobody had asked: what do people do when the floor moves?

When a floor moves sideways, people widen their steps to keep balance. They push back.
When a floor moves sideways, people widen their steps to keep balance. They push back.
Each push made the swing bigger. A bigger swing made more people push. A loop.
Each push made the swing bigger. A bigger swing made more people push. A loop.
THE STORY IN TIME
1996

A competition is held for a new footbridge over the River Thames in London.

10 Jun 2000

The Millennium Bridge opens. Around 90,000 people cross on the first day.

12 Jun 2000

It is closed after two days because of the sideways sway.

2000-01

Tests with volunteers identify synchronous lateral excitation.

22 Feb 2002

The bridge reopens with dampers fitted underneath. It has been steady since.

The repair was dampers — dozens of them, bolted underneath to soak up the movement.
The repair was dampers — dozens of them, bolted underneath to soak up the movement.
Twenty months later the bridge reopened. It has been steady ever since.
Twenty months later the bridge reopened. It has been steady ever since.
Charlie found her still staring at it.
Charlie found her still staring at it.
So the engineers made a mistake.
Did they? Which calculation was wrong?
None of them. Every sum was right. What was missing was not an answer. It was a question.
None of them. Every sum was right. What was missing was not an answer. It was a question.
SIRI’S NOTEBOOK

Nobody was lazy. Nobody was stupid. They tested the bridge for weight and for wind, because those are the things bridges are tested for. So: what makes a question invisible?

CHAPTER 2

The Puzzle With One Rule

Charlie laid three numbers on the table. Two, four, six. They followed a rule he knew.
Charlie laid three numbers on the table. Two, four, six. They followed a rule he knew.
2GUESS MY RULE
Siri could test as many sets of three as she liked. He would only answer yes or no.
Siri could test as many sets of three as she liked. He would only answer yes or no.
She had it in four seconds.
She had it in four seconds.
Eight, ten, twelve. Add two each time.
Yes. That set fits the rule.
So she tried again. Twenty, twenty-two, twenty-four.
So she tried again. Twenty, twenty-two, twenty-four.
Yes. A hundred, a hundred and two, a hundred and four. Yes. Yes. Yes.
Yes. A hundred, a hundred and two, a hundred and four. Yes. Yes. Yes.
PAUSE & THINK

Siri has had seven yeses in a row and she is about to be told she is wrong. Before you read on: what is the one kind of set she has not tried even once?

The table filled with accepted guesses. She announced her rule. It was wrong.
The table filled with accepted guesses. She announced her rule. It was wrong.
Every single test had said yes. She had never been closer to the answer, or further from it.
Every single test had said yes. She had never been closer to the answer, or further from it.
Look at what she collected. A card full of yeses. It proves almost nothing.
Look at what she collected. A card full of yeses. It proves almost nothing.
She had never once offered a set she expected him to refuse. Not once.
She had never once offered a set she expected him to refuse. Not once.
WORTH REMEMBERING

The rule was any three numbers that go up. One, two, ninety. Three, four, five. Siri’s rule fitted every test she ran — because she only ever ran tests it would pass.

The real rule was enormous. She had been defending a tiny corner of it.
The real rule was enormous. She had been defending a tiny corner of it.
Charlie would not tell her whether her new rule was right either.
Charlie would not tell her whether her new rule was right either.
Just tell me if I have it now.
What would you have to try to be proved wrong?
She had been guarding one small pen in the middle of an open field.
She had been guarding one small pen in the middle of an open field.
Looking harder at the same place is not the same as looking somewhere else.
Looking harder at the same place is not the same as looking somewhere else.
The door she never opened was the one with the answer behind it.
The door she never opened was the one with the answer behind it.
Hunting for yeses had felt exactly like investigating. That was the whole trap.
Hunting for yeses had felt exactly like investigating. That was the whole trap.
It felt like checking. It really did.
SIRI’S NOTEBOOK

A hundred years later this has a name: confirmation bias. Looking for what would agree with me, and calling it evidence. Next question: if it feels the same from the inside, how would I ever catch myself doing it?

Five roads out of one crossing. Every test she ran had gone down the same one.
Five roads out of one crossing. Every test she ran had gone down the same one.
CHAPTER 3

The Perfect Wrong Mirror

It was the smoothest large mirror anyone had ever made. That part is still true.
It was the smoothest large mirror anyone had ever made. That part is still true.
3THE SHARPEST EYE EVER BUILT
A telescope the size of a bus, built to look at the sky from above the blur of the air.
A telescope the size of a bus, built to look at the sky from above the blur of the air.
The first pictures came back like this. Every star a small smudge of light.
The first pictures came back like this. Every star a small smudge of light.
This is what the same stars should have looked like.
This is what the same stars should have looked like.
Siri had an ordinary explanation ready.
Siri had an ordinary explanation ready.
Something must have broken on the way up.
SIRI’S NOTEBOOK

A rocket launch is violent. Things shake loose. That is the boring answer and boring answers are usually right — so I looked for what had come loose.

Nothing had come loose. The telescope was exactly where it should be, working perfectly.
Nothing had come loose. The telescope was exactly where it should be, working perfectly.
The mirror was measured again. And again. Its shape was beautiful. It was also wrong.
The mirror was measured again. And again. Its shape was beautiful. It was also wrong.
Wrong by about two thousandths of a millimetre at the rim. A fiftieth the width of a hair.
Wrong by about two thousandths of a millimetre at the rim. A fiftieth the width of a hair.
The cause was a measuring rod. A small worn patch let a laser bounce off the wrong surface.
The cause was a measuring rod. A small worn patch let a laser bounce off the wrong surface.
WORTH REMEMBERING

The instrument used to check the mirror was itself out of true by about 1.3 millimetres. So the mirror was polished, very carefully and very accurately, to the wrong shape.

Two other instruments had been used as well. Both said the mirror was wrong.
Two other instruments had been used as well. Both said the mirror was wrong.
Both results were set aside. The main instrument was the trusted one, so it was believed.
Both results were set aside. The main instrument was the trusted one, so it was believed.
Which measurement did they want to be right?
Three years later a crew flew up and fitted corrective optics — spectacles for a telescope.
Three years later a crew flew up and fitted corrective optics — spectacles for a telescope.
It has been sending back the sharpest pictures ever taken from above the air ever since.
It has been sending back the sharpest pictures ever taken from above the air ever since.
Siri was furious on behalf of the tests nobody read.
Siri was furious on behalf of the tests nobody read.
The warning was right there!
It was. Why was the other answer nicer?
Because it agreed. Chapter two, wearing a lab coat and a budget of a billion dollars.
Because it agreed. Chapter two, wearing a lab coat and a budget of a billion dollars.
THE STORY IN TIME
1979-81

The primary mirror is ground and polished.

24 Apr 1990

The telescope is launched.

Jun 1990

The images are blurred. The mirror is found to be the wrong shape.

Nov 1990

An investigation reports that two other tests had shown the error and were discounted.

Dec 1993

A servicing crew installs corrective optics. The telescope works as intended.

CHAPTER 4

The Seam Nobody Owned

Nine months of flight. Two hundred million kilometres. Everything working.
Nine months of flight. Two hundred million kilometres. Everything working.
4A PERFECT FLIGHT, AND THEN NOTHING
The spacecraft went behind the planet to slow down. It never came out the other side.
The spacecraft went behind the planet to slow down. It never came out the other side.
The path had been almost perfect for nine months. It drifted only at the very end.
The path had been almost perfect for nine months. It drifted only at the very end.
Siri went for the obvious cause.
Siri went for the obvious cause.
The engine failed. It has to be the engine.
It was not the engine. Two teams had been measuring the same push in two different units.
It was not the engine. Two teams had been measuring the same push in two different units.
PAUSE & THINK

One team wrote the small nudges from the thrusters in pound-force seconds. The other read them as newton-seconds. One pound-force is about 4.45 newtons. If every nudge is counted as about a quarter of its real size, where does the spacecraft think it is?

Two jugs. Both called one measure. One holds about four and a half times the other.
Two jugs. Both called one measure. One holds about four and a half times the other.
The error was tiny each time and it added up for nine months. The path came in far too low.
The error was tiny each time and it added up for nine months. The path came in far too low.
Navigators had noticed something odd. They said so. There was no formal place for it to go.
Navigators had noticed something odd. They said so. There was no formal place for it to go.
Whose job was it to be worried?
A worry that is mentioned but never written into the process is a worry that vanishes.
A worry that is mentioned but never written into the process is a worry that vanishes.
WORTH REMEMBERING

The planned closest approach was about 226 km above the surface. The spacecraft came in at roughly 57 km — far below the level anything could survive.

Two teams. Two buildings. Each doing its own job correctly, inside its own walls.
Two teams. Two buildings. Each doing its own job correctly, inside its own walls.
The failure lived in the join between them. And the join belonged to nobody.
The failure lived in the join between them. And the join belonged to nobody.
Siri wanted a name to blame. Charlie would not give her one.
Siri wanted a name to blame. Charlie would not give her one.
Somebody must have got it wrong.
Name them. Which person made an error?
A long chain of correct steps, with one open link nobody was standing next to.
A long chain of correct steps, with one open link nobody was standing next to.
The fix afterwards was dull and powerful. A check that crosses the wall between the rooms.
The fix afterwards was dull and powerful. A check that crosses the wall between the rooms.
A system can fail in a place where no single person failed. Siri wrote that down twice.
A system can fail in a place where no single person failed. Siri wrote that down twice.
WHAT EACH SIDE BELIEVED · AND WHAT WAS TRUE

🧭 WHAT THE FIRST TEAM BELIEVED

The file we send out is in the units our own software has always used.

🧮 WHAT THE SECOND TEAM BELIEVED

The file we receive is in metric units, because the mission standard says so.

⚠️ WHAT WAS ACTUALLY TRUE

Both were right about their own half. Nobody owned the sentence between them.

🔧 WHAT CHANGED AFTERWARD

Checks that deliberately cross team boundaries, and a route for a small worry to travel.

A chain is checked at its links. Somebody has to be standing in the doorway.
A chain is checked at its links. Somebody has to be standing in the doorway.
CHAPTER 5

The Line That Wasn't

A card with three lines on it. One of them plainly matches the line beside it.
A card with three lines on it. One of them plainly matches the line beside it.
5AN EASY QUESTION, ASKED OUT LOUD
Everyone answers out loud, one at a time, going round the room. You are last.
Everyone answers out loud, one at a time, going round the room. You are last.
The first person picks a line that is obviously too short. So does the second. And the third.
The first person picks a line that is obviously too short. So does the second. And the third.
Siri knew exactly what was going on here.
Siri knew exactly what was going on here.
They are being polite. Nobody believes that.
But the question is not hard. Alone, on paper, almost nobody gets it wrong.
But the question is not hard. Alone, on paper, almost nobody gets it wrong.
SIRI’S NOTEBOOK

The others in the room were in on it. Only the last person was really being asked. Alone, people got it right nearly every time. In the group, about three quarters went along with the wrong answer at least once.

A wall of agreement, all of it pointing at the wrong line, all of it perfectly confident.
A wall of agreement, all of it pointing at the wrong line, all of it perfectly confident.
Some went along to avoid standing out. Some began to doubt their own eyes.
Some went along to avoid standing out. Some began to doubt their own eyes.
Alone, with nobody listening, the same people were right about ninety-nine times in a hundred.
Alone, with nobody listening, the same people were right about ninety-nine times in a hundred.
Then one change. Give the last person a single other voice that disagrees.
Then one change. Give the last person a single other voice that disagrees.
WORTH REMEMBERING

With one other person disagreeing, going along with the crowd collapsed to a small fraction of what it had been. Not a big campaign. One voice.

A flock turns together without any bird deciding to lead. That is not a plan. It is copying.
A flock turns together without any bird deciding to lead. That is not a plan. It is copying.
One bird leaves the flock, and the sky does not fall in.
One bird leaves the flock, and the sky does not fall in.
Charlie asked the question she was avoiding.
Charlie asked the question she was avoiding.
Anyone would have spoken up. I would.
What would it cost you to be the only one?
That is the honest question. Not what is true. What is expensive to say.
That is the honest question. Not what is true. What is expensive to say.
Counting the voices is not the same as weighing the evidence.
Counting the voices is not the same as weighing the evidence.
Agreement is not evidence. It can just be everyone copying everyone.
Agreement is not evidence. It can just be everyone copying everyone.
SIRI’S NOTEBOOK

This one has a name too: social proof. The useful bit is not that people copy. It is how cheap the cure was. One other voice. So: who in a room is allowed to be that voice?

CHAPTER 6

The Wheel That Changed the Answer

A wheel of fortune. Plain coloured segments. It has nothing to do with anything.
A wheel of fortune. Plain coloured segments. It has nothing to do with anything.
6A NUMBER FROM NOWHERE
You spin it. Everyone watches it slow down and stop. Then you are asked a question.
You spin it. Everyone watches it slow down and stop. Then you are asked a question.
The question is about the world: what percentage of countries in the UN are African?
The question is about the world: what percentage of countries in the UN are African?
Siri was completely certain the wheel could not possibly matter.
Siri was completely certain the wheel could not possibly matter.
It is a wheel. It knows nothing.
Everybody writes their answer on a blank card. Nobody is rushed. Nobody is tricked.
Everybody writes their answer on a blank card. Nobody is rushed. Nobody is tricked.
WHAT WOULD YOU DO?

You are selling a bicycle. A stranger asks what you want for it. You could name a price first, ask them to name one first, or say you have not decided. Which do you choose, and what does each one give away? There is no obvious right answer here.

The wheel was rigged. For half the people it stopped at 10. For the other half, at 65.
The wheel was rigged. For half the people it stopped at 10. For the other half, at 65.
The group who saw 10 answered, on average, about 25 percent. The group who saw 65: about 45.
The group who saw 10 answered, on average, about 25 percent. The group who saw 65: about 45.
Same question. Same people. A meaningless number changed the answer by twenty points.
Same question. Same people. A meaningless number changed the answer by twenty points.
It works like an anchor. Wherever it drops, that is where your thinking is moored.
It works like an anchor. Wherever it drops, that is where your thinking is moored.
WORTH REMEMBERING

They knew the wheel was random. They watched it spin. It moved them anyway. Knowing about an anchor does not cut the rope.

Now think about a price tag. Somebody chose that first number, and they chose it on purpose.
Now think about a price tag. Somebody chose that first number, and they chose it on purpose.
Charlie let her sit with that.
Charlie let her sit with that.
So a shop is a rigged wheel.
Who spins it? And who profits from the spin?
Every stall opens with a number, and every haggle starts from wherever that number landed.
Every stall opens with a number, and every haggle starts from wherever that number landed.
Without a scale of your own, you will use whichever one you are handed.
Without a scale of your own, you will use whichever one you are handed.
The cure is unglamorous: work out your own number before you look at theirs.
The cure is unglamorous: work out your own number before you look at theirs.
Siri had been sure a random number could not move her. It had moved her by twenty points.
Siri had been sure a random number could not move her. It had moved her by twenty points.
SIRI’S NOTEBOOK

Name for the file: anchoring. And a rule I can actually use — decide what a thing is worth to me before anybody tells me what they want for it.

On dry land you can see exactly how heavy an anchor is. In the water you only feel it.
On dry land you can see exactly how heavy an anchor is. In the water you only feel it.
CHAPTER 7

Everyone Is in the Better Half

An ordinary road. Ordinary drivers. Everyone paying attention, more or less.
An ordinary road. Ordinary drivers. Everyone paying attention, more or less.
7A SURVEY WITH AN IMPOSSIBLE ANSWER
Ask every driver on this road one question. Are you safer than the average driver here?
Ask every driver on this road one question. Are you safer than the average driver here?
In one well-known study, close to nine in ten put themselves in the better half.
In one well-known study, close to nine in ten put themselves in the better half.
Siri checked the arithmetic twice, because it cannot be right.
Siri checked the arithmetic twice, because it cannot be right.
Nine in ten cannot be in the top half.
Half must be below the middle. That is what the middle means. It is not an opinion.
Half must be below the middle. That is what the middle means. It is not an opinion.
PAUSE & THINK

Write down a number you are 98 percent sure the truth falls inside — how many kilometres it is from London to Sydney, say. Give a lowest and a highest. Now: how often do you think a range like that actually contains the answer? Write your guess down first.

The answers pile up on one side of a see-saw that has to balance. Somebody is wrong.
The answers pile up on one side of a see-saw that has to balance. Somebody is wrong.
In one classic study people gave ranges they were 98 percent sure of.
In one classic study people gave ranges they were 98 percent sure of.
The true answer fell outside those ranges about four times in ten. Not two in a hundred.
The true answer fell outside those ranges about four times in ten. Not two in a hundred.
Tight, consistent, confident — and nowhere near the middle of the target.
Tight, consistent, confident — and nowhere near the middle of the target.
WORTH REMEMBERING

Being sure is a feeling. Being right is a measurement. They come from completely different places, and only one of them can be checked.

Confidence is usually highest exactly where the ground stops being visible.
Confidence is usually highest exactly where the ground stops being visible.
One more sure step is how a very sure person arrives at the edge.
One more sure step is how a very sure person arrives at the edge.
SIRI’S NOTEBOOK

Nobody in that survey was lying. Almost everyone can think of a moment they drove carefully. Almost nobody can watch themselves being careless — you are busy driving.

Charlie asked the only question that helps here.
Charlie asked the only question that helps here.
I am usually right about this sort of thing.
When did anybody last check that?
A needle that always points somewhere is not the same as a needle that is right.
A needle that always points somewhere is not the same as a needle that is right.
So the cure is boring. Write the guess down first, with a date. Then go and look.
So the cure is boring. Write the guess down first, with a date. Then go and look.
A page of predictions you can be marked against later. It is the only honest scoreboard.
A page of predictions you can be marked against later. It is the only honest scoreboard.
WORTH REMEMBERING

You cannot feel your way to knowing how good your judgment is. You have to keep score — and the score has to be written down before the answer arrives.

CHAPTER 8

The Ward Next Door

One hospital corridor. Two doors. Two clinics, taking mothers in turn, day about.
One hospital corridor. Two doors. Two clinics, taking mothers in turn, day about.
8TWO DOORS, TWO VERY DIFFERENT RECORDS
Mothers begged to be sent to the second clinic. Some gave birth in the street instead.
Mothers begged to be sent to the second clinic. Some gave birth in the street instead.
The second clinic was staffed by midwives. The first, by doctors and medical students.
The second clinic was staffed by midwives. The first, by doctors and medical students.
Siri read the two columns and read them again.
Siri read the two columns and read them again.
The better-trained ward is the worse one?
Somebody was already counting. The difference was written down, every single month.
Somebody was already counting. The difference was written down, every single month.
SIRI’S NOTEBOOK

In one bad year about one mother in ten died in the doctors’ clinic. Next door, with midwives, it was closer to one in twenty-five. My first guess was that the doctors there were simply worse. They were the best-trained people in the building.

A young doctor noticed one difference. The doctors came straight from the autopsy room.
A young doctor noticed one difference. The doctors came straight from the autopsy room.
The midwives did not. That was, very nearly, the whole of the difference.
The midwives did not. That was, very nearly, the whole of the difference.
From May 1847 he made everyone wash their hands in a chlorine solution first.
From May 1847 he made everyone wash their hands in a chlorine solution first.
Within a year the deaths in his clinic had fallen to around one in a hundred.
Within a year the deaths in his clinic had fallen to around one in a hundred.
WORTH REMEMBERING

The evidence was counted, written down and published. It still took the profession decades to accept it. Not because doctors were wicked. Because the fact accused them.

One curve drops steeply. The other stays roughly where it always was.
One curve drops steeply. The other stays roughly where it always was.
This is not a story about villains. Nobody yet knew that germs existed.
This is not a story about villains. Nobody yet knew that germs existed.
THE STORY IN TIME
1846

In Vienna, the clinic staffed by doctors loses far more mothers than the one staffed by midwives. The gap is recorded monthly.

Mar 1847

A colleague dies after a scalpel wound received during an autopsy, with symptoms like the mothers'.

May 1847

Handwashing in a chlorine solution is made compulsory in the clinic.

1848

Deaths in that clinic fall to about one in a hundred.

1861

He publishes the full results, fifteen years after the observation.

1860s-70s

Germ theory gives the explanation he never had, and the practice spreads.

Siri was angry that nobody listened. Charlie made it harder.
Siri was angry that nobody listened. Charlie made it harder.
Why would anyone argue with that?
What would believing it have cost them?
It would have meant accepting they had been carrying the cause on their own hands.
It would have meant accepting they had been carrying the cause on their own hands.
SIRI’S NOTEBOOK

He also could not say why washing worked, and he was not gentle with the people he needed to persuade. Being right is not the same as being convincing. Both things are true at once, and neither of them makes anybody a villain.

The hardest fact to see is the one that is about you.
The hardest fact to see is the one that is about you.
A fact can be true, checkable, written down — and still be expensive to believe.
A fact can be true, checkable, written down — and still be expensive to believe.
WORTH REMEMBERING

Ask of any uncomfortable fact: what would it cost me to accept this? The bigger the answer, the harder you should look at your reasons for rejecting it.

A basin, a cloth, and a fact the profession took decades to accept.
A basin, a cloth, and a fact the profession took decades to accept.
CHAPTER 9

The Praise That Seemed to Backfire

Flight instructors, teaching landings, watching the same pilots week after week.
Flight instructors, teaching landings, watching the same pilots week after week.
9A RULE EVERY INSTRUCTOR BELIEVED
Praise a pilot for a beautiful landing, they said, and the next one is worse.
Praise a pilot for a beautiful landing, they said, and the next one is worse.
Criticise a bad one sharply, they said, and the next one improves. Every time.
Criticise a bad one sharply, they said, and the next one improves. Every time.
Siri had a tidy explanation ready within seconds.
Siri had a tidy explanation ready within seconds.
Praise makes people relax and get sloppy.
So test it on something that cannot hear a word you say. Two dice.
So test it on something that cannot hear a word you say. Two dice.
PAUSE & THINK

Roll two dice and write the total down. Suppose you roll a 12 — the highest there is. Now praise the dice warmly and roll again. What will most likely happen to the total? And what if you had shouted at them instead?

Roll. Write it down. Roll again. Write it down. Keep going for twenty rolls.
Roll. Write it down. Roll again. Write it down. Keep going for twenty rolls.
After a 12 the next roll is almost always lower. Praise or no praise.
After a 12 the next roll is almost always lower. Praise or no praise.
After a 2, the next roll is almost always higher. Shouting is not required.
After a 2, the next roll is almost always higher. Shouting is not required.
Extreme results are rare. Ordinary ones are common. So an extreme is usually followed by one.
Extreme results are rare. Ordinary ones are common. So an extreme is usually followed by one.
WORTH REMEMBERING

The instructors were watching something real. Their explanation was the problem. The world seemed to punish praise and reward criticism — by arithmetic alone.

One sunflower stands far above the row this season.
One sunflower stands far above the row this season.
Next season the row is much more even. Nobody did anything to it.
Next season the row is much more even. Nobody did anything to it.
Charlie took away the last of her comfort.
Charlie took away the last of her comfort.
So the praise did nothing at all?
What would they have seen if they had said nothing?
One unusually high bounce.
One unusually high bounce.
Then a lower one. The ball was not told off in between.
Then a lower one. The ball was not told off in between.
We invent causes for patterns that are only arithmetic. Then we teach them to other people.
We invent causes for patterns that are only arithmetic. Then we teach them to other people.
SIRI’S NOTEBOOK

Name for the file: regression to the mean. Frightening thought — how many of the rules people live by are arithmetic wearing a costume? Next question: what else does my mind count badly?

CHAPTER 10

The Dangers We Can Picture

Ask people which is more common. Almost everyone can picture this one instantly.
Ask people which is more common. Almost everyone can picture this one instantly.
10WHAT THE MIND COUNTS
Now picture the other kind of danger. Nothing to see. Nothing to picture. Nothing at all.
Now picture the other kind of danger. Nothing to see. Nothing to picture. Nothing at all.
One of them gets printed. The other happens far more often and never makes a page.
One of them gets printed. The other happens far more often and never makes a page.
Siri had a reasonable explanation, and it was wrong.
Siri had a reasonable explanation, and it was wrong.
People just have not seen the numbers.
The quiet jar is far fuller. It is also the one nobody ever looks at.
The quiet jar is far fuller. It is also the one nobody ever looks at.
SIRI’S NOTEBOOK

In a 1978 study people were asked to judge how often different things kill. Accidents were judged about as deadly as disease. Disease actually killed something like fifteen times more people. Same people, same year, same country.

Most of what happens to people happens quietly, in corridors like this one.
Most of what happens to people happens quietly, in corridors like this one.
A calm river is remembered for the one year it flooded, and for nothing else.
A calm river is remembered for the one year it flooded, and for nothing else.
The mind does not survey the field. It searches where the light already is.
The mind does not survey the field. It searches where the light already is.
Near and vivid looks bigger than far and ordinary. That is not a fault of the eyes.
Near and vivid looks bigger than far and ordinary. That is not a fault of the eyes.
WORTH REMEMBERING

The mind does not count events. It counts how easily an example comes to mind — and then reports that feeling back to you as if it were a number.

One red pebble is remembered. A thousand grey ones are not.
One red pebble is remembered. A thousand grey ones are not.
Charlie asked the question that turns the chapter around.
Charlie asked the question that turns the chapter around.
So we are just bad at counting.
Who chooses which pictures reach you?
Memory keeps an album. It does not keep an even sample of the world.
Memory keeps an album. It does not keep an even sample of the world.
And whoever fills the album is quietly deciding what you will be afraid of.
And whoever fills the album is quietly deciding what you will be afraid of.
Counting the jar gives a completely different answer from remembering it.
Counting the jar gives a completely different answer from remembering it.
So Siri added a rule. When something feels common, go and find the count.
So Siri added a rule. When something feels common, go and find the count.
WHAT WE PICTURE · AND WHAT ACTUALLY HAPPENS MOST

📰 WHAT COMES TO MIND FIRST

The rare, sudden, dramatic event. It is memorable precisely because it is rare.

📊 WHAT THE COUNT ACTUALLY SAYS

The quiet, slow, ordinary cause, many times over, almost never reported.

🧠 WHY THE GAP OPENS

The mind measures how easily an example arrives, not how often the thing happens.

🔍 THE CHECK THAT CLOSES IT

Ask for the number. If nobody has the number, treat the feeling as a guess.

Thousands of grey beads. One red one. Only the red one is ever remembered.
Thousands of grey beads. One red one. Only the red one is ever remembered.
CHAPTER 11

Ten Mysteries, One Shape

Ten cases. Different centuries, different countries, different jobs entirely.
Ten cases. Different centuries, different countries, different jobs entirely.
11LOOKING FOR THE PATTERN
A bridge. A number puzzle. A mirror. A spacecraft. A room of chairs. A wheel.
A bridge. A number puzzle. A mirror. A spacecraft. A room of chairs. A wheel.
A survey of drivers. A hospital corridor. A flight school. A jar of beads.
A survey of drivers. A hospital corridor. A flight school. A jar of beads.
Siri pinned them all up and drew string between them. Nothing came into focus.
Siri pinned them all up and drew string between them. Nothing came into focus.
Then she offered Charlie the tidy version, and he would not take it.
Then she offered Charlie the tidy version, and he would not take it.
People are careless. That is the pattern.
Careless? Show me the careless one.
SIRI’S NOTEBOOK

I went back through my own board. Bridge engineers who ran every test. Opticians who measured a mirror three ways. Navigators who noticed and said so. Doctors who counted deaths every month. Not one careless person on the whole board.

Ten different centuries, ten different fields. And the same shape, over and over.
Ten different centuries, ten different fields. And the same shape, over and over.
Looking closer at one case is not the same as looking across all ten.
Looking closer at one case is not the same as looking across all ten.
Charlie gave her his one tool, which is not an answer.
Charlie gave her his one tool, which is not an answer.
How did each of them fail?
Wrong way round. How could each one have failed?
Turn the question round. Do not ask how it succeeds. Ask how it could break.
Turn the question round. Do not ask how it succeeds. Ask how it could break.
WORTH REMEMBERING

Every single case had clever, careful, well-equipped people in it. That is not a coincidence in the stories. It is the finding.

From the top of the ladder, everything looks perfectly safe.
From the top of the ladder, everything looks perfectly safe.
From the bottom you can see the foot of it is standing on something smooth.
From the bottom you can see the foot of it is standing on something smooth.
INTELLIGENCE · AND JUDGMENT

🧠 INTELLIGENCE

Solving the problem in front of you, quickly, and being right about it.

⚖️ JUDGMENT

Knowing which problem you are actually looking at, and what you are assuming.

🚫 WHY BEING CLEVER IS NOT ENOUGH

A quick mind builds a better argument for the wrong answer, not a slower one.

✅ WHAT ACTUALLY HELPS

A check you did not invent yourself, applied before you are sure.

And there it was. Every mistake on the board was a mistake somebody had made before.
And there it was. Every mistake on the board was a mistake somebody had made before.
Which means they can be written down. And a thing written down can be looked for.
Which means they can be written down. And a thing written down can be looked for.
SIRI’S NOTEBOOK

The list so far: am I hunting for yeses? What would prove me wrong? Whose number is this? Who else is in the room? What would it cost me to believe this? Is this a cause, or is it arithmetic? Where is the count?

You do not wear a helmet because you expect to fall. You wear it because you might.
You do not wear a helmet because you expect to fall. You wear it because you might.
She asked Charlie whether she had it right. Of course, he would not say.
She asked Charlie whether she had it right. Of course, he would not say.
Am I right, though? Just this once.
Ask someone who thinks in decades.
WORTH REMEMBERING

Intelligence and judgment are two different tools. Being good with the first one does nothing whatsoever to sharpen the second.

CHAPTER 12

And Then What?

The last afternoon of the investigation, and the longest walk of it.
The last afternoon of the investigation, and the longest walk of it.
12THE LONG VIEW
Siri had ten solved mysteries in her notebook and no answer to the question underneath.
Siri had ten solved mysteries in her notebook and no answer to the question underneath.
Charlie was no help, cheerfully.
Charlie was no help, cheerfully.
You never answer anything.
No. But I know who to walk you to.
Warren does not think in afternoons. He thinks in decades, which is a different shape.
Warren does not think in afternoons. He thinks in decades, which is a different shape.
He listened to all ten. Then he asked about the eleventh, which has not happened yet.
He listened to all ten. Then he asked about the eleventh, which has not happened yet.
So how do I stop making these mistakes?
You won't. What happens when you make one?
WHAT WOULD YOU DO?

You have a plan that works brilliantly ninety-nine days out of a hundred. On the hundredth day it loses everything you have. You could run it, run it smaller, run it with a reserve set aside, or not run it at all. What do you choose — and what does your answer assume about how many hundred-day runs you get?

A dam is built for far more water than anyone expects to arrive. That is the point of it.
A dam is built for far more water than anyone expects to arrive. That is the point of it.
The spare oars are dead weight on every trip except one.
The spare oars are dead weight on every trip except one.
Warren's question was not how to be right. It was how to still be here after being wrong.
Warren's question was not how to be right. It was how to still be here after being wrong.
That sounds like giving up on being right.
It is what lets you keep trying to be.
The bridge is built for a load that has not turned up yet. Room to be wrong in.
The bridge is built for a load that has not turned up yet. Room to be wrong in.
WORTH REMEMBERING

Good decisions do not guarantee good outcomes, and good outcomes do not prove good decisions. What a habit of checking does is improve the odds, and let you survive the times it does not.

A young tree is staked and guarded for years before it is asked to stand alone.
A young tree is staked and guarded for years before it is asked to stand alone.
What survives long enough gets to keep growing. Nothing else does.
What survives long enough gets to keep growing. Nothing else does.
THE STORY IN TIME
1847

A Vienna clinic starts washing hands, and the deaths fall.

1951

A room of people call a short line long, and one dissenting voice breaks the spell.

1960

A three-number puzzle shows people testing only what they hope will pass.

1974

A rigged wheel of fortune moves people's answers about the world.

1978

A survey finds people judge dangers by how easily they can picture them.

1981

Nearly nine in ten drivers place themselves in the better half.

1990

A telescope mirror is polished, accurately, to the wrong shape.

1999

Two units of measurement meet at a seam nobody owned.

2000

A new bridge sways because a crowd walks in time with it.

One more thing, he said. Money can usually be rebuilt. Trust is much slower.
One more thing, he said. Money can usually be rebuilt. Trust is much slower.
Because people remember?
Because they were counting all along.
Siri opened her notebook to a blank page, which is where investigations start.
Siri opened her notebook to a blank page, which is where investigations start.
SIRI’S NOTEBOOK

Something in every one of the ten cases that I have not written down yet. Somebody was paid to build the bridge. Somebody was paid to launch the telescope. Somebody was rewarded for agreeing. In not one single case was anybody paid to go and find the error.

One card left over at the end of the string. It is not an answer. It is the next question.
One card left over at the end of the string. It is not an answer. It is the next question.
Being clever is not the same as being careful. She had needed all twelve chapters for that.
Being clever is not the same as being careful. She had needed all twelve chapters for that.
WORTH REMEMBERING

Intelligence does not protect anybody from a predictable mistake. But a mistake that is predictable is a mistake you can go looking for on purpose — and that is a skill, not a gift.

She closed the notebook on ten answers, and opened it again on one question.
She closed the notebook on ten answers, and opened it again on one question.
THE MORAL SCIENCE MAP · Book 1
HOW IT HAPPENS, EVERY TIMEA CLEVER, CAREFUL PERSONAN ASSUMPTION NOBODY THINKS TO TESTEVIDENCE THAT DISAGREES IS EXPLAINED AWAYCONFIDENCE GROWS WITHOUT BEING CHECKEDA DECISION IS MADETHE SYSTEM CARRIES IT FURTHER THAN ONE PERSON COULDTHE CONSEQUENCE ARRIVESTHE MISTAKE IS GIVEN A NAMEA CHECK THAT CATCHES IT NEXT TIME
A glimpse of the next book in the series

NEXT: THE INCENTIVE TRAP

In every story Siri found, someone was paid to build, to launch, to agree. Nobody was paid to find the mistake. What happens to a number when a person's reward depends on it?

Book 2 · Show Me How You’re Paid