Tim Harford: Trial, error and the God complex

It's the Second World War.

A German prison camp.

And this man,

Archie Cochrane,

is a prisoner of war and a doctor,

and he has a problem.

The problem is that the men under his care

are suffering

from an excruciating and debilitating condition

that Archie doesn't really understand.

The symptoms

are this horrible swelling up of fluids under the skin.

But he doesn't know whether it's an infection, whether it's to do with malnutrition.

He doesn't know how to cure it.

And he's operating in a hostile environment.

And people do terrible things in wars.

The German camp guards, they've got bored.

They've taken to just firing into the prison camp at random

for fun.

On one particular occasion,

one of the guards threw a grenade into the prisoners' lavatory

while it was full of prisoners.

He said he heard suspicious laughter.

And Archie Cochrane, as the camp doctor,

was one of the first men in

to clear up the mess.

And one more thing:

Archie was suffering from this illness himself.

So the situation seemed pretty desperate.

But Archie Cochrane

was a resourceful person.

He'd already smuggled vitamin C into the camp,

and now he managed

to get hold of supplies of marmite

on the black market.

Now some of you will be wondering what marmite is.

Marmite is a breakfast spread beloved of the British.

It looks like crude oil.

It tastes ...

zesty.

And importantly,

it's a rich source

of vitamin B12.

So Archie splits the men under his care as best he can

into two equal groups.

He gives half of them vitamin C.

He gives half of them vitamin B12.

He very carefully and meticulously notes his results

in an exercise book.

And after just a few days,

it becomes clear

that whatever is causing this illness,

marmite is the cure.

So Cochrane then goes to the Germans who are running the prison camp.

Now you've got to imagine at the moment --

forget this photo, imagine this guy

with this long ginger beard and this shock of red hair.

He hasn't been able to shave -- a sort of Billy Connolly figure.

Cochrane, he starts ranting at these Germans

in this Scottish accent --

in fluent German, by the way, but in a Scottish accent --

and explains to them how German culture was the culture

that gave Schiller and Goethe to the world.

And he can't understand

how this barbarism can be tolerated,

and he vents his frustrations.

And then he goes back to his quarters,

breaks down and weeps

because he's convinced that the situation is hopeless.

But a young German doctor

picks up Archie Cochrane's exercise book

and says to his colleagues,

"This evidence is incontrovertible.

If we don't supply vitamins to the prisoners,

it's a war crime."

And the next morning,

supplies of vitamin B12 are delivered to the camp,

and the prisoners begin to recover.

Now I'm not telling you this story

because I think Archie Cochrane is a dude,

although Archie Cochrane is a dude.

I'm not even telling you the story

because I think we should be running

more carefully controlled randomized trials

in all aspects of public policy,

although I think that would also be completely awesome.

I'm telling you this story

because Archie Cochrane, all his life,

fought against a terrible affliction,

and he realized it was debilitating to individuals

and it was corrosive to societies.

And he had a name for it.

He called it the God complex.

Now I can describe the symptoms of the God complex very, very easily.

So the symptoms of the complex

are, no matter how complicated the problem,

you have an absolutely overwhelming belief

that you are infallibly right in your solution.

Now Archie was a doctor,

so he hung around with doctors a lot.

And doctors suffer from the God complex a lot.

Now I'm an economist, I'm not a doctor,

but I see the God complex around me all the time

in my fellow economists.

I see it in our business leaders.

I see it in the politicians we vote for --

people who, in the face of an incredibly complicated world,

are nevertheless absolutely convinced

that they understand the way that the world works.

And you know, with the future billions that we've been hearing about,

the world is simply far too complex

to understand in that way.

Well let me give you an example.

Imagine for a moment

that, instead of Tim Harford in front of you,

there was Hans Rosling presenting his graphs.

You know Hans:

the Mick Jagger of TED.

(Laughter)

And he'd be showing you these amazing statistics,

these amazing animations.

And they are brilliant; it's wonderful work.

But a typical Hans Rosling graph:

think for a moment, not what it shows,

but think instead about what it leaves out.

So it'll show you GDP per capita,

population, longevity,

that's about it.

So three pieces of data for each country --

three pieces of data.

Three pieces of data is nothing.

I mean, have a look at this graph.

This is produced by the physicist Cesar Hidalgo.

He's at MIT.

Now you won't be able to understand a word of it,

but this is what it looks like.

Cesar has trolled the database

of over 5,000 different products,

and he's used techniques of network analysis

to interrogate this database

and to graph relationships between the different products.

And it's wonderful, wonderful work.

You show all these interconnections, all these interrelations.

And I think it'll be profoundly useful

in understanding how it is that economies grow.

Brilliant work.

Cesar and I tried to write a piece for The New York Times Magazine

explaining how this works.

And what we learned

is Cesar's work is far too good to explain

in The New York Times Magazine.

Five thousand products --

that's still nothing.

Five thousand products --

imagine counting every product category

in Cesar Hidalgo's data.

Imagine you had one second

per product category.

In about the length of this session,

you would have counted all 5,000.

Now imagine doing the same thing

for every different type of product on sale in Walmart.

There are 100,000 there. It would take you all day.

Now imagine trying to count

every different specific product and service

on sale in a major economy

such as Tokyo, London or New York.

It's even more difficult in Edinburgh

because you have to count all the whisky and the tartan.

If you wanted to count every product and service

on offer in New York --

there are 10 billion of them --

it would take you 317 years.

This is how complex the economy we've created is.

And I'm just counting toasters here.

I'm not trying to solve the Middle East problem.

The complexity here is unbelievable.

And just a piece of context --

the societies in which our brains evolved

had about 300 products and services.

You could count them in five minutes.

So this is the complexity of the world that surrounds us.

This perhaps is why

we find the God complex so tempting.

We tend to retreat and say, "We can draw a picture,

we can post some graphs,

we get it, we understand how this works."

And we don't.

We never do.

Now I'm not trying to deliver a nihilistic message here.

I'm not trying to say we can't solve

complicated problems in a complicated world.

We clearly can.

But the way we solve them

is with humility --

to abandon the God complex

and to actually use a problem-solving technique that works.

And we have a problem-solving technique that works.

Now you show me

a successful complex system,

and I will show you a system

that has evolved through trial and error.

Here's an example.

This baby was produced through trial and error.

I realize that's an ambiguous statement.

Maybe I should clarify it.

This baby is a human body: it evolved.

What is evolution?

Over millions of years, variation and selection,

variation and selection --

trial and error,

trial and error.

And it's not just biological systems

that produce miracles through trial and error.

You could use it in an industrial context.

So let's say you wanted to make detergent.

Let's say you're Unilever

and you want to make detergent in a factory near Liverpool.

How do you do it?

Well you have this great big tank full of liquid detergent.

You pump it at a high pressure through a nozzle.

You create a spray of detergent.

Then the spray dries. It turns into powder.

It falls to the floor.

You scoop it up. You put it in cardboard boxes.

You sell it at a supermarket.

You make lots of money.

How do you design that nozzle?

It turns out to be very important.

Now if you ascribe to the God complex,

what you do is you find yourself a little God.

You find yourself a mathematician; you find yourself a physicist --

somebody who understands the dynamics of this fluid.

And he will, or she will,

calculate the optimal design of the nozzle.

Now Unilever did this and it didn't work --

too complicated.

Even this problem, too complicated.

But the geneticist Professor Steve Jones

describes how Unilever actually did solve this problem --

trial and error,

variation and selection.

You take a nozzle

and you create 10 random variations on the nozzle.

You try out all 10; you keep the one that works best.

You create 10 variations on that one.

You try out all 10. You keep the one that works best.

You try out 10 variations on that one.

You see how this works, right?

And after 45 generations,

you have this incredible nozzle.

It looks a bit like a chess piece --

functions absolutely brilliantly.

We have no idea

why it works,

no idea at all.

And the moment you step back from the God complex --

let's just try to have a bunch of stuff;

let's have a systematic way of determining what's working and what's not --

you can solve your problem.

Now this process of trial and error

is actually far more common in successful institutions

than we care to recognize.

And we've heard a lot about how economies function.

The U.S. economy is still the world's greatest economy.

How did it become the world's greatest economy?

I could give you all kinds of facts and figures

about the U.S. economy,

but I think the most salient one is this:

ten percent of American businesses

disappear every year.

That is a huge failure rate.

It's far higher than the failure rate of, say, Americans.

Ten percent of Americans don't disappear every year.

Which leads us to conclude

American businesses fail faster than Americans,

and therefore American businesses are evolving faster than Americans.

And eventually, they'll have evolved to such a high peak of perfection

that they will make us all their pets --

(Laughter)

if, of course, they haven't already done so.

I sometimes wonder.

But it's this process of trial and error

that explains this great divergence,

this incredible performance of Western economies.

It didn't come because you put some incredibly smart person in charge.

It's come through trial and error.

Now I've been sort of banging on about this

for the last couple of months,

and people sometimes say to me,

"Well Tim, it's kind of obvious.

Obviously trial and error is very important.

Obviously experimentation is very important.

Now why are you just wandering around saying this obvious thing?"

So I say, okay, fine.

You think it's obvious?

I will admit it's obvious

when schools

start teaching children

that there are some problems that don't have a correct answer.

Stop giving them lists of questions

every single one of which has an answer.

And there's an authority figure in the corner

behind the teacher's desk who knows all the answers.

And if you can't find the answers,

you must be lazy or stupid.

When schools stop doing that all the time,

I will admit that, yes,

it's obvious that trial and error is a good thing.

When a politician stands up

campaigning for elected office

and says, "I want to fix our health system.

I want to fix our education system.

I have no idea how to do it.

I have half a dozen ideas.

We're going to test them out. They'll probably all fail.

Then we'll test some other ideas out.

We'll find some that work. We'll build on those.

We'll get rid of the ones that don't." --

when a politician campaigns on that platform,

and more importantly, when voters like you and me

are willing to vote for that kind of politician,

then I will admit

that it is obvious that trial and error works, and that -- thank you.

(Applause)

Until then, until then

I'm going to keep banging on about trial and error

and why we should abandon the God complex.

Because it's so hard

to admit our own fallibility.

It's so uncomfortable.

And Archie Cochrane understood this as well as anybody.

There's this one trial he ran

many years after World War II.

He wanted to test out

the question of, where is it

that patients should recover

from heart attacks?

Should they recover in a specialized cardiac unit in hospital,

or should they recover at home?

All the cardiac doctors tried to shut him down.

They had the God complex in spades.

They knew that their hospitals were the right place for patients,

and they knew it was very unethical

to run any kind of trial or experiment.

Nevertheless, Archie managed to get permission to do this.

He ran his trial.

And after the trial had been running for a little while,

he gathered together all his colleagues

around his table,

and he said, "Well, gentlemen,

we have some preliminary results.

They're not statistically significant.

But we have something.

And it turns out that you're right and I'm wrong.

It is dangerous for patients

to recover from heart attacks at home.

They should be in hospital."

And there's this uproar, and all the doctors start pounding the table

and saying, "We always said you were unethical, Archie.

You're killing people with your clinical trials. You need to shut it down now.

Shut it down at once."

And there's this huge hubbub.

Archie lets it die down.

And then he says, "Well that's very interesting, gentlemen,

because when I gave you the table of results,

I swapped the two columns around.

It turns out your hospitals are killing people,

and they should be at home.

Would you like to close down the trial now,

or should we wait until we have robust results?"

Tumbleweed

rolls through the meeting room.

But Cochrane would do that kind of thing.

And the reason he would do that kind of thing

is because he understood

it feels so much better

to stand there and say,

"Here in my own little world,

I am a god, I understand everything.

I do not want to have my opinions challenged.

I do not want to have my conclusions tested."

It feels so much more comfortable

simply to lay down the law.

Cochrane understood

that uncertainty, that fallibility,

that being challenged, they hurt.

And you sometimes need to be shocked out of that.

Now I'm not going to pretend that this is easy.

It isn't easy.

It's incredibly painful.

And since I started talking about this subject

and researching this subject,

I've been really haunted by something

a Japanese mathematician said on the subject.

So shortly after the war,

this young man, Yutaka Taniyama,

developed this amazing conjecture

called the Taniyama-Shimura Conjecture.

It turned out to be absolutely instrumental

many decades later

in proving Fermat's Last Theorem.

In fact, it turns out it's equivalent

to proving Fermat's Last Theorem.

You prove one, you prove the other.

But it was always a conjecture.

Taniyama tried and tried and tried

and he could never prove that it was true.

And shortly before his 30th birthday in 1958,

Yutaka Taniyama killed himself.

His friend, Goro Shimura --

who worked on the mathematics with him --

many decades later, reflected on Taniyama's life.

He said,

"He was not a very careful person

as a mathematician.

He made a lot of mistakes.

But he made mistakes in a good direction.

I tried to emulate him,

but I realized

it is very difficult

to make good mistakes."

Thank you.

(Applause)