Bruce Schneier: The security mirage

So, security is two different things:

it's a feeling, and it's a reality.

And they're different.

You could feel secure even if you're not.

And you can be secure

even if you don't feel it.

Really, we have two separate concepts

mapped onto the same word.

And what I want to do in this talk is to split them apart --

figuring out when they diverge and how they converge.

And language is actually a problem here.

There aren't a lot of good words

for the concepts we're going to talk about.

So if you look at security from economic terms,

it's a trade-off.

Every time you get some security, you're always trading off something.

Whether this is a personal decision --

whether you're going to install a burglar alarm in your home --

or a national decision,

where you're going to invade a foreign country --

you're going to trade off something: money or time, convenience, capabilities,

maybe fundamental liberties.

And the question to ask when you look at a security anything

is not whether this makes us safer,

but whether it's worth the trade-off.

You've heard in the past several years, the world is safer

because Saddam Hussein is not in power.

That might be true, but it's not terribly relevant.

The question is: Was it worth it?

And you can make your own decision,

and then you'll decide whether the invasion was worth it.

That's how you think about security: in terms of the trade-off.

Now, there's often no right or wrong here.

Some of us have a burglar alarm system at home and some of us don't.

And it'll depend on where we live,

whether we live alone or have a family,

how much cool stuff we have,

how much we're willing to accept the risk of theft.

In politics also, there are different opinions.

A lot of times, these trade-offs are about more than just security,

and I think that's really important.

Now, people have a natural intuition about these trade-offs.

We make them every day.

Last night in my hotel room, when I decided to double-lock the door,

or you in your car when you drove here;

when we go eat lunch

and decide the food's not poison and we'll eat it.

We make these trade-offs again and again,

multiple times a day.

We often won't even notice them.

They're just part of being alive; we all do it.

Every species does it.

Imagine a rabbit in a field, eating grass.

And the rabbit sees a fox.

That rabbit will make a security trade-off:

"Should I stay, or should I flee?"

And if you think about it,

the rabbits that are good at making that trade-off

will tend to live and reproduce,

and the rabbits that are bad at it

will get eaten or starve.

So you'd think

that us, as a successful species on the planet -- you, me, everybody --

would be really good at making these trade-offs.

Yet it seems, again and again, that we're hopelessly bad at it.

And I think that's a fundamentally interesting question.

I'll give you the short answer.

The answer is, we respond to the feeling of security

and not the reality.

Now, most of the time, that works.

Most of the time,

feeling and reality are the same.

Certainly that's true for most of human prehistory.

We've developed this ability

because it makes evolutionary sense.

One way to think of it is that we're highly optimized

for risk decisions

that are endemic to living in small family groups

in the East African Highlands in 100,000 BC.

2010 New York, not so much.

Now, there are several biases in risk perception.

A lot of good experiments in this.

And you can see certain biases that come up again and again.

I'll give you four.

We tend to exaggerate spectacular and rare risks

and downplay common risks --

so, flying versus driving.

The unknown is perceived to be riskier than the familiar.

One example would be:

people fear kidnapping by strangers,

when the data supports that kidnapping by relatives is much more common.

This is for children.

Third, personified risks are perceived to be greater

than anonymous risks.

So, Bin Laden is scarier because he has a name.

And the fourth is:

people underestimate risks in situations they do control

and overestimate them in situations they don't control.

So once you take up skydiving or smoking,

you downplay the risks.

If a risk is thrust upon you -- terrorism is a good example --

you'll overplay it,

because you don't feel like it's in your control.

There are a bunch of other of these cognitive biases,

that affect our risk decisions.

There's the availability heuristic,

which basically means we estimate the probability of something

by how easy it is to bring instances of it to mind.

So you can imagine how that works.

If you hear a lot about tiger attacks, there must be a lot of tigers around.

You don't hear about lion attacks, there aren't a lot of lions around.

This works, until you invent newspapers,

because what newspapers do is repeat again and again

rare risks.

I tell people: if it's in the news, don't worry about it,

because by definition, news is something that almost never happens.

(Laughter)

When something is so common, it's no longer news.

Car crashes, domestic violence --

those are the risks you worry about.

We're also a species of storytellers.

We respond to stories more than data.

And there's some basic innumeracy going on.

I mean, the joke "One, two, three, many" is kind of right.

We're really good at small numbers.

One mango, two mangoes, three mangoes,

10,000 mangoes, 100,000 mangoes --

it's still more mangoes you can eat before they rot.

So one half, one quarter, one fifth -- we're good at that.

One in a million, one in a billion --

they're both almost never.

So we have trouble with the risks that aren't very common.

And what these cognitive biases do

is they act as filters between us and reality.

And the result is that feeling and reality get out of whack,

they get different.

Now, you either have a feeling -- you feel more secure than you are,

there's a false sense of security.

Or the other way, and that's a false sense of insecurity.

I write a lot about "security theater,"

which are products that make people feel secure,

but don't actually do anything.

There's no real word for stuff that makes us secure,

but doesn't make us feel secure.

Maybe it's what the CIA is supposed to do for us.

So back to economics.

If economics, if the market, drives security,

and if people make trade-offs based on the feeling of security,

then the smart thing for companies to do for the economic incentives

is to make people feel secure.

And there are two ways to do this.

One, you can make people actually secure

and hope they notice.

Or two, you can make people just feel secure

and hope they don't notice.

Right?

So what makes people notice?

Well, a couple of things:

understanding of the security,

of the risks, the threats,

the countermeasures, how they work.

But if you know stuff, you're more likely

to have your feelings match reality.

Enough real-world examples helps.

We all know the crime rate in our neighborhood,

because we live there, and we get a feeling about it

that basically matches reality.

Security theater is exposed

when it's obvious that it's not working properly.

OK. So what makes people not notice?

Well, a poor understanding.

If you don't understand the risks, you don't understand the costs,

you're likely to get the trade-off wrong,

and your feeling doesn't match reality.

Not enough examples.

There's an inherent problem with low-probability events.

If, for example, terrorism almost never happens,

it's really hard to judge the efficacy of counter-terrorist measures.

This is why you keep sacrificing virgins,

and why your unicorn defenses are working just great.

There aren't enough examples of failures.

Also, feelings that cloud the issues --

the cognitive biases I talked about earlier: fears, folk beliefs --

basically, an inadequate model of reality.

So let me complicate things.

I have feeling and reality.

I want to add a third element. I want to add "model."

Feeling and model are in our head,

reality is the outside world; it doesn't change, it's real.

Feeling is based on our intuition,

model is based on reason.

That's basically the difference.

In a primitive and simple world,

there's really no reason for a model,

because feeling is close to reality.

You don't need a model.

But in a modern and complex world,

you need models to understand a lot of the risks we face.

There's no feeling about germs.

You need a model to understand them.

This model is an intelligent representation of reality.

It's, of course, limited by science, by technology.

We couldn't have a germ theory of disease

before we invented the microscope to see them.

It's limited by our cognitive biases.

But it has the ability to override our feelings.

Where do we get these models? We get them from others.

We get them from religion, from culture, teachers, elders.

A couple years ago, I was in South Africa on safari.

The tracker I was with grew up in Kruger National Park.

He had some very complex models of how to survive.

And it depended on if you were attacked by a lion, leopard, rhino, or elephant --

and when you had to run away, when you couldn't run away,

when you had to climb a tree, when you could never climb a tree.

I would have died in a day.

But he was born there, and he understood how to survive.

I was born in New York City.

I could have taken him to New York, and he would have died in a day.

(Laughter)

Because we had different models based on our different experiences.

Models can come from the media,

from our elected officials ...

Think of models of terrorism,

child kidnapping,

airline safety, car safety.

Models can come from industry.

The two I'm following are surveillance cameras,

ID cards,

quite a lot of our computer security models come from there.

A lot of models come from science.

Health models are a great example.

Think of cancer, bird flu, swine flu, SARS.

All of our feelings of security about those diseases

come from models given to us, really, by science filtered through the media.

So models can change.

Models are not static.

As we become more comfortable in our environments,

our model can move closer to our feelings.

So an example might be,

if you go back 100 years ago,

when electricity was first becoming common,

there were a lot of fears about it.

There were people who were afraid to push doorbells,

because there was electricity in there, and that was dangerous.

For us, we're very facile around electricity.

We change light bulbs without even thinking about it.

Our model of security around electricity is something we were born into.

It hasn't changed as we were growing up.

And we're good at it.

Or think of the risks on the Internet across generations --

how your parents approach Internet security,

versus how you do,

versus how our kids will.

Models eventually fade into the background.

"Intuitive" is just another word for familiar.

So as your model is close to reality and it converges with feelings,

you often don't even know it's there.

A nice example of this came from last year and swine flu.

When swine flu first appeared,

the initial news caused a lot of overreaction.

Now, it had a name,

which made it scarier than the regular flu,

even though it was more deadly.

And people thought doctors should be able to deal with it.

So there was that feeling of lack of control.

And those two things made the risk more than it was.

As the novelty wore off and the months went by,

there was some amount of tolerance; people got used to it.

There was no new data, but there was less fear.

By autumn,

people thought the doctors should have solved this already.

And there's kind of a bifurcation:

people had to choose between fear and acceptance --

actually, fear and indifference --

and they kind of chose suspicion.

And when the vaccine appeared last winter,

there were a lot of people -- a surprising number --

who refused to get it.

And it's a nice example of how people's feelings of security change,

how their model changes,

sort of wildly,

with no new information, with no new input.

This kind of thing happens a lot.

I'm going to give one more complication.

We have feeling, model, reality.

I have a very relativistic view of security.

I think it depends on the observer.

And most security decisions have a variety of people involved.

And stakeholders with specific trade-offs will try to influence the decision.

And I call that their agenda.

And you see agenda -- this is marketing, this is politics --

trying to convince you to have one model versus another,

trying to convince you to ignore a model

and trust your feelings,

marginalizing people with models you don't like.

This is not uncommon.

An example, a great example, is the risk of smoking.

In the history of the past 50 years,

the smoking risk shows how a model changes,

and it also shows how an industry fights against a model it doesn't like.

Compare that to the secondhand smoke debate --

probably about 20 years behind.

Think about seat belts.

When I was a kid, no one wore a seat belt.

Nowadays, no kid will let you drive if you're not wearing a seat belt.

Compare that to the airbag debate,

probably about 30 years behind.

All examples of models changing.

What we learn is that changing models is hard.

Models are hard to dislodge.

If they equal your feelings,

you don't even know you have a model.

And there's another cognitive bias

I'll call confirmation bias,

where we tend to accept data that confirms our beliefs

and reject data that contradicts our beliefs.

So evidence against our model, we're likely to ignore,

even if it's compelling.

It has to get very compelling before we'll pay attention.

New models that extend long periods of time are hard.

Global warming is a great example.

We're terrible at models that span 80 years.

We can do "to the next harvest."

We can often do "until our kids grow up."

But "80 years," we're just not good at.

So it's a very hard model to accept.

We can have both models in our head simultaneously --

that kind of problem where we're holding both beliefs together,

the cognitive dissonance.

Eventually, the new model will replace the old model.

Strong feelings can create a model.

September 11 created a security model in a lot of people's heads.

Also, personal experiences with crime can do it,

personal health scare,

a health scare in the news.

You'll see these called "flashbulb events" by psychiatrists.

They can create a model instantaneously,

because they're very emotive.

So in the technological world,

we don't have experience to judge models.

And we rely on others. We rely on proxies.

And this works, as long as it's the correct others.

We rely on government agencies

to tell us what pharmaceuticals are safe.

I flew here yesterday.

I didn't check the airplane.

I relied on some other group

to determine whether my plane was safe to fly.

We're here, none of us fear the roof is going to collapse on us,

not because we checked,

but because we're pretty sure the building codes here are good.

It's a model we just accept

pretty much by faith.

And that's OK.

Now, what we want is people to get familiar enough with better models,

have it reflected in their feelings,

to allow them to make security trade-offs.

When these go out of whack, you have two options.

One, you can fix people's feelings, directly appeal to feelings.

It's manipulation, but it can work.

The second, more honest way

is to actually fix the model.

Change happens slowly.

The smoking debate took 40 years -- and that was an easy one.

Some of this stuff is hard.

Really, though, information seems like our best hope.

And I lied.

Remember I said feeling, model, reality; reality doesn't change?

It actually does.

We live in a technological world;

reality changes all the time.

So we might have, for the first time in our species:

feeling chases model, model chases reality, reality's moving --

they might never catch up.

We don't know.

But in the long term,

both feeling and reality are important.

And I want to close with two quick stories to illustrate this.

1982 -- I don't know if people will remember this --

there was a short epidemic of Tylenol poisonings

in the United States.

It's a horrific story.

Someone took a bottle of Tylenol,

put poison in it, closed it up, put it back on the shelf,

someone else bought it and died.

This terrified people.

There were a couple of copycat attacks.

There wasn't any real risk, but people were scared.

And this is how the tamper-proof drug industry was invented.

Those tamper-proof caps? That came from this.

It's complete security theater.

As a homework assignment, think of 10 ways to get around it.

I'll give you one: a syringe.

But it made people feel better.

It made their feeling of security more match the reality.

Last story: a few years ago, a friend of mine gave birth.

I visit her in the hospital.

It turns out, when a baby's born now,

they put an RFID bracelet on the baby, a corresponding one on the mother,

so if anyone other than the mother takes the baby out of the maternity ward,

an alarm goes off.

I said, "Well, that's kind of neat.

I wonder how rampant baby snatching is out of hospitals."

I go home, I look it up.

It basically never happens.

(Laughter)

But if you think about it, if you are a hospital,

and you need to take a baby away from its mother,

out of the room to run some tests,

you better have some good security theater,

or she's going to rip your arm off.

(Laughter)

So it's important for us,

those of us who design security,

who look at security policy --

or even look at public policy in ways that affect security.

It's not just reality; it's feeling and reality.

What's important

is that they be about the same.

It's important that, if our feelings match reality,

we make better security trade-offs.

Thank you.

(Applause)