Can you solve the false positive riddle? - Alex Gendler

Mining unobtainium is hard work.

The rare mineral appears in only 1% of rocks in the mine.

But your friend Tricky Joe has something up his sleeve.

The unobtainium detector he’s been perfecting for months is finally ready.

The device never fails to detect unobtainium if any is present.

Otherwise, it’s still highly reliable,

returning accurate readings 90% of the time.

On his first day trying it out in the field,

the device goes off, and Joe happily places the rock in his cart.

As the two of you head back to camp where the ore can be examined,

Joe makes you an offer:

he’ll sell you the ore for just $200.

You know that a piece of unobtanium that size would easily be worth $1000,

but any other minerals would be effectively worthless.

Should you make the trade?

Pause here if you want to figure it out for yourself.

Answer in: 3

Answer in: 2

Answer in: 1

Intuitively, it seems like a good deal.

Since the detector is correct most of the time,

shouldn’t you be able to trust its reading?

Unfortunately, no.

Here’s why.

Imagine the mine has exactly 1,000 pieces of ore.

An unobtainium rarity of 1%

means that there are only 10 rocks with the precious mineral inside.

All 10 would set off the detector.

But what about the other 990 rocks without unobtainium?

Well, 90% of them, 891 rocks, to be exact,

won’t set off anything.

But 10%, or 99 rocks, will set off the detector

despite not having unobtanium,

a result known as a false positive.

Why does that matter?

Because it means that all in all,

109 rocks will have triggered the detector.

And Joe’s rock could be any one of them,

from the 10 that contain the mineral

to the 99 that don’t,

which means the chances of it containing unobtainium are 10 out of 109 – about 9%.

And paying $200 for a 9% chance of getting $1000 isn’t great odds.

So why is this result so unexpected,

and why did Joe’s rock seem like such a sure bet?

The key is something called the base rate fallacy.

While we’re focused on the relatively high accuracy of the detector,

our intuition makes us forget to account

for how rare the unobtanium was in the first place.

But because the device’s error rate of 10%

is still higher than the mineral’s overall occurrence,

any time it goes off is still more likely to be a false positive

than a real finding.

This problem is an example of conditional probability.

The answer lies neither in the overall chance of finding unobtainium,

nor the overall chance of receiving a false positive reading.

This kind of background information that we’re given before anything happens

is known as unconditional, or prior probability.

What we’re looking for, though, is the chance of finding unobtainium

once we know that the device did return a positive reading.

This is known as the conditional, or posterior probability,

determined once the possibilities have been narrowed down through observation.

Many people are confused by the false positive paradox

because we have a bias for focusing on specific information

over the more general,

especially when immediate decisions come into play.

And while in many cases it’s better to be safe than sorry,

false positives can have real negative consequences.

False positives in medical testing are preferable to false negatives,

but they can still lead to stress or unnecessary treatment.

And false positives in mass surveillance

can cause innocent people to be wrongfully arrested, jailed, or worse.

As for this case, the one thing you can be positive about

is that Tricky Joe is trying to take you for a ride.