3 ways to spot a bad statistic | Mona Chalabi

翻译人员: Wilde Luo 校对人员: Lipeng Chen

I'm going to be talking about statistics today.

今天我想讲一下统计学。

If that makes you immediately feel a little bit wary, that's OK,

如果你马上有所警觉，没关系，

that doesn't make you some kind of crazy conspiracy theorist,

我不会让你变成那种 疯狂的阴谋论者，

it makes you skeptical.

只会让你学会质疑。

And when it comes to numbers, especially now, you should be skeptical.

尤其是现在，说到数字、数据， 你应该时刻保持怀疑态度。

But you should also be able to tell which numbers are reliable

但你需要能够辨别哪些数字是可信的，

and which ones aren't.

哪些存在问题。

So today I want to try to give you some tools to be able to do that.

因此为了让你们具有这种甄别能力， 我会试着提供一些方法。

But before I do,

但在我开始之前，

I just want to clarify which numbers I'm talking about here.

我想说明我所说的统计 数据是什么类型的。

我讨论的并不是这种广告：

I'm not talking about claims like,

"9 out of 10 women recommend this anti-aging cream."

“10个女人中有9个 会推荐这款抗衰老乳霜。”

I think a lot of us always roll our eyes at numbers like that.

我想我们中很多人对 这样的数字不以为然。

What's different now is people are questioning statistics like,

但不同的是，现在人们对这样的 数字也开始抱有怀疑：

“美国的失业率是 5%。”

"The US unemployment rate is five percent."

What makes this claim different is it doesn't come from a private company,

这个数字不是出于一个私人公司，

it comes from the government.

而是来自政府。

事实上，如今10个美国人中差不多有4个 不相信政府公布的

About 4 out of 10 Americans distrust the economic data

that gets reported by government.

那些经济数据。

Among supporters of President Trump it's even higher;

在特朗普总统的支持者中，情况更糟，

it's about 7 out of 10.

10人中有7人不相信那些数据。

I don't need to tell anyone here

无须我告诉大家，

that there are a lot of dividing lines in our society right now,

当今我们的社会中 划分了许多的分界线，

and a lot of them start to make sense,

一旦你明白“人”和那些“政府数据”

之间的关系，这些分界线中的 大多数就不言自明了。

once you understand people's relationships with these government numbers.

On the one hand, there are those who say these statistics are crucial,

一方面，有些人说， 这些统计数据很重要，

that we need them to make sense of society as a whole

我们需要它们来理解整个社会，

in order to move beyond emotional anecdotes

进而脱离情感喜好、偏见等，

and measure progress in an [objective] way.

以一种客观的方式衡量发展程度。

And then there are the others,

正相反，另一方面，有一些人说

who say that these statistics are elitist,

这些数据隶属于精英统治，

maybe even rigged;

它们甚至有可能是被操纵的；

they don't make sense and they don't really reflect

它们没什么意义，也不能真正反映

what's happening in people's everyday lives.

人们的日常生活中发生了什么。

It kind of feels like that second group is winning the argument right now.

现在看来，似乎是第二种人 在这两个阵营的针锋相对中获得了胜利。

We're living in a world of alternative facts,

我们所生活的世界里， 胡说八道已成常态，

where people don't find statistics this kind of common ground,

人们对统计数据不能达成共识，

this starting point for debate.

也不会把数据作为争论时的出发点。

This is a problem.

这是一个问题。

There are actually moves in the US right now

如今在美国，人们有一个举措是

to get rid of some government statistics altogether.

完全丢弃政府的某项统计数据。

Right now there's a bill in congress about measuring racial inequality.

现在，国会有一项关于 衡量种族不平等程度的议案。

The draft law says that government money should not be used

草案规定，政府资金不能被用在

搜集关于种族隔离的数据上。

to collect data on racial segregation.

This is a total disaster.

这完全是一场灾难。

If we don't have this data,

如果我们没有相关数据，

how can we observe discrimination,

我们怎样观察种族歧视现象，

let alone fix it?

更不要说解决它了？

In other words:

换句话说：

How can a government create fair policies

如果政府都不能衡量 如今的种族不平等程度，

那他们又如何制定合理的政策呢？

if they can't measure current levels of unfairness?

This isn't just about discrimination,

不只是在种族歧视方面，

it's everything -- think about it.

还包括其他方面—— 想一想吧，

如果我们没有关于医疗健康

How can we legislate on health care

以及贫穷问题的可靠数据， 要如何为医疗保健问题立法？

if we don't have good data on health or poverty?

如果我们甚至对进入或离开 这个国家的人数都无法达成共识，

How can we have public debate about immigration

if we can't at least agree

on how many people are entering and leaving the country?

我们又如何

关于移民问题进行公众讨论？

Statistics come from the state; that's where they got their name.

“统计”（statistics）来自于 “国家事务”（state），是它的词源所在。

它的重点在于更准确地衡量人口

The point was to better measure the population

来更好地造福于人民。

in order to better serve it.

So we need these government numbers,

所以我们需要这些政府数据，

but we also have to move beyond either blindly accepting

但我们也不能盲目接受

or blindly rejecting them.

或者盲目排斥它们。

We need to learn the skills to be able to spot bad statistics.

我们需要学会甄别不可信的数据。

I started to learn some of these

当我在统计部门工作时——

when I was working in a statistical department

它是联合国的一部分，

我开始学会了一些甄别的技巧。

that's part of the United Nations.

Our job was to find out how many Iraqis had been forced from their homes

我们的工作是弄清楚 有多少伊拉克人是因为战争

as a result of the war,

而被迫离家，

and what they needed.

以及他们需要什么。

这的确是一项非常重要的工作， 但其难度也让人难以置信。

It was really important work, but it was also incredibly difficult.

Every single day, we were making decisions

每一天，我们都在做出

that affected the accuracy of our numbers --

会影响我们统计数据精确度的决定——

decisions like which parts of the country we should go to,

决定我们应该去 这个国家的哪一块区域调查，

who we should speak to,

和哪种人交谈，

which questions we should ask.

我们应该询问他们什么问题。

And I started to feel really disillusioned with our work,

我开始对我们的工作感到失望，

because we thought we were doing a really good job,

因为我们以为我们的 工作非常棒、很有意义，

but the one group of people who could really tell us were the Iraqis,

但是真正能告诉我们 实情的是那些伊拉克人，

and they rarely got the chance to find our analysis, let alone question it.

他们却很难有机会看到我们的 分析结果，更别提质疑它了。

So I started to feel really determined

所以我开始坚信，

能让数据更加精确可信的一种方法，

that the one way to make numbers more accurate

is to have as many people as possible be able to question them.

就是让尽可能多的人能够去质疑它。

So I became a data journalist.

所以我成为了一名“数据记者”。

My job is finding these data sets and sharing them with the public.

我的工作是找到这些数据集， 并且向公众发布。

Anyone can do this, you don't have to be a geek or a nerd.

任何人都可以做到这件事； 你无须是一个极客或者老手。

你可以忽视那些术语； 那些想要表现得很聪明

You can ignore those words; they're used by people

又假装谦虚的人才喜欢这些专业词汇。

trying to say they're smart while pretending they're humble.

毫无疑问任何人都能做到这些事。

Absolutely anyone can do this.

I want to give you guys three questions

我想向你们提出3个问题，

that will help you be able to spot some bad statistics.

会帮助你甄别那些不可靠的数据。

So, question number one is: Can you see uncertainty?

那么第一个问题是： 你能看到不确定性吗？

One of things that's really changed people's relationship with numbers,

有一件事真正改变了 人们和数据的关系，

and even their trust in the media,

甚至改变了人们对媒体的信任，

has been the use of political polls.

那就是政治上的民意调查的滥用。

I personally have a lot of issues with political polls

我个人对政治民意调查颇有微词，

because I think the role of journalists is actually to report the facts

因为我认为，记者的任务就是 原原本本地报道事实，

and not attempt to predict them,

而不应该尝试去预测什么，

especially when those predictions can actually damage democracy

尤其是当这些“预测” 实际上会有损民主的时候，

它向人们示意： 不用费心给那个家伙投票了，

by signaling to people: don't bother to vote for that guy,

他是没机会入选的。

he doesn't have a chance.

我们把这事先搁到一边， 来谈谈这种做法的精确度如何。

Let's set that aside for now and talk about the accuracy of this endeavor.

Based on national elections in the UK, Italy, Israel

根据英国、意大利、 以色列的国家选举情况，

and of course, the most recent US presidential election,

当然，还有近期的 美国总统选举情况，

using polls to predict electoral outcomes

它们都表明：使用民意 调查来预测选举结果

is about as accurate as using the moon to predict hospital admissions.

无异于夜观天象来预测是否 住院——都是不可靠的。

No, seriously, I used actual data from an academic study to draw this.

说真的，我用一份学术研究中的 真实数据画出了这幅图。

There are a lot of reasons why polling has become so inaccurate.

民意调查变得如此不准确 是有很多原因的。

Our societies have become really diverse,

我们的社会已经变得相当多元化，

which makes it difficult for pollsters to get a really nice representative sample

这使得民意调查者很难在 人群中为他们的调查得到一份

不错的、有代表性的样本。

of the population for their polls.

People are really reluctant to answer their phones to pollsters,

人们不是很情愿地 回答民意调查电话，

and also, shockingly enough, people might lie.

另外，令人震惊的是， 人们可能会说谎。

But you wouldn't necessarily know that to look at the media.

但在媒体的报道中，你不一定会 看到并了解这些过程。

For one thing, the probability of a Hillary Clinton win

例如，希拉里 · 克林顿 赢得选举的可能性

was communicated with decimal places.

在媒体的报道中精确到了小数点。

We don't use decimal places to describe the temperature.

我们描述气温都不会这么精确。

How on earth can predicting the behavior of 230 million voters in this country

所以，对这个国家里的 2亿3千万选民的行为的预测，

be that precise?

怎么可能会如此精确？

And then there were those sleek charts.

另外，还有一些美观的图表。

See, a lot of data visualizations will overstate certainty, and it works --

知道吗，很多数据可视化的方法会 夸大数据的准确性，而且很有效——

these charts can numb our brains to criticism.

这些图表会麻痹我们的大脑， 让我们无法对其准确性生疑。

When you hear a statistic, you might feel skeptical.

当你听到一项统计数据， 你或许会怀疑。

As soon as it's buried in a chart,

但当它整合进了图表中，

它看起来仿佛就成了客观的科学，

it feels like some kind of objective science,

and it's not.

但实际上正好相反。

So I was trying to find ways to better communicate this to people,

所以我试着找出一些方法 来告诉人们这些数据背后的事实，

to show people the uncertainty in our numbers.

为人们展示这些数据的不确定性。

What I did was I started taking real data sets,

我所做的，就是先采集数据，

and turning them into hand-drawn visualizations,

然后把它们转换为手绘的示意图，

so that people can see how imprecise the data is;

所以人们可以看到 这些数据的不精确性；

so people can see that a human did this,

可以看到这份图表是人画出来的，

有人搜集数据并把它可视化了。

a human found the data and visualized it.

For example, instead of finding out the probability

例如，我们不去计算出每个月

of getting the flu in any given month,

可能患流感的概率，

you can see the rough distribution of flu season.

但我们可以得到流感 在每个季节的大致分布。

This is --

这就是——

(Laughter)

（笑声）

现在就是二月，展示 这张图不是很合适。

a bad shot to show in February.

But it's also more responsible data visualization,

但这是更加负责任的数据可视图表，

because if you were to show the exact probabilities,

因为，你如果在这张图中 展示了精确的概率，

maybe that would encourage people to get their flu jabs

那么它就有可能鼓励人们 在不合适的时间段

接种流感疫苗。

at the wrong time.

The point of these shaky lines

这些摇摇晃晃的线条

is so that people remember these imprecisions,

是想向人们说明 这些数据的不精确性，

but also so they don't necessarily walk away with a specific number,

它也不会让人们简单地 得到一个鸡肋的具体数字，

but they can remember important facts.

而是让他们记住重要的事实。

Facts like injustice and inequality leave a huge mark on our lives.

不公正、不平等这样的事实 给我们的生活带来了巨大的影响。

Facts like Black Americans and Native Americans have shorter life expectancies

“美国黑人以及原住民的预期寿命 比其他种族的人短一些”，

than those of other races,

这样的事实，

and that isn't changing anytime soon.

短时间内也难以改变。

Facts like prisoners in the US can be kept in solitary confinement cells

还包括“美国监狱中被单独监禁的 囚犯的活动空间

that are smaller than the size of an average parking space.

比起一般的停车位面积还要小” 这样的事实。

The point of these visualizations is also to remind people

这些数据可视化的 意义也在于提醒人们

of some really important statistical concepts,

一些极其重要的统计概念，

concepts like averages.

例如“平均”的概念。

So let's say you hear a claim like,

比方说你听到了这样一句话：

"The average swimming pool in the US contains 6.23 fecal accidents."

“在美国，一般大小的游泳池 平均含有6.23次大便。”

That doesn't mean every single swimming pool in the country

这并不意味着这个 国家里每一座游泳池

contains exactly 6.23 turds.

刚好有6.23次大便。

So in order to show that,

所以为了展示这一点，

I went back to the original data, which comes from the CDC,

我找到疾病预防控制 中心的原始数据，

who surveyed 47 swimming facilities.

他们调查了47座游泳池。

And I just spent one evening redistributing poop.

我花了一个晚上 “重新分配这些大便”。

So you can kind of see how misleading averages can be.

所以你可以看到， “平均”是多么误导人。

(Laughter)

（笑声）

OK, so the second question that you guys should be asking yourselves

好，第二个问题， 你们在甄别数据时

to spot bad numbers is:

应当反问自己的是：

Can I see myself in the data?

我能在这些数据中 看到自己的身影吗？

This question is also about averages in a way,

这个问题在某种程度上 也是关于“平均”的，

because part of the reason why people are so frustrated

因为，人们对政府 公布的数据感到失望的

with these national statistics,

一部分原因就是，

is they don't really tell the story of who's winning and who's losing

他们无法分清，在国家政策下

from national policy.

谁损失、谁获益。

It's easy to understand why people are frustrated with global averages

很容易理解为什么当人们发现 自己的个人经验与全球平均数据

不相符时，他们会感到非常失望。

when they don't match up with their personal experiences.

I wanted to show people the way data relates to their everyday lives.

我想为人们展示：数据是如何 与他们的日常生活相关联的。

我开设了一个咨询专栏， 叫做《亲爱的Mona》，

I started this advice column called "Dear Mona,"

人们会写信询问 他们所关心的事情，

where people would write to me with questions and concerns

而我会尝试借着数据回答他们。

and I'd try to answer them with data.

这些问题包罗万象，

People asked me anything.

例如，“和老婆分床睡是正常的吗？”

questions like, "Is it normal to sleep in a separate bed to my wife?"

"Do people regret their tattoos?"

“人们会为身上的刺青感到后悔吗？”

“什么是‘自然死亡’？”

"What does it mean to die of natural causes?"

All of these questions are great, because they make you think

这些问题都很棒， 因为它们会让你思考，

如何去寻找并转达这些数据。

about ways to find and communicate these numbers.

If someone asks you, "How much pee is a lot of pee?"

如果有人问你： “ 排多少尿才算 ‘尿量多’ ？”

这是我曾被问及的一个问题，

which is a question that I got asked,

you really want to make sure that the visualization makes sense

你真的会确信：将数据图示化表达

to as many people as possible.

能让更多的人容易理解。

These numbers aren't unavailable.

这些统计数据并不难以得到。

Sometimes they're just buried in the appendix of an academic study.

有时它们只是被淹没在 学术研究的附录里。

And they're certainly not inscrutable;

它们也绝非高深莫测；

如果你真的想要检验 这些有关尿量的数据，

if you really wanted to test these numbers on urination volume,

抓一个瓶子自己试试吧。

you could grab a bottle and try it for yourself.

(Laughter)

（笑声）

The point of this isn't necessarily

重点在于，每一份数据集

不一定要跟你有专门的关联。

that every single data set has to relate specifically to you.

我对“法国有多少女人 因为戴面纱或头巾而被罚款”

I'm interested in how many women were issued fines in France

这样的问题很感兴趣，

for wearing the face veil, or the niqab,

即使我不住在法国也不戴面纱。

even if I don't live in France or wear the face veil.

The point of asking where you fit in is to get as much context as possible.

问自己是否符合数据当中的情况， 是为了更好地了解其环境背景。

So it's about zooming out from one data point,

所以我们从一小块 数据出发来综观全体，

like the unemployment rate is five percent,

就像“失业率是5%”这样的数据，

观察它随着时间如何变化，

and seeing how it changes over time,

or seeing how it changes by educational status --

或者观察它随着教育程度的 差异而如何改变——

this is why your parents always wanted you to go to college --

这就是你的父母总是 希望你上大学的原因——

or seeing how it varies by gender.

或者观察它随着 性别差异而如何变化。

Nowadays, male unemployment rate is higher

如今，男性的失业率

than the female unemployment rate.

比女性的失业率高。

Up until the early '80s, it was the other way around.

但是在80年代初期之前， 情况与此相反。

This is a story of one of the biggest changes

这是美国社会至今为止

that's happened in American society,

发生的最大的改变，

一旦你不再局限于“平均”， 就会发现这些信息都隐含在在图表中。

and it's all there in that chart, once you look beyond the averages.

The axes are everything;

轴线能代表一切；

once you change the scale, you can change the story.

当你改变观察的尺度， 你就得到了新的信息。

OK, so the third and final question that I want you guys to think about

好，第三个也是最后 一个问题，我希望

when you're looking at statistics is:

你们在观察数据时思考的是：

How was the data collected?

这些数据是如何被收集的？

So far, I've only talked about the way data is communicated,

目前为止，我只谈到 数据被传达的方式，

但收集这些数据的 方式也同样重要。

but the way it's collected matters just as much.

我知道这很困难，

I know this is tough,

因为收集数据的方法一般是 不透明而且有些无聊的。

because methodologies can be opaque and actually kind of boring,

但你能使用一些简单的 步骤来检视数据。

but there are some simple steps you can take to check this.

I'll use one last example here.

这里我要举最后一个例子。

One poll found that 41 percent of Muslims in this country support jihad,

一份民意调查指出，这个国家内 41%的穆斯林支持伊斯兰教的“圣战”，

which is obviously pretty scary,

这听起来相当吓人，

and it was reported everywhere in 2015.

这个数字在2015年被大肆报道。

When I want to check a number like that,

当我想要检查一下这个数据时，

I'll start off by finding the original questionnaire.

我会首先寻找原始的调查问卷。

It turns out that journalists who reported on that statistic

结果表明， 那些报道此事的记者

ignored a question lower down on the survey

忽视了调查问卷后面的一个问题，

that asked respondents how they defined "jihad."

关于受调查者如何定义“圣战”。

And most of them defined it as,

大多数人是这样定义的：

"Muslims' personal, peaceful struggle to be more religious."

“为了变得更加虔诚， 穆斯林们个人的、和平的奋斗。”

Only 16 percent defined it as, "violent holy war against unbelievers."

只有16%的人如此认为： “反对不信教者的暴力的、神圣的战争。”

This is the really important point:

这是很重要的一点：

based on those numbers, it's totally possible

根据这些数据， 完全有可能发生的是：

that no one in the survey who defined it as violent holy war

在调查中那些把“圣战”解释为 “暴力的、神圣的战争”的人

also said they support it.

根本就不支持“圣战”。

Those two groups might not overlap at all.

这两组人可能根本就不重叠。

It's also worth asking how the survey was carried out.

调查进行的方式也是 一个值得探讨的问题。

This was something called an opt-in poll,

这份调查使用了一种 “自愿参与”的调查方式，

which means anyone could have found it on the internet and completed it.

意味着任何一个人都可以在 网络上找到并完成这份调查。

我们甚至无从知道这些 被调查者是否是穆斯林。

There's no way of knowing if those people even identified as Muslim.

And finally, there were 600 respondents in that poll.

最后，有600人参与了这份调查。

根据皮尤研究中心统计，这个国家里

There are roughly three million Muslims in this country,

according to Pew Research Center.

差不多有3百万穆斯林。

这意味着，在这个国家中， 每5000个穆斯林中只有一个

That means the poll spoke to roughly one in every 5,000 Muslims

参与了这项调查。

in this country.

This is one of the reasons

这也是为什么政府的统计数据

why government statistics are often better than private statistics.

通常比私人机构的调查 数据更为准确的原因之一。

A poll might speak to a couple hundred people, maybe a thousand,

一份民意调查可能 有数百人参与，甚至一千人，

or if you're L'Oreal, trying to sell skin care products in 2005,

或者你在欧莱雅公司， 想要在2005年销售护肤产品，

then you spoke to 48 women to claim that they work.

那么你只需要调查48名 认为此产品有效的女性就行了。

(Laughter)

（笑声）

很显然，私人公司对于 追求数据的准确性兴趣不大，

Private companies don't have a huge interest in getting the numbers right,

they just need the right numbers.

他们只想得到想要的 数字来进行宣传。

Government statisticians aren't like that.

政府的统计学家就不是如此了。

In theory, at least, they're totally impartial,

理论上，他们是完全公正的，

not least because most of them do their jobs regardless of who's in power.

主要是因为他们的工作 不会受掌权者所影响。

They're civil servants.

他们是公务员。

And to do their jobs properly,

为了做好本职工作，

they don't just speak to a couple hundred people.

他们不能只调查几百人而已。

Those unemployment numbers I keep on referencing

我一直提到的这些失业率数字，

come from the Bureau of Labor Statistics,

来自于美国劳工统计局，

and to make their estimates,

为了进行这项估计，

they speak to over 140,000 businesses in this country.

他们调查了全国14万家以上的企业。

I get it, it's frustrating.

我知道，这令人沮丧。

If you want to test a statistic that comes from a private company,

如果你想检验来自 一家私人企业的统计数据，

你可以为自己和朋友们 购买一款面霜拿来使用，

you can buy the face cream for you and a bunch of friends, test it out,

如果它没有效果， 你可以说那些统计数字有误。

if it doesn't work, you can say the numbers were wrong.

但是你如何质疑 政府的统计数据呢？

But how do you question government statistics?

你需要把一切都检查一遍。

You just keep checking everything.

了解他们是如何收集数据的。

Find out how they collected the numbers.

观察一下，你是否在图表中 看到了你需要了解的一切信息。

Find out if you're seeing everything on the chart you need to see.

但是也不要完全不相信这些数据，

But don't give up on the numbers altogether, because if you do,

否则我们只能在一无所知中 制定公共政策，

we'll be making public policy decisions in the dark,

引导我们的除了私人利益，别无他物。

using nothing but private interests to guide us.

Thank you.

谢谢。

（掌声）

(Applause)