The price of a "clean" internet | Hans Block and Moritz Riesewieck

翻译人员: Wanting Zhong 校对人员: Jiasi Hao

[This talk contains mature content]

【本演讲包含成人内容】

Moritz Riesewieck: On March 23, 2013,

莫里兹·里斯维克： 2013 年 3 月 23 日，

users worldwide discovered in their news feed

世界各地的用户 在他们的新闻推送里

a video of a young girl being raped by an older man.

发现了一个年轻女孩 被年长男性强奸的视频。

Before this video was removed from Facebook,

在这个视频从 Facebook 上被移除前，

it was already shared 16,000 times,

它已经被转发了 1.6 万次，

and it was even liked 4,000 times.

甚至被点赞了 4 千次。

This video went viral and infected the net.

这个视频被疯转， 像病毒一样侵染了网络。

Hans Block: And that was the moment we asked ourselves

汉斯·布洛克： 也正是在这一刻，我们问自己，

how could something like this get on Facebook?

这种东西是怎么得以 出现在 Facebook 上的？

And at the same time, why don't we see such content more often?

同时，为什么我们没有 更加频繁地看见这种内容？

After all, there's a lot of revolting material online,

毕竟网络上有很多 令人反胃的资料信息，

but why do we so rarely see such crap on Facebook, Twitter or Google?

但为什么我们很少在 Facebook 、 推特，或谷歌上看到这样的垃圾？

MR: While image-recognition software

莫：虽说图像识别软件

can identify the outlines of sexual organs,

可以在图片和视频中

blood or naked skin in images and videos,

分辨性器官、血或者裸体，

it has immense difficulties to distinguish pornographic content

它很难从度假照片、阿多尼斯雕像，

或乳腺癌检查的宣传活动中，

from holiday pictures, Adonis statues

or breast-cancer screening campaigns.

区分出色情内容。

It can't distinguish Romeo and Juliet dying onstage

它无法区分舞台上 罗密欧与朱丽叶的死亡，

from a real knife attack.

和现实中的持刀袭击。

It can't distinguish satire from propaganda

它无法区分讽喻和煽动，

or irony from hatred, and so on and so forth.

反语和仇恨， 如此种种。

Therefore, humans are needed to decide

因此，需要人类来判断

which of the suspicious content should be deleted,

可疑内容中哪些应被删除，

and which should remain.

哪些可以保留。

Humans whom we know almost nothing about,

我们对这些人几乎一无所知，

because they work in secret.

因为他们进行的是秘密工作。

They sign nondisclosure agreements,

他们签了保密协议，

which prohibit them from talking and sharing

禁止他们谈论与分享

what they see on their screens and what this work does to them.

自己在屏幕上看到了什么， 以及这份工作对他们造成的影响。

They are forced to use code words in order to hide who they work for.

他们被迫使用暗号 以隐藏他们的雇主。

They are monitored by private security firms

他们被私人安保公司监控，

in order to ensure that they don't talk to journalists.

以确保他们不会同记者交谈。

而要是他们发声， 便会被威胁处以罚款。

And they are threatened by fines in case they speak.

All of this sounds like a weird crime story,

这些听起来像是 某个离奇的犯罪故事，

但这是真实的。

but it's true.

These people exist,

这些人是存在的，

and they are called content moderators.

他们被称为“网络审查员”。

HB: We are the directors of the feature documentary film "The Cleaners,"

汉：我们是专题纪录片《The Cleaners》 （《网络清道夫》）的导演。

and we would like to take you

请让我们将你们带往

to a world that many of you may not know yet.

一个你们大多数人 可能还未曾知晓的世界。

这是我们电影的一个片段。

Here's a short clip of our film.

（音乐）

(Music)

(Video) Moderator: I need to be anonymous, because we have a contract signed.

（视频）审查员：我必须匿名， 因为我们签了合同。

We are not allowed to declare whom we are working with.

我们不被允许透露 我们在为谁工作。

The reason why I speak to you

我之所以和你对话的原因

is because the world should know that we are here.

是因为世界应当知道 我们在这里。

There is somebody who is checking the social media.

有人在检查社交媒体。

We are doing our best to make this platform

我们在尽自己所能

为他们所有人 维持一个安全的平台。

safe for all of them.

Delete.

删除。

Ignore.

忽略。

Delete.

删除。

Ignore.

忽略。

Delete.

删除。

Ignore.

忽略。

Ignore.

忽略。

Delete.

删除。

HB: The so-called content moderators

汉：这个被称作“网络审查员”的群体

don't get their paychecks from Facebook, Twitter or Google themselves,

并不是直接从Facebook、 推特，或谷歌拿工资，

而是受雇于世界各地的外包公司，

but from outsourcing firms around the world

in order to keep the wages low.

以降低时薪成本。

Tens of thousands of young people

成千上万的年轻人

looking at everything we are not supposed to see.

看着我们不应当看到的一切。

我们指的是斩首、残割、

And we are talking about decapitations, mutilations,

executions, necrophilia, torture, child abuse.

处决、尸奸、酷刑、儿童虐待。

Thousands of images in one shift --

一次轮值要处理几千张图像——

忽略，删除—— 不论昼夜。

ignore, delete, day and night.

And much of this work is done in Manila,

这项工作大部分是在 马尼拉进行的，

where the analog toxic waste from the Western world

多年来西方世界 都用集装箱船

将有毒的电子垃圾 输向这里，

was transported for years by container ships,

now the digital waste is dumped there via fiber-optic cable.

如今数字垃圾正通过 光纤电缆倾倒在同一个地方。

And just as the so-called scavengers

而正如同所谓的拾荒者

rummage through gigantic tips on the edge of the city,

在城市边缘的 巨大垃圾山里翻捡一样，

the content moderators click their way through an endless toxic ocean

网络审查员点击着鼠标， 趟过一片无边无际的、

of images and videos and all manner of intellectual garbage,

由图像、视频和各种知识垃圾 构成的充满毒素的汪洋，

so that we don't have to look at it.

感谢于此， 我们无需亲自面对这些内容。

MR: But unlike the wounds of the scavengers,

莫：但和拾荒者们身上的伤口不同，

those of the content moderators remain invisible.

网络审查员的伤口是看不见的。

Full of shocking and disturbing content,

这些图片和视频充斥着 令人震惊与不安的内容，

these pictures and videos burrow into their memories

烙印在他们的记忆里，

where, at any time, they can have unpredictable effects:

随时可能造成难以预计的影响：

eating disorders, loss of libido,

饮食失调、性欲丧失、

anxiety disorders, alcoholism,

焦虑症、酗酒、

depression, which can even lead to suicide.

抑郁症，甚至可能造成自杀。

The pictures and videos infect them,

那些图片和视频感染了他们，

and often never let them go again.

往往再也不会放过他们。

If they are unlucky, they develop post-traumatic stress disorders,

如果不幸的话，他们会像 从战场归来的士兵一样，

like soldiers after war missions.

患上创伤后应激障碍（PTSD）。

In our film, we tell the story of a young man

在影片里，我们讲述了 一个年轻人的故事：

他的工作是监控 自残以及自杀企图的直播，

who had to monitor livestreams of self-mutilations and suicide attempts,

again and again,

周而复始，

然而最终，他也以自杀的方式 结束了自己的生命。

and who eventually committed suicide himself.

It's not an isolated case, as we've been told.

我们被告知的是， 这样的事并非个例。

这是我们所有人，

This is the price all of us pay

为了我们所谓的干净、安全、 且“健康”的社交媒体环境，

for our so-called clean and safe and "healthy"

environments on social media.

付出的代价。

Never before in the history of mankind

在人类历史中， 从未有哪个时代

能像现在这样轻易地

has it been easier to reach millions of people around the globe

在数秒之内便触及 全球各地的数百万人。

in a few seconds.

在社交媒体上发布的内容 传递得如此之快，

What is posted on social media spreads so quickly,

迅速爆红疯转， 刺激全球所有人的神经。

becomes viral and excites the minds of people all around the globe.

Before it is deleted,

在它被删除之前，

it is often already too late.

往往已为时晚矣。

Millions of people have already been infected

数百万人已经 被憎恨和愤怒感染，

with hatred and anger,

他们抑或在网上变得活跃，

and they either become active online,

by spreading or amplifying hatred,

继续传播或放大憎恨，

抑或走上街头， 诉诸暴力。

or they take to the streets and take up arms.

HB: Therefore, an army of content moderators

汉：因此， 一支由网络审查员形成的军队，

sit in front of a screen to avoid new collateral damage.

守在屏幕前， 防止新的附带损害产生。

And they are deciding, as soon as possible,

他们必须尽快做出决断，

whether the content stays on the platform -- ignore;

是否保留某条内容——忽略；

or disappears -- delete.

还是让它消失——删除。

But not every decision is as clear

但并不是每个决定

都能像对儿童虐待的视频那样 迅速做出清晰明了的判断。

as the decision about a child-abuse video.

What about controversial content, ambivalent content,

对于由民权活动人士、 公民记者上传的

uploaded by civil rights activists or citizen journalists?

有争议的、模棱两可的内容， 该怎么处理呢？

网络审查员在判断这些案例时，

The content moderators often decide on such cases

at the same speed as the [clear] cases.

通常和处理泾渭分明的案例时 使用同样的速度。

MR: We will show you a video now,

莫：下面我们会让大家 观看一段视频，

and we would like to ask you to decide:

我们想让你们决定：

Would you delete it,

你们会删除它吗？

还是不删除它呢？

or would you not delete it?

（视频）（空袭声）

(Video) (Air strike sounds)

(Explosion)

（爆炸）

(People speaking in Arabic)

（人们用阿拉伯语交谈）

莫：没错，我们已为大家 对视频进行了打码处理。

MR: Yeah, we did some blurring for you.

A child would potentially be dangerously disturbed

一个孩子要是看到这样的内容，

可能会感到严重不安， 以及极度恐惧。

and extremely frightened by such content.

So, you rather delete it?

那么，你们觉得删了更好？

But what if this video could help investigate the war crimes in Syria?

但要是说，这段视频能帮助调查 在叙利亚发生的战争罪行呢？

What if nobody would have heard about this air strike,

要是因为Facebook、Youtube、 推特都决定撤除这段视频，

because Facebook, YouTube, Twitter would have decided to take it down?

导致无人得知这场空袭呢？

Airwars, a nongovernmental organization based in London,

Airwars 是一个位于伦敦的 非政府组织，

tries to find those videos as quickly as possible

他们试图在这些视频 上传到社交网络时，

whenever they are uploaded to social media,

尽快找到这些视频，

in order to archive them.

以便对它们进行归档记录。

Because they know, sooner or later,

因为他们知道，这些内容迟早 会被Facebook、Youtube、推特删除，

Facebook, YouTube, Twitter would take such content down.

People armed with their mobile phones

拥有手机的人们能曝光

can make visible what journalists often do not have access to.

记者们通常难以接触的事情。

Civil rights groups often do not have any better option

人权组织为了让他们的录像

to quickly make their recordings accessible to a large audience

能迅速向广大观众公开，

除了上传到社交媒体 常常没有更好的选择。

than by uploading them to social media.

Wasn't this the empowering potential the World Wide Web should have?

这难道不是万维网应当拥有的 能够赋予力量的潜力吗？

Weren't these the dreams

这难道不是万维网初具雏形时，

people in its early stages had about the World Wide Web?

人们对它抱有的梦想吗？

Can't pictures and videos like these

这样的图片和视频

persuade people who have become insensitive to facts

难道无法劝说已对事实 变得麻木的人们

to rethink?

开始反思吗？

HB: But instead, everything that might be disturbing is deleted.

汉：然而，一切可能 造成不安的内容都被删除了。

And there's a general shift in society.

在社会中还有这样的 一种变化趋势。

Media, for example, more and more often use trigger warnings

比如说，媒体更加频繁地

在有人可能感到冒犯 或者不安的文章顶部

at the top of articles

which some people may perceive as offensive or troubling.

使用“敏感警告”。

Or more and more students at universities in the United States

美国的大学校园内 有越来越多的学生

demand the banishment of antique classics

要求从课程中剔除

which depict sexual violence or assault from the curriculum.

描写性暴力或性侵犯的古典内容。

But how far should we go with that?

但这些行为的尺度该如何把握？

Physical integrity is guaranteed as a human right

在世界各地的宪法中，

身体健全是被保障的基本人权。

in constitutions worldwide.

In the Charter of Fundamental Rights of the European Union,

欧盟的《基本权利宪章》明文规定，

this right expressly applies to mental integrity.

这项权利同样适用于心理健全。

But even if the potentially traumatic effect

但即使图像和视频 带来的潜在创伤难以预测，

of images and videos is hard to predict,

do we want to become so cautious

我们是否想变得如此谨小慎微，

that we risk losing social awareness of injustice?

以至于要冒险失去 对不公的社会意识？

So what to do?

那么该怎么做呢？

Mark Zuckerberg recently stated that in the future,

马克·扎克伯格最近声明

the users, we, or almost everybody,

在未来，用户们， 即我们，或者几乎是任何人，

will decide individually

将会通过个人过滤设定，

what they would like to see on the platform,

个人独立决定在平台上 想看到的内容。

by personal filter settings.

也就是说任何人能轻松地声称

So everyone could easily claim to remain undisturbed

看到战争和暴力冲突的图像时 能不为所动，比如说——

by images of war or other violent conflicts, like ...

MR: I'm the type of guy who doesn't mind seeing breasts

莫：我是那种不介意 看到胸部的男人，

and I'm very interested in global warming,

我对全球变暖很感兴趣，

但不怎么喜欢战争。

but I don't like war so much.

HB: Yeah, I'm more the opposite,

汉：嗯，我就比较相反，

I have zero interest in naked breasts or naked bodies at all.

我对胸部或者裸体 压根没有一点兴趣。

But why not guns? I like guns, yes.

但何不谈谈枪支？ 没错，我喜欢枪。

MR: Come on, if we don't share a similar social consciousness,

莫：看嘛，如果我们没有 共享相似的社会意识，

how shall we discuss social problems?

我们该如何讨论社会问题？

How shall we call people to action?

我们该如何呼吁人们行动？

Even more isolated bubbles would emerge.

更多互相孤立的泡泡会浮现。

One of the central questions is: "How, in the future,

核心问题之一是：

“在未来，我们该如何平衡 言论自由与人们对保护的需求。”

freedom of expression will be weighed against the people's need for protection."

It's a matter of principle.

这是个原则性的问题。

Do we want to design an either open or closed society

我们想为数字空间

设计一个相较开放 或是封闭的社会？

for the digital space?

问题的核心是 “自由 vs. 安全感”。

At the heart of the matter is "freedom versus security."

Facebook has always wanted to be a "healthy" platform.

Facebook 一直想成为 一个“健康”的平台。

Above all, users should feel safe and secure.

重中之重的是， 用户应当感到安全。

It's the same choice of words

在我们的很多采访中，

the content moderators in the Philippines used

菲律宾的网络审查员们

也使用了同样的遣词。

in a lot of our interviews.

(Video) The world that we are living in right now,

（视频）我相信， 我们正生活的世界，

I believe, is not really healthy.

并不是真的健康。

(Music)

（音乐）

In this world, there is really an evil who exists.

在这世界上， 确实存在着邪恶。

(Music)

（音乐）

We need to watch for it.

我们需要警惕它。

(Music)

（音乐）

We need to control it -- good or bad.

我们需要控制它——无论好坏。

(Music)

（音乐）

[Look up, Young man! --God]

[年轻人，抬头看！——上帝]

MR: For the young content moderators in the strictly Catholic Philippines,

莫：这些来自信奉天主教的菲律宾的 年轻网络审查员们，

this is linked to a Christian mission.

对于他们来说，这份工作 和基督教的使命有所联系。

To counter the sins of the world

为了对抗在网络上传播的

which spread across the web.

这个世界的罪恶。

"Cleanliness is next to godliness,"

“清洁近于圣洁”，

is a saying everybody in the Philippines knows.

这个说法在菲律宾人尽皆知。

汉：其他人则将自己与他们的总统 罗德里戈·杜特尔特相比较，

HB: And others motivate themselves

by comparing themselves with their president, Rodrigo Duterte.

以此激励自身。

He has been ruling the Philippines since 2016,

他自 2016 年当选以来 一直掌权菲律宾，

and he won the election with the promise: "I will clean up."

他凭借“我会进行清扫”的承诺 在当年的选举中胜出。

And what that means is eliminating all kinds of problems

而这个承诺的意思是

通过杀掉街上被视为罪犯的人， 不管这是什么意思，

by literally killing people on the streets

who are supposed to be criminals, whatever that means.

从而达到 排除社会上各种问题的目的。

And since he was elected,

自从他当选以后，

an estimated 20,000 people have been killed.

估计有 2 万人被杀。

And one moderator in our film says,

我们影片中的一位审查员说：

“杜特尔特在街头上怎么做，

"What Duterte does on the streets,

I do for the internet."

我在网络上也怎么做。”

And here they are, our self-proclaimed superheroes,

这就是他们， 我们的“自我标榜的超级英雄”，

who enforce law and order in our digital world.

在数字世界里维持法制与秩序。

They clean up, they polish everything clean,

他们进行扫除， 把一切擦拭得干干净净，

they free us from everything evil.

他们将我们 从一切邪恶中解放出来。

Tasks formerly reserved to state authorities

曾经为国家机关保留的任务

如今落到了二十岁出头的 大学毕业生肩上，

have been taken over by college graduates in their early 20s,

equipped with three- to five-day training --

他们接受完三天到五天的训练，

this is the qualification --

——这便是他们的资格证，

who work on nothing less than the world's rescue.

他们的工作不亚于拯救世界。

MR: National sovereignties have been outsourced to private companies,

莫：国家权能被外包给私人公司，

and they pass on their responsibilities to third parties.

他们又将自己的责任 托付给第三方。

情况就是—— 外包，再外包，再外包。

It's an outsourcing of the outsourcing of the outsourcing,

which takes place.

With social networks,

对于社交网络，

我们要处理的是一个全新的架构，

we are dealing with a completely new infrastructure,

它有着自己的运行机制，

with its own mechanisms,

its own logic of action

自己的行为逻辑，

and therefore, also, its own new dangers,

因而也有其特定的潜在新危险。

which had not yet existed in the predigitalized public sphere.

这些危险在电子化时代以前的 公共领域中不曾存在过。

HB: When Mark Zuckerberg was at the US Congress

汉：当马克·扎克伯格在美国国会

or at the European Parliament,

或者欧洲议会时，

he was confronted with all kinds of critics.

他面对的是各式各样的批评。

And his reaction was always the same:

而他的反应总是千篇一律的：

"We will fix that,

“这一点我们会改进，

and I will follow up on that with my team."

那一点我们团队会跟进。”

可是这样的辩论

But such a debate shouldn't be held in back rooms of Facebook,

不应该在 Facebook 、推特 或谷歌的幕后进行——

Twitter or Google --

such a debate should be openly discussed in new, cosmopolitan parliaments,

这样的辩论应当被公开探讨， 在崭新的、国际化的议会中，

在新的机构中—— 它们应当能反映

in new institutions that reflect the diversity of people

“为全球化网络理想工程 做出贡献的人们的多元化”。

contributing to a utopian project of a global network.

And while it may seem impossible to consider the values

考虑到全球用户的价值观 虽说看上去不可能

of users worldwide,

it's worth believing

但值得相信的是，

that there's more that connects us than separates us.

我们之间的 联系将比隔阂更强大。

MR: Yeah, at a time when populism is gaining strength,

莫：没错， 在这个民粹主义抬头的时点，

it becomes popular to justify the symptoms,

为症状辩解、

将它们消除、将它们隐形，

to eradicate them,

to make them invisible.

这样的做法变得普及。

This ideology is spreading worldwide,

这种观念正在全世界扩散，

analog as well as digital,

无论在现实里还是在网络上，

and it's our duty to stop it

而我们的义务

是在为时已晚前阻止它。

before it's too late.

The question of freedom and democracy

自由和民主的问题

must not only have these two options.

并不能只有这两个选项。

汗：删除。

HB: Delete.

MR: Or ignore.

莫：或者忽略。

HB: Thank you very much.

汗：谢谢大家。

(Applause)

（掌声）