The price of a "clean" internet | Hans Block and Moritz Riesewieck

譯者: Lilian Chiu 審譯者: Helen Chang

[This talk contains mature content]

〔本演說有成人內容〕

Moritz Riesewieck: On March 23, 2013,

莫里茲·里斯維克： 2013 年 3 月 23 日，

users worldwide discovered in their news feed

全世界的使用者 在他們的新聞提要中

a video of a young girl being raped by an older man.

看到一個年輕女孩 被年長男子強暴的影片。

Before this video was removed from Facebook,

在這支影片被臉書移除之前，

it was already shared 16,000 times,

它已經被分享了一萬六千次，

and it was even liked 4,000 times.

它甚至得到了四千個讚。

This video went viral and infected the net.

這支影片在網絡上被瘋傳。

Hans Block: And that was the moment we asked ourselves

漢斯·布拉克：那時，我們自問，

how could something like this get on Facebook?

這樣的東西怎麼會出現在臉書上？

And at the same time, why don't we see such content more often?

此外，我們為什麼不會 更常見到這種內容？

After all, there's a lot of revolting material online,

畢竟，網路上有許多 令人反感的素材，

but why do we so rarely see such crap on Facebook, Twitter or Google?

但我們為什麼很少會在臉書、 推特，或 Google 看到這類內容？

MR: While image-recognition software

莫：雖然影像辨識軟體

can identify the outlines of sexual organs,

可以在影像和影片中辨認出

性器官的外形、血， 或者裸露的皮膚，

blood or naked skin in images and videos,

it has immense difficulties to distinguish pornographic content

但它非常難區別色情內容

from holiday pictures, Adonis statues

和渡假清涼照片、阿多尼斯雕像，

or breast-cancer screening campaigns.

或者乳癌篩選活動的差別。

It can't distinguish Romeo and Juliet dying onstage

它無法區別羅密歐 和茱麗葉死在舞台上

from a real knife attack.

與真正用刀攻擊的差別。

It can't distinguish satire from propaganda

它無法區別諷刺作品 和宣傳的不同，

or irony from hatred, and so on and so forth.

或者諷刺和仇恨的不同，諸如此類。

Therefore, humans are needed to decide

因此，必須要由人類來決定

which of the suspicious content should be deleted,

哪些可疑的內容應該被刪除，

and which should remain.

哪些應該留下。

Humans whom we know almost nothing about,

我們完全不知道這些人的任何事，

because they work in secret.

因為他們的工作很秘密。

They sign nondisclosure agreements,

他們要簽署保密協定，

which prohibit them from talking and sharing

保密協定會禁止他們談論及分享

what they see on their screens and what this work does to them.

他們在螢幕上看到什麼， 及這份工作對他們的影響。

They are forced to use code words in order to hide who they work for.

他們被迫要使用代號， 以隱瞞他們為誰工作。

They are monitored by private security firms

他們會被私人保全公司監視，

in order to ensure that they don't talk to journalists.

以確保他們沒有和記者交談。

And they are threatened by fines in case they speak.

如果他們說出來，還有罰金的威脅。

All of this sounds like a weird crime story,

這一切聽起來就像是個 怪異的犯罪故事，

but it's true.

但這是真的。

These people exist,

這些人真的存在，

and they are called content moderators.

他們被稱為內容審查員。

HB: We are the directors of the feature documentary film "The Cleaners,"

漢：我們是紀錄長片 《網路監護人》的導演，

and we would like to take you

我們想要帶大家前往

to a world that many of you may not know yet.

一個許多人都還不知道的世界。

Here's a short clip of our film.

這個片段取自我們的紀錄片。

(Music)

（音樂）

(Video) Moderator: I need to be anonymous, because we have a contract signed.

（影片）審查員：我必須要匿名， 因為我們有簽合約。

We are not allowed to declare whom we are working with.

我們不可以聲明我們和誰一起工作。

The reason why I speak to you

我跟你談的理由是因為

is because the world should know that we are here.

世界應該要知道我們的存在。

There is somebody who is checking the social media.

有人在負責檢查社交媒體。

We are doing our best to make this platform

我們在盡力確保這個平台

safe for all of them.

對大家都是安全的。

Delete.

刪除。

Ignore.

忽略。

Delete.

刪除。

Ignore.

忽略。

Delete.

刪除。

Ignore.

忽略。

Ignore.

忽略。

Delete.

刪除。

HB: The so-called content moderators

漢：所謂的內容審查員，

don't get their paychecks from Facebook, Twitter or Google themselves,

他們的薪水並不是來自臉書、 推特，或 Google 本身，

but from outsourcing firms around the world

而是世界各地的外包公司， 以確保維持低薪資。

in order to keep the wages low.

Tens of thousands of young people

數萬名年輕人

looking at everything we are not supposed to see.

在負責看我們不應該看的內容。

And we are talking about decapitations, mutilations,

不該看的內容指的是斬首、殘害、

executions, necrophilia, torture, child abuse.

處決、戀屍癖、拷打、孩童虐待。

Thousands of images in one shift --

輪一次班就要看數千張 影像——忽略、刪除，

ignore, delete, day and night.

日以繼夜，夜以繼日。

And much of this work is done in Manila,

這些工作大部分在馬尼拉進行，

where the analog toxic waste from the Western world

來自西方世界的類比有毒廢棄物

was transported for years by container ships,

數年來一直用貨櫃船運輸到那裡，

now the digital waste is dumped there via fiber-optic cable.

現在數位廢棄物則是 透過光纖網路丟過去。

And just as the so-called scavengers

當所謂的拾荒者

rummage through gigantic tips on the edge of the city,

在廣大的城市邊緣翻找，

the content moderators click their way through an endless toxic ocean

在影像、影片，和各種智慧 所形成的無邊無際有毒海洋中，

of images and videos and all manner of intellectual garbage,

內容審查員則是地不斷點擊滑鼠，

so that we don't have to look at it.

讓我們不需要去看這些內容。

MR: But unlike the wounds of the scavengers,

莫：但，不像拾荒者的傷口，

those of the content moderators remain invisible.

內容審查員的傷口一直是隱形的。

Full of shocking and disturbing content,

這些影像和影片中滿是 震撼、讓人不安的內容，

these pictures and videos burrow into their memories

這些全都潛入了他們的記憶中，

where, at any time, they can have unpredictable effects:

它們隨時都可能會在記憶中 產生無法預測的影響：

eating disorders, loss of libido,

飲食失調、性慾喪失、

anxiety disorders, alcoholism,

焦慮相關疾病、酗酒、

depression, which can even lead to suicide.

憂鬱，甚至可能會導致自殺。

The pictures and videos infect them,

這些影像和影片會感染他們，

and often never let them go again.

且通常不會再放他們走。

If they are unlucky, they develop post-traumatic stress disorders,

如果他們不夠幸運，

他們會得到創傷後壓力症候群，

like soldiers after war missions.

和戰後的士兵一樣。

In our film, we tell the story of a young man

我們的紀錄片說了 一個年輕人的故事，

who had to monitor livestreams of self-mutilations and suicide attempts,

他必須要監看自殘 和嘗試自殺的直播，

again and again,

不斷地監看，

and who eventually committed suicide himself.

最後，他自己也自殺了。

It's not an isolated case, as we've been told.

我們聽說，這並非獨立個案。

This is the price all of us pay

我們所有人要付出這樣的代價，

for our so-called clean and safe and "healthy"

才能有我們所謂乾淨、安全，

和「健康」的社交媒體環境。

environments on social media.

Never before in the history of mankind

人類歷史上從來沒有像現在

has it been easier to reach millions of people around the globe

這麼容易在幾秒鐘內就能 接觸到全世界數百萬人。

in a few seconds.

What is posted on social media spreads so quickly,

社交媒體上刊登的內容 能非常快速地散播出去，

becomes viral and excites the minds of people all around the globe.

瞬間爆紅，刺激全球 各地使用者的大腦。

Before it is deleted,

在它被刪除之前，

it is often already too late.

通常都已經太遲了。

Millions of people have already been infected

數百萬人已經感染到了仇恨和憤怒，

with hatred and anger,

and they either become active online,

他們可能會在網路上採取主動，

by spreading or amplifying hatred,

將仇恨散播出去或放大，

or they take to the streets and take up arms.

要不他們就會走上街頭使用武力。

HB: Therefore, an army of content moderators

漢：因此，一支內容審查員軍團

sit in front of a screen to avoid new collateral damage.

坐在螢幕前面， 避免新的間接傷害發生。

And they are deciding, as soon as possible,

他們得要盡快判定

whether the content stays on the platform -- ignore;

這些內容要留在平台上——忽略；

or disappears -- delete.

或者應該消失——刪除。

But not every decision is as clear

但並非每個決定

都跟兒童虐待影片的決定 一樣黑白分明。

as the decision about a child-abuse video.

What about controversial content, ambivalent content,

民權活動家或者公民記者 上傳的爭議性內容、

uploaded by civil rights activists or citizen journalists?

矛盾內容又該如何判定？

The content moderators often decide on such cases

內容審查員判定這些情況的速度，

at the same speed as the [clear] cases.

通常跟判定黑白分明的情況一樣快。

MR: We will show you a video now,

莫：我們先給各位看一段影片，

and we would like to ask you to decide:

接著想請各位決定：

Would you delete it,

你會刪除它，

or would you not delete it?

或者不會刪除它？

(Video) (Air strike sounds)

（影片）（空襲聲）

(Explosion)

（爆炸）

（阿拉伯語）

(People speaking in Arabic)

MR: Yeah, we did some blurring for you.

莫：是的，我們 為大家打了些馬賽克。

A child would potentially be dangerously disturbed

這樣的內容有可能造成孩子 不安，到了有危險的程度，

and extremely frightened by such content.

且受到極度的驚嚇。

So, you rather delete it?

所以，你想要刪除它嗎？

But what if this video could help investigate the war crimes in Syria?

但如果這支影片能夠協助 調查敘利亞的戰爭犯罪呢？

What if nobody would have heard about this air strike,

如果因為臉書、YouTube、 推特都決定把這支影片下架，

because Facebook, YouTube, Twitter would have decided to take it down?

因此就沒有人知道這次空襲呢？

Airwars, a nongovernmental organization based in London,

在倫敦有一個非政府 組織叫做 Airwars，

tries to find those videos as quickly as possible

每當有這類影片 被上傳到社交媒體上，

whenever they are uploaded to social media,

這個組織就會嘗試 盡快找出這些影片，

in order to archive them.

以將它們建檔。

Because they know, sooner or later,

因為，他們知道，遲早，

Facebook, YouTube, Twitter would take such content down.

臉書、YouTube、推特 都會把這類內容移除。

People armed with their mobile phones

有手機的人，

can make visible what journalists often do not have access to.

能夠讓記者通常無法 取得的資訊曝光。

Civil rights groups often do not have any better option

民權團體若想要很快 把它們錄下的內容

to quickly make their recordings accessible to a large audience

提供給大眾，通常它們最好的選擇

than by uploading them to social media.

就是將這些內容上傳到社交媒體。

Wasn't this the empowering potential the World Wide Web should have?

這不就是網際網路潛能： 可以讓使用者有更多能力？

Weren't these the dreams

這不就是網際網路剛開始發展時，

people in its early stages had about the World Wide Web?

大家對它懷抱的夢想嗎？

Can't pictures and videos like these

像這樣的影像和影片不能夠

persuade people who have become insensitive to facts

說服已經對事實麻木的人

to rethink?

重新思考嗎？

HB: But instead, everything that might be disturbing is deleted.

漢：但卻變成，凡是 讓人不安的內容都會被刪除。

And there's a general shift in society.

社會上發生了普遍的轉變。

Media, for example, more and more often use trigger warnings

比如，媒體越來越常在文章的開頭

at the top of articles

使用觸發警告，

which some people may perceive as offensive or troubling.

有些人可能會覺得被冒犯或不舒服。

Or more and more students at universities in the United States

或者，越來越多美國大學生

demand the banishment of antique classics

要求將描述暴力或攻擊性的古典文學

which depict sexual violence or assault from the curriculum.

從課程中除去。

But how far should we go with that?

但，我們要做到什麼程度？

Physical integrity is guaranteed as a human right

全世界的憲法都會保證

身體完整性這項人權。

in constitutions worldwide.

In the Charter of Fundamental Rights of the European Union,

在歐洲聯盟基本權利憲章中，

this right expressly applies to mental integrity.

這項權利明確適用於心理完整性。

But even if the potentially traumatic effect

但，就算很難預測影像和影片

of images and videos is hard to predict,

可能造成的創傷性影響，

do we want to become so cautious

我們想要那麼謹慎嗎？

that we risk losing social awareness of injustice?

要做到甚至會有風險，可能失去 對於不公正的社會意識嗎？

So what to do?

所以，該怎麼辦？

Mark Zuckerberg recently stated that in the future,

馬克·祖克柏最近表示，在未來，

the users, we, or almost everybody,

使用者，即我們， 或幾乎可說是所有人，

will decide individually

會能夠自己做出個人決定，

what they would like to see on the platform,

決定想要在平台上看到什麼，

by personal filter settings.

只要調整個人過濾設定即可。

So everyone could easily claim to remain undisturbed

所以，人人都能輕易地要求不要受到

by images of war or other violent conflicts, like ...

戰爭或其他暴力衝突影像的打擾……

MR: I'm the type of guy who doesn't mind seeing breasts

莫：我是那種不介意看到胸部的人，

and I'm very interested in global warming,

且我對全球暖化非常感興趣，

but I don't like war so much.

但我就不那麼喜歡戰爭了。

HB: Yeah, I'm more the opposite,

漢：我則正好相反，

I have zero interest in naked breasts or naked bodies at all.

我對裸露的胸部或身體完全沒興趣。

But why not guns? I like guns, yes.

但，為什麼不看槍枝呢？ 是的，我喜歡槍枝。

MR: Come on, if we don't share a similar social consciousness,

莫：拜託，如果沒有 相近的社會意識，

how shall we discuss social problems?

我們要如何討論社會問題？

How shall we call people to action?

我們要如何呼籲大家採取行動？

Even more isolated bubbles would emerge.

更孤立的同溫層會出現。

One of the central questions is: "How, in the future,

很重要的問題之一是：在未來，

freedom of expression will be weighed against the people's need for protection."

在大家對於保護的需求之下，

自由表述會有多大的重要性？

It's a matter of principle.

這是原則問題。

Do we want to design an either open or closed society

我們想要為數位空間

設計一個非開即關的社會嗎？

for the digital space?

At the heart of the matter is "freedom versus security."

那件事的核心就是

「自由 vs. 安全」。

Facebook has always wanted to be a "healthy" platform.

臉書一直想要成為 「健康」的平台。

Above all, users should feel safe and secure.

最重要的是，使用者 應該要感到安心、安全。

It's the same choice of words

我們在訪談菲律賓的 內容審查員時，

the content moderators in the Philippines used

也常聽到他們選用同樣這些字眼。

in a lot of our interviews.

(Video) The world that we are living in right now,

（影片）我相信，我們現在 所生活的世界並非真的很健康。

I believe, is not really healthy.

(Music)

（音樂）

In this world, there is really an evil who exists.

在這個世界裡，確實有邪惡存在。

(Music)

（音樂）

We need to watch for it.

我們必須要小心它。

(Music)

（音樂）

We need to control it -- good or bad.

我們必須要控制它——不論好壞。

(Music)

（音樂）

[Look up, Young man! --God]

〔向上看，年輕人！——上帝〕

MR: For the young content moderators in the strictly Catholic Philippines,

莫：在嚴格信奉天主教的菲律賓， 對這些年輕的內容審查員來說，

this is linked to a Christian mission.

這和基督教的使命有關。

To counter the sins of the world

是要反擊散佈在網路上的、

which spread across the web.

這個世界的罪惡。

"Cleanliness is next to godliness,"

「乾淨就是非常接近神聖。」

is a saying everybody in the Philippines knows.

在菲律賓，人人都知道這個說法。

HB: And others motivate themselves

漢：其他人的動機則是

by comparing themselves with their president, Rodrigo Duterte.

將他們自己和他們的 總統杜特蒂做比較。

He has been ruling the Philippines since 2016,

這位總統從 2016 年 便開始統治菲律賓，

and he won the election with the promise: "I will clean up."

他贏得選舉是因為他保證： 「我會清理乾淨。」

And what that means is eliminating all kinds of problems

那句話的意思是要 把各種問題都清除，

by literally killing people on the streets

且他的做法是殺光街上所有

who are supposed to be criminals, whatever that means.

應該是罪犯的人， 不論那是什麼意思。

And since he was elected,

他當選之後，

an estimated 20,000 people have been killed.

估計有兩萬人遭殺害。

And one moderator in our film says,

我們電影中的一位審查員說：

"What Duterte does on the streets,

「杜特蒂在街頭所做的事，

I do for the internet."

就是我在網路上所做的事。」

And here they are, our self-proclaimed superheroes,

這就是我們自命的超級英雄，

who enforce law and order in our digital world.

他們在我們的數位世界中 執行法律維護秩序。

They clean up, they polish everything clean,

他們清理，把一切都擦乾淨，

they free us from everything evil.

讓我們不用接觸任何邪惡。

Tasks formerly reserved to state authorities

以前是國家當權機關要做的工作，

have been taken over by college graduates in their early 20s,

現在已經被二十多歲的 大學畢業生接手，

equipped with three- to five-day training --

他們只受過三到五天的訓練——

this is the qualification --

這就是必要資格——

who work on nothing less than the world's rescue.

他們的工作不亞於拯救世界。

MR: National sovereignties have been outsourced to private companies,

莫：國家統治權已經 被外包給私人公司，

and they pass on their responsibilities to third parties.

而他們又把他們的責任 轉給了第三方。

It's an outsourcing of the outsourcing of the outsourcing,

真正發生的狀況， 是外包給外包再做外包。

which takes place.

With social networks,

隨著社交網路出現，我們 要處理的是全新的基礎結構，

we are dealing with a completely new infrastructure,

with its own mechanisms,

它有自己的機制、自己的行動邏輯，

its own logic of action

and therefore, also, its own new dangers,

因此，也有自己的新危險，

which had not yet existed in the predigitalized public sphere.

在數位化之前的公共領域中 不曾有過這些危險。

HB: When Mark Zuckerberg was at the US Congress

漢：當馬克·祖伯克出席 美國國會或者歐洲議會時，

or at the European Parliament,

he was confronted with all kinds of critics.

他受到各種批評。

And his reaction was always the same:

而他的反應始終如一：

"We will fix that,

「我們會處理它，

and I will follow up on that with my team."

我和團隊會做後續追蹤。」

But such a debate shouldn't be held in back rooms of Facebook,

但這種討論不應該在臉書、推特，

Twitter or Google --

或 Google 私下進行——

such a debate should be openly discussed in new, cosmopolitan parliaments,

應該要在新的、國際性的 議會中公開討論，

in new institutions that reflect the diversity of people

在新機構中進行，要能夠 反映出對於全球網路的

contributing to a utopian project of a global network.

烏托邦計畫做出貢獻的人的多樣性。

And while it may seem impossible to consider the values

雖然，似乎不可能考量到

of users worldwide,

全世界使用者的價值觀，

it's worth believing

但仍然值得相信，

that there's more that connects us than separates us.

將我們連結在一起的東西， 比將我們分化的東西更多。

MR: Yeah, at a time when populism is gaining strength,

莫：是的，在民粹主義 越來越強大的時代，

it becomes popular to justify the symptoms,

似乎大家都會將症狀正當化，

to eradicate them,

根絕它們，讓它們隱形。

to make them invisible.

This ideology is spreading worldwide,

這種意識形態散播到全世界，

analog as well as digital,

類比和數位都是如此，

and it's our duty to stop it

我們有責任

要在太遲之前阻止它。

before it's too late.

The question of freedom and democracy

自由和民主的問題

must not only have these two options.

不能只有這兩個選項。

HB: Delete.

漢：刪除。

MR: Or ignore.

莫：或者忽略。

HB: Thank you very much.

漢：非常謝謝。

(Applause)

（掌聲）