How technology can fight extremism and online harassment | Yasmin Green

翻译人员: Phyllis Lin 校对人员: 睿哲 王

My relationship with the internet reminds me of the setup

我与互联网的关系，使我想起

恐怖片的老套情节。

to a clichéd horror movie.

You know, the blissfully happy family moves in to their perfect new home,

幸福快乐的家庭乔迁新居，

excited about their perfect future,

憧憬着他们美好的未来，

and it's sunny outside and the birds are chirping ...

窗外阳光明媚，鸟儿嘤嘤成韵...

And then it gets dark.

突然，电影的气氛变得阴暗起来。

And there are noises from the attic.

阁楼传来阵阵嘈杂的声响。

And we realize that that perfect new house isn't so perfect.

我们这才意识到，那间美好的 新房子，其实问题重重。

When I started working at Google in 2006,

2006年，当我开始在谷歌工作的时候，

Facebook was just a two-year-old,

脸书刚刚推出两年，

and Twitter hadn't yet been born.

推特还未问世。

And I was in absolute awe of the internet and all of its promise

我对互联网和它给予的承诺充满赞叹。

它承诺让我们

关系更亲密，变得更聪明，

to make us closer

and smarter

and more free.

并给予我们更多自由。

But as we were doing the inspiring work of building search engines

但是当我们开始进行这项 鼓舞人心的工作，开发搜索引擎，

视频分享和社交网络，

and video-sharing sites and social networks,

criminals, dictators and terrorists were figuring out

罪犯，独裁者和恐怖分子，

也找到了相同的平台，来攻击我们。

how to use those same platforms against us.

And we didn't have the foresight to stop them.

我们并未预见到这一点， 因而对此毫无抵抗力。

Over the last few years, geopolitical forces have come online to wreak havoc.

过去几年里，地缘政治势力 开始在网络上肆虐。

And in response,

作为反击，

Google supported a few colleagues and me to set up a new group called Jigsaw,

在谷歌的支持下，我和几位同事 成了一个名叫“Jigsaw”的新组织，

with a mandate to make people safer from threats like violent extremism,

我们肩负重任，去维护人们的安全， 让大众免受暴力极端主义，

censorship, persecution --

审查制度和迫害的威胁——

我对这些威胁深有体会， 因为我在伊朗出生，

threats that feel very personal to me because I was born in Iran,

and I left in the aftermath of a violent revolution.

在暴力革命的余波中，我离开了伊朗。

But I've come to realize that even if we had all of the resources

但我逐渐意识到，

即便我们拥有世界上 所有科技公司的资源，

of all of the technology companies in the world,

we'd still fail

我们仍然会失败，

if we overlooked one critical ingredient:

如果我们忽视那个决定性的因素：

the human experiences of the victims and perpetrators of those threats.

带来这些威胁的犯罪分子 和受害群众背后的经历。

There are many challenges I could talk to you about today.

我今天可以和各位谈很多 我们所面临的挑战。

不过我打算只着重于两点。

I'm going to focus on just two.

The first is terrorism.

其一是恐怖主义。

So in order to understand the radicalization process,

为了了解那种激进的过程，

we met with dozens of former members of violent extremist groups.

我们同数十名暴力极端主义的 前成员进行了接触。

One was a British schoolgirl,

其中有一位英国的女学生，

who had been taken off of a plane at London Heathrow

在伦敦希思罗机场，被迫下飞机，

as she was trying to make her way to Syria to join ISIS.

因为她打算到叙利亚加入ISIS组织。

And she was 13 years old.

当时她只有13岁。

So I sat down with her and her father, and I said, "Why?"

我同她和她父亲坐下来谈话， 我问到：“你为什么要这么做？”

And she said,

她说：

"I was looking at pictures of what life is like in Syria,

“我看到了叙利亚的生活写照，

and I thought I was going to go and live in the Islamic Disney World."

我以为我会住进伊斯兰的迪斯尼乐园。”

That's what she saw in ISIS.

这是她所看到ISIS组织。

She thought she'd meet and marry a jihadi Brad Pitt

她觉得，她会邂逅并嫁给 圣战分子中的布拉德 · 皮特，

and go shopping in the mall all day and live happily ever after.

整天在商场购物， 从此过上幸福快乐的生活。

ISIS understands what drives people,

ISIS组织对于驾驭人心了如指掌，

and they carefully craft a message for each audience.

他们为每一位聆听者 精心策划了要传递的信息。

Just look at how many languages

光看他们的营销材料

they translate their marketing material into.

被译成了多少种语言，便能一目了然。

They make pamphlets, radio shows and videos

他们还制作小册子，广播节目和影片，

in not just English and Arabic,

不但有英语和阿拉伯语，

but German, Russian, French, Turkish, Kurdish,

还有德语，俄语，法语， 土耳其语，库尔德语，

Hebrew,

希伯来语

Mandarin Chinese.

和汉语。

I've even seen an ISIS-produced video in sign language.

我甚至看到了ISIS组织 制作的手语视频。

Just think about that for a second:

想想吧，ISIS组织耗费了 这么多时间和精力，

ISIS took the time and made the effort

做出种种行为

to ensure their message is reaching the deaf and hard of hearing.

以确保他们的信息 能够传达给残障人士。

It's actually not tech-savviness

其实ISIS组织赢得人心与信赖，

that is the reason why ISIS wins hearts and minds.

并非因为他们精通科技。

It's their insight into the prejudices, the vulnerabilities, the desires

事实上，他们洞悉力很强， 深知他们试图接触的人

所拥有的偏见，脆弱和欲望，

of the people they're trying to reach

that does that.

因此才能实现其目的。

That's why it's not enough

这就为什么

for the online platforms to focus on removing recruiting material.

对网络平台来说，把焦点放在 移除他们的招募材料上是远远不够的。

If we want to have a shot at building meaningful technology

如果我们想要尝试建立有意义的技术，

that's going to counter radicalization,

用以打击激进主义，

we have to start with the human journey at its core.

我们就必须从人心着手。

So we went to Iraq

因此，我们去了伊拉克，

to speak to young men who'd bought into ISIS's promise

和年轻人交谈，他们深信ISIS组织

关于英勇与正义的承诺，

of heroism and righteousness,

who'd taken up arms to fight for them

曾为ISIS组织斗争，

接着，他们在目睹ISIS组织

and then who'd defected

after they witnessed the brutality of ISIS's rule.

统治的残忍无情后，选择变节。

And I'm sitting there in this makeshift prison in the north of Iraq

在伊拉克北部的临时监狱中，

with this 23-year-old who had actually trained as a suicide bomber

我面见了一位年仅23岁， 在变节前受过自杀式炸弹训练的

before defecting.

年轻人。

And he says,

他说，

"I arrived in Syria full of hope,

“我满怀希望来到叙利亚，

and immediately, I had two of my prized possessions confiscated:

但马上，我被没收了 两项最重要的东西：

my passport and my mobile phone."

我的护照和手机。”

The symbols of his physical and digital liberty

从他到达叙利亚的那刻，象征他人身自由

were taken away from him on arrival.

和电子自由的东西，就被剥夺了。

And then this is the way he described that moment of loss to me.

接着，他向我描述了那个迷失的时刻。

He said,

他说，

"You know in 'Tom and Jerry,'

“在《猫和老鼠》中，

when Jerry wants to escape, and then Tom locks the door

当杰瑞想要逃脱时，汤姆总是锁上门，

and swallows the key

并吞下钥匙，

and you see it bulging out of his throat as it travels down?"

还可以从外观看出钥匙 沿着汤姆的喉咙下滑，记得吗？

And of course, I really could see the image that he was describing,

当然，我完全想象得到 他所描绘的那种画面，

and I really did connect with the feeling that he was trying to convey,

也完全能体会到他想要传达的感受，

which was one of doom,

一种在劫难逃的感觉，

when you know there's no way out.

当你知道无路可走了。

And I was wondering:

我很纳闷：

What, if anything, could have changed his mind

在他离开家的那天，是否有任何东西

the day that he left home?

有可能改变他的想法？

So I asked,

所以我问：

"If you knew everything that you know now

“如果你当时知晓现在面临的苦难，

about the suffering and the corruption, the brutality --

腐败和残酷的状况——

that day you left home,

在你离开家的那天就知道，

would you still have gone?"

你还会选择离开吗？“

And he said, "Yes."

他说：“会的。“

And I thought, "Holy crap, he said 'Yes.'"

我心想：“我的天啊，他尽然说‘会’。”

And then he said,

接着他说：

"At that point, I was so brainwashed,

“我那时被完全洗脑了，

I wasn't taking in any contradictory information.

没有接受任何与之矛盾的信息。

I couldn't have been swayed."

没人能动摇我的决心。“

"Well, what if you knew everything that you know now

“好吧，如果在你离开的六个月前，

six months before the day that you left?"

就已知道现在发生的事情呢？”

"At that point, I think it probably would have changed my mind."

“我想那时我可能会改变心意。”

Radicalization isn't this yes-or-no choice.

极端激进并不是一道选择题。

It's a process, during which people have questions --

它是人充满疑惑时的一个过程，

about ideology, religion, the living conditions.

关于思想意识，宗教信仰，和生活条件。

And they're coming online for answers,

人们会在网上寻求答案，

which is an opportunity to reach them.

这就为我们提供了接触他们的机会。

And there are videos online from people who have answers --

有答案的人在网上提供影片——

defectors, for example, telling the story of their journey

比如，叛逃者讲述他们 投入和摆脱暴力的

into and out of violence;

心路历程；

stories like the one from that man I met in the Iraqi prison.

就像伊拉克监狱那名男子的故事。

There are locals who've uploaded cell phone footage

有当地人会上传手机影片，

of what life is really like in the caliphate under ISIS's rule.

呈现在ISIS组织统治下， 穆斯林的生活状态。

There are clerics who are sharing peaceful interpretations of Islam.

有教会圣职人士分享关于 伊斯兰教的和平诠释。

But you know what?

但你们知道吗？

这些人通常并不具备ISIS组织 那样高超的营销本领。

These people don't generally have the marketing prowess of ISIS.

They risk their lives to speak up and confront terrorist propaganda,

他们冒着生命危险， 与恐怖主义宣传对质，

and then they tragically don't reach the people

不幸的是，他们无法接触到

who most need to hear from them.

最需要听到他们声音的人。

And we wanted to see if technology could change that.

我们想试试看， 科技是否能够改变这一点。

So in 2016, we partnered with Moonshot CVE

所以在2016年，我们与 Moonshot CVE合作，

to pilot a new approach to countering radicalization

试着采用新方案打击激进主义，

called the "Redirect Method."

该方法名叫“重新定向法。”

It uses the power of online advertising

它利用线上广告的力量，

to bridge the gap between those susceptible to ISIS's messaging

在摇摆不的人们和 ISIS组织的信息之间搭建桥梁，

那些可靠的声音揭露了 大量信息的真面目。

and those credible voices that are debunking that messaging.

And it works like this:

它是这样运作的：

someone looking for extremist material --

当有人在寻找极端主义的材料——

say they search for "How do I join ISIS?" --

比如，搜索 “我如何 能加入ISIS组织？”——

will see an ad appear

就会看见一则广告出现，

that invites them to watch a YouTube video of a cleric, of a defector --

邀请他们观看圣职人士和变节人员 在Youtube网站上的视频——

someone who has an authentic answer.

那些拥有真实答案的人的心声。

And that targeting is based not on a profile of who they are,

这个方法锁定目标的方式， 不是依据他们的个人资料，

but of determining something that's directly relevant

而是由与询问和质疑

to their query or question.

直接相关的经历来决定。

During our eight-week pilot in English and Arabic,

在用英语和阿拉伯语 进行的八周试验中，

we reached over 300,000 people

我们接触了超过30万

who had expressed an interest in or sympathy towards a jihadi group.

对圣战组织表现出兴趣或者同情的人。

These people were now watching videos

这些人现在观看的影片，

that could prevent them from making devastating choices.

能预防他们做出毁灭性的选择。

And because violent extremism isn't confined to any one language,

因为暴力极端主义 并不局限于任何一种语言，

religion or ideology,

宗教和思想，

the Redirect Method is now being deployed globally

“重新定向法”现已在全球实施，

to protect people being courted online by violent ideologues,

保护大家上网时不被 暴力的意识形态诱惑，

whether they're Islamists, white supremacists

不论是伊斯兰教，白人至上主义

or other violent extremists,

或者其他暴力极端主义，

with the goal of giving them the chance to hear from someone

我们的目标是给他们机会，去聆听那些

on the other side of that journey;

旅程另一端的人怎么说；

to give them the chance to choose a different path.

帮助他们去选择不一样的人生道路。

It turns out that often the bad guys are good at exploiting the internet,

结果表明，恶人通常擅长利用网络，

not because they're some kind of technological geniuses,

并不是因为他们是科技天才，

but because they understand what makes people tick.

而是因为他们理解什么能引人注意。

I want to give you a second example:

我再举个例子：

online harassment.

网络骚扰。

Online harassers also work to figure out what will resonate

线网络骚扰者也致力于发掘能够

与他人引起共鸣的事物。

with another human being.

But not to recruit them like ISIS does,

但他们的目的不是 像ISIS组织一样招募人，

but to cause them pain.

而是给他人带来痛苦。

Imagine this:

想象一下这个状况：

you're a woman,

你是个女人，

you're married,

已婚，

you have a kid.

有一个孩子。

You post something on social media,

你在社交媒体上发表了一篇文章，

and in a reply, you're told that you'll be raped,

在评论中，有人说你会被强暴，

that your son will be watching,

你的儿子

details of when and where.

会在何时何地被监视。

In fact, your home address is put online for everyone to see.

事实上，所有人都能在网上 搜索到你的家庭地址。

That feels like a pretty real threat.

那种威胁的感觉十分真实。

Do you think you'd go home?

你觉得你还会回家吗？

Do you think you'd continue doing the thing that you were doing?

你觉得你会对此无动于衷吗？

Would you continue doing that thing that's irritating your attacker?

你会继续做惹恼了 攻击你的人的那件事吗？

Online abuse has been this perverse art

网络欺凌已经变成了 一种变态的艺术，

of figuring out what makes people angry,

发掘让人愤怒，

what makes people afraid,

让人害怕，

what makes people insecure,

让人们失去安全感的事物，

and then pushing those pressure points until they're silenced.

然后给他们施压， 直至他们永远保持沉默。

When online harassment goes unchecked,

当线上骚扰者肆意妄为时，

free speech is stifled.

言论自由就会被扼杀。

And even the people hosting the conversation

即使是主持对话的人，

throw up their arms and call it quits,

也举手投降并宣布到此为止，

closing their comment sections and their forums altogether.

把他们的留言区以及论坛全部关闭。

That means we're actually losing spaces online

这意味着我们其实失去了在网上

to meet and exchange ideas.

让思想碰撞与交流的空间。

And where online spaces remain,

在网络剩下的空间里，

we descend into echo chambers with people who think just like us.

我们被困在了“回音室”，只和 想法一致的人聚在一起。

But that enables the spread of disinformation;

但那会导致虚假信息的传播；

that facilitates polarization.

这会更加促成两极分化。

What if technology instead could enable empathy at scale?

反之，如果能用科技产生大量共鸣呢？

This was the question that motivated our partnership

这驱使我们

with Google's Counter Abuse team,

与谷歌反虐待小组，

Wikipedia

维基百科，

和《纽约时报》这类报刊合作。

and newspapers like the New York Times.

We wanted to see if we could build machine-learning models

我们想要看看是否可以获悉

that could understand the emotional impact of language.

机器学习的模型来理解 语言所造成的情感影响。

Could we predict which comments were likely to make someone else leave

我们能否推测一下， 什么样的评论最可能让人

the online conversation?

逃离网络聊天？

And that's no mean feat.

这个问题很难回答。

That's no trivial accomplishment

对人工智能来说，

能完成这些并非理所当然。

for AI to be able to do something like that.

I mean, just consider these two examples of messages

想想这两个例子，

我上周可能收到的信息。

that could have been sent to me last week.

"Break a leg at TED!"

“祝TED演讲顺利！”

... and

以及

"I'll break your legs at TED."

“我会在TED现场打断你的腿。”

(Laughter)

(笑声)

你们是人，

You are human,

that's why that's an obvious difference to you,

所以能够明显感受到 两句话意思的不同，

even though the words are pretty much the same.

即使字句基本一致。

但对于人工智能来说， 要对模型进行一定的训练，

But for AI, it takes some training to teach the models

to recognize that difference.

才能辨析那些不同之处。

The beauty of building AI that can tell the difference

创建能分辨差异的人工智能， 其美妙之处在于，

is that AI can then scale to the size of the online toxicity phenomenon,

人工智能可以大规模 防治网络毒害现象，

and that was our goal in building our technology called Perspective.

为了实现这一目标， 我们打造了名叫“观点”的技术。

With the help of Perspective,

在“观点”的帮助下，

the New York Times, for example,

以《纽约时报》为例，

他们增加了线上讨论的空间。

has increased spaces online for conversation.

Before our collaboration,

在与我们合作前，

they only had comments enabled on just 10 percent of their articles.

他们的文章只有约10%的开放评论。

With the help of machine learning,

在机器学习的协助下，

they have that number up to 30 percent.

这个数字增加到了30%。

So they've tripled it,

涨了两倍，

and we're still just getting started.

这项合作才刚刚开始而已。

But this is about way more than just making moderators more efficient.

但这绝不仅仅是让版主更有效率而已。

Right now I can see you,

现在，我可以看见你们，

and I can gauge how what I'm saying is landing with you.

我可以估量我的用词 将对你们产生怎样的影响。

You don't have that opportunity online.

线上并没有这样的机会。

Imagine if machine learning could give commenters,

想象一下，在网友打字的时候，

as they're typing,

如果机器学习能够

real-time feedback about how their words might land,

对他们的话语可能造成的影响 提供即时的反馈，会如何。

just like facial expressions do in a face-to-face conversation.

就像面对面交谈时的脸部表情。

Machine learning isn't perfect,

机器学习并不完美，

and it still makes plenty of mistakes.

它仍然会犯许多的错误。

But if we can build technology

但如果我们能够打造

that understands the emotional impact of language,

理解语言中的情感影响力的技术，

我们就能建立更多的同理心。

we can build empathy.

That means that we can have dialogue between people

那就意味着，我们可以让两人进行对话，

即使他们所属的政治体系不同，

with different politics,

different worldviews,

世界观不同，

different values.

价值观不同。

And we can reinvigorate the spaces online that most of us have given up on.

我们可以让大多数人已经 放弃的网络空间得到复苏。

When people use technology to exploit and harm others,

当人们利用科技剥削和伤害他人时，

they're preying on our human fears and vulnerabilities.

他们靠着人类的恐惧与脆弱，肆意掠夺。

If we ever thought that we could build an internet

如果我们想当然，认为能够建立一个

insulated from the dark side of humanity,

和人性阴暗面绝缘的网络，

we were wrong.

那就大错特错了。

If we want today to build technology

现如今，如果我们想要发展

that can overcome the challenges that we face,

能够克服当前所面临的挑战的科技，

we have to throw our entire selves into understanding the issues

就必须全身心投入，深刻理解问题，

and into building solutions

进而找到解决方法，

that are as human as the problems they aim to solve.

来克服人类自身的种种问题。

Let's make that happen.

让我们一起实现它吧。

Thank you.

谢谢大家！

(Applause)

(掌声)