Damon Horowitz calls for a "moral operating system"

翻译人员: Xiaoqiao Xie 校对人员: Angelia King

Power.

能力

That is the word that comes to mind.

是我们时时在想的

We're the new technologists.

我们是新时代的技术人员

We have a lot of data, so we have a lot of power.

我们有很多信息，这样我们就有能力

How much power do we have?

多少能力呢？

Scene from a movie: "Apocalypse Now" -- great movie.

想像电影“现代启示录”， 这电影绝了

We've got to get our hero, Captain Willard, to the mouth of the Nung River

里面有个英雄，卫尔沃德上尉，来到怒河的入海口

so he can go pursue Colonel Kurtz.

去追踪科特兹上校

The way we're going to do this is fly him in and drop him off.

想像他飞到那里，下飞机

So the scene:

想像这个情景：

the sky is filled with this fleet of helicopters carrying him in.

天空布满了陪他来的直升飞机

And there's this loud, thrilling music in the background,

背景音乐震人心肺

this wild music.

非常野性

♫ Dum da ta da dum ♫

♫ Dum da ta da dum ♫

♫ Dum da ta da dum ♫

♫ Dum da ta da dum ♫

♫ Da ta da da ♫

♫ Da ta da da ♫

That's a lot of power.

充满了力量

That's the kind of power I feel in this room.

这就是我在这个房间里感觉到的能量

That's the kind of power we have

这就是我们拥有的能量

because of all of the data that we have.

就因为我们拥有信息

Let's take an example.

举个例子

What can we do

用个人信息

with just one person's data?

我们能做什么呢？

What can we do

用这位先生的信息

with that guy's data?

我们能做什么？

I can look at your financial records.

我可以查看你的财政历史

I can tell if you pay your bills on time.

看你是不是按时付账单

I know if you're good to give a loan to.

来决定给不给你贷款

I can look at your medical records; I can see if your pump is still pumping --

我可以浏览你的医疗病历，看你的心脏好不好

see if you're good to offer insurance to.

来决定给不给你保险

I can look at your clicking patterns.

我可以跟踪你上网的习惯

When you come to my website, I actually know what you're going to do already

当你访问我的网站时，我已经知道你会怎么做了

because I've seen you visit millions of websites before.

因为我已经看到你以前是怎么访问成千上万个网站的

And I'm sorry to tell you,

我不得不说

you're like a poker player, you have a tell.

你就像是个赌客，你能被看透

I can tell with data analysis what you're going to do

就凭你的信息我能看透你将会做什么

before you even do it.

在你还没做之前

I know what you like. I know who you are,

我知道你喜欢什么，我知道你是谁

and that's even before I look at your mail

在我看你的邮件之前

or your phone.

看你的电话之前

Those are the kinds of things we can do

现今我们能这样做了

with the data that we have.

凭着我们有的信息

But I'm not actually here to talk about what we can do.

但我来这里不是谈我们能做什么

I'm here to talk about what we should do.

我是来谈谈我们应该做什么

What's the right thing to do?

如何善用信息？

Now I see some puzzled looks

我看到一些人迷惑了

like, "Why are you asking us what's the right thing to do?

好像在想：“为什么你问我们应该如何善用信息？

We're just building this stuff. Somebody else is using it."

我们只管把技术推出来，用是别人的事情。”

Fair enough.

就算你对

But it brings me back.

但这正是我要说的

I think about World War II --

想想二次世界大战

some of our great technologists then,

很多我们最优秀的技术人员

some of our great physicists,

最优秀的物理学家

studying nuclear fission and fusion --

在研究核裂变和核聚变

just nuclear stuff.

都是关于核的

We gather together these physicists in Los Alamos

我们把这些科学家请到洛斯阿拉莫斯国家实验室

to see what they'll build.

来看看他们想用这些技术来做什么

We want the people building the technology

我们希望，人们能够在发明新技术时

thinking about what we should be doing with the technology.

时时想着我们应该怎么用这些技术

So what should we be doing with that guy's data?

话说回来，我们应该怎么用这位先生的信息？

Should we be collecting it, gathering it,

我们应不应该收集他的信息

so we can make his online experience better?

来使他的上网冲浪的体验更好？

So we can make money?

我们应不应该用这些信息来赚钱？

So we can protect ourselves

应不应该用这些信息来自我保护

if he was up to no good?

以防万一他干坏事？

Or should we respect his privacy,

还是我们应该尊重他的隐私

protect his dignity and leave him alone?

为他保留尊严，不要烦他？

Which one is it?

哪一个办法更好？

How should we figure it out?

怎么做决定？

I know: crowdsource. Let's crowdsource this.

我知道：集思广益。让我们来听听大家的意见

So to get people warmed up,

先来个热身题

let's start with an easy question --

简单的问题——

something I'm sure everybody here has an opinion about:

这个问题我相信大家都有倾向

iPhone versus Android.

苹果电话还是谷歌Android电话？

Let's do a show of hands -- iPhone.

让我们举手表决，支持苹果的

Uh huh.

恩

Android.

现在支持谷歌Android的

You'd think with a bunch of smart people

原以为这里都是聪明人

we wouldn't be such suckers just for the pretty phones.

不会只为了漂亮徒有其表而盲目消费。

(Laughter)

（笑声）

Next question,

下一个问题，

a little bit harder.

有点难

Should we be collecting all of that guy's data

我们应不应该收集这位先生的信息

to make his experiences better

来让他的体验更好

and to protect ourselves in case he's up to no good?

同时来保护我们自身，万一他干坏事？

Or should we leave him alone?

还是我们应该不管他？

Collect his data.

支持收集信息的

Leave him alone.

支持不管他的

You're safe. It's fine.

（这位先生）你是安全的

(Laughter)

（笑声）

Okay, last question --

现在，最后一个问题——

harder question --

更难一点——

when trying to evaluate

当我们试图决定

what we should do in this case,

我们应该怎么做的时候

should we use a Kantian deontological moral framework,

是应该用康德的义务论道德框架，

or should we use a Millian consequentialist one?

还是用米利安的结果主义论？

Kant.

支持康德的

Mill.

支持米利安的

Not as many votes.

没什么人知道哦。

(Laughter)

（笑声）

Yeah, that's a terrifying result.

这是个惊人的结果

Terrifying, because we have stronger opinions

惊人，因为我们关心

about our hand-held devices

手持设备的电话

than about the moral framework

比关心道德理论更多

we should use to guide our decisions.

我们应该用这些理论来指导我们的决定

How do we know what to do with all the power we have

如何决定我们该怎么来使用我们所拥有的能力

if we don't have a moral framework?

如果我们连道德框架都没有？

We know more about mobile operating systems,

我们对移动电话操作系统知道得越来越多

but what we really need is a moral operating system.

但是我们需要知道的是道德的操作系统

What's a moral operating system?

什么是道德操作系统？

We all know right and wrong, right?

我们知道什么是对，什么是错

You feel good when you do something right,

你做对了的时候，自我感觉挺好

you feel bad when you do something wrong.

如果你做错了你觉得自责

Our parents teach us that: praise with the good, scold with the bad.

父母亲教导我们，奖对惩错

But how do we figure out what's right and wrong?

但是怎么才能知道什么是对什么是错？

And from day to day, we have the techniques that we use.

尤其是当每天新技术都层出不穷

Maybe we just follow our gut.

说不定我们可以就靠本能

Maybe we take a vote -- we crowdsource.

说不定我们需要推选——群众意见集思广益

Or maybe we punt --

或者我们赌一把——

ask the legal department, see what they say.

问法律专家的意见，看他们怎么说

In other words, it's kind of random,

也就是说，我们作出决定的办法

kind of ad hoc,

很随机

how we figure out what we should do.

很即兴

And maybe, if we want to be on surer footing,

或者，如果我们想更保险些

what we really want is a moral framework that will help guide us there,

只想要一个好的道德框架，我们能用来自己弄明白的

that will tell us what kinds of things are right and wrong in the first place,

能帮我们明辨对错

and how would we know in a given situation what to do.

帮我们决定什么情况下做什么

So let's get a moral framework.

让我们找到一个道德框架

We're numbers people, living by numbers.

我们用很多数字，生活在数字中

How can we use numbers

怎么用数字

as the basis for a moral framework?

来建立一个道德框架？

I know a guy who did exactly that.

我知道一个人专门干这个的

A brilliant guy --

很聪明的人——

he's been dead 2,500 years.

2500年前就死了

Plato, that's right.

柏拉图。对了

Remember him -- old philosopher?

还记得他？老哲学家？

You were sleeping during that class.

在他的教学课上，你可能睡过去了。

And Plato, he had a lot of the same concerns that we did.

柏拉图，他有很多我们有的担心

He was worried about right and wrong.

他担心如何辨明是非

He wanted to know what is just.

他希望知道怎么衡量

But he was worried that all we seem to be doing

他担心我们现在做的

is trading opinions about this.

不过是交换意见

He says something's just. She says something else is just.

你一个主意我一个主意

It's kind of convincing when he talks and when she talks too.

听起来都很有理

I'm just going back and forth; I'm not getting anywhere.

我只是前瞻后顾，最后还是没结果

I don't want opinions; I want knowledge.

我不需要意见，我需要的是真理

I want to know the truth about justice --

我需要知道什么是有关正义的真理--

like we have truths in math.

就像我们研究什么是数学的真理一样

In math, we know the objective facts.

在数学中，我们使用客观的事实

Take a number, any number -- two.

比如将一个数字，任何数字——二

Favorite number. I love that number.

个人最爱

There are truths about two.

二有一些事实可讲

If you've got two of something,

如果你有两个东西

you add two more, you get four.

你再加上两个，就是四个

That's true no matter what thing you're talking about.

这是真理，不管你讨论的东西是什么

It's an objective truth about the form of two,

这是关于二的客观真理

the abstract form.

抽象性

When you have two of anything -- two eyes, two ears, two noses,

当你有两个东西，任何东西——两只眼睛，两只耳朵，两个鼻子

just two protrusions --

或者仅仅是两个小突起——

those all partake of the form of two.

他们都是二的表现

They all participate in the truths that two has.

都分享了关于二的事实

They all have two-ness in them.

都有二在其中

And therefore, it's not a matter of opinion.

所以，个人情绪没关系

What if, Plato thought,

如果柏拉图考虑

ethics was like math?

道德，就像数学一样，会怎么样？

What if there were a pure form of justice?

如果真有纯粹的公正，会怎么样？

What if there are truths about justice,

如果公正真有绝对的事实，

and you could just look around in this world

你可以环顾四周

and see which things participated,

看能用在什么上

partook of that form of justice?

什么东西有着绝对的公正，会怎么样？

Then you would know what was really just and what wasn't.

这样你就能知道什么是真的对，什么是真的错

It wouldn't be a matter

和事情的外表

of just opinion or just appearances.

和个人的观点都没有关系

That's a stunning vision.

这是个诱人的观点

I mean, think about that. How grand. How ambitious.

我是说，想象一下，多么壮观，多么充满雄心

That's as ambitious as we are.

这是我们最大的野心了

He wants to solve ethics.

他希望解决道德的问题

He wants objective truths.

他希望客观的事实

If you think that way,

如果你这么想

you have a Platonist moral framework.

你就有柏拉图式的道德观。

If you don't think that way,

如果你不这么认为

well, you have a lot of company in the history of Western philosophy,

那你就在西方哲学史上有很多同僚

because the tidy idea, you know, people criticized it.

因为人们总爱挑剔这简明的观点。

Aristotle, in particular, he was not amused.

特别是亚里士多德，他就不买帐

He thought it was impractical.

他觉得这不实际

Aristotle said, "We should seek only so much precision in each subject

亚里士多德说：“我们看每个物体

as that subject allows."

都只能精确到这个物体允许的程度。”

Aristotle thought ethics wasn't a lot like math.

亚里士多德认为道德不像是数学

He thought ethics was a matter of making decisions in the here-and-now

他认为道德是我在当时当地

using our best judgment

作出的最好的判断

to find the right path.

从而引导我们的决定

If you think that, Plato's not your guy.

如果你这么想，柏拉图就不是你这边的了

But don't give up.

但是别放弃

Maybe there's another way

或者还有一条路

that we can use numbers as the basis of our moral framework.

我们能用数字，作为我们道德观的基础。

How about this:

这个怎么样：

What if in any situation you could just calculate,

如果你能在任何情况下都算出

look at the choices,

衡量所有的选择

measure out which one's better and know what to do?

算出怎么做更好，什么应该做？

That sound familiar?

听起来熟悉是不是？

That's a utilitarian moral framework.

这就是实用主义的道德观

John Stuart Mill was a great advocate of this --

约翰·斯图尔特·密尔是大支持者——

nice guy besides --

很友善的一个人——

and only been dead 200 years.

可惜死了两百年了

So basis of utilitarianism --

所以实用主义的基础——

I'm sure you're familiar at least.

我详细你们都略知一二

The three people who voted for Mill before are familiar with this.

刚才三个米尔的支持者肯定知道

But here's the way it works.

我还是讲讲这个怎么运作的

What if morals, what if what makes something moral

假如道德，道德的内涵

is just a matter of if it maximizes pleasure

只是为了最大的享受

and minimizes pain?

和最少的痛苦？

It does something intrinsic to the act.

这在根本上决定了我们的行为

It's not like its relation to some abstract form.

看起来和任何抽象的形式有关

It's just a matter of the consequences.

只是一系列的必然结果

You just look at the consequences

你只看必然结果

and see if, overall, it's for the good or for the worse.

看总的来说是好结果还是坏结果

That would be simple. Then we know what to do.

这看起来简单，然后我们就知道怎么做

Let's take an example.

让我们看一个例子

Suppose I go up

假设我上前

and I say, "I'm going to take your phone."

说：“我要没收你的手机。”

Not just because it rang earlier,

不仅仅是因为刚才你的手机响了

but I'm going to take it because I made a little calculation.

而是因为我刚算计了一下

I thought, that guy looks suspicious.

我觉得，这个人看起来可疑

And what if he's been sending little messages to Bin Laden's hideout --

说不定他是在给本拉登的藏身处发消息——

or whoever took over after Bin Laden --

或者是跟本拉登的接班人发消息——

and he's actually like a terrorist, a sleeper cell.

他看起来真像是恐怖分子，一个卧底

I'm going to find that out, and when I find that out,

我得把他揪出来，当我揪他时

I'm going to prevent a huge amount of damage that he could cause.

我就能防止他将带来一个大危害

That has a very high utility to prevent that damage.

防止这个危害的结果非常有意义

And compared to the little pain that it's going to cause --

和我的行为将带来的小小痛苦相比——

because it's going to be embarrassing when I'm looking on his phone

因为在我检查他的手机的时候

and seeing that he has a Farmville problem and that whole bit --

要是看到他只是在玩线上游戏，是挺丢人的

that's overwhelmed

但是这不抵

by the value of looking at the phone.

检查手机能带来的价值

If you feel that way,

如果你这么想

that's a utilitarian choice.

就是实用主义的选择

But maybe you don't feel that way either.

可能你也并不赞同这个做法

Maybe you think, it's his phone.

可能你会想，这是他的手机

It's wrong to take his phone

拿他的手机是不对的

because he's a person

他是个人

and he has rights and he has dignity,

他有人权，有尊严

and we can't just interfere with that.

我们不能干涉

He has autonomy.

他有自主权

It doesn't matter what the calculations are.

不管计算结果是什么

There are things that are intrinsically wrong --

这个举动在本质上是错误的——

like lying is wrong,

就像撒谎是错误的

like torturing innocent children is wrong.

像是折磨无辜的孩童是错误的

Kant was very good on this point,

康德非常坚持这一点

and he said it a little better than I'll say it.

他解释得更好一点

He said we should use our reason

他说我们应该用我们自己的理由

to figure out the rules by which we should guide our conduct,

来弄明白决定该怎么做的准则

and then it is our duty to follow those rules.

接下来我们的责任是遵守这些准则

It's not a matter of calculation.

不是关于计算。

So let's stop.

别再算了

We're right in the thick of it, this philosophical thicket.

我们正在这错综复杂的哲学中间

And this goes on for thousands of years,

而且已经像这样上千年了

because these are hard questions,

因为这些是很难的问题

and I've only got 15 minutes.

我只有十五分钟

So let's cut to the chase.

所以让我们直奔主题

How should we be making our decisions?

我们应该怎么做决定？

Is it Plato, is it Aristotle, is it Kant, is it Mill?

是听从柏拉图，亚里士多德，康德，还是米尔？

What should we be doing? What's the answer?

我们该做什么？答案是什么？

What's the formula that we can use in any situation

能在任何情况下计算出我们应该怎么做的

to determine what we should do,

万灵的公式在哪里？

whether we should use that guy's data or not?

我们该还是不该用这位先生的数据？

What's the formula?

公式是什么？

There's not a formula.

没有公式

There's not a simple answer.

没有简单明了的答案

Ethics is hard.

道德观是很难的

Ethics requires thinking.

道德观需要思想

And that's uncomfortable.

这是不太容易的

I know; I spent a lot of my career

我知道。在我的职业生涯中我花了很多时间

in artificial intelligence,

研究机器人

trying to build machines that could do some of this thinking for us,

试图造出机器来替我们做决定

that could give us answers.

来给我们答案

But they can't.

但是他们做不到

You can't just take human thinking

你不能就拿人的思维方式

and put it into a machine.

放在机器里

We're the ones who have to do it.

我们要靠自己思考

Happily, we're not machines, and we can do it.

好的方面是我们不是机器，我们能思考

Not only can we think,

不光能思考

we must.

我们必须思考

Hannah Arendt said,

汉娜·阿伦特说过

"The sad truth

“悲哀的是

is that most evil done in this world

世上大多数的罪恶

is not done by people

不是由有恶意的人

who choose to be evil.

故意造成的

It arises from not thinking."

而是人没有想清楚。”

That's what she called the "banality of evil."

这就是她所谓的“平庸的罪恶”

And the response to that

对策是

is that we demand the exercise of thinking

我们需要练习思考

from every sane person.

每个平常人都要

So let's do that. Let's think.

让我们来做吧，来思考

In fact, let's start right now.

事实上，让我们现在就做

Every person in this room do this:

房间里每个人都这样做：

think of the last time you had a decision to make

想像上一次你需要做一个决定的时候

where you were worried to do the right thing,

当你担心不能做出正确的决定

where you wondered, "What should I be doing?"

当你怀疑：“到底该怎么做呢？”

Bring that to mind,

想象那个时刻

and now reflect on that

之后再想

and say, "How did I come up that decision?

“我当时是怎么做决定的？”

What did I do? Did I follow my gut?

我做了什么？是随我意愿的么？

Did I have somebody vote on it? Or did I punt to legal?"

征求他人意见了么？征求法律意见了么？

Or now we have a few more choices.

现在我们有更多选择

"Did I evaluate what would be the highest pleasure

“我有没有像米尔会做的那样

like Mill would?

衡量最大的享受？

Or like Kant, did I use reason to figure out what was intrinsically right?"

或者像康德，借助原因来弄明白什么是本质上对的？”

Think about it. Really bring it to mind. This is important.

想象一下，真的深入其中，这是很重要的

It is so important

有多重要

we are going to spend 30 seconds of valuable TEDTalk time

我们将会用三十秒，非常宝贵的TED的时间

doing nothing but thinking about this.

什么也不做，就思考

Are you ready? Go.

准备好了么？开始

Stop. Good work.

结束，不错

What you just did,

你刚做的

that's the first step towards taking responsibility

是向着为我们使用我们的能力

for what we should do with all of our power.

负责任迈出的第一步

Now the next step -- try this.

现在第二步——试试这个

Go find a friend and explain to them

找到一个朋友，向他们解释

how you made that decision.

你是怎么做决定的

Not right now. Wait till I finish talking.

不是现在，等我讲完了之后。

Do it over lunch.

午饭的时候做

And don't just find another technologist friend;

别找另一个搞技术的朋友

find somebody different than you.

找一个和你很不同的

Find an artist or a writer --

艺术家或者作家——

or, heaven forbid, find a philosopher and talk to them.

或者，哲学家，和他们谈

In fact, find somebody from the humanities.

事实上找个人文主义者谈

Why? Because they think about problems

为什么？因为他们考虑问题的角度

differently than we do as technologists.

和我们技术人员是很不一样的

Just a few days ago, right across the street from here,

仅仅几天前，街对面

there was hundreds of people gathered together.

成百上千人聚集起来

It was technologists and humanists

都是技术人员和人文主义者

at that big BiblioTech Conference.

在开一个大的研讨会

And they gathered together

他们聚集起来

because the technologists wanted to learn

因为技术人员想了解

what it would be like to think from a humanities perspective.

从人文主义者的角度想是怎么样的

You have someone from Google

有从谷歌来的人

talking to someone who does comparative literature.

和研究比较文学的人谈

You're thinking about the relevance of 17th century French theater --

你在想十七世纪的法国戏剧的影响——

how does that bear upon venture capital?

是怎么在风险投资下存活的？

Well that's interesting. That's a different way of thinking.

这很有趣，是另一个角度看问题。

And when you think in that way,

我们在这样想的时候

you become more sensitive to the human considerations,

就开始对人类的考量更敏感

which are crucial to making ethical decisions.

这对于做出道德的决定是很重要的

So imagine that right now

想像现在

you went and you found your musician friend.

你去找个音乐家的朋友

And you're telling him what we're talking about,

和他说我们现在的话题

about our whole data revolution and all this --

这些信息革命等等——

maybe even hum a few bars of our theme music.

甚至哼哼我们的一些电影主题音乐。

♫ Dum ta da da dum dum ta da da dum ♫

♫ Dum ta da da dum dum ta da da dum ♫

Well, your musician friend will stop you and say,

你的音乐家朋友会打断你说：

"You know, the theme music

“你知道，你技术革命的

for your data revolution,

主题音乐

that's an opera, that's Wagner.

是个戏剧，是瓦格纳的作品。

It's based on Norse legend.

关于一个北欧传说

It's Gods and mythical creatures

是关于上帝和神话人物

fighting over magical jewelry."

为了魔术珍宝而争斗。”

That's interesting.

这很有趣

Now it's also a beautiful opera,

现在这也是个动人的戏剧

and we're moved by that opera.

我们都被感动了

We're moved because it's about the battle

因为这是关于一场战争

between good and evil,

善与恶之间的

about right and wrong.

对与错之间的

And we care about right and wrong.

我们关心对和错

We care what happens in that opera.

我们关心在戏剧里发生了什么

We care what happens in "Apocalypse Now."

我们关心在“现代启示录”里发生了什么

And we certainly care

我们也关心

what happens with our technologies.

用我们的技术会发生什么

We have so much power today,

当下我们有这么多的能力

it is up to us to figure out what to do,

这是我们的责任来搞清楚该怎么做。

and that's the good news.

这是好消息

We're the ones writing this opera.

我们是写剧本的人

This is our movie.

这就是我们的电影

We figure out what will happen with this technology.

我们清楚这个技术将会带给我们什么。

We determine how this will all end.

我们认为事情会这样结束

Thank you.

谢谢

(Applause)

（掌声）