Damon Horowitz calls for a "moral operating system"

譯者: Ana Choi 審譯者: Shelley Krishna Tsang

Power.

權力。

That is the word that comes to mind.

這就是在我腦海呈現的詞語。

We're the new technologists.

我們是新的科技專家。

We have a lot of data, so we have a lot of power.

我們有很多數據資料，所以我們有很多權力。

How much power do we have?

我們究竟有幾大威力？

Scene from a movie: "Apocalypse Now" -- great movie.

這是從電影＜現代啟示錄＞其中一幕--很棒的電影。

We've got to get our hero, Captain Willard, to the mouth of the Nung River

我們必須讓我們的英雄，韋勒上尉，到儂河的河口

so he can go pursue Colonel Kurtz.

去追尋寇茲上校。

The way we're going to do this is fly him in and drop him off.

我們要帶他飛往目的地然後讓他前行。

So the scene:

場景如下：

the sky is filled with this fleet of helicopters carrying him in.

天空佈滿了運載他的直升機艦隊。

And there's this loud, thrilling music in the background,

附帶這個響亮，驚險的背景音樂，

this wild music.

這瘋狂的音樂。

♫ Dum da ta da dum ♫

♫ 音樂聲 ♫

♫ Dum da ta da dum ♫

♫ 音樂聲 ♫

♫ Da ta da da ♫

♫ 音樂聲 ♫

That's a lot of power.

這是強勁的威力。

That's the kind of power I feel in this room.

這就是在我在這個房間感覺到的力量。

That's the kind of power we have

這就我們擁有的一種力量，

because of all of the data that we have.

因為我們有所有的資料數據。

Let's take an example.

讓我們舉一個例子。

What can we do

若我們有著一個人的資料數據,

with just one person's data?

我們可以用它做什麼？

What can we do

我們可以用此人

with that guy's data?

的資料做什麼？

I can look at your financial records.

我可以看看你的財務記錄。

I can tell if you pay your bills on time.

我可知道你是否按時支付帳單。

I know if you're good to give a loan to.

我會知道應否給你予貸款。

I can look at your medical records; I can see if your pump is still pumping --

我可以看看你的醫療記錄，看看你的身體機能是否運行如常--

see if you're good to offer insurance to.

看看應否給你推銷健康保險。

I can look at your clicking patterns.

我可以看看你的點擊模式。

When you come to my website, I actually know what you're going to do already

當你來到我的網站，其實我已經知道你要做的事情，

because I've seen you visit millions of websites before.

因為我已經看過你之前到過的數百萬個網站。

And I'm sorry to tell you,

而且我很抱歉地告訴你，

you're like a poker player, you have a tell.

你像一個撲克玩家，你有一個「偷雞」破綻。

I can tell with data analysis what you're going to do

我可以使用分析你的數據

before you even do it.

來知道你的下一步。

I know what you like. I know who you are,

我知道你的喜嗜。我知道你是誰。

and that's even before I look at your mail

而且我還未看你的電郵

or your phone.

或手機。

Those are the kinds of things we can do

這都我們可以用數據

with the data that we have.

達到的事情。

But I'm not actually here to talk about what we can do.

但其實我不是來談我們能夠做些什麼。

I'm here to talk about what we should do.

我是來談談我們應該做什麼。

What's the right thing to do?

什麼是正確的事情？

Now I see some puzzled looks

我看到你們有些困惑的樣子

like, "Why are you asking us what's the right thing to do?

像在問：「你為何問我們應該做什麼?

We're just building this stuff. Somebody else is using it."

我們只是建造這種東西供給別人所用。」

Fair enough.

合理的。

But it brings me back.

但它使我回到這一點。

I think about World War II --

我想到第二次世界大戰 --

some of our great technologists then,

我們有一些重要的技術人員，

some of our great physicists,

我們的一些偉大的物理學家，

studying nuclear fission and fusion --

研究核裂變和核聚變 --

just nuclear stuff.

只是核子的科學。

We gather together these physicists in Los Alamos

我們聚集這些物理學家在洛斯阿拉莫斯國家實驗室，

to see what they'll build.

看看他們將構建些什麼。

We want the people building the technology

我們要建設技術的人

thinking about what we should be doing with the technology.

想想我們應該用技術建做些什麼。

So what should we be doing with that guy's data?

那麼，我們應該用那傢伙的數據做什麼？

Should we be collecting it, gathering it,

我們是否應該收集它，薈集它，

so we can make his online experience better?

用來使他有更好的上網體驗？

So we can make money?

用來使我們可以賺錢？

So we can protect ourselves

用來保護我們自己，

if he was up to no good?

若假他實行一些壞主意？

Or should we respect his privacy,

或我們應該尊重他的隱私，

protect his dignity and leave him alone?

保護他的尊嚴，讓他獨自自我？

Which one is it?

我們該怎樣?

How should we figure it out?

我們應該如何看著辦？

I know: crowdsource. Let's crowdsource this.

我知道：眾包。讓我們用眾包的方法解決問題。

So to get people warmed up,

好，為了讓你們做好準備就绪，

let's start with an easy question --

讓我們先從一個簡單的問題開始 --

something I'm sure everybody here has an opinion about:

一個我敢肯定在座每個人都有異議的問題：

iPhone versus Android.

iPhone 抑或 Android電話。

Let's do a show of hands -- iPhone.

讓我們舉手 -- iPhone。

Uh huh.

嗯。

Android.

Android。

You'd think with a bunch of smart people

哼,還以為一班聰明人

we wouldn't be such suckers just for the pretty phones.

我們不會出現這種只顧漂亮手機的人。

(Laughter)

<笑聲>

Next question,

接下來的問題，

a little bit harder.

有點更困難。

Should we be collecting all of that guy's data

我們應否該收集那傢伙所有的資料數據

to make his experiences better

使他有更好的上網體驗

and to protect ourselves in case he's up to no good?

並用來保護我們自己若假他實行一些壞主意？

Or should we leave him alone?

或我們應該讓他獨自自我？

Collect his data.

收集他的資料數據。

Leave him alone.

讓他獨自自我。

You're safe. It's fine.

你是安全的。沒有問題。

(Laughter)

<笑聲>

Okay, last question --

好吧，最後一個問題--

harder question --

更困難的問題。

when trying to evaluate

當我們在這種情況下

what we should do in this case,

試圖評估我們應該如何，

should we use a Kantian deontological moral framework,

我們應否使用康德的道義論的道德框架，

or should we use a Millian consequentialist one?

或我們應否使用莉恩結果主義的道德框架，

Kant.

康德的。

Mill.

莉恩的。

Not as many votes.

比上一次少表決。

(Laughter)

<笑聲>

Yeah, that's a terrifying result.

是的，這是一個可怕的結果。

Terrifying, because we have stronger opinions

可怕，因為我們對手機的意見

about our hand-held devices

比我們對用來指導

than about the moral framework

我們決定的道德框架

we should use to guide our decisions.

的意見更強。

How do we know what to do with all the power we have

如果我們沒有一個道德框架，

if we don't have a moral framework?

我們怎麼知道用我們所有的力量該怎麼做？

We know more about mobile operating systems,

我們對關於手機的操作系統知道更十分多，

but what we really need is a moral operating system.

但我們真正需要的是一個道德的操作系統。

What's a moral operating system?

什麼是道德的操作系統?

We all know right and wrong, right?

我們都知道是,非。

You feel good when you do something right,

你做一些好的事會有好的感覺，

you feel bad when you do something wrong.

你做一些錯事會有不好的感覺。

Our parents teach us that: praise with the good, scold with the bad.

我們的父母教導我們：讚美好的，罵壞的。

But how do we figure out what's right and wrong?

但是，我們究境如何找出什麼是正確的和錯誤的？

And from day to day, we have the techniques that we use.

而每過一天又一天，我們用我們的方法權衡。

Maybe we just follow our gut.

也許我們只要按照我們的膽量。

Maybe we take a vote -- we crowdsource.

也許我們需要投票 - 我們的人群的來源。

Or maybe we punt --

或者，也許我們用篙撐 --

ask the legal department, see what they say.

問問法律部門，看他們說些什麼。

In other words, it's kind of random,

換句話說，它是一種隨機的，

kind of ad hoc,

即席的,

how we figure out what we should do.

的權衡方法。

And maybe, if we want to be on surer footing,

也或許，若我們想在基礎上更有把握，

what we really want is a moral framework that will help guide us there,

我們真正需要的是一種道德框架，有助於引導我們到那裡，

that will tell us what kinds of things are right and wrong in the first place,

指導我們從一開始什麼樣是正確和錯誤的,

and how would we know in a given situation what to do.

以及指導我們在特定情況下該怎麼辦。

So let's get a moral framework.

因此，讓我們弄一個道德框架。

We're numbers people, living by numbers.

我們數字的人，生活於數字裡。

How can we use numbers

我們如何使用數字

as the basis for a moral framework?

作為道德框架的基礎？

I know a guy who did exactly that.

我知道一個傢伙正恰恰如此辦，

A brilliant guy --

一個傑出的傢伙 --

he's been dead 2,500 years.

他已經死了2500年。

Plato, that's right.

不錯, 是柏拉圖。

Remember him -- old philosopher?

老哲學家--記得他嗎？

You were sleeping during that class.

你在那課時正在睡覺。

And Plato, he had a lot of the same concerns that we did.

柏拉圖，他有很多和我們相同的問題。

He was worried about right and wrong.

他憂慮着對與錯。

He wanted to know what is just.

他想知道什麼是正義的。

But he was worried that all we seem to be doing

但他擔心，似乎我們所做的

is trading opinions about this.

只是交流有關這一點的意見。

He says something's just. She says something else is just.

公說公有理。 婆說婆有理。

It's kind of convincing when he talks and when she talks too.

雙方首都非常有說服力。

I'm just going back and forth; I'm not getting anywhere.

來來回回; 沒有結果。

I don't want opinions; I want knowledge.

我不想要意見，我想要知識。

I want to know the truth about justice --

我想知道公義的真相 --

like we have truths in math.

像我們在數學有的真理。

In math, we know the objective facts.

在數學，我們知道客觀的事實。

Take a number, any number -- two.

例如一個數字，任何數字 -- 「二」。

Favorite number. I love that number.

最喜愛的數字。我愛這個數字。

There are truths about two.

有關於「二」的真理。

If you've got two of something,

如果你有兩個東西，

you add two more, you get four.

你加兩個，你會得到四個。

That's true no matter what thing you're talking about.

這是真的，無論你在說什麼事情。

It's an objective truth about the form of two,

這是一個有關「二」客觀的真理，

the abstract form.

抽象的形式。

When you have two of anything -- two eyes, two ears, two noses,

當你有兩個東西 -- 兩隻眼睛，兩隻耳朵，兩個鼻子，

just two protrusions --

只有兩個突起的物體 --

those all partake of the form of two.

所有參與「二」的形式。

They all participate in the truths that two has.

他們都參與了「二」。

They all have two-ness in them.

「二」是他們所有。

And therefore, it's not a matter of opinion.

因此，它不是一個見仁見智的局勢。

What if, Plato thought,

如果，柏拉圖在想，

ethics was like math?

道德是好比數學？

What if there were a pure form of justice?

假如有一個純粹的正義形式呢？

What if there are truths about justice,

如果公義有真理，

and you could just look around in this world

你可以看看周圍環顧這個世界，

and see which things participated,

看看哪些東西參與，

partook of that form of justice?

參加這種形式的正義？

Then you would know what was really just and what wasn't.

那你便會知道什麼是真正公正，什麼不是。

It wouldn't be a matter

這不會是只有關意見

of just opinion or just appearances.

或表面的問題。

That's a stunning vision.

這是一個驚人的遠見。

I mean, think about that. How grand. How ambitious.

試想想,這一點。這是如何宏偉, 如何雄心勃勃。

That's as ambitious as we are.

這是如我們的雄心勃勃。

He wants to solve ethics.

他要解決道德。

He wants objective truths.

他要客觀真理。

If you think that way,

如果你這樣思想，

you have a Platonist moral framework.

你便擁有一個柏拉圖主義的道德框架。

If you don't think that way,

如果你不認為這樣，

well, you have a lot of company in the history of Western philosophy,

那，你在歷史上西方哲學史便有很多支持者，

because the tidy idea, you know, people criticized it.

因為整齊的想法 --- 眾所週知，人們會批評。

Aristotle, in particular, he was not amused.

尤其是亞里士多德，他便不覺得有趣。

He thought it was impractical.

他認為這是不切實際的。

Aristotle said, "We should seek only so much precision in each subject

亞里士多德說，「我們只應該尋求每個主題允許

as that subject allows."

我們能夠尋求的問題。」

Aristotle thought ethics wasn't a lot like math.

亞里士多德不認為道德好比數學。

He thought ethics was a matter of making decisions in the here-and-now

他認為道德是一個在此時此地用我們最好的

using our best judgment

判斷決策的問題,

to find the right path.

來找到正確的道路。

If you think that, Plato's not your guy.

如果你這樣認為，柏拉圖不是你的同道。

But don't give up.

但不要放棄。

Maybe there's another way

也許有另一種方式，

that we can use numbers as the basis of our moral framework.

我們可以使用數字作為我們道德框架的依據。

How about this:

這樣如何：

What if in any situation you could just calculate,

如果在任何情況下你可以計算，

look at the choices,

看看選擇，

measure out which one's better and know what to do?

測算出哪一個更好，知道該怎麼辦？

That sound familiar?

這聽起來很熟悉？

That's a utilitarian moral framework.

這是一個功利主義的道德框架。

John Stuart Mill was a great advocate of this --

穆勒是便主張這一點 --

nice guy besides --

好好的傢伙 --

and only been dead 200 years.

而且只死了200年。

So basis of utilitarianism --

所以功利主義的基礎 --

I'm sure you're familiar at least.

我敢肯定你會熟悉。

The three people who voted for Mill before are familiar with this.

至少剛才那三個舉手贊同穆勒的人對他熟悉。

But here's the way it works.

現在讓我說說該怎麼樣。

What if morals, what if what makes something moral

若果我說道德，或某一件擁有道德的東西,

is just a matter of if it maximizes pleasure

只不過是擴大快樂限度

and minimizes pain?

以及減少痛苦？

It does something intrinsic to the act.

這種思想方式便有一些固定性。

It's not like its relation to some abstract form.

它不好比一些抽象的形式。

It's just a matter of the consequences.

這只是一個後果的問題。

You just look at the consequences

你只要看看後果，

and see if, overall, it's for the good or for the worse.

看看總體而言，它是好還是壞。

That would be simple. Then we know what to do.

這便簡單。我們知道該怎麼做。

Let's take an example.

讓我們舉一個例子。

Suppose I go up

假如我說，

and I say, "I'm going to take your phone."

「我要拿帶走你的手機。」

Not just because it rang earlier,

不是因為它剛才響過，

but I'm going to take it because I made a little calculation.

而是因為我做了一個小小的計算。

I thought, that guy looks suspicious.

我想，那傢伙看起來有可疑。

And what if he's been sending little messages to Bin Laden's hideout --

如果他一直有消息發送到拉登藏身之處 --

or whoever took over after Bin Laden --

或誰接手拉登之位 --

and he's actually like a terrorist, a sleeper cell.

或者他實際上像個恐怖分子，臥底細胞。

I'm going to find that out, and when I find that out,

我會去了解，當我查出了，

I'm going to prevent a huge amount of damage that he could cause.

我將會防止他構成的巨大損害。

That has a very high utility to prevent that damage.

具有很高的實用工具，以防止該傷害。

And compared to the little pain that it's going to cause --

這相比於小小不便的痛苦--

because it's going to be embarrassing when I'm looking on his phone

因為這將會是個令人尷尬的時候,

and seeing that he has a Farmville problem and that whole bit --

當我在他的電話，看到他竟然沉迷於Farmville遊戲 --

that's overwhelmed

這是不堪相比於

by the value of looking at the phone.

看他手機的價值。

If you feel that way,

如果你這樣認為，

that's a utilitarian choice.

這是一個功利思想的選擇。

But maybe you don't feel that way either.

但也許你覺得都不應該是這樣。

Maybe you think, it's his phone.

也許你認為，這是他的電話。

It's wrong to take his phone

拿他的電話看是不對，

because he's a person

因為他是一個人，

and he has rights and he has dignity,

他有他的權利和尊嚴，

and we can't just interfere with that.

我們不能干擾。

He has autonomy.

他有自主權。

It doesn't matter what the calculations are.

什麼計算根本不緊要。

There are things that are intrinsically wrong --

有些事情是一件固的壞事 --

like lying is wrong,

例如說謊是不對的，

like torturing innocent children is wrong.

例如虐待無辜的孩子是錯的。

Kant was very good on this point,

康德於這一點非常滿意，

and he said it a little better than I'll say it.

而且他說這一點比我會說得較好。

He said we should use our reason

他說我們應該用我們的原因找出規律，

to figure out the rules by which we should guide our conduct,

應該以這種方式引導我們的行為。

and then it is our duty to follow those rules.

然後，我們有責任遵循這些規則。

It's not a matter of calculation.

這不是一個計算的問題。

So let's stop.

因此，讓我們停下來。

We're right in the thick of it, this philosophical thicket.

我們就在它的要緊點，在這哲學叢林內。

And this goes on for thousands of years,

而這任其發展了千百年，

because these are hard questions,

因為這些都是難以回答的問題，

and I've only got 15 minutes.

而我只有15分鐘。

So let's cut to the chase.

因此，讓我們切入正題。

How should we be making our decisions?

我們應該如何營造我們的決策？

Is it Plato, is it Aristotle, is it Kant, is it Mill?

是柏拉圖，亞里士多德，是康德，抑或是密爾？

What should we be doing? What's the answer?

我們應該怎麼做？答案該是什麼？

What's the formula that we can use in any situation

什麼是我們可以在任何情況下都使用的公式,

to determine what we should do,

來確定了我們應該做的，

whether we should use that guy's data or not?

我們是否應該使用那傢伙的數據?

What's the formula?

有什麼公式？

There's not a formula.

根本沒有一個公式。

There's not a simple answer.

沒有一個簡單的答案。

Ethics is hard.

道德是艱難的。

Ethics requires thinking.

道德要求思想。

And that's uncomfortable.

這是不舒服的地帶。

I know; I spent a lot of my career

我知道，在我的職業生涯中

in artificial intelligence,

我花了很多時間在人工智能上，

trying to build machines that could do some of this thinking for us,

試圖建立一些可以替我們做這些思想的機器，

that could give us answers.

給我們一些答案。

But they can't.

但它們不能。

You can't just take human thinking

你不能拿人的思想

and put it into a machine.

把它輸入機器。

We're the ones who have to do it.

那些是我們要自己做。

Happily, we're not machines, and we can do it.

令人高興的是，我們不是機器，我們可以做到這一點。

Not only can we think,

我們不僅可以思想，

we must.

我們必須要思想。

Hannah Arendt said,

漢娜阿倫特說:

"The sad truth

「可悲的是，

is that most evil done in this world

這個世界最邪惡的事

is not done by people

不是由選擇作惡

who choose to be evil.

的人所做。

It arises from not thinking."

而它是產生於不思想。」

That's what she called the "banality of evil."

這就是她所謂的「平庸的邪惡」。

And the response to that

而根據這一要求，

is that we demand the exercise of thinking

是我們要要求每一個思維健全的人

from every sane person.

鍛煉思想。

So let's do that. Let's think.

因此，讓我們這樣做。讓我們思考。

In fact, let's start right now.

事實上，我們現在開始。

Every person in this room do this:

每個人在這個房間裡要這樣做：

think of the last time you had a decision to make

想起你上次要做出決定，

where you were worried to do the right thing,

當你擔心是否做正確的決定，

where you wondered, "What should I be doing?"

你在想，「我應該怎樣做？」

Bring that to mind,

想想。

and now reflect on that

反思一下

and say, "How did I come up that decision?

並問，「我是怎麼作出這個決定？

What did I do? Did I follow my gut?

我做了什麼？難道我跟著我的直覺？

Did I have somebody vote on it? Or did I punt to legal?"

是否有人投票呢？還是我舉法律？」

Or now we have a few more choices.

或者，現在我們有更多些選擇。

"Did I evaluate what would be the highest pleasure

「我是否會使用穆勒的最高快樂

like Mill would?

來評價決定？

Or like Kant, did I use reason to figure out what was intrinsically right?"

或像康德，我是否會使用因由弄清楚什麼是本質？」

Think about it. Really bring it to mind. This is important.

真正想想看。這一點很重要。

It is so important

它是如此的重要，

we are going to spend 30 seconds of valuable TEDTalk time

我們將要花30秒寶貴的TEDTalk時間

doing nothing but thinking about this.

什麼也不做，並去思考這一點。

Are you ready? Go.

你準備好嗎？開始。

Stop. Good work.

停止。好。

What you just did,

你剛才做的，

that's the first step towards taking responsibility

這就用是我們所有力量的責任,

for what we should do with all of our power.

承擔對於我們應該做的第一步。

Now the next step -- try this.

現在，試試這下一個步驟。

Go find a friend and explain to them

去找一個朋友，向他們解釋

how you made that decision.

你如何作出這個決定。

Not right now. Wait till I finish talking.

不是現在。等到我談話結束。

Do it over lunch.

可以在午餐談談。

And don't just find another technologist friend;

而且不要只是找另一個技師朋友;

find somebody different than you.

找一個與你不同的人。

Find an artist or a writer --

找一個藝術家或作家 --

or, heaven forbid, find a philosopher and talk to them.

或者，上天保佑，找到一個哲學家交談。

In fact, find somebody from the humanities.

或者，找一個從事文科學的人。

Why? Because they think about problems

為什麼？因為他們思考問題

differently than we do as technologists.

不同於我們做技術的人。

Just a few days ago, right across the street from here,

就在數天前，從這裡在對面街，

there was hundreds of people gathered together.

有數百人聚集在一起。

It was technologists and humanists

這是技術人員和人道主義者

at that big BiblioTech Conference.

在那個BiblioTech大會議。

And they gathered together

他們聚集在一起，

because the technologists wanted to learn

因為技術人員想了解

what it would be like to think from a humanities perspective.

從文人會有什麼樣的視角遠景。

You have someone from Google

有人從谷歌

talking to someone who does comparative literature.

與做比較文學的人交談。

You're thinking about the relevance of 17th century French theater --

你在考慮17世紀法國戲劇的相關性 --

how does that bear upon venture capital?

如何會對風險投資有影響？

Well that's interesting. That's a different way of thinking.

這是有趣的。這是一個不同的思維方式。

And when you think in that way,

而當你以這種方式思想，

you become more sensitive to the human considerations,

你變得更加對人的考慮敏感，

which are crucial to making ethical decisions.

這是對決策倫理至關重要的。

So imagine that right now

所以現在想想

you went and you found your musician friend.

你去找你的音樂家朋友。

And you're telling him what we're talking about,

告訴他我們在說什麼，

about our whole data revolution and all this --

有關我們整個數據革命的一切 --

maybe even hum a few bars of our theme music.

甚至哼了幾節我們的主題音樂。

♫ Dum ta da da dum dum ta da da dum ♫

♫音樂♫

Well, your musician friend will stop you and say,

那麼，你的音樂家朋友會對你說，

"You know, the theme music

「你要知道，為你數據的革命

for your data revolution,

的主題音樂,

that's an opera, that's Wagner.

是一部歌劇，是瓦格納的。

It's based on Norse legend.

它是基於北歐傳說。

It's Gods and mythical creatures

是神和神話動物

fighting over magical jewelry."

爭奪神奇首飾。」

That's interesting.

那很有意思。

Now it's also a beautiful opera,

當然，它也是一個美麗的歌劇。

and we're moved by that opera.

而我們對歌劇感動。

We're moved because it's about the battle

我們很感動，因為它是

between good and evil,

對善惡之間的戰鬥，

about right and wrong.

對與錯的戰鬥。

And we care about right and wrong.

而我們是關心是與非。

We care what happens in that opera.

我們關心在歌劇會發生什麼。

We care what happens in "Apocalypse Now."

我們關心在<現代啟示錄>會發生什麼。

And we certainly care

我們當然關心

what happens with our technologies.

我們的技術會發生什麼。

We have so much power today,

我們今天有這麼大的權力，

it is up to us to figure out what to do,

它是由我們來搞清楚怎樣做。

and that's the good news.

這就是好消息。

We're the ones writing this opera.

我們是寫這部歌劇的作家。

This is our movie.

這是我們的電影。

We figure out what will happen with this technology.

我們會弄清楚這種技術將會發生什麼。

We determine how this will all end.

我們確定全部如何將結束。

Thank you.

謝謝。

(Applause)

(掌聲)