What moral decisions should driverless cars make? | Iyad Rahwan

譯者: Lilian Chiu 審譯者: NAN-KUN WU

Today I'm going to talk about technology and society.

今天我要談的是科技與社會。

The Department of Transport estimated that last year

交通部估計去年

35,000 people died from traffic crashes in the US alone.

在美國有 35,000 人死於交通事故。

Worldwide, 1.2 million people die every year in traffic accidents.

全球，每年有 120 萬人 死於交通意外。

If there was a way we could eliminate 90 percent of those accidents,

如果有方法可以避免 90% 的那些意外，

would you support it?

你們願意支持嗎？

Of course you would.

當然你們會願意。

This is what driverless car technology promises to achieve

這就是無人駕駛車技術 許諾要達成的目標，

by eliminating the main source of accidents --

做法是消除意外事故的主要源頭：

human error.

人類錯誤。

Now picture yourself in a driverless car in the year 2030,

想像一下，2030 年， 你坐在一臺無人駕駛的車內，

sitting back and watching this vintage TEDxCambridge video.

你靠坐著，看著這支極佳的 TEDxCambridge 影片。

(Laughter)

（笑聲）

All of a sudden,

突然間，

the car experiences mechanical failure and is unable to stop.

這臺車發生機械故障，無法停下來。

If the car continues,

如果車繼續前進，

it will crash into a bunch of pedestrians crossing the street,

它會撞上過馬路的一群行人，

but the car may swerve,

但可以把車突然轉向，

hitting one bystander,

只撞死一個旁觀者，

killing them to save the pedestrians.

撞死他就可以救一群行人。

What should the car do, and who should decide?

這臺車會怎麼做？誰來決定？

What if instead the car could swerve into a wall,

如果改成，這臺車可以 突然轉向去撞牆，

crashing and killing you, the passenger,

撞毀後只有乘客──也就是你── 會身亡，

in order to save those pedestrians?

這樣就能夠救那些行人，如何？

This scenario is inspired by the trolley problem,

這個情境的靈感是來自於電車問題，

which was invented by philosophers a few decades ago

那是幾十年前哲學家所發明的問題，

to think about ethics.

做道德思考用。

Now, the way we think about this problem matters.

我們思考這個問題的方式 是很重要的。

We may for example not think about it at all.

比如，我們有可能完全不去想它。

We may say this scenario is unrealistic,

我們可能會說，這個情境不實際、

incredibly unlikely, or just silly.

非常不可能發生或很蠢。

But I think this criticism misses the point

但我認為這種批判是失焦了，

because it takes the scenario too literally.

因為它看這個情境看得太表面。

Of course no accident is going to look like this;

當然不會有像這樣子的意外發生；

no accident has two or three options

沒有意外會有兩、三個選項，

where everybody dies somehow.

且每個選項都有人死。

Instead, the car is going to calculate something

情況是，這臺車去做計算，

like the probability of hitting a certain group of people,

比如計算撞到某一群人的機率，

if you swerve one direction versus another direction,

如果你向這個方向急轉彎， 跟另一個方向做比較，

you might slightly increase the risk to passengers or other drivers

你可能會稍微增加乘客 或其他駕駛人的風險，

versus pedestrians.

跟行人的風險比較等等。

It's going to be a more complex calculation,

這會是個比較複雜的計算，

but it's still going to involve trade-offs,

但它仍然會涉及取捨，

and trade-offs often require ethics.

而取捨通常都會需要用到道德。

We might say then, "Well, let's not worry about this.

我們可能會說：「別擔心這個。

Let's wait until technology is fully ready and 100 percent safe."

我們可以等到技術完全 準備好且 100% 安全。」

Suppose that we can indeed eliminate 90 percent of those accidents,

假設我們的確可以減少 90% 的意外事故，

or even 99 percent in the next 10 years.

或是在接下來十年甚至達到 99%。

What if eliminating the last one percent of accidents

如果說要把最後 1% 的 意外事故都消除，

requires 50 more years of research?

會需要再多做 50 年的研究呢？

Should we not adopt the technology?

我們不該採用這個技術嗎？

That's 60 million people dead in car accidents

如果我們繼續用目前的方式，

那就會有 6000 萬人死亡。

if we maintain the current rate.

So the point is,

所以重點是，

waiting for full safety is also a choice,

等到完全安全也是一個選擇，

and it also involves trade-offs.

而這個選擇也涉及取捨。

People online on social media have been coming up with all sorts of ways

人們在網路社會媒體上 提出各式各樣的方式，

to not think about this problem.

來避免思考這個問題。

One person suggested the car should just swerve somehow

有個人建議，車應該要急轉彎，

in between the passengers --

穿過這些路人──

(Laughter)

（笑聲）

and the bystander.

和旁觀者之間。

Of course if that's what the car can do, that's what the car should do.

當然，如果這臺車能這麼做， 那它是應該這麼做。

We're interested in scenarios in which this is not possible.

我們感興趣的是 沒辦法這樣做的情況。

And my personal favorite was a suggestion by a blogger

我個人的最愛， 是一個部落客的建議，

to have an eject button in the car that you press --

他建議車要裝設一個 彈射按鈕，讓你可以──

(Laughter)

（笑聲）

just before the car self-destructs.

在車自毀之前按下它。

(Laughter)

（笑聲）

So if we acknowledge that cars will have to make trade-offs on the road,

所以，如果我們承認車在 路上行駛時會需要做取捨，

how do we think about those trade-offs,

我們要如何去想那些取捨？

and how do we decide?

我們要如何做決定？

Well, maybe we should run a survey to find out what society wants,

也許我們該做個調查 來了解社會想要什麼，

because ultimately,

因為，最終，

regulations and the law are a reflection of societal values.

規定和法律都是反應出社會價值。

So this is what we did.

所以我們就這麼做了。

With my collaborators,

我和我的共同研究者，

Jean-François Bonnefon and Azim Shariff,

尚方斯華．邦納方和阿辛．夏利夫，

we ran a survey

一起進行了一項調查，

in which we presented people with these types of scenarios.

調查中，我們給人們看這些情境。

We gave them two options inspired by two philosophers:

我們給予他們兩個選項， 選項靈感來自兩位哲學家：

Jeremy Bentham and Immanuel Kant.

傑瑞米．邊沁及伊曼努爾．康德。

Bentham says the car should follow utilitarian ethics:

邊沁說，車應該要遵循特定的道德：

it should take the action that will minimize total harm --

它應該要採取 能讓傷害最小的行為──

even if that action will kill a bystander

即使那個行為會害死一名旁觀者，

and even if that action will kill the passenger.

即使那個行為會害死一名乘客。

Immanuel Kant says the car should follow duty-bound principles,

康德說，車應該要遵循 責無旁貸原則，

like "Thou shalt not kill."

比如「你不該殺人」。

So you should not take an action that explicitly harms a human being,

所以你不能採取很明確 會傷害到人類的行為，

and you should let the car take its course

你應該讓車就照它的路去走，

even if that's going to harm more people.

即使那最後會傷害到更多人。

What do you think?

你們認為如何？

Bentham or Kant?

邊沁或康德？

Here's what we found.

我們發現的結果如下。

Most people sided with Bentham.

大部分人站在邊沁這一邊。

So it seems that people want cars to be utilitarian,

所以，似乎人們會希望 車是功利主義的，

minimize total harm,

把總傷害降到最低，

and that's what we should all do.

我們所有人都該這麼做。

Problem solved.

問題解決。

But there is a little catch.

但有一個小難處。

When we asked people whether they would purchase such cars,

當我們問人們， 他們是否會買這樣的車，

they said, "Absolutely not."

他們說：「絕對不會。」

(Laughter)

（笑聲）

They would like to buy cars that protect them at all costs,

他們會買不計代價來保護他們的車，

but they want everybody else to buy cars that minimize harm.

但他們希望其他人都買 能把傷害減到最低的車。

(Laughter)

（笑聲）

We've seen this problem before.

我們以前就看過這個問題。

It's called a social dilemma.

它叫做社會兩難。

And to understand the social dilemma,

要了解社會兩難，

we have to go a little bit back in history.

我們得要回溯一下歷史。

In the 1800s,

在 1800 年代，

English economist William Forster Lloyd published a pamphlet

英國經濟學家威廉．佛斯特．洛伊 出版了一本小冊子，

which describes the following scenario.

小冊子描述了下列情境：

You have a group of farmers --

有一群農夫

English farmers --

──英國農夫──

who are sharing a common land for their sheep to graze.

他們在一塊共用地上面 放牧他們的羊。

Now, if each farmer brings a certain number of sheep --

每個農夫都會帶來一定數量的羊──

let's say three sheep --

比如說三隻羊──

the land will be rejuvenated,

這塊地會更生，

the farmers are happy,

農夫都很快樂，

the sheep are happy,

羊也很快樂，

everything is good.

一切都很好。

Now, if one farmer brings one extra sheep,

現在，如果一個農夫多帶了一隻羊，

that farmer will do slightly better, and no one else will be harmed.

那位農夫的生計會更好一些， 其他人也沒受害。

But if every farmer made that individually rational decision,

但如果每位農夫都各別 做出了那個理性的決定，

the land will be overrun, and it will be depleted

就會超過這塊地的限度， 資源會被用盡，

to the detriment of all the farmers,

所有農夫都會受害，

and of course, to the detriment of the sheep.

當然，羊也會受害。

We see this problem in many places:

我們在許多地方都會看到這類問題：

in the difficulty of managing overfishing,

比如管理過度捕魚的困難，

or in reducing carbon emissions to mitigate climate change.

或是減少碳排放來緩和氣候變遷。

When it comes to the regulation of driverless cars,

當用到規範無人駕駛車的情況時，

the common land now is basically public safety --

這塊共用地基本上就是公共安全──

that's the common good --

也就是共善──

and the farmers are the passengers

農夫就是乘客，

or the car owners who are choosing to ride in those cars.

或是選擇坐這種車的車主。

And by making the individually rational choice

他們各自做出理性的選擇，

of prioritizing their own safety,

把他們自己的安全擺在第一，

they may collectively be diminishing the common good,

那麼整體來說， 他們就可能會削減了共善，

which is minimizing total harm.

共善就是把總傷害減到最低。

It's called the tragedy of the commons,

這就是所謂的「公地悲劇」，

traditionally,

傳統是這麼稱呼的。

but I think in the case of driverless cars,

但我認為在無人駕駛車的情況中，

the problem may be a little bit more insidious

問題可能還有涉及其他隱患，

because there is not necessarily an individual human being

因為並沒有特別一個人類

making those decisions.

在做那些決定。

So car manufacturers may simply program cars

汽車製造商很可能就會把他們的車

that will maximize safety for their clients,

設計成以確保客戶安全為第一要務，

and those cars may learn automatically on their own

而那些車可能會自己發現，

that doing so requires slightly increasing risk for pedestrians.

保護客戶會稍微增加行人的風險。

So to use the sheep metaphor,

若用放牧羊的比喻，

it's like we now have electric sheep that have a mind of their own.

就像是有電子羊，自己會思考。

(Laughter)

（笑聲）

And they may go and graze even if the farmer doesn't know it.

他們可能會在農夫不知道的 情況下自己去吃草。

So this is what we may call the tragedy of the algorithmic commons,

所以我們可以稱之為 「演算法公地悲劇」，

and if offers new types of challenges.

它會帶來新的挑戰。

Typically, traditionally,

通常，傳統上，

we solve these types of social dilemmas using regulation,

我們會用規制來解決這些社會兩難，

so either governments or communities get together,

所以政府或是社區會集合起來，

and they decide collectively what kind of outcome they want

他們共同決定想要什麼樣的結果、

and what sort of constraints on individual behavior

以及對於個人行為，他們需要實施

they need to implement.

什麼樣的限制。

And then using monitoring and enforcement,

接著透過監控和執行，

they can make sure that the public good is preserved.

他們就能確保公善能被維護。

So why don't we just,

身為管理者，

as regulators,

我們為什麼不

require that all cars minimize harm?

要求所有的車都要把 傷害降至最低就好了？

After all, this is what people say they want.

畢竟，這是人民要的。

And more importantly,

更重要的，

I can be sure that as an individual,

我可以確定，身為個人，

if I buy a car that may sacrifice me in a very rare case,

如果我買了一臺有可能在 非常少數情況下把我犧牲的車，

I'm not the only sucker doing that

至少我不會是唯一的冤大頭，

while everybody else enjoys unconditional protection.

因為人人都享有無條件保護。

In our survey, we did ask people whether they would support regulation

在我們的調查中， 我們有問人們是否支持規制，

and here's what we found.

這是我們的發現。

First of all, people said no to regulation;

首先，人們對規制說不；

and second, they said,

第二，他們說：

"Well if you regulate cars to do this and to minimize total harm,

「如果你們規定車都要這樣做， 並把總傷害降至最低，

I will not buy those cars."

我不會去買那些車。」

So ironically,

所以，很諷刺，

by regulating cars to minimize harm,

藉由規範汽車來將傷害降至最小，

we may actually end up with more harm

結果卻會是更多的傷害，

because people may not opt into the safer technology

因為人們可能不會選擇 比較安全的科技，

even if it's much safer than human drivers.

即使科技比人類駕駛還安全。

I don't have the final answer to this riddle,

關於這個謎，我沒有最終的答案，

but I think as a starting point,

但我認為，做為一個起始點，

we need society to come together

我們需要社會能團結起來，

to decide what trade-offs we are comfortable with

來決定我們對何種取捨 會覺得比較舒服，

and to come up with ways in which we can enforce those trade-offs.

並提出方法來執行這些取捨。

As a starting point, my brilliant students,

做為一個起始點，我的出色學生們，

Edmond Awad and Sohan Dsouza,

艾德蒙．艾瓦和索漢．達蘇札，

built the Moral Machine website,

建立了「道德機器」網站，

which generates random scenarios at you --

它會為你產生隨機的情境，

basically a bunch of random dilemmas in a sequence

基本上是一連串的隨機兩難，

where you have to choose what the car should do in a given scenario.

你得要選擇在給定的情境中， 車要怎麼做。

And we vary the ages and even the species of the different victims.

我們會改變受害者的年齡和物種。

So far we've collected over five million decisions

目前我們已經從 來自全世界超過 100 萬人，

by over one million people worldwide

收集到了超過 500 萬個決定，

from the website.

透過網站取得的。

And this is helping us form an early picture

這能協助我們勾勒出初步的狀況，

of what trade-offs people are comfortable with

了解人們對怎樣的取捨 感到比較舒服、

and what matters to them --

什麼對他們比較重要──

even across cultures.

且是跨文化的。

But more importantly,

但更重要的，

doing this exercise is helping people recognize

做這個演練，能協助人們認清

the difficulty of making those choices

做那些決定有多困難、

and that the regulators are tasked with impossible choices.

而管理者的工作就是要 做不可能的選擇。

And maybe this will help us as a society understand the kinds of trade-offs

也許這能協助我們這個社會 去了解這類的取捨，

that will be implemented ultimately in regulation.

終究這類的取捨在將來也會 被納入到規制當中。

And indeed, I was very happy to hear

的確，我非常高興聽到

that the first set of regulations

來自交通部的

that came from the Department of Transport --

第一組規定

announced last week --

在上週宣佈，

included a 15-point checklist for all carmakers to provide,

內容有一張清單，上面有 15 個項目， 是要汽車製造商避免的，

and number 14 was ethical consideration --

而第 14 項就是道德考量──

how are you going to deal with that.

你要如何去處理它。

We also have people reflect on their own decisions

也有人會反思他們自己的決定，

by giving them summaries of what they chose.

我們會把人們所做的決定 歸納給他們。

I'll give you one example --

我舉一個例子給各位──

I'm just going to warn you that this is not your typical example,

我要先警告各位， 這不是典型的例子，

your typical user.

使用者也不是典型的。

This is the most sacrificed and the most saved character for this person.

這個人最會犧牲的角色（小孩） 及最會拯救的角色（貓）是這些。

(Laughter)

（笑聲）

Some of you may agree with him,

有些人可能認同他，

or her, we don't know.

或她，性別不詳。

But this person also seems to slightly prefer passengers over pedestrians

但這個人對於乘客的偏好 也略高於行人，

in their choices

從他們的選擇看得出來，

and is very happy to punish jaywalking.

且非常樂意懲罰闖紅燈的人。

(Laughter)

（笑聲）

So let's wrap up.

來總結收尾一下。

We started with the question -- let's call it the ethical dilemma --

我們一開頭問的問題， 就姑且稱之為道德兩難，

of what the car should do in a specific scenario:

在特定的情境中， 車該怎麼做：

swerve or stay?

急轉彎或保持原路？

But then we realized that the problem was a different one.

但接著我們了解到， 這個問題有所不同。

It was the problem of how to get society to agree on and enforce

問題是要如何讓社會認同，

the trade-offs they're comfortable with.

並且執行讓他們覺得舒服的取捨。

It's a social dilemma.

這是個社會兩難。

In the 1940s, Isaac Asimov wrote his famous laws of robotics --

在四〇年代，以撒．艾西莫夫 寫了著名的機器人學法則──

the three laws of robotics.

機器人學的三大法則。

A robot may not harm a human being,

機器人不得傷害人類，

a robot may not disobey a human being,

機器人不得違背人類，

and a robot may not allow itself to come to harm --

且機器人不得 讓它自己受到傷害──

in this order of importance.

重要性就是依這個順序。

But after 40 years or so

約四十年後，

and after so many stories pushing these laws to the limit,

許多故事已經把 這些法則推到了極限，

Asimov introduced the zeroth law

艾西莫夫又提出了第零條法則，

which takes precedence above all,

其重要性高於前述的所有法則，

and it's that a robot may not harm humanity as a whole.

這條法則是， 機器人不得傷害整體人性。

I don't know what this means in the context of driverless cars

我不知道在無人駕駛車的情況中， 或任何明確的情境中，

or any specific situation,

這條法則是什麼意思，

and I don't know how we can implement it,

我也不知道我們要如何實施它，

but I think that by recognizing

但我認為，透過認清

that the regulation of driverless cars is not only a technological problem

無人駕駛車的規範不只是科技問題，

but also a societal cooperation problem,

也是社會合作問題，

I hope that we can at least begin to ask the right questions.

我希望我們至少能 開始問出對的問題。

Thank you.

謝謝。

(Applause)

（掌聲）