What moral decisions should driverless cars make? | Iyad Rahwan

Prevoditelj: Senzos Osijek Recezent: Sanda L

Today I'm going to talk about technology and society.

Danas ću govoriti o tehnologiji i društvu.

The Department of Transport estimated that last year

Odjel za transport je procijenio da je prošle godine

35,000 people died from traffic crashes in the US alone.

35,000 ljudi umrlo u prometnim nesrećama samo u SAD-u.

Worldwide, 1.2 million people die every year in traffic accidents.

Diljem svijeta, 1,2 milijuna ljudi umre svake godine u prometnim nesrećama.

If there was a way we could eliminate 90 percent of those accidents,

Kad bi postojao način na koji bismo mogli eliminirati 90 posto tih nesreća,

would you support it?

biste li ga podržali?

Of course you would.

Naravno da biste.

This is what driverless car technology promises to achieve

Upravo ovo obećava postići tehnologija automobila bez vozača

by eliminating the main source of accidents --

eliminirajući glavni uzrok nesreća --

human error.

ljudsku pogrešku.

Now picture yourself in a driverless car in the year 2030,

Sada zamislite sebe u automobilu bez vozača u 2030. godini,

sitting back and watching this vintage TEDxCambridge video.

kako sjedite otraga i gledate ovaj zastarjeli TEDxCambridge video.

(Laughter)

(Smijeh)

All of a sudden,

Odjednom,

the car experiences mechanical failure and is unable to stop.

automobilu se dogodio mehanički kvar i ne može se zaustaviti.

If the car continues,

Ako automobil nastavi voziti,

it will crash into a bunch of pedestrians crossing the street,

sudarit će se s hrpom pješaka koji prelaze ulicu,

but the car may swerve,

ali automobil može i skrenuti,

hitting one bystander,

udarajući nekoga tko stoji sa strane,

killing them to save the pedestrians.

usmrćujući ga kako bi spasio pješake.

What should the car do, and who should decide?

Što automobil treba učiniti i tko o tome treba odlučiti?

What if instead the car could swerve into a wall,

Što ako bi, umjesto toga, automobil mogao skrenuti u zid,

crashing and killing you, the passenger,

sudarajući se i ubijajući tebe, putnika,

in order to save those pedestrians?

kako bi spasio te pješake?

This scenario is inspired by the trolley problem,

Ovaj scenarij je inspiriran problemom trolejbusa,

which was invented by philosophers a few decades ago

kojeg su se dosjetili filozofi prije nekoliko desetljeća,

to think about ethics.

da bi promišljali o etici.

Now, the way we think about this problem matters.

Zato je važan način na koji razmišljamo o ovom problemu.

We may for example not think about it at all.

Možemo, na primjer, uopće ne razmišljati o njemu.

We may say this scenario is unrealistic,

Možemo reći kako je taj scenarij nerealan,

incredibly unlikely, or just silly.

iznimno nevjerojatan ili jednostavno blesav.

But I think this criticism misses the point

Ali, ja smatram da takva osuda promašuje poantu

because it takes the scenario too literally.

jer shvaća taj scenarij previše doslovno.

Of course no accident is going to look like this;

Naravno da nijedna nesreća neće izgledati ovako;

no accident has two or three options

nijedna nesreća nema dvije ili tri opcije

where everybody dies somehow.

u kojima svi nekako stradaju.

Instead, the car is going to calculate something

Umjesto toga, automobil će izračunati nešto

like the probability of hitting a certain group of people,

poput vjerojatnosti udaranja u određenu skupinu ljudi,

if you swerve one direction versus another direction,

ako skrenete u jednom smjeru naspram drugog smjera,

you might slightly increase the risk to passengers or other drivers

mogli biste neznatno povećati rizik za putnike ili druge vozače

versus pedestrians.

naspram pješaka.

It's going to be a more complex calculation,

To će biti složenija kalkulacija,

but it's still going to involve trade-offs,

ali će i dalje uključivati kompromise,

and trade-offs often require ethics.

a za kompromis je često potrebna moralna prosudba.

We might say then, "Well, let's not worry about this.

Možemo onda reći „Pa, nemojmo se brinuti oko toga.

Let's wait until technology is fully ready and 100 percent safe."

Pričekajmo dok tehnologija ne bude potpuno spremna i 100 posto sigurna.“

Suppose that we can indeed eliminate 90 percent of those accidents,

Pretpostavimo da zaista možemo uspjeti eliminirati 90 posto tih nesreća,

or even 99 percent in the next 10 years.

ili čak 99 posto u idućih 10 godina.

What if eliminating the last one percent of accidents

Što ako uklanjanje tog zadnjeg postotka nesreća

requires 50 more years of research?

zahtijeva još dodatnih 50 godina istraživanja?

Should we not adopt the technology?

Zar ne trebamo usvojiti tehnologiju?

That's 60 million people dead in car accidents

To je 60 milijuna ljudi poginulih u automobilskim nesrećama,

if we maintain the current rate.

ako zadržimo trenutnu stopu.

So the point is,

Stoga poanta je da,

waiting for full safety is also a choice,

čekanje potpune sigurnosti je također izbor

and it also involves trade-offs.

i isto uključuje kompromise.

People online on social media have been coming up with all sorts of ways

Ljudi prisutni na društvenim mrežama pronalaze svakakve načine

to not think about this problem.

kako ne misliti o ovom problemu.

One person suggested the car should just swerve somehow

Jedna je osoba predložila da automobil samo skrene nekako

in between the passengers --

između putnika --

(Laughter)

(Smijeh)

and the bystander.

i promatrača.

Of course if that's what the car can do, that's what the car should do.

Naravno, ako to automobil može učiniti, onda to i treba učiniti.

We're interested in scenarios in which this is not possible.

Nas zanimaju scenariji u kojima ovo nije moguće.

And my personal favorite was a suggestion by a blogger

I moj osobni favorit bio je prijedlog blogera

to have an eject button in the car that you press --

da u automobilu imamo gumb za izbacivanje koji pritisnemo --

(Laughter)

(Smijeh)

just before the car self-destructs.

netom prije no što se automobil sam uništi.

(Laughter)

(Smijeh)

So if we acknowledge that cars will have to make trade-offs on the road,

Dakle, ako priznamo da će automobili morati raditi kompromise na cesti,

how do we think about those trade-offs,

kako ćemo razmišljati o tim kompromisima

and how do we decide?

i kako ćemo donijeti odluku?

Well, maybe we should run a survey to find out what society wants,

Pa, možda bismo trebali provesti anketu da saznamo što društvo želi,

because ultimately,

jer naposljetku,

regulations and the law are a reflection of societal values.

regulacije i zakon su odraz društvenih vrijednosti.

So this is what we did.

Stoga smo učinili ovo.

With my collaborators,

S mojim suradnicima,

Jean-François Bonnefon and Azim Shariff,

Jean-Françoisem Bonnefonom i Azimom Shariffom,

we ran a survey

proveli smo istraživanje

in which we presented people with these types of scenarios.

u kojem smo predstavili ljudima ove vrste scenarija.

We gave them two options inspired by two philosophers:

Dali smo im dvije mogućnosti inspirirane dvama filozofima:

Jeremy Bentham and Immanuel Kant.

Jeremyem Benthamom i Immanuelom Kantom.

Bentham says the car should follow utilitarian ethics:

Bentham kaže da automobil treba pratiti utilitarnu etiku:

it should take the action that will minimize total harm --

treba poduzeti akciju koja će minimalizirati totalnu štetu --

even if that action will kill a bystander

čak i ako će ta akcija ubiti promatrača

and even if that action will kill the passenger.

te čak i ako će ta akcija ubiti putnika.

Immanuel Kant says the car should follow duty-bound principles,

Immanuel Kant kaže da automobil treba pratiti moralna načela,

like "Thou shalt not kill."

poput „Ne ubij.“

So you should not take an action that explicitly harms a human being,

Zato ne bismo trebali poduzeti akciju koja eksplicitno šteti ljudskom biću

and you should let the car take its course

i trebali bismo pustiti da automobil ide svojim putem,

even if that's going to harm more people.

čak i ako će to ozlijediti više ljudi.

What do you think?

Što vi mislite?

Bentham or Kant?

Bentham ili Kant?

Here's what we found.

Evo što smo mi pronašli.

Most people sided with Bentham.

Većina ljudi je stala na stranu Benthama.

So it seems that people want cars to be utilitarian,

Stoga se čini kako većina želi da automobili budu korisni,

minimize total harm,

da smanje totalnu štetu

and that's what we should all do.

i to bismo svi trebali činiti.

Problem solved.

Riješen problem.

But there is a little catch.

Ali, postoji mala kvaka.

When we asked people whether they would purchase such cars,

Kad smo pitali ljude bi li kupili takve automobile,

they said, "Absolutely not."

odgovorili su: „ Apsolutno ne.“

(Laughter)

(Smijeh)

They would like to buy cars that protect them at all costs,

Oni bi htjeli kupiti automobile koji njih štite pod svaku cijenu,

but they want everybody else to buy cars that minimize harm.

ali žele da svi ostali kupe automobile koji minimaliziraju štetu.

(Laughter)

(Smijeh)

We've seen this problem before.

Već prije smo se sreli s ovim problemom.

It's called a social dilemma.

Zove se društvena dilema.

And to understand the social dilemma,

I da bismo razumjeli društvena dilemu,

we have to go a little bit back in history.

moramo se vratiti malo u prošlost.

In the 1800s,

U 1800-ima,

English economist William Forster Lloyd published a pamphlet

engleski ekonomist William Forster Lloyd objavio je pamflet

which describes the following scenario.

koji opisuje idući scenarij.

You have a group of farmers --

Imate grupu farmera,

English farmers --

engleskih farmera,

who are sharing a common land for their sheep to graze.

koji dijele zajedničko zemljište za ispašu njihovih ovaca.

Now, if each farmer brings a certain number of sheep --

Sada, ako svaki farmer dovodi određeni broj ovaca --

let's say three sheep --

recimo tri ovce --

the land will be rejuvenated,

zemljište će biti pomlađeno,

the farmers are happy,

farmeri sretni,

the sheep are happy,

ovce sretne,

everything is good.

sve je dobro.

Now, if one farmer brings one extra sheep,

Sada, ako jedan farmer dovede jednu ovcu više,

that farmer will do slightly better, and no one else will be harmed.

tom će farmeru ići malo bolje i nitko drugi neće biti oštećen.

But if every farmer made that individually rational decision,

Međutim, ako svaki farmer donese tu individualno racionalnu odluku,

the land will be overrun, and it will be depleted

zemljište će postati pretrpano i bit će istrošeno

to the detriment of all the farmers,

na štetu svih farmera

and of course, to the detriment of the sheep.

i naravno, na štetu ovaca.

We see this problem in many places:

Možemo vidjeti ovaj problem u raznim područjima:

in the difficulty of managing overfishing,

u poteškoćama u vođenju pretjeranog ribolova

or in reducing carbon emissions to mitigate climate change.

ili u smanjenju emisija ugljika za ublažavanje klimatskih promjena.

When it comes to the regulation of driverless cars,

Kada je u pitanju regulacija automobila bez vozača,

the common land now is basically public safety --

zajedničko je, zapravo, javna sigurnost --

that's the common good --

to je opće dobro --

and the farmers are the passengers

a farmeri su putnici

or the car owners who are choosing to ride in those cars.

ili vlasnici automobila koji odabiru voziti se u tim automobilima.

And by making the individually rational choice

I odabirući individualno racionalni odabir

of prioritizing their own safety,

davanja prioriteta vlastitoj sigurnosti,

they may collectively be diminishing the common good,

oni bi kolektivno mogli umanjiti opće dobro,

which is minimizing total harm.

koje je smanjene totalne štete.

It's called the tragedy of the commons,

To se naziva tragedija zajedništva,

traditionally,

tradicionalno,

but I think in the case of driverless cars,

ali ja mislim da u slučaju automobila bez vozača,

the problem may be a little bit more insidious

problem je možda malo više podmukao

because there is not necessarily an individual human being

jer ne postoji nužno individualno ljudsko biće

making those decisions.

koje donosi te odluke.

So car manufacturers may simply program cars

Proizvođači automobila mogu jednostavno programirati automobile

that will maximize safety for their clients,

tako da maksimalno povećaju sigurnost svojih klijenata

and those cars may learn automatically on their own

te bi takvi automobili mogli automatski samostalno naučiti

that doing so requires slightly increasing risk for pedestrians.

da to zahtijeva povećanje rizika za pješake.

So to use the sheep metaphor,

Da upotrijebim metaforu s ovcom,

it's like we now have electric sheep that have a mind of their own.

to je kao da sada imamo električnu ovcu s vlastitim umom.

(Laughter)

(Smijeh)

And they may go and graze even if the farmer doesn't know it.

I one mogu otići i jesti bez da uopće njihov farmer to zna.

So this is what we may call the tragedy of the algorithmic commons,

Dakle, ovo bismo mogli nazvati tragedijom zajedničkog algoritma

and if offers new types of challenges.

te nam ona nudi nove vrste izazova.

Typically, traditionally,

Obično, tradicionalno,

we solve these types of social dilemmas using regulation,

ove vrste društvenih dilema rješavamo koristeći propise,

so either governments or communities get together,

stoga se sastanu ili vlade ili zajednice

and they decide collectively what kind of outcome they want

i kolektivno odluče kakav ishod žele

and what sort of constraints on individual behavior

i kakva ograničenja individualnog ponašanja

they need to implement.

trebaju provoditi.

And then using monitoring and enforcement,

Te tada, koristeći praćenje i provedbu,

they can make sure that the public good is preserved.

mogu osigurati očuvanje javne sigurnosti.

So why don't we just,

Stoga, zašto mi ne bismo samo,

as regulators,

kao regulatori,

require that all cars minimize harm?

zahtijevali da svi automobili potpuno smanje štetu?

After all, this is what people say they want.

Uostalom, to je ono što ljudi kažu da žele.

And more importantly,

I još važnije,

I can be sure that as an individual,

mogu biti siguran da kao pojedinac,

if I buy a car that may sacrifice me in a very rare case,

ukoliko kupim automobil koji bi me mogao žrtvovati u vrlo rijetkom slučaju,

I'm not the only sucker doing that

nisam jedina naivčina čineći to

while everybody else enjoys unconditional protection.

dok svi ostali uživaju bezuvjetnu zaštitu.

In our survey, we did ask people whether they would support regulation

U našoj anketi, pitali smo ljude bi li podržali propise

and here's what we found.

i evo što smo saznali.

First of all, people said no to regulation;

Prvo, ljudi su rekli ne propisima;

and second, they said,

i drugo, rekli su:

"Well if you regulate cars to do this and to minimize total harm,

„Pa, ako regulirate automobile da to čine i da potpuno smanje štetu,

I will not buy those cars."

ja neću kupiti te automobile.“

So ironically,

Dakle, ironično,

by regulating cars to minimize harm,

regulirajući automobile u svrhu potpunog smanjenja štete,

we may actually end up with more harm

možemo zapravo završiti s više štete

because people may not opt into the safer technology

jer se možda ljudi neće odlučiti za sigurniju tehnologiju,

even if it's much safer than human drivers.

čak i ako je puno sigurnija od ljudskih vozača.

I don't have the final answer to this riddle,

Nemam konačan odgovor na ovu zagonetku,

but I think as a starting point,

ali mislim da, za početak,

we need society to come together

trebamo društvo da se zajedno okupi

to decide what trade-offs we are comfortable with

kako bismo odlučili s kojim kompromisima se slažemo

and to come up with ways in which we can enforce those trade-offs.

i kako bismo osmislili načine donošenja tih kompromisa.

As a starting point, my brilliant students,

Za početak, moji sjajni studenti,

Edmond Awad and Sohan Dsouza,

Edmond Awad i Sohan Dsouza,

built the Moral Machine website,

napravili su internetsku stranicu Moral Machine,

which generates random scenarios at you --

koja vam postavlja nasumične događaje --

basically a bunch of random dilemmas in a sequence

obično gomilu nasumičnih dilema u nizu,

where you have to choose what the car should do in a given scenario.

gdje vi trebate odlučiti što bi automobil trebao napraviti u određenoj situaciji.

And we vary the ages and even the species of the different victims.

U ponudi imamo žrtve raznih dobi i vrsta.

So far we've collected over five million decisions

Dosad smo prikupili više od pet milijuna odluka

by over one million people worldwide

od preko milijun ljudi diljem svijeta

from the website.

putem te internetske stranice.

And this is helping us form an early picture

Ovo nam od početka pomaže u razumijevanju

of what trade-offs people are comfortable with

s kakvim kompromisima se ljudi slažu

and what matters to them --

te što je njima važno --

even across cultures.

čak i u različitim kulturama.

But more importantly,

No, važnije od toga,

doing this exercise is helping people recognize

ljudima se time pomaže u shvaćanju

the difficulty of making those choices

težine donošenja tih odluka

and that the regulators are tasked with impossible choices.

te da su oni koji donose odluke zaduženi za nemoguće izbore.

And maybe this will help us as a society understand the kinds of trade-offs

I možda će upravo ovo pomoći nama kao društvu u razumijevanju kompromisa

that will be implemented ultimately in regulation.

koji će biti ostvareni u konačnici pri donošenju propisa.

And indeed, I was very happy to hear

I zaista, bio sam sretan saznajući

that the first set of regulations

da je prvi skup propisa

that came from the Department of Transport --

iz Odjela za transport --

announced last week --

objavljen prošli tjedan --

included a 15-point checklist for all carmakers to provide,

obuhvaćao kontrolni popis of 15 točaka za sve proizvođače automobila

and number 14 was ethical consideration --

te da je broj 14 uzimao u obzir etiku --

how are you going to deal with that.

kako ćete se vi nositi s tim.

We also have people reflect on their own decisions

Također, potičemo ljude da promišljaju o vlastitim odlukama

by giving them summaries of what they chose.

tako što im dajemo sažetak njihovih odabira.

I'll give you one example --

Dat ću vam jedan primjer --

I'm just going to warn you that this is not your typical example,

samo ću vas upozoriti da ovo nije tipičan primjer,

your typical user.

tipičan korisnik.

This is the most sacrificed and the most saved character for this person.

Ovo je najviše žrtvovan i najviše spašen lik za ovu osobu.

(Laughter)

(Smijeh)

Some of you may agree with him,

Neki od vas bi se mogli složiti s njim,

or her, we don't know.

ili s njom, ne znamo.

But this person also seems to slightly prefer passengers over pedestrians

No također, čini se da ova osoba ipak malo više preferira putnike nego pješake

in their choices

u svojim odlukama

and is very happy to punish jaywalking.

te da vrlo rado kažnjava neoprezno prelaženje ulice.

(Laughter)

(Smijeh)

So let's wrap up.

Dakle, zaključimo.

We started with the question -- let's call it the ethical dilemma --

Započeli smo pitanjem -- nazovimo to moralnom dilemom --

of what the car should do in a specific scenario:

kako bi automobil trebao postupiti u određenoj situaciji:

swerve or stay?

skrenuti ili ostati?

But then we realized that the problem was a different one.

Ali onda smo shvatili da je problem bio u nečem drugom.

It was the problem of how to get society to agree on and enforce

U tome kako pridobiti društvo da donese zajedničku odluku i provede

the trade-offs they're comfortable with.

kompromise s kojim se slaže.

It's a social dilemma.

To je društvena dilema.

In the 1940s, Isaac Asimov wrote his famous laws of robotics --

1940., Isaac Asimov je napisao svoje slavne zakone robotike --

the three laws of robotics.

tri zakona robotike.

A robot may not harm a human being,

Robot nikada ne smije naškoditi ljudskom biću,

a robot may not disobey a human being,

robot nikada ne smije ne poslušati ljudsko biće

and a robot may not allow itself to come to harm --

i robot nikada ne smije dopustiti sebi štetu --

in this order of importance.

tim redoslijedom važnosti.

But after 40 years or so

No, nakon otprilike 40 godina

and after so many stories pushing these laws to the limit,

i nakon toliko slučajeva koji guraju ove zakone do krajnjih granica,

Asimov introduced the zeroth law

Asimov je predstavio nulti zakon,

which takes precedence above all,

koji ima prednost spram svih ostalih zakona

and it's that a robot may not harm humanity as a whole.

i glasi da robot nikada ne smije naškoditi čovječnosti u cijelosti.

I don't know what this means in the context of driverless cars

Ne znam što ovo znači u kontekstu automobila bez vozača

or any specific situation,

ili u bilo kojoj specifičnoj situaciji,

and I don't know how we can implement it,

i ne znam kako bismo to mogli primijeniti,

but I think that by recognizing

ali smatram da prepoznavanjem

that the regulation of driverless cars is not only a technological problem

kako regulacija automobila bez vozača nije samo problem tehnologije

but also a societal cooperation problem,

već i čitavog društva i njegove suradnje,

I hope that we can at least begin to ask the right questions.

nadam se, možemo barem početi postavljati prava pitanja.

Thank you.

Hvala vam.

(Applause)

(Pljesak)