Can we build AI without losing control over it? | Sam Harris

Prevodilac: Vesna Radovic Lektor: Mile Živković

I'm going to talk about a failure of intuition

Pričaću vam o porazu intuicije

that many of us suffer from.

od koga mnogi patimo.

It's really a failure to detect a certain kind of danger.

Zista je porazno opaziti određenu vrstu opasnosti.

I'm going to describe a scenario

Opisaću vam jedan scenario

that I think is both terrifying

za koji mislim da je i zastrašujuć

and likely to occur,

i koji se realno može dogoditi,

and that's not a good combination,

a to, kako se čini,

as it turns out.

nije dobra kombinacija.

And yet rather than be scared, most of you will feel

A opet umesto da se uplaše, mnogi od vas će misliti

that what I'm talking about is kind of cool.

da je ono o čemu pričam nešto kul.

I'm going to describe how the gains we make

Opisaću vam kako bi napredak koji pravimo

in artificial intelligence

u veštačkoj inteligenciji

could ultimately destroy us.

mogao na kraju da nas uništi.

And in fact, I think it's very difficult to see how they won't destroy us

Zapravo, mislim da je jako teško zamisliti kako nas neće uništiti,

or inspire us to destroy ourselves.

ili navesti da se samouništimo.

And yet if you're anything like me,

A opet ako ste imalo kao ja,

you'll find that it's fun to think about these things.

shvatićete da je zabavno razmišljati o ovim stvarima.

And that response is part of the problem.

I taj odgovor je deo ovog problema.

OK? That response should worry you.

OK? Taj odgovor treba da vas brine.

And if I were to convince you in this talk

Kad bih vas u ovom govoru ubedio

that we were likely to suffer a global famine,

da ćemo vrlo verovatno globalno gladovati,

either because of climate change or some other catastrophe,

bilo zbog klimatskih promena ili neke druge katastrofe,

and that your grandchildren, or their grandchildren,

i da će vaši unuci, ili njihovi unuci,

are very likely to live like this,

vrlo verovatno živeti ovako,

you wouldn't think,

ne biste mislili:

"Interesting.

"Zanimljivo.

I like this TED Talk."

Dobar ovaj TED Talk."

Famine isn't fun.

Glad nije zabavna.

Death by science fiction, on the other hand, is fun,

Smrt u naučnoj fantastici, sa druge strane, jeste zabavna,

and one of the things that worries me most about the development of AI at this point

i nešto što me najviše brine u ovom trenutku u vezi sa razvojem VI

is that we seem unable to marshal an appropriate emotional response

je što izgleda da ne možemo da pružimo odgovarajući emocionalni odgovor

to the dangers that lie ahead.

na opasnosti koje dolaze.

I am unable to marshal this response, and I'm giving this talk.

Ni ja ne mogu da pružim ovaj odgovor, a držim ovaj govor.

It's as though we stand before two doors.

To je kao da stojimo ispred dvoja vrata.

Behind door number one,

Iza vrata broj jedan,

we stop making progress in building intelligent machines.

zaustavljamo napredak u pravljenju inteligentnih mašina.

Our computer hardware and software just stops getting better for some reason.

Kompjuterski hardver i softver prosto prestaju da budu bolji iz nekog razloga.

Now take a moment to consider why this might happen.

Sada na trenutak razmislite zašto bi se ovo moglo desiti.

I mean, given how valuable intelligence and automation are,

Mislim, s obzirom koliku vrednost imaju inteligencija i automatizacija,

we will continue to improve our technology if we are at all able to.

nastavićemo da unapređujemo tehnologiju ako smo uopšte kadri za to.

What could stop us from doing this?

Šta bi nas moglo sprečiti u tome?

A full-scale nuclear war?

Svetski nuklearni rat?

A global pandemic?

Globalna pandemija?

An asteroid impact?

Pad asteroida?

Justin Bieber becoming president of the United States?

Ako Džastin Biber postane predsednik Amerike?

(Laughter)

(Smeh)

The point is, something would have to destroy civilization as we know it.

Poenta je u tome, da bi nešto moralo da uništi civilizaciju ovakvu kakva je.

You have to imagine how bad it would have to be

Morate zamisliti kako bi zaista loše moralo biti to

to prevent us from making improvements in our technology

što bi nas trajno sprečilo da unapređujemo,

permanently,

tehnologiju,

generation after generation.

generaciju za generacijom.

Almost by definition, this is the worst thing

Skoro po definiciji, ovo je najgora stvar

that's ever happened in human history.

koja se ikada desila u ljudskoj istoriji.

So the only alternative,

Pa je jedina alternativa,

and this is what lies behind door number two,

a ovo se nalazi iza vrata broj dva,

is that we continue to improve our intelligent machines

da nastavimo da unapređujemo naše inteligentne mašine

year after year after year.

bez prestanka.

At a certain point, we will build machines that are smarter than we are,

U jednom trenutku, napravićemo mašine koje su pametnije od nas,

and once we have machines that are smarter than we are,

i tad kad budemo imali mašine koje su pametnije od nas,

they will begin to improve themselves.

one će same početi da se razvijaju.

And then we risk what the mathematician IJ Good called

I onda rizikujemo ono što je matematičar I.Dž. Gud nazvao

an "intelligence explosion,"

"eksplozija inteligencije",

that the process could get away from us.

da proces može da nam pobegne.

Now, this is often caricatured, as I have here,

E, sad, ovo se često karikira, kao što ću i ja uraditi,

as a fear that armies of malicious robots

kao strah od napada armije

will attack us.

zlih robota.

But that isn't the most likely scenario.

Ali to nije najverovatniji scenario.

It's not that our machines will become spontaneously malevolent.

Nije kao da će naše mašine spontano postati zle.

The concern is really that we will build machines

Brine nas, zapravo, da ćemo mi napraviti mašine

that are so much more competent than we are

koje su mnogo, mnogo kompetentnije od nas,

that the slightest divergence between their goals and our own

brinemo da će nas uništiti i najmanja razlika između

could destroy us.

naših i njihovih ciljeva.

Just think about how we relate to ants.

Zamislite kakvi smo mi prema mravima.

We don't hate them.

Ne mrzimo ih.

We don't go out of our way to harm them.

Ne povređujemo ih namerno.

In fact, sometimes we take pains not to harm them.

Zapravo, ponekad se trudimo se da ih ne povredimo.

We step over them on the sidewalk.

Preskačemo ih na trotoaru.

But whenever their presence

Ali kada god njihovo prisustvo

seriously conflicts with one of our goals,

ozbiljno ugrozi neke naše ciljeve,

let's say when constructing a building like this one,

recimo kada konstruišemo neku zgradu poput ove,

we annihilate them without a qualm.

bez po muke ih istrebimo.

The concern is that we will one day build machines

Brine nas da ćemo jednog dana napraviti mašine

that, whether they're conscious or not,

koje će, bilo da su svesne ili ne,

could treat us with similar disregard.

nas tretirati na isti način.

Now, I suspect this seems far-fetched to many of you.

E, sad, cenim da većini vas ovo zvuči kao daleka budućnost.

I bet there are those of you who doubt that superintelligent AI is possible,

Kladim se da ima onih koji ne veruju da je superinteligentna VI moguća,

much less inevitable.

a kamoli neizbežna.

But then you must find something wrong with one of the following assumptions.

Ali onda morate naći neku grešku u sledećim pretpostavkama.

And there are only three of them.

A samo ih je tri.

Intelligence is a matter of information processing in physical systems.

Inteligencija je stvar obrade informacija u fizičkim sistemima.

Actually, this is a little bit more than an assumption.

U stvari, ovo je malo više od neke pretpostavke.

We have already built narrow intelligence into our machines,

Već smo ugradili specifičnu inteligenciju u naše mašine,

and many of these machines perform

i mnoge od ovih mašina već

at a level of superhuman intelligence already.

imaju performanse na nivou superljudske inteligencije.

And we know that mere matter

I znamo da sama materija

can give rise to what is called "general intelligence,"

može da omogući takozvanu "uopštenu inteligenciju",

an ability to think flexibly across multiple domains,

sposobnost da se fleksibilno razmišlja u različitim domenima,

because our brains have managed it. Right?

jer to može i naš mozak. Je l' tako?

I mean, there's just atoms in here,

Mislim, ovde postoje samo atomi,

and as long as we continue to build systems of atoms

i sve dok mi nastavljamo da gradimo sisteme atoma

that display more and more intelligent behavior,

koji su sve inteligentniji i inteligentniji,

we will eventually, unless we are interrupted,

mi ćemo na kraju, osim ako nas prekinu,

we will eventually build general intelligence

mi ćemo na kraju ugraditi uopštenu inteligenciju

into our machines.

u naše mašine.

It's crucial to realize that the rate of progress doesn't matter,

Od suštinske je važnosti da shvatimo da stopa napretka nije važna,

because any progress is enough to get us into the end zone.

jer bilo kakav napredak je dovoljan da nas dovede do cilja.

We don't need Moore's law to continue. We don't need exponential progress.

Ne treba nam Murov zakon. Ne treba nam eksponencijalni napredak.

We just need to keep going.

Samo treba da nastavimo dalje.

The second assumption is that we will keep going.

Druga pretpostavka je da ćemo nastaviti dalje.

We will continue to improve our intelligent machines.

Nastavićemo da poboljšavamo naše inteligentne mašine.

And given the value of intelligence --

I uz postojeću vrednost inteligencije -

I mean, intelligence is either the source of everything we value

mislim, inteligencija je ili izvor svega što je vredno

or we need it to safeguard everything we value.

ili nam treba da sačuvamo sve ono što nam je vredno.

It is our most valuable resource.

To je naš najvredniji resurs.

So we want to do this.

I zato želimo da uradimo ovo.

We have problems that we desperately need to solve.

Imamo probleme koje očajnički moramo da rešimo.

We want to cure diseases like Alzheimer's and cancer.

Želimo da izlečimo bolesti poput Alchajmera i raka.

We want to understand economic systems. We want to improve our climate science.

Želimo da shvatimo ekonomske sisteme. Želimo da poboljšamo nauku o klimi.

So we will do this, if we can.

Tako da, ako možemo, to ćemo i uraditi.

The train is already out of the station, and there's no brake to pull.

Voz je već napustio stanicu, i nema kočnice.

Finally, we don't stand on a peak of intelligence,

Konačno, mi ne stojimo na vrhuncu inteligencije,

or anywhere near it, likely.

a verovatno ni blizu toga.

And this really is the crucial insight.

I ovo je zaista suštinski uvid.

This is what makes our situation so precarious,

Zbog ovoga je naša situacija tako nepouzdana,

and this is what makes our intuitions about risk so unreliable.

i zbog toga je naša intuicija o riziku tako nepouzdana.

Now, just consider the smartest person who has ever lived.

E, sad, samo razmislite o najpametnijoj osobi ikada.

On almost everyone's shortlist here is John von Neumann.

Na skoro svačijem užem spisku ovde je Džon fon Nojman.

I mean, the impression that von Neumann made on the people around him,

Mislim, utisak koji je Nojman ostavio na ljude oko sebe,

and this included the greatest mathematicians and physicists of his time,

uključujući tu najveće matematičare i fizičare tog vremena,

is fairly well-documented.

je prilično dobro dokumentovan.

If only half the stories about him are half true,

I ako je samo pola onih priča o njemu polutačna,

there's no question

nema dileme da je on

he's one of the smartest people who has ever lived.

jedan od najpametnijih ljudi koji su ikada živeli.

So consider the spectrum of intelligence.

Samo zamislite opseg inteligencije.

Here we have John von Neumann.

Ovde imamo Džona fon Nojmana.

And then we have you and me.

Onda smo tu vi i ja.

And then we have a chicken.

I onda imamo neko pile.

(Laughter)

(Smeh)

Sorry, a chicken.

Pardon, neko pile.

(Laughter)

(Smeh)

There's no reason for me to make this talk more depressing than it needs to be.

Nema potrebe da idem dalje u dubiozu u ovom govoru nego što treba.

(Laughter)

(Smeh)

It seems overwhelmingly likely, however, that the spectrum of intelligence

Međutim, čini se da je više nego verovatno da se spektar inteligencije

extends much further than we currently conceive,

proteže mnogo više nego što trenutno možemo da shvatimo,

and if we build machines that are more intelligent than we are,

i ako napravimo mašine koje su mnogo inteligentnije od nas,

they will very likely explore this spectrum

one će skoro sigurno istražiti ovaj spektar

in ways that we can't imagine,

onako kako to ne možemo ni zamisliti,

and exceed us in ways that we can't imagine.

i preteći nas na načine koje ne možemo ni zamisliti.

And it's important to recognize that this is true by virtue of speed alone.

I važno je shvatiti da je ovo moguće samo putem brzine.

Right? So imagine if we just built a superintelligent AI

U redu? Zamislite samo da napravimo jednu superinteligentnu VI

that was no smarter than your average team of researchers

koja nije ništa pametnija od vašeg prosečnog tima istraživača

at Stanford or MIT.

na Stanfordu ili MIT-u.

Well, electronic circuits function about a million times faster

Pa, električna kola funkcionišu oko milion puta brže

than biochemical ones,

od biohemijskih,

so this machine should think about a million times faster

tako da bi ove mašine razmišljale oko milion puta brže

than the minds that built it.

od umova koje su ih sastavile.

So you set it running for a week,

Pustite ih da rade nedelju dana,

and it will perform 20,000 years of human-level intellectual work,

i izvešće 20.000 godina ljudskog intelektualnog rada,

week after week after week.

nedelju za nedeljom.

How could we even understand, much less constrain,

Kako čak možemo i razumeti, a kamoli ograničiti

a mind making this sort of progress?

um koji bi mogao ovo da uradi?

The other thing that's worrying, frankly,

Iskreno, ono drugo što je zabrinjavajuće

is that, imagine the best case scenario.

je da, zamislite najbolji mogući scenario.

So imagine we hit upon a design of superintelligent AI

Zamislite da nabasamo na neki dizajn superinteligentne VI

that has no safety concerns.

čija nas sigurnost ne brine.

We have the perfect design the first time around.

Po prvi put imamo savršeni dizajn.

It's as though we've been handed an oracle

To je kao da nam je dato proročanstvo

that behaves exactly as intended.

koje se odvija baš kako je zapisano.

Well, this machine would be the perfect labor-saving device.

Ta mašina bi bila savršeni uređaj za uštedu radne snage.

It can design the machine that can build the machine

Dizajnirala bi mašinu koja može napraviti mašine

that can do any physical work,

za bilo kakav fizički posao,

powered by sunlight,

a koje bi napajalo sunce,

more or less for the cost of raw materials.

manje-više po ceni sirovina.

So we're talking about the end of human drudgery.

Pričamo o kraju teškog ljudskog rada.

We're also talking about the end of most intellectual work.

Takođe pričamo o kraju većeg dela intelektualnog rada.

So what would apes like ourselves do in this circumstance?

Pa šta će onda majmuni poput nas da rade u takvim okolnostima?

Well, we'd be free to play Frisbee and give each other massages.

Pa, imali bismo vremena za frizbi i da masiramo jedni druga.

Add some LSD and some questionable wardrobe choices,

Dodajmo malo LSD-a i neke diskutabilne garderobe,

and the whole world could be like Burning Man.

i ceo svet bi mogao biti poput festivala Burning Man.

(Laughter)

(Smeh)

Now, that might sound pretty good,

E, sad, to možda zvuči kao dobra ideja,

but ask yourself what would happen

ali zapitajte se šta bi se desilo

under our current economic and political order?

pod našim trenutnim ekonomskim i političkim poretkom?

It seems likely that we would witness

Čini se verovatnim da bismo bili svedoci

a level of wealth inequality and unemployment

određenog nivoa novčane nejednakosti i nezaposlenosti

that we have never seen before.

koji ranije nismo viđali.

Absent a willingness to immediately put this new wealth

Uz odsustvo volje da se odmah ovo novo bogatstvo stavi

to the service of all humanity,

na raspolaganje čovečanstvu,

a few trillionaires could grace the covers of our business magazines

nekoliko bilionera bi poziralo na omotima biznis magazina

while the rest of the world would be free to starve.

dok bi ostatak sveta slobodno mogao umreti od gladi.

And what would the Russians or the Chinese do

I šta bi radili Rusi ili Kinezi

if they heard that some company in Silicon Valley

kada bi čuli da tamo neka firma iz Silicijumske doline

was about to deploy a superintelligent AI?

treba da izbaci superinteligentnu VI?

This machine would be capable of waging war,

Ova mašina bi bila sposobna da vodi rat,

whether terrestrial or cyber,

bilo klasični, bilo sajber rat,

with unprecedented power.

nadmoćna bez presedana.

This is a winner-take-all scenario.

Ovo je scenario pobednik nosi sve.

To be six months ahead of the competition here

Biti šest meseci ispred konkurencije ovde

is to be 500,000 years ahead,

je isto biti 500 000 godina ispred,

at a minimum.

u najmanju ruku.

So it seems that even mere rumors of this kind of breakthrough

Tako se čini da bi čak i samo glasina o proboju ovog tipa

could cause our species to go berserk.

izazvala ludnicu među ljudima.

Now, one of the most frightening things,

I, sad, jedna od najstarašnijih stvari,

in my view, at this moment,

prema mom mišljenju, trenutno,

are the kinds of things that AI researchers say

su one stvari koje istraživači VI kažu

when they want to be reassuring.

kada žele da vas ohrabre.

And the most common reason we're told not to worry is time.

I najuobičajeniji razlog zbog koga nam kažu da ne brinemo jeste vreme.

This is all a long way off, don't you know.

Sve je to daleko, znate.

This is probably 50 or 100 years away.

To je verovatno 50 ili 100 godina daleko.

One researcher has said,

Jedan istraživač je rekao:

"Worrying about AI safety

"Briga o bezbednosti VI

is like worrying about overpopulation on Mars."

je kao briga o prenaseljenosti Marsa."

Ovo je kao kada vam iz Silicijumske doline kažu:

This is the Silicon Valley version

of "don't worry your pretty little head about it."

"Nemojte vi zamarati vaše glavice time."

(Laughter)

(Smeh)

No one seems to notice

Izgleda da niko ne primećuje

that referencing the time horizon

da je odnos prema vremenu

is a total non sequitur.

potpuni non sequitur.

If intelligence is just a matter of information processing,

Ako je inteligencija samo stvar procesuiranja informacija,

and we continue to improve our machines,

i mi nastavimo da poboljšavamo mašine,

we will produce some form of superintelligence.

proizvešćemo neki oblik superinteligencije.

And we have no idea how long it will take us

I nemamo pojma koliko će nam trebati vremena

to create the conditions to do that safely.

da stvorimo uslove da to uradimo bezbedno.

Let me say that again.

Dozvolite mi da ponovim.

We have no idea how long it will take us

Nemamo pojma koliko vremena će nam trebati

to create the conditions to do that safely.

da stvorimo uslove da to uradimo bezbedno.

And if you haven't noticed, 50 years is not what it used to be.

Ako niste primetili, 50 godina nije kao što je bilo ranije.

This is 50 years in months.

Ovo je 50 godina u mesecima.

This is how long we've had the iPhone.

Ovoliko dugo već imamo iPhone.

This is how long "The Simpsons" has been on television.

Ovoliko se već prikazuju "Simpsonovi".

Fifty years is not that much time

Pedeset godina nije mnogo vremena

to meet one of the greatest challenges our species will ever face.

da se ostvari jedan od najvećih izazova naše vrste.

Once again, we seem to be failing to have an appropriate emotional response

Još jedanput, izgleda da nam nedostaje odgovarajući emocionalni odgovor

to what we have every reason to believe is coming.

na ono na šta imamo svako pravo da verujemo da dolazi.

The computer scientist Stuart Russell has a nice analogy here.

Kompjuterski naučnik Stjuart Rasel ima lepu analogiju za ovo.

He said, imagine that we received a message from an alien civilization,

On kaže, zamislite da smo primili neku poruku od vanzemaljaca,

which read:

u kojoj stoji:

"People of Earth,

"Zemljani,

we will arrive on your planet in 50 years.

stižemo na vašu planetu za 50 godina.

Get ready."

Spremite se."

And now we're just counting down the months until the mothership lands?

I mi ćemo samo da odbrojavamo mesece dok se njihov brod ne spusti?

We would feel a little more urgency than we do.

Osetili bismo malo više straha nego obično.

Another reason we're told not to worry

Drugo zbog čega kažu da ne brinemo

is that these machines can't help but share our values

je da ove mašine ne mogu drugačije nego da dele naše vrednosti

because they will be literally extensions of ourselves.

zato što će one bukvalno biti produžeci nas samih.

They'll be grafted onto our brains,

One će biti nakalamljene na naš mozak

and we'll essentially become their limbic systems.

i mi ćemo suštinski postati limbički sistem.

Now take a moment to consider

Sad na trenutak razmislite

that the safest and only prudent path forward,

da je najbezbednija i jedina razborita i preporučena,

recommended,

putanja napred,

is to implant this technology directly into our brains.

da ovu tehnologiju implementiramo direktno u naš mozak.

Now, this may in fact be the safest and only prudent path forward,

Sad, ovo može biti najbezbednija i jedina razborita putanja napred,

but usually one's safety concerns about a technology

ali obično se bezbednost neke tehnologije

have to be pretty much worked out before you stick it inside your head.

mora baš dobro razmotriti pre nego što vam se zabode u mozak.

(Laughter)

(Smeh)

The deeper problem is that building superintelligent AI on its own

Veći problem je što pravljenje superinteligentne VI same od sebe

seems likely to be easier

izgleda lakše

than building superintelligent AI

od pravljenja superinteligentne VI

and having the completed neuroscience

i kompletiranja neuronauke

that allows us to seamlessly integrate our minds with it.

koja nam omogućava da integrišemo naše umove sa njom.

And given that the companies and governments doing this work

I ako imamo to da kompanije i vlade rade ovaj posao,

are likely to perceive themselves as being in a race against all others,

one će verovatno to shvatiti kao da se takmiče protiv svih drugih,

given that to win this race is to win the world,

jer pobediti u ovoj trci znači osvojiti svet,

provided you don't destroy it in the next moment,

uz uslov da ga sledećeg trenutka ne uništite,

then it seems likely that whatever is easier to do

a onda će verovatno prvo biti urađeno

will get done first.

ono što je najlakše.

Now, unfortunately, I don't have a solution to this problem,

E, sad, nažalost, ja nemam rešenje za ovaj problem,

apart from recommending that more of us think about it.

osim toga da preporučim da više nas promisli o tome.

I think we need something like a Manhattan Project

Smatram da nam je potrebno nešto poput Projekta Menhetn

on the topic of artificial intelligence.

na temu veštačke inteligencije.

Not to build it, because I think we'll inevitably do that,

Ne da bismo je izgradili, jer mislim da je to neizbežno,

but to understand how to avoid an arms race

već da razumemo kako da izbegnemo trku naoružanja

and to build it in a way that is aligned with our interests.

i da je napravimo na način koji je u skladu sa našim interesima.

When you're talking about superintelligent AI

Kada se razgovara o superinteligentnoj VI

that can make changes to itself,

koja sama sebe može da menja,

it seems that we only have one chance to get the initial conditions right,

čini se da imamo samo jednu šansu da početne uslove postavimo kako treba,

and even then we will need to absorb

a čak i onda ćemo morati da prihvatimo

the economic and political consequences of getting them right.

ekonomske i političke posledice pravilne upotrebe.

But the moment we admit

Ali trenutak kada priznamo

that information processing is the source of intelligence,

da je obrada informacija izvor inteligencije,

that some appropriate computational system is what the basis of intelligence is,

da je neki odgovarajući računarski sistem osnova onoga što je inteligencija,

and we admit that we will improve these systems continuously,

i prihvatimo da ćemo stalno unapređivati ove sisteme,

and we admit that the horizon of cognition very likely far exceeds

i prihvatimo da će horizont shvatanja vrlo verovatno daleko nadmašiti

what we currently know,

ono što trenutno znamo,

then we have to admit

onda moramo da prihvatimo

that we are in the process of building some sort of god.

da smo u procesu pravljenja neke vrste božanstva.

Now would be a good time

Sada bi bilo dobro vreme

to make sure it's a god we can live with.

da se postaramo da je to božanstvo sa kojim bismo mogli živeti.

Thank you very much.

Hvala vam mnogo.

(Applause)

(Aplauz)