Can we build AI without losing control over it? | Sam Harris

Translator: 潘 可儿 Reviewer: Alan Watson

我想講下 一個好多人都經歷過嘅感官錯覺

I'm going to talk about a failure of intuition

that many of us suffer from.

It's really a failure to detect a certain kind of danger.

當呢個錯覺嚟嗰陣

我哋會唔識得留意危險

I'm going to describe a scenario

我亦都想講下一個我認為駭人聽聞

that I think is both terrifying

and likely to occur,

同時又好有可能會發生嘅情景

and that's not a good combination,

呢個情景發生嘅話，唔係一件好事嚟

as it turns out.

你哋可能唔覺得我依家講緊嘅嘢恐怖

And yet rather than be scared, most of you will feel

that what I'm talking about is kind of cool.

反而覺得好型

I'm going to describe how the gains we make

所以我想講下 我哋人類喺人工智能方面取得嘅成就

in artificial intelligence

最终會點樣摧毀我哋

could ultimately destroy us.

And in fact, I think it's very difficult to see how they won't destroy us

而事實上，我認為好難會見到

佢哋唔會摧毀我哋 或者導致我哋自我毀滅

or inspire us to destroy ourselves.

And yet if you're anything like me,

依家你哋或者同我一樣

you'll find that it's fun to think about these things.

覺得諗呢啲嘢好得意

And that response is part of the problem.

正因為覺得得意

亦都成為咗問題嘅一部份

OK? That response should worry you.

你哋應該擔心你哋嘅反應至真！

And if I were to convince you in this talk

如果我喺呢場演講度話畀你哋聽

that we were likely to suffer a global famine,

因為氣候變化或者大災難嘅原因

我哋會遭遇一場饑荒

either because of climate change or some other catastrophe,

而你嘅孫，或者佢哋嘅孫 會好似咁樣生活

and that your grandchildren, or their grandchildren,

are very likely to live like this,

你就唔會覺得

you wouldn't think,

「好有趣，我鍾意呢場 TED 演講。」

"Interesting.

I like this TED Talk."

Famine isn't fun.

饑荒一啲都唔有趣

Death by science fiction, on the other hand, is fun,

但科幻小說描繪嘅死亡就好有趣

呢一刻，人工智能發展 最令我最困擾嘅係

and one of the things that worries me most about the development of AI at this point

我哋面對近在眼前嘅危險似乎無動於衷

is that we seem unable to marshal an appropriate emotional response

to the dangers that lie ahead.

雖然我喺你哋面前演講

I am unable to marshal this response, and I'm giving this talk.

但我同你哋一樣都係冇反應

成件事就好似我哋企喺兩道門前面

It's as though we stand before two doors.

Behind door number one,

喺一號門後面，我哋唔再發展智能機器

we stop making progress in building intelligent machines.

因為某啲原因

Our computer hardware and software just stops getting better for some reason.

我哋電腦嘅硬件同軟件都停滯不前

依家嚟諗一下點解呢種情況會發生

Now take a moment to consider why this might happen.

即係話，因為智能同自動化好重要

I mean, given how valuable intelligence and automation are,

we will continue to improve our technology if we are at all able to.

所以我哋會喺許可嘅情況之下 繼續改善科技

咁究竟係乜嘢會阻止我哋？

What could stop us from doing this?

A full-scale nuclear war?

一個全面嘅核戰爭？

一個全球流行病？

A global pandemic?

一個小行星撞擊？

An asteroid impact?

Justin Bieber becoming president of the United States?

Justin Bieber 做咗美國總統？

（笑聲）

(Laughter)

The point is, something would have to destroy civilization as we know it.

之但係，如我哋所知

有一啲嘢會摧毀文明

You have to imagine how bad it would have to be

你必須要想像

如果我哋一代又一代人 永遠改善唔到科技

to prevent us from making improvements in our technology

permanently,

情況會有幾嚴重

generation after generation.

幾乎可以確定嘅係

Almost by definition, this is the worst thing

呢個係人類史上最壞嘅事

that's ever happened in human history.

So the only alternative,

所以唯一嘅選擇 就係二號門後嘅做法

and this is what lies behind door number two,

我哋繼續年復一年升級改造智能機器

is that we continue to improve our intelligent machines

year after year after year.

到咗某個地步

At a certain point, we will build machines that are smarter than we are,

我哋就會整出比我哋仲要聰明嘅機器

一旦我哋有咗比我哋自己 仲聰明嘅機器

and once we have machines that are smarter than we are,

they will begin to improve themselves.

佢哋就會自我改良

And then we risk what the mathematician IJ Good called

到時我哋就會面臨數學家 IJ Good 講嘅「智能爆炸」危機

an "intelligence explosion,"

即係話，改良過程唔再需要人類

that the process could get away from us.

依家，經常會有人學呢張諷刺漫畫咁

Now, this is often caricatured, as I have here,

as a fear that armies of malicious robots

描繪叛變嘅機器人會攻擊我哋

will attack us.

但係呢個唔係最有可能發生嘅情景

But that isn't the most likely scenario.

It's not that our machines will become spontaneously malevolent.

我哋嘅機器唔會自動變惡

所以問題在於我哋製造出 比我哋更加做到嘢嘅機器嘅時候

The concern is really that we will build machines

that are so much more competent than we are

that the slightest divergence between their goals and our own

佢哋目標上同我哋嘅細微分歧 會置我哋於死地

could destroy us.

Just think about how we relate to ants.

就諗下我哋同螞蟻之間嘅關係︰

We don't hate them.

我哋唔討厭佢哋

我哋唔會傷害佢哋

We don't go out of our way to harm them.

In fact, sometimes we take pains not to harm them.

甚至我哋為咗唔傷害佢哋 而會受一啲苦

We step over them on the sidewalk.

例如我哋會為咗唔踩到佢哋 而跨過佢哋

But whenever their presence

但係一旦佢哋嘅存在 同我哋嘅其中一個目標有嚴重衝突

seriously conflicts with one of our goals,

let's say when constructing a building like this one,

譬如話要起一棟咁樣嘅樓

we annihilate them without a qualm.

我哋諗都唔諗就殺死佢哋

The concern is that we will one day build machines

問題係，我哋終有一日整出嘅機器——

that, whether they're conscious or not,

無論佢哋自己有冇意識都好

同樣會冷漠咁對待我哋

could treat us with similar disregard.

Now, I suspect this seems far-fetched to many of you.

依家，我估對於你哋大部份人嚟講 呢件情景都係遙不可及嘅

I bet there are those of you who doubt that superintelligent AI is possible,

我賭你哋當中有人質疑 超級智能嘅可能性

much less inevitable.

更加唔好講 人類要避免超級智能

But then you must find something wrong with one of the following assumptions.

但係你哋肯定會喺下面嘅假設當中 搵到一啲謬誤

呢度一共有三個假設

And there are only three of them.

Intelligence is a matter of information processing in physical systems.

喺物理系統裏面，智能等如訊息處理

Actually, this is a little bit more than an assumption.

但係，呢個超過咗假設

We have already built narrow intelligence into our machines,

因為我哋已經喺我哋嘅機器裏面 植入咗弱人工智能

and many of these machines perform

而且呢啲機器好多 已經處於一個超人類智能水平

at a level of superhuman intelligence already.

And we know that mere matter

同時我哋知道僅僅係物質 就可以產生所謂嘅「一般智能」

can give rise to what is called "general intelligence,"

一種可以喺唔同領域之間 靈活思考嘅能力

an ability to think flexibly across multiple domains,

because our brains have managed it. Right?

咁係因為我哋嘅大腦 已經可以做到，係唔係？

I mean, there's just atoms in here,

我嘅意思係，大腦凈係得原子

and as long as we continue to build systems of atoms

只要我哋繼續加設原子系統

that display more and more intelligent behavior,

機器就可以有更加多智能行為

除非進度有咩停頓

we will eventually, unless we are interrupted,

we will eventually build general intelligence

否則我哋最終會喺機器裏面 建構出一般智能

into our machines.

明白進度嘅快慢並唔影響係好重要

It's crucial to realize that the rate of progress doesn't matter,

因為任何過程都足以令我哋返唔到轉頭

because any progress is enough to get us into the end zone.

We don't need Moore's law to continue. We don't need exponential progress.

我哋唔需要按照摩爾定律進行

我哋唔需要指數式增長

我哋只需要繼續做

We just need to keep going.

The second assumption is that we will keep going.

第二個假設就係我哋會繼續做

我哋會繼續改造我哋嘅智能機器

We will continue to improve our intelligent machines.

And given the value of intelligence --

而考慮到智能嘅價值…

我係話，因為有智能 我哋至會珍重事物

I mean, intelligence is either the source of everything we value

or we need it to safeguard everything we value.

或者我哋需要智能 去保護我哋珍重嘅一切

It is our most valuable resource.

智能係我哋最有寶貴嘅資源

所以我哋想繼續發展智能

So we want to do this.

We have problems that we desperately need to solve.

我哋有極需解決嘅問題

We want to cure diseases like Alzheimer's and cancer.

例如我哋想治療類似阿茲海默症 同癌症嘅疾病

We want to understand economic systems. We want to improve our climate science.

我哋想認識經濟系統

我哋想改善我哋嘅氣候科學

So we will do this, if we can.

所以如果可以做到嘅話 我哋會繼續發展智能

The train is already out of the station, and there's no brake to pull.

件事亦都可以比喻為︰ 列車已經開出，但冇刹車掣可以踩

Finally, we don't stand on a peak of intelligence,

最終，我哋唔會去到 智能嘅頂峰或者高智能水平

or anywhere near it, likely.

And this really is the crucial insight.

而呢個就係非常重要嘅觀察結果

This is what makes our situation so precarious,

就係呢個結果 將我哋置於岌岌可危嘅境地

亦令到我哋對於危險嘅觸覺唔可靠

and this is what makes our intuitions about risk so unreliable.

依家，就諗下史上最聰明嘅人

Now, just consider the smartest person who has ever lived.

幾乎喺每個人嘅名單上面 都會有 John von Neumann

On almost everyone's shortlist here is John von Neumann.

I mean, the impression that von Neumann made on the people around him,

我嘅意思係 John von Neumann 畀佢周圍嘅人嘅印象

包括佢畀嗰個時代最犀利嘅數學家 同物理學家嘅印象

and this included the greatest mathematicians and physicists of his time,

is fairly well-documented.

都係有紀錄低嘅

If only half the stories about him are half true,

如果一半關於佢嘅故事有一半係真嘅

there's no question

咁毫無疑問

he's one of the smartest people who has ever lived.

佢係有史以來其中一個最聰明嘅人

所以當我哋畫一幅比較智力嘅圖

So consider the spectrum of intelligence.

喺右邊高分嘅位置 我哋有 John von Neumann

Here we have John von Neumann.

And then we have you and me.

喺中間有你同我

And then we have a chicken.

去到最左邊，我哋有雞仔

(Laughter)

（笑聲）

係吖，就係一隻雞仔

Sorry, a chicken.

(Laughter)

（笑聲）

There's no reason for me to make this talk more depressing than it needs to be.

我冇理由將呢個演講搞到咁灰㗎

（笑聲）

(Laughter)

It seems overwhelmingly likely, however, that the spectrum of intelligence

但好有可能智力分佈 遠比我哋目前認知嘅廣

extends much further than we currently conceive,

and if we build machines that are more intelligent than we are,

如果我哋建造出 比我哋擁有更高智慧嘅機器

佢哋嘅智力好有可能會 超越我哋認知嘅最高智力

they will very likely explore this spectrum

in ways that we can't imagine,

and exceed us in ways that we can't imagine.

同埋以無法想像嘅方式超越我哋

同樣重要嘅係 單憑運算速度就可以超越我哋

And it's important to recognize that this is true by virtue of speed alone.

啱唔啱？諗下如果我哋整咗一個

Right? So imagine if we just built a superintelligent AI

冇哈佛或者麻省理工研究人員 咁聰明嘅超級人工智能

that was no smarter than your average team of researchers

at Stanford or MIT.

但電路運行速度大概 比生化電路快一百萬倍

Well, electronic circuits function about a million times faster

than biochemical ones,

so this machine should think about a million times faster

所以呢個機器嘅思考速度應該會 比佢嘅創造者快大概一百萬倍

than the minds that built it.

So you set it running for a week,

所以如果佢運行一個星期

and it will perform 20,000 years of human-level intellectual work,

佢就可以完成人類要兩萬年 先至完成得到嘅工作

week after week after week.

How could we even understand, much less constrain,

而我哋又點會明白

人工智能係點樣完成咁龐大嘅運算呢？

a mind making this sort of progress?

The other thing that's worrying, frankly,

另一個令人擔憂嘅事，老實講

is that, imagine the best case scenario.

就係…不如想像一下最好嘅情形

So imagine we hit upon a design of superintelligent AI

想像一下我哋設計咗一個 冇安全問題嘅超級人工智能

that has no safety concerns.

We have the perfect design the first time around.

我哋第一次擁有完美嘅設計

It's as though we've been handed an oracle

就好似我哋摞住 一個按照預期發展嘅神諭

that behaves exactly as intended.

Well, this machine would be the perfect labor-saving device.

呢個機器仲會變成完美嘅慳力設備

It can design the machine that can build the machine

事關機器可以生產另一款機器出嚟

做任何體力勞動

that can do any physical work,

powered by sunlight,

兼由太陽能驅動

more or less for the cost of raw materials.

成本仲同買原材料差唔多

所以，我哋唔單止講緊咕哩勞力嘅終結

So we're talking about the end of human drudgery.

We're also talking about the end of most intellectual work.

我哋同時講緊大部份用腦工作嘅終結

So what would apes like ourselves do in this circumstance?

咁我哋人類面對工作削減 應該何去何從？

Well, we'd be free to play Frisbee and give each other massages.

我哋會好自由咁去掟飛盤 、同人按摩

服食一啲 LSD 精神藥 同埋著上怪異服飾

Add some LSD and some questionable wardrobe choices,

and the whole world could be like Burning Man.

於是成個世界都會變成火人節嘅人咁

(Laughter)

（笑聲）

Now, that might sound pretty good,

頭先講到嘅嘢聽起上嚟好似好好咁

but ask yourself what would happen

但係撫心自問

面對目前嘅經濟政治秩序 乜嘢會發生呢？

under our current economic and political order?

It seems likely that we would witness

似乎我哋會目睹

a level of wealth inequality and unemployment

我哋從未見過咁嚴重嘅 貧富懸殊同失業率

that we have never seen before.

Absent a willingness to immediately put this new wealth

如果呢筆新財富唔即時用嚟服務全人類

to the service of all humanity,

a few trillionaires could grace the covers of our business magazines

就算一啲億萬富翁使好多錢 㨘靚商業雜誌嘅封面

世界上其他人都要挨餓

while the rest of the world would be free to starve.

咁如果俄羅斯人或者中國人

And what would the Russians or the Chinese do

聽到矽谷嘅一啲公司 打算使用一個超級人工智能

if they heard that some company in Silicon Valley

was about to deploy a superintelligent AI?

佢哋會點諗？

This machine would be capable of waging war,

呢個機器有能力用未見過嘅力度 發動地面或者網絡戰爭

whether terrestrial or cyber,

with unprecedented power.

呢個係「勝者全取」嘅情況

喺呢場人工智能較量中有六個月嘅優勢

This is a winner-take-all scenario.

To be six months ahead of the competition here

就係至少要做多人類五十萬年做到嘅嘢

is to be 500,000 years ahead,

at a minimum.

So it seems that even mere rumors of this kind of breakthrough

甚至只係關於人工智能突破嘅謠言 就可以令到人類亂起上嚟

could cause our species to go berserk.

Now, one of the most frightening things,

依家最驚人嘅一件事，我覺得

in my view, at this moment,

就係人工智能研究人員 安定人心時講嘅說話

are the kinds of things that AI researchers say

when they want to be reassuring.

佢哋成日話，因為我哋有時間 所以我哋唔需要擔心

And the most common reason we're told not to worry is time.

This is all a long way off, don't you know.

「乜你唔知有排咩？

This is probably 50 or 100 years away.

仲有五十年或者一百年先到。」

One researcher has said,

一位研究人員曾經咁講︰

「擔心人工智能嘅安全就好似 擔心火星人口爆棚一樣。」

"Worrying about AI safety

is like worrying about overpopulation on Mars."

呢句嘢等如矽谷同你講︰

This is the Silicon Valley version

of "don't worry your pretty little head about it."

「你十八廿二就杞人憂天！」

(Laughter)

（笑聲）

冇人意識到 攞時間嚟到講完全係無稽之談

No one seems to notice

that referencing the time horizon

is a total non sequitur.

如果智能凈係同處理訊息有關

If intelligence is just a matter of information processing,

and we continue to improve our machines,

同埋我哋繼續改良我哋嘅機器嘅話

we will produce some form of superintelligence.

我哋最終會生產到超級智能

但我哋唔知道要用幾長時間

And we have no idea how long it will take us

先可以生產安全嘅超級智能

to create the conditions to do that safely.

Let me say that again.

等我再講多一次

We have no idea how long it will take us

我哋唔知道要用幾長時間

先可以生產安全嘅超級智能

to create the conditions to do that safely.

And if you haven't noticed, 50 years is not what it used to be.

如果你仲未意識到

五十年嘅概念已經唔同咗喇

This is 50 years in months.

呢幅圖顯示咗以月份計嘅五十年

This is how long we've had the iPhone.

先係 iPhone 面世至今嘅時間

This is how long "The Simpsons" has been on television.

再係阿森一族出現係電視上嘅時間

Fifty years is not that much time

五十年不足以畀人類應對最大挑戰

to meet one of the greatest challenges our species will ever face.

Once again, we seem to be failing to have an appropriate emotional response

再一次，我哋對於有理由發生嘅事

to what we have every reason to believe is coming.

未有採取適當嘅情緒反應

The computer scientist Stuart Russell has a nice analogy here.

對此，電腦科學家 Stuart Russell 有一個好嘅比喻

He said, imagine that we received a message from an alien civilization,

佢話︰想像一下我哋收到 一個來自外星文明嘅信息

上面寫住：

which read:

「地球上嘅人類，

"People of Earth,

we will arrive on your planet in 50 years.

我哋五十年之後會到達你哋嘅星球。

Get ready."

請準備好。」

And now we're just counting down the months until the mothership lands?

咁我哋依家凈係會倒數外星人來臨？

We would feel a little more urgency than we do.

我哋應該更加緊張至係

Another reason we're told not to worry

另一個我哋被告知唔使擔心嘅原因係

is that these machines can't help but share our values

呢啲機器只會識得 將我哋嘅價值觀傳開

because they will be literally extensions of ourselves.

因為佢哋係我哋人類嘅附屬嘅一部分

但同時佢哋會被植入我哋嘅大腦

They'll be grafted onto our brains,

and we'll essentially become their limbic systems.

所以我哋會成為佢哋嘅邊緣系統

依家使啲時間諗下 最安全同唯一審慎嘅做法

Now take a moment to consider

that the safest and only prudent path forward,

recommended,

而推薦嘅做法就係 直接將呢種科技植入我哋嘅大腦

is to implant this technology directly into our brains.

Now, this may in fact be the safest and only prudent path forward,

呢種做法可能係最安全同唯一審慎嘅

但係喺你將佢植入你個腦之前

but usually one's safety concerns about a technology

have to be pretty much worked out before you stick it inside your head.

科技嘅安全問題需要解決

(Laughter)

（笑聲）

The deeper problem is that building superintelligent AI on its own

更深一層嘅問題係

人工智能自己整超級人工智能

似乎比整一個可以喺神經科學上

seems likely to be easier

than building superintelligent AI

and having the completed neuroscience

同我哋腦部無縫接合嘅 超級人工智能簡單

that allows us to seamlessly integrate our minds with it.

And given that the companies and governments doing this work

考慮到從事研發人工智能嘅公司 同政府好可能會互相競爭

are likely to perceive themselves as being in a race against all others,

given that to win this race is to win the world,

考慮到要贏呢場比賽就要贏成個世界

provided you don't destroy it in the next moment,

同埋先假設如果你下一刻 唔會糟塌人工智能嘅成果

then it seems likely that whatever is easier to do

咁樣，似乎更加簡單嘅事會完成咗先

will get done first.

Now, unfortunately, I don't have a solution to this problem,

但唔好彩嘅係 我除咗叫大家反思呢個問題

我就再冇辦法解決呢個問題

apart from recommending that more of us think about it.

我覺得我哋喺人工智能方面

I think we need something like a Manhattan Project

需要好似「曼哈頓計劃」咁嘅計劃

on the topic of artificial intelligence.

Not to build it, because I think we'll inevitably do that,

唔係講點樣整人工智能 因為我認為人工智能終有一日會整到

but to understand how to avoid an arms race

而係搞清楚點樣避免一場軍備競賽

and to build it in a way that is aligned with our interests.

同埋往符合我哋利益嘅方向 發展人工智能

當你講緊可以自我改造嘅超級人工智能

When you're talking about superintelligent AI

that can make changes to itself,

it seems that we only have one chance to get the initial conditions right,

我哋似乎只有一個機會 令到人工智能發展得安全

就算發展得安全

and even then we will need to absorb

我哋都要接受 人工智能對經濟同政治產生嘅結果

the economic and political consequences of getting them right.

But the moment we admit

但係當我哋同意 訊息處理係智能嘅起步點

that information processing is the source of intelligence,

that some appropriate computational system is what the basis of intelligence is,

同意一啲適當嘅計算系統係智能嘅基礎

同意我哋會不斷完善人工智能

and we admit that we will improve these systems continuously,

同意將來有好多嘢超越我哋認知嘅

and we admit that the horizon of cognition very likely far exceeds

what we currently know,

咁我哋就必須要承認 我哋正喺度創造緊某種神明

then we have to admit

that we are in the process of building some sort of god.

依家會係一個好時機 確保佢係可以同我哋共存嘅神明

Now would be a good time

to make sure it's a god we can live with.

好多謝你哋

Thank you very much.

(Applause)

（掌聲）