Can we build AI without losing control over it? | Sam Harris

譯者: Hans Chiang 審譯者: Qiyun Xing

I'm going to talk about a failure of intuition

我要談一種我們很多人 遭受的、直覺上的失誤。

that many of us suffer from.

It's really a failure to detect a certain kind of danger.

那其實是一種使你無法察覺到 特定種類危險的失誤。

I'm going to describe a scenario

我會描述一個情境

that I think is both terrifying

是我認為很可怕

and likely to occur,

而且很有機會發生的，

and that's not a good combination,

這不是很好的組合，

as it turns out.

一如預期。

And yet rather than be scared, most of you will feel

然而比起感到害怕，

大部分的人會覺得 我正在說的東西有點酷。

that what I'm talking about is kind of cool.

I'm going to describe how the gains we make

我將會描述我們在 人工智能領域的進展

in artificial intelligence

could ultimately destroy us.

如何能最終消滅我們。

And in fact, I think it's very difficult to see how they won't destroy us

事實上，我認為很難看不出 他們為何不會消滅我們，

或者驅使我們消滅自己。

or inspire us to destroy ourselves.

And yet if you're anything like me,

如果你是和我類似的人，

you'll find that it's fun to think about these things.

你會發現思考這類事情很有趣。

And that response is part of the problem.

那種反應也是問題的一部分。

OK? That response should worry you.

對嗎？那種反應應該讓你感到擔心。

And if I were to convince you in this talk

如果我是打算在這個裡演講說服你，

that we were likely to suffer a global famine,

我們很可能會遭受全球性的飢荒，

either because of climate change or some other catastrophe,

無論是因為氣候變遷或某種大災難，

and that your grandchildren, or their grandchildren,

而你的孫子們或者孫子的孫子們

are very likely to live like this,

非常可能要這樣生活，

you wouldn't think,

你不會覺得：

"Interesting.

「有意思，

I like this TED Talk."

我喜歡這個 TED 演講。」

Famine isn't fun.

飢荒並不有趣。

Death by science fiction, on the other hand, is fun,

另一方面來說， 科幻式的死亡，是有趣的。

and one of the things that worries me most about the development of AI at this point

而現階段人工智能的發展 最讓我擔心的是，

is that we seem unable to marshal an appropriate emotional response

我們似乎無法組織出 一個適當的情緒反應，

to the dangers that lie ahead.

針對眼前的威脅。

I am unable to marshal this response, and I'm giving this talk.

我無法組織出這個回應， 可是我在這裡講這個。

It's as though we stand before two doors.

就像我們站在兩扇門前面。

Behind door number one,

一號門後面，

we stop making progress in building intelligent machines.

我們停止發展製造有智能的機器。

Our computer hardware and software just stops getting better for some reason.

我們的電腦硬體和軟體 就因故停止變得更好。

Now take a moment to consider why this might happen.

現在花一點時間想想 為什麼這會發生。

I mean, given how valuable intelligence and automation are,

我的意思是，人工智能 和自動化如此有價值，

we will continue to improve our technology if we are at all able to.

我們會持續改善我們的科技， 只要我們有能力做。

What could stop us from doing this?

有什麼東西能阻止我們這麼做呢？

A full-scale nuclear war?

一場全面性的核子戰爭？

A global pandemic?

一場全球性的流行病？

An asteroid impact?

一次小行星撞擊地球？

Justin Bieber becoming president of the United States?

小賈斯汀成為美國總統？

(Laughter)

（笑聲）

The point is, something would have to destroy civilization as we know it.

重點是：必須有什麼東西 會毀滅我們所知的文明。

You have to imagine how bad it would have to be

你必須想像到底能有多糟

to prevent us from making improvements in our technology

才能阻止我們持續改善我們的科技，

permanently,

永久地，

generation after generation.

一代又一代人。

Almost by definition, this is the worst thing

幾乎從定義上，這就是

人類歷史上發生過的最糟的事。

that's ever happened in human history.

So the only alternative,

所以唯一的替代選項，

and this is what lies behind door number two,

這是在二號門之後的東西，

is that we continue to improve our intelligent machines

是我們繼續改善我們的智能機器，

year after year after year.

年復一年，年復一年。

At a certain point, we will build machines that are smarter than we are,

到某個時間點，我們會造出 比我們還聰明的機器，

and once we have machines that are smarter than we are,

而我們一旦造出比我們聰明的機器，

they will begin to improve themselves.

它們就會開始改善自己。

And then we risk what the mathematician IJ Good called

然後我們承擔數學家 IJ Good 稱為

an "intelligence explosion,"

「人工智能爆發」的風險，

that the process could get away from us.

那個過程會脫離我們的掌握。

Now, this is often caricatured, as I have here,

這時常被漫畫化，如我的這張圖，

as a fear that armies of malicious robots

一種恐懼：充滿惡意的機械人軍團

will attack us.

會攻擊我們。

But that isn't the most likely scenario.

但這不是最可能發生的情境。

It's not that our machines will become spontaneously malevolent.

並不是說我們的機器會變得 自然地帶有敵意。

The concern is really that we will build machines

問題在於我們將會造出

that are so much more competent than we are

遠比我們更有競爭力的機器，

that the slightest divergence between their goals and our own

只要我們和他們的目標 有些微的歧異，

could destroy us.

就會讓我們被毀滅。

Just think about how we relate to ants.

就想想我們和螞蟻的關係。

We don't hate them.

我們不討厭牠們。

We don't go out of our way to harm them.

我們不會特別去傷害牠們。

In fact, sometimes we take pains not to harm them.

甚至有時我們為了 不傷害牠們而承受痛苦。

We step over them on the sidewalk.

我們在人行道跨越他們。

But whenever their presence

但當牠們的存在

seriously conflicts with one of our goals,

和我們的目標嚴重衝突，

let's say when constructing a building like this one,

譬如當我們要建造一棟 和這裡一樣的建築物，

we annihilate them without a qualm.

我們會毫無不安地除滅牠們。

The concern is that we will one day build machines

問題在於有一天我們會造出機器，

that, whether they're conscious or not,

無論他們是有意識或者沒有意識，

could treat us with similar disregard.

會對我們如螞蟻般的不予理會。

Now, I suspect this seems far-fetched to many of you.

現在，我懷疑這種說法 對這裡大部分的人來說不著邊際。

I bet there are those of you who doubt that superintelligent AI is possible,

我確信你們有些人懷疑 超級人工智能出現的可能，

much less inevitable.

更別說它必然出現。

But then you must find something wrong with one of the following assumptions.

但接著你一點會發現 接下來其中一個假設有點問題。

And there are only three of them.

以下只有三個假設。

Intelligence is a matter of information processing in physical systems.

智能是關於資訊 在物質系統裡處理的過程。

Actually, this is a little bit more than an assumption.

其實這個陳述已經不只是一個假設，

We have already built narrow intelligence into our machines,

我們已經在我們的機器裡 安裝了有限的智能，

and many of these machines perform

而很多這樣的機器已經表現出

at a level of superhuman intelligence already.

某種程度的超人類智能。

And we know that mere matter

而我們知道這個現象

can give rise to what is called "general intelligence,"

可能導致被稱為「通用智能」的東西，

an ability to think flexibly across multiple domains,

一種能跨多個領域靈活思考的能力，

because our brains have managed it. Right?

因為我們的腦 已經掌握了這個，對吧？

I mean, there's just atoms in here,

我的意思是，裡面都只是原子，

and as long as we continue to build systems of atoms

只要我們繼續製造基於原子的系統，

that display more and more intelligent behavior,

越來越能表現智能的行為，

we will eventually, unless we are interrupted,

我們終究會，除非我們被打斷，

we will eventually build general intelligence

我們終究會造出通用智能

into our machines.

裝進我們的機器裡。

It's crucial to realize that the rate of progress doesn't matter,

關鍵是理解到發展的速率無關緊要，

because any progress is enough to get us into the end zone.

因為任何進展都足以 帶我們到終結之境。

We don't need Moore's law to continue. We don't need exponential progress.

我們不需要摩爾定律才能繼續。 我們不需要指數型的發展。

We just need to keep going.

我們只需要繼續前進。

The second assumption is that we will keep going.

第二個假設是我們會繼續前進。

We will continue to improve our intelligent machines.

我們會持續改善我們的智能機器。

And given the value of intelligence --

而因為智能的價值──

I mean, intelligence is either the source of everything we value

我的意思是，智能是所有 我們珍視的事物的源頭，

or we need it to safeguard everything we value.

或者我們需要智能 來保護我們珍視的事物。

It is our most valuable resource.

智能是我們最珍貴的資源。

So we want to do this.

所以我們想要這麼做。

We have problems that we desperately need to solve.

我們有許多亟需解決的問題。

We want to cure diseases like Alzheimer's and cancer.

我們想要治癒疾病 如阿茲海默症和癌症。

We want to understand economic systems. We want to improve our climate science.

我們想要了解經濟系統。 我們想要改進我們的氣候科學。

So we will do this, if we can.

所以我們會這麼做，只要我們可以。

The train is already out of the station, and there's no brake to pull.

火車已經出站，而沒有煞車可以拉。

Finally, we don't stand on a peak of intelligence,

最後一點，我們不站在智能的巔峰，

or anywhere near it, likely.

或者根本不在那附近。

And this really is the crucial insight.

而這真的是一種重要的洞察。

This is what makes our situation so precarious,

正是這個讓我們的處境如此危險可疑，

and this is what makes our intuitions about risk so unreliable.

這也讓我們對風險的直覺 變得很不可靠。

Now, just consider the smartest person who has ever lived.

現在，想想這世界上最聰明的人。

On almost everyone's shortlist here is John von Neumann.

每個人的清單上幾乎都會有 約翰·馮·諾伊曼。

I mean, the impression that von Neumann made on the people around him,

我是指， 馮·諾伊曼 對他周圍的人造成的印象，

and this included the greatest mathematicians and physicists of his time,

而這包括和他同時代 最棒的數學家和物理學家，

is fairly well-documented.

被好好地記錄了。

If only half the stories about him are half true,

只要有一半關於他的故事一半是真的，

there's no question

那毫無疑問

he's one of the smartest people who has ever lived.

他是世界上活過最聰明的人之一。

So consider the spectrum of intelligence.

所以考慮智能的頻譜。

Here we have John von Neumann.

約翰·馮·諾伊曼在這裡。

And then we have you and me.

然後你和我在這裡。

And then we have a chicken.

然後雞在這裡。

(Laughter)

（笑聲）

Sorry, a chicken.

抱歉，雞應該在那裡。

(Laughter)

（笑聲）

There's no reason for me to make this talk more depressing than it needs to be.

我實在無意把這個演講 弄得比它本身更讓人感到沮喪。

(Laughter)

（笑聲）

It seems overwhelmingly likely, however, that the spectrum of intelligence

智能的頻譜似乎勢不可擋地

extends much further than we currently conceive,

往比我們能理解的更遠的地方延伸，

and if we build machines that are more intelligent than we are,

如果我們造出 比我們更有智能的機器，

they will very likely explore this spectrum

他們很可能會探索這個頻譜，

in ways that we can't imagine,

以我們無法想像的方式，

and exceed us in ways that we can't imagine.

然後以我們無法想像的方式超越我們。

And it's important to recognize that this is true by virtue of speed alone.

重要的是認識到這說法 僅因速度的優勢即為真。

Right? So imagine if we just built a superintelligent AI

對吧？請想像如果我們造出了 一個超級人工智能，

that was no smarter than your average team of researchers

它不比你一般在史丹佛或麻省理工 遇到的研究團隊聰明。

at Stanford or MIT.

Well, electronic circuits function about a million times faster

電子電路作用的速率 比起生化作用快一百萬倍，

than biochemical ones,

so this machine should think about a million times faster

所以這個機器思考應該 比製造它的心智快一百萬倍。

than the minds that built it.

So you set it running for a week,

如果你設定讓它運行一星期，

and it will perform 20,000 years of human-level intellectual work,

他會執行人類兩萬年的智能工作，

week after week after week.

一週接著一週接著一週。

How could we even understand, much less constrain,

我們如何可能理解，較不嚴格地說，

a mind making this sort of progress?

一個達成如此進展的心智？

The other thing that's worrying, frankly,

另一個另人擔心的事，老實說，

is that, imagine the best case scenario.

是想像最好的情況。

So imagine we hit upon a design of superintelligent AI

想像我們想到一個沒有安全顧慮的 超級人工智能的設計，

that has no safety concerns.

We have the perfect design the first time around.

我們第一次就做出了完美的設計。

It's as though we've been handed an oracle

如同我們被給予了一個神諭，

that behaves exactly as intended.

完全照我們的預期地動作。

Well, this machine would be the perfect labor-saving device.

這個機器會是完美的人力節約裝置。

It can design the machine that can build the machine

它能設計一個機器，

那機器能製造出能做任何人工的機器，

that can do any physical work,

powered by sunlight,

以太陽能驅動，

more or less for the cost of raw materials.

幾乎只需要原料的成本。

So we're talking about the end of human drudgery.

所以我們是在談人類苦役的終結。

We're also talking about the end of most intellectual work.

我們也是在談大部分 智力工作的終結。

So what would apes like ourselves do in this circumstance?

像我們一樣的猩猩 在這種情況下會做什麼？

Well, we'd be free to play Frisbee and give each other massages.

我們可能可以自由地 玩飛盤和互相按摩。

Add some LSD and some questionable wardrobe choices,

加上一點迷幻藥和可議的服裝選擇，

and the whole world could be like Burning Man.

整個世界都可以像在過火人祭典。

(Laughter)

（笑聲）

Now, that might sound pretty good,

那聽起來也許很不錯，

but ask yourself what would happen

但請問，在我們目前的經濟和政治 秩序下，會發生什麼事情？

under our current economic and political order?

It seems likely that we would witness

我們很可能會見證

a level of wealth inequality and unemployment

一種我們從未見過的 財富不均和失業程度。

that we have never seen before.

Absent a willingness to immediately put this new wealth

缺乏一種意願來把這份新財富馬上

to the service of all humanity,

放在服務全人類，

a few trillionaires could grace the covers of our business magazines

少數幾個萬億富翁 能登上我們的財經雜誌，

while the rest of the world would be free to starve.

而其他人可以自由地選擇挨餓。

And what would the Russians or the Chinese do

而俄國和中國會怎麼做？

if they heard that some company in Silicon Valley

當他們聽說矽谷的某個公司

was about to deploy a superintelligent AI?

即將部署一個超級人工智能，

This machine would be capable of waging war,

這個機器能夠發動戰爭，

whether terrestrial or cyber,

無論是領土侵略或者網路電子戰，

with unprecedented power.

以前所未見的威力。

This is a winner-take-all scenario.

這是個贏者全拿的劇本。

To be six months ahead of the competition here

在這個競爭領先六個月

is to be 500,000 years ahead,

等於領先五十萬年，

at a minimum.

最少。

So it seems that even mere rumors of this kind of breakthrough

所以即使僅僅是這種突破的謠言

could cause our species to go berserk.

都能使我們這個種族走向狂暴。

Now, one of the most frightening things,

現在，最讓人驚恐的事情，

in my view, at this moment,

在我的看法，在這個時刻，

are the kinds of things that AI researchers say

是人工智慧研究者在試著表現得 讓人安心時說的那類話。

when they want to be reassuring.

And the most common reason we're told not to worry is time.

而最常用來告訴我們 現在不要擔心的理由是時間。

This is all a long way off, don't you know.

這還有很長的路要走，你不知道嗎，

This is probably 50 or 100 years away.

起碼還要 50 到 100 年。

One researcher has said,

一個研究人員曾說，

"Worrying about AI safety

「憂心人工智慧安全

is like worrying about overpopulation on Mars."

如同憂心火星人口爆炸。」

This is the Silicon Valley version

這是矽谷版本的

of "don't worry your pretty little head about it."

「別杞人憂天。」

（笑聲）

(Laughter)

No one seems to notice

似乎沒人注意到

that referencing the time horizon

以時間當參考

is a total non sequitur.

是一個不合理的推論。

If intelligence is just a matter of information processing,

如果智能只是關於資訊的處理，

and we continue to improve our machines,

而我們持續改善我們的機器，

we will produce some form of superintelligence.

我們會製作出某種形式的超級智能。

And we have no idea how long it will take us

而且我們不知道要花我們多長的時間

to create the conditions to do that safely.

來創造安全地這麼做的條件。

Let me say that again.

讓我再說一次，

We have no idea how long it will take us

我們不知道要花我們多長的時間

to create the conditions to do that safely.

來創造安全地這麼做的條件。

And if you haven't noticed, 50 years is not what it used to be.

而且如果你還沒注意到， 50 年已經不像以前的概念。

This is 50 years in months.

這是 50 年以月來表示。

This is how long we've had the iPhone.

這是我們有了 iPhone 的時間。

This is how long "The Simpsons" has been on television.

這是《辛普森家庭》 在電視上播映的時間。

Fifty years is not that much time

50 年不是那麼長的時間

to meet one of the greatest challenges our species will ever face.

來面對對我們這個種族來說 最巨大的挑戰之一。

Once again, we seem to be failing to have an appropriate emotional response

再一次說，我們似乎 無法產生適當的情緒反應，

to what we have every reason to believe is coming.

對應我們有所有的理由 相信將發生的事。

The computer scientist Stuart Russell has a nice analogy here.

資訊科學家斯圖亞特·羅素 有個很好的比喻。

He said, imagine that we received a message from an alien civilization,

他說，想像我們收到一則 外星文明的訊息，

which read:

寫道：

"People of Earth,

「地球的人們，

we will arrive on your planet in 50 years.

我們 50 年內會到達你們的星球。

Get ready."

作好準備。」

And now we're just counting down the months until the mothership lands?

而現在我們只是在倒數 外星母艦還剩幾個月登陸？

We would feel a little more urgency than we do.

我們會比我們現在稍微感到緊迫。

Another reason we're told not to worry

另一個我們被告知不用擔心的原因

is that these machines can't help but share our values

是這些機器不得不和我們 有一樣的價值觀，

because they will be literally extensions of ourselves.

因為他們實際上只是我們的延伸。

They'll be grafted onto our brains,

它們會被植入我們的大腦裡，

and we'll essentially become their limbic systems.

而我們基本上變成 他們大腦的邊緣系統。

Now take a moment to consider

現在用一點時間想想

that the safest and only prudent path forward,

這最安全而且唯一謹慎的往前的路，

recommended,

被推薦的，

is to implant this technology directly into our brains.

是將這個科技植入我們的腦內。

Now, this may in fact be the safest and only prudent path forward,

這也許的確是最安全 而且唯一謹慎的往前的路，

but usually one's safety concerns about a technology

但通常科技的安全性問題

have to be pretty much worked out before you stick it inside your head.

應該在把東西插到你腦袋裡之前 就該大部分解決了。

(Laughter)

（笑聲）

The deeper problem is that building superintelligent AI on its own

更深層的問題是， 打造超級人工智能本身，

seems likely to be easier

似乎相對容易於

than building superintelligent AI

打造超級人工智慧

and having the completed neuroscience

並擁有完整的神經科學，

that allows us to seamlessly integrate our minds with it.

讓我們可以把我們的心智 無縫與之整合。

And given that the companies and governments doing this work

而假設正在從事人工智能 研發的許多公司和政府，

are likely to perceive themselves as being in a race against all others,

很可能察覺他們 正在和所有其他人競爭，

given that to win this race is to win the world,

假設贏了這個競爭就是贏得世界，

provided you don't destroy it in the next moment,

假設你在下一刻不會毀了世界，

then it seems likely that whatever is easier to do

那麼很可能比較容易做的事

will get done first.

就會先被做完。

Now, unfortunately, I don't have a solution to this problem,

現在，很不幸地， 我沒有這個問題的解決方法，

apart from recommending that more of us think about it.

除了建議我們更多人思考這個問題。

我想我們需要類似曼哈頓計畫的東西，

I think we need something like a Manhattan Project

on the topic of artificial intelligence.

針對人工智能這個課題。

Not to build it, because I think we'll inevitably do that,

不是因為我們不可避免地 要這麼做而做，

而是試著理解如何避免軍備競賽，

but to understand how to avoid an arms race

and to build it in a way that is aligned with our interests.

而且用一種符合 我們利益的方式打造之。

When you're talking about superintelligent AI

當你在談論能夠對其本身 造成改變的超級人工智能，

that can make changes to itself,

it seems that we only have one chance to get the initial conditions right,

這似乎說明我們只有一次機會 把初始條件做對，

and even then we will need to absorb

而且我們會必須承受

the economic and political consequences of getting them right.

為了將它們做對的經濟和政治後果。

But the moment we admit

但一旦我們承認

that information processing is the source of intelligence,

資訊處理是智能的源頭，

that some appropriate computational system is what the basis of intelligence is,

某些適當的電腦系統是智能的基礎，

and we admit that we will improve these systems continuously,

而且我們承認我們會 持續改進這些系統，

and we admit that the horizon of cognition very likely far exceeds

而且我們承認認知的極限 有可能遠遠超越

what we currently know,

我們目前所知，

then we have to admit

我們就會承認

that we are in the process of building some sort of god.

我們正在打造某種神明的過程裡。

Now would be a good time

現在是個好時機

to make sure it's a god we can live with.

來確保那是個我們能夠 與之共存的神明。

Thank you very much.

謝謝大家。

(Applause)