Don't fear superintelligent AI | Grady Booch

譯者: Melody Tang 審譯者: Helen Chang

When I was a kid, I was the quintessential nerd.

我小時候是個典型的書呆子。

I think some of you were, too.

你們有些人也曾經是書呆子。

(Laughter)

（笑聲）

And you, sir, who laughed the loudest, you probably still are.

那位笑最大聲的也許現在還是書呆子。

(Laughter)

（笑聲）

I grew up in a small town in the dusty plains of north Texas,

我在北德州一座塵土飛揚的 平原小鎮長大，

the son of a sheriff who was the son of a pastor.

父親是警長，祖父是牧師。

Getting into trouble was not an option.

所以我「惹上麻煩」是不可能的事。

And so I started reading calculus books for fun.

因此我開始讀微積分的書當消遣。

(Laughter)

（笑聲）

You did, too.

你也讀過。

That led me to building a laser and a computer and model rockets,

這引導我去製作雷射、 電腦和火箭模型，

and that led me to making rocket fuel in my bedroom.

然後我在臥室裡製造火箭推進燃料。

Now, in scientific terms,

如果用科學上的說法來形容，

we call this a very bad idea.

我們把這叫做「糟糕透頂的主意」。

(Laughter)

（笑聲）

Around that same time,

大概在那段時期，

Stanley Kubrick's "2001: A Space Odyssey" came to the theaters,

史丹利·庫柏力克執導的 《2001 太空漫遊》上映了，

and my life was forever changed.

我的人生也永遠地改變了。

I loved everything about that movie,

我愛上了這部電影裡的一切，

especially the HAL 9000.

尤其是「豪爾－9000」。

Now, HAL was a sentient computer

豪爾是一台知覺電腦，

designed to guide the Discovery spacecraft

設計來引導「發現號」太空船，

from the Earth to Jupiter.

從地球前往木星。

HAL was also a flawed character,

豪爾也是個有缺陷的角色，

for in the end he chose to value the mission over human life.

因為它最終選擇 任務優先、人命其次。

Now, HAL was a fictional character,

儘管豪爾只是個虛構的角色，

but nonetheless he speaks to our fears,

卻直指我們內心的恐懼，

our fears of being subjugated

我們害怕被征服的恐懼，

by some unfeeling, artificial intelligence

臣服於某個沒有情感，

對於人類漠不關心的人工智能電腦。

who is indifferent to our humanity.

I believe that such fears are unfounded.

我認為這種恐懼只是杞人憂天。

Indeed, we stand at a remarkable time

沒錯，我們現在處於

人類史上一個偉大的時代。

in human history,

where, driven by refusal to accept the limits of our bodies and our minds,

我們拒絕接受肉體和心靈上的限制，

we are building machines

我們製造細緻、精美，複雜

of exquisite, beautiful complexity and grace

又優雅的機器。

that will extend the human experience

這些機器將透過各種超乎想像的方式，

in ways beyond our imagining.

拓展人類的經驗範圍。

After a career that led me from the Air Force Academy

我曾任職於美國空軍學院，

to Space Command to now,

現在服務於美國空軍太空司令部。

I became a systems engineer,

我成了系統工程師，

and recently I was drawn into an engineering problem

最近我被派去解決一個

與美國太空總署的 火星任務有關的工程問題。

associated with NASA's mission to Mars.

Now, in space flights to the Moon,

目前，前往月球的太空航行，

we can rely upon mission control in Houston

我們可以仰賴位於休士頓的控制中心

to watch over all aspects of a flight.

來監控這段旅程的所有層面。

However, Mars is 200 times further away,

然而，火星的距離比月球遠了200倍，

and as a result it takes on average 13 minutes

一個訊號平均要花 13 分鐘，

for a signal to travel from the Earth to Mars.

才能從地球傳送到火星。

If there's trouble, there's not enough time.

如果發生了任何狀況， 根本不夠時間解決。

And so a reasonable engineering solution

所以一個合理的工程解決方案，

calls for us to put mission control

就是把控制中心的位置，

inside the walls of the Orion spacecraft.

放在「獵戶座」太空船裡面。

Another fascinating idea in the mission profile

在任務方面，還有另一個絕妙的點子，

places humanoid robots on the surface of Mars

就是提早在人類之前抵達前，

先在火星表面部署人型機器人。

before the humans themselves arrive,

first to build facilities

它們首先建造設備，

and later to serve as collaborative members of the science team.

以後擔任科學小組的協助角色。

Now, as I looked at this from an engineering perspective,

當我從工程師的角度來看這件事，

it became very clear to me that what I needed to architect

很明顯我需要做的

was a smart, collaborative,

就是製作一個聰明、善於合作、

socially intelligent artificial intelligence.

具備社交智能的人工智能電腦。

In other words, I needed to build something very much like a HAL

換句話說，我需要製作一個很像豪爾，

but without the homicidal tendencies.

但是沒有殺人的癖好的電腦。

(Laughter)

（笑聲）

Let's pause for a moment.

讓我們暫停一下。

Is it really possible to build an artificial intelligence like that?

真的有可能，製作出那樣的 人工智慧電腦嗎？

Actually, it is.

確實有可能。

In many ways,

就許多方面來說，

this is a hard engineering problem

這是一個困難的工程問題，

with elements of AI,

其中包含了人工智慧的元素，

not some wet hair ball of an AI problem that needs to be engineered.

而不是一個難解的人工智能問題。

To paraphrase Alan Turing,

就如同艾倫·圖靈所說的，

I'm not interested in building a sentient machine.

我沒有興趣製造一台情識的機器。

I'm not building a HAL.

我所製作的並不是豪爾，

All I'm after is a simple brain,

我想要的是一個簡單的大腦，

something that offers the illusion of intelligence.

它能夠營造出「具備智慧」的錯覺。

The art and the science of computing have come a long way

在豪爾登台亮相之後，

藝術和計算科學已有了長足的進步。

since HAL was onscreen,

and I'd imagine if his inventor Dr. Chandra were here today,

我想如果它的發明者 強德拉（Chandra）博士今天也在現場，

he'd have a whole lot of questions for us.

他會有很多問題要問我們。

Is it really possible for us

我們是否真的有可能

to take a system of millions upon millions of devices,

讓一個擁有無數元件的系統，

to read in their data streams,

去讀自己的數據流，

to predict their failures and act in advance?

然後預測故障以及提前預防？

Yes.

這是可能的。

Can we build systems that converse with humans in natural language?

我們可否做出可使用自然語言 交談的系統？

Yes.

可以。

Can we build systems that recognize objects, identify emotions,

我們可否做出能夠 辨別物體、辨識情緒、

emote themselves, play games and even read lips?

模擬情緒、玩遊戲， 甚至能讀唇語的系統？

Yes.

可以。

Can we build a system that sets goals,

我們可否做出一個能夠設定目標，

that carries out plans against those goals and learns along the way?

並一面執行朝向目標的計劃， 同時在過程中學習的系統？

Yes.

可以。

Can we build systems that have a theory of mind?

我們可否做出具備心智理論的系統？

This we are learning to do.

這個目前我們還在學習做。

Can we build systems that have an ethical and moral foundation?

我們可否做出擁有 倫理與道德基礎的系統？

This we must learn how to do.

這個我們必須摸索如何做。

So let's accept for a moment

所以讓我們姑且相信

that it's possible to build such an artificial intelligence

建造這樣的人工智能系統有可能成真，

for this kind of mission and others.

以用在諸如此類和其他的任務。

The next question you must ask yourself is,

接下來我們必須捫心自問，

should we fear it?

我們是否應該感到害怕？

Now, every new technology

誠然，每一項新的科技

brings with it some measure of trepidation.

都會帶來某種程度的擔憂。

When we first saw cars,

當汽車問世的時候，

people lamented that we would see the destruction of the family.

眾人都在哀嘆家庭可能因此而毀滅。

When we first saw telephones come in,

當電話問世的時候，

people were worried it would destroy all civil conversation.

眾人擔心日常的對話會不復存在。

At a point in time we saw the written word become pervasive,

當書面文字風行的時候，

people thought we would lose our ability to memorize.

眾人以為我們會失去記憶的能力。

These things are all true to a degree,

這些擔憂在某種程度上都有所根據，

but it's also the case that these technologies

但同時這些科技

brought to us things that extended the human experience

以某些深刻的方式，

帶給我們許多拓展人類經驗的事物。

in some profound ways.

So let's take this a little further.

讓我們再繼續探討。

I do not fear the creation of an AI like this,

我並不懼怕創造 像這樣的人工智能系統，

because it will eventually embody some of our values.

因為它最終會 體現我們的一些價值觀。

Consider this: building a cognitive system is fundamentally different

思考一下：建造認知系統

與過去建造傳統 軟體密集型的系統根本不同。

than building a traditional software-intensive system of the past.

We don't program them. We teach them.

我們不寫電腦程式。我們教它們。

In order to teach a system how to recognize flowers,

為了教導系統如何辨識花，

I show it thousands of flowers of the kinds I like.

我給它們看數以千計 我喜歡的花的圖片。

In order to teach a system how to play a game --

為了教系統如何玩遊戲－－

Well, I would. You would, too.

我會，你也會。

I like flowers. Come on.

我喜歡花。你也是吧！

To teach a system how to play a game like Go,

為了教系統如何玩遊戲，例如圍棋，

I'd have it play thousands of games of Go,

我會讓它一面下數千局圍棋，

but in the process I also teach it

也在下棋的過程中

how to discern a good game from a bad game.

教它如何分辨好的、不好的棋局。

If I want to create an artificially intelligent legal assistant,

如果我要造個人工智能的法律助理，

I will teach it some corpus of law

我會教它一些法律，

but at the same time I am fusing with it

同時我會將它與憐憫的感覺

the sense of mercy and justice that is part of that law.

和法律正義融合在一起。

In scientific terms, this is what we call ground truth,

以科學術語方面， 這就是我們所謂的真理，

and here's the important point:

重點來了：

in producing these machines,

在製造這些機器時，

we are therefore teaching them a sense of our values.

我們因此教它們我們的價值感。

To that end, I trust an artificial intelligence

為此，我相信人工智能

the same, if not more, as a human who is well-trained.

至少不會輸給訓練有素的人。

But, you may ask,

但是，你可能問，

what about rogue agents,

那些流氓代理，

some well-funded nongovernment organization?

那些資金豐富的非政府組織呢？

I do not fear an artificial intelligence in the hand of a lone wolf.

我不擔心獨狼（獨行俠） 手上的人工智能。

Clearly, we cannot protect ourselves against all random acts of violence,

很顯然，我們無法防禦 所有隨機的暴力行為，

but the reality is such a system

但是現實是這種系統，

requires substantial training and subtle training

需要大量的訓練和微妙的訓練，

far beyond the resources of an individual.

非個人的資源所能及。

And furthermore,

此外，

it's far more than just injecting an internet virus to the world,

遠遠超出像把互聯網病毒注入世界，

where you push a button, all of a sudden it's in a million places

只要按個按鈕， 頃刻之間病毒就會散播各處，

and laptops start blowing up all over the place.

所有地方的筆電開始當機。

Now, these kinds of substances are much larger,

這些實質要大得多，

and we'll certainly see them coming.

我們確定會看到它們的到來。

Do I fear that such an artificial intelligence

我擔心這樣的人工智能

might threaten all of humanity?

會威脅全體人類嗎？

If you look at movies such as "The Matrix," "Metropolis,"

電影如《駭客任務》、 《大都會》、《魔鬼終結者》，

"The Terminator," shows such as "Westworld,"

電視劇像是《西方極樂園》，

they all speak of this kind of fear.

都是在談這種恐懼。

Indeed, in the book "Superintelligence" by the philosopher Nick Bostrom,

的確，在哲學家尼克· 博斯特倫 寫的 《超級智能》書裡，

he picks up on this theme

他以此主題，

and observes that a superintelligence might not only be dangerous,

主張超智能不僅危險，

it could represent an existential threat to all of humanity.

還可能威脅全人類的存亡。

Dr. Bostrom's basic argument

博斯特倫博士的基本論點是：

is that such systems will eventually

這種系統最終

have such an insatiable thirst for information

將會不屈不撓地渴望資訊，

that they will perhaps learn how to learn

它們或許會學習到如何學習的方法，

and eventually discover that they may have goals

最終發現它們的目標

可能與人類的背道而馳。

that are contrary to human needs.

Dr. Bostrom has a number of followers.

博斯特倫博士有不少追隨者。

He is supported by people such as Elon Musk and Stephen Hawking.

他得到伊隆·馬斯克和 史蒂芬·霍金等人的支持。

With all due respect

我非常尊重

to these brilliant minds,

這些非常聰明的人，

I believe that they are fundamentally wrong.

但是我相信他們從根本上就是錯誤的，

Now, there are a lot of pieces of Dr. Bostrom's argument to unpack,

博斯格羅姆博士的許多說法 需要被詳細分析，

and I don't have time to unpack them all,

但在此我沒有時間分別解說。

but very briefly, consider this:

簡要的說，就考慮這個：

super knowing is very different than super doing.

超智能與超執行完全是兩回事。

HAL was a threat to the Discovery crew

只有當豪爾全面掌控「發現號」時，

only insofar as HAL commanded all aspects of the Discovery.

才對「發現號」的組員造成威脅。

So it would have to be with a superintelligence.

所以它就必須是有超智慧。

It would have to have dominion over all of our world.

它必須擁有對我們全世界的統治權。

This is the stuff of Skynet from the movie "The Terminator"

這是電影《魔鬼終結者》 裡的 「天網」所具有的。

in which we had a superintelligence

它有超智能

that commanded human will,

來指揮的人的意志，

that directed every device that was in every corner of the world.

來操控世界每一個角落的每個設備。

Practically speaking,

實際上，

it ain't gonna happen.

這不會發生。

We are not building AIs that control the weather,

我們不是製造人工智能來控制氣候、

that direct the tides,

控制海潮，

that command us capricious, chaotic humans.

指揮反覆無常和混亂的人類。

And furthermore, if such an artificial intelligence existed,

此外，如果這樣的人工智能真的存在，

it would have to compete with human economies,

它必須與人類的經濟競爭，

and thereby compete for resources with us.

從而與我們競爭資源。

And in the end --

最後，

don't tell Siri this --

不要告訴 Siri 這個——

we can always unplug them.

我們可以隨時拔掉它們的插頭。

(Laughter)

（笑聲）

We are on an incredible journey

我們正處於與我們的機器共同演化的

of coevolution with our machines.

一個難以置信的旅程。

The humans we are today

今天的人類

are not the humans we will be then.

不是屆時的人類。

To worry now about the rise of a superintelligence

現在擔心超級智能的出現，

is in many ways a dangerous distraction

會使我們在許多方面危險地分心，

because the rise of computing itself

因為計算機本身的興起，

brings to us a number of human and societal issues

帶來的許多人類和社會問題，

to which we must now attend.

我們現在就必須解決。

How shall I best organize society

當人力的需要逐漸減少時，

when the need for human labor diminishes?

我們如何重整這個社會？

How can I bring understanding and education throughout the globe

我該如何為全球帶來理解和教育

and still respect our differences?

而仍然尊重我們彼此的分歧？

How might I extend and enhance human life through cognitive healthcare?

我們如何通過認知保健 延伸和增強人的生命？

How might I use computing

我如何使用計算技術

to help take us to the stars?

來幫助我們去到其他星球？

And that's the exciting thing.

這是令人興奮的事。

The opportunities to use computing

透過計算技術的運用，

to advance the human experience

來拓展人類經驗的機會，

are within our reach,

就在我們眼前。

here and now,

此時、此刻，

and we are just beginning.

我們才正要起步而已。

Thank you very much.

非常感謝各位。

(Applause)

（掌聲）