Don't fear superintelligent AI | Grady Booch

Translator: Danly Deng Reviewer: Alan Watson

When I was a kid, I was the quintessential nerd.

我細個嗰陣係個書蟲

I think some of you were, too.

我估你哋一啲人都係

(Laughter)

（笑聲）

And you, sir, who laughed the loudest, you probably still are.

仲有你呀，阿生 笑得最大聲嗰個，睇嚟宜家都仲係

(Laughter)

（笑聲）

I grew up in a small town in the dusty plains of north Texas,

我喺德州北部平原一個鎮仔大嘅

the son of a sheriff who was the son of a pastor.

我阿爺係牧師，我老豆係警察

Getting into trouble was not an option.

所以自細就係乖乖仔

And so I started reading calculus books for fun.

睇微積分書當娛樂

(Laughter)

（笑聲）

You did, too.

你都係咧

That led me to building a laser and a computer and model rockets,

於是我自己整 激光啦、電腦啦、火箭模型啦

and that led me to making rocket fuel in my bedroom.

最後仲喺房裏邊整埋火箭燃料

Now, in scientific terms,

用我哋宜家科學嘅講法

we call this a very bad idea.

係搞搞震無幫襯

(Laughter)

（笑聲）

Around that same time,

就喺嗰陣

Stanley Kubrick 拍嗰部 《2001太空漫遊》上畫

Stanley Kubrick's "2001: A Space Odyssey" came to the theaters,

and my life was forever changed.

我嘅生活從此改寫

I loved everything about that movie,

我係部片嘅忠實擁躉

especially the HAL 9000.

特別中意哈爾 9000

Now, HAL was a sentient computer

哈爾係一部機械人

designed to guide the Discovery spacecraft

專為太空船導航而設計

from the Earth to Jupiter.

指引太空船由地球飛往木星

HAL was also a flawed character,

但哈爾都有缺點

for in the end he chose to value the mission over human life.

佢最後為實現目的而罔顧人類安全

Now, HAL was a fictional character,

哈爾雖然係虛構

but nonetheless he speaks to our fears,

但佢反映咗我哋嘅恐懼

our fears of being subjugated

就係怕無感情嘅人工智能 以後會操控人類

by some unfeeling, artificial intelligence

who is indifferent to our humanity.

I believe that such fears are unfounded.

我認為呢種恐懼唔成立

Indeed, we stand at a remarkable time

講真，人類呢一刻嘅歷史 可以話係最輝煌嘅

in human history,

where, driven by refusal to accept the limits of our bodies and our minds,

我哋冇向身體同大腦限制低頭

we are building machines

反而製造出精良、複雜又美觀嘅機器

of exquisite, beautiful complexity and grace

that will extend the human experience

協助人類逹至超乎想象嘅領域

in ways beyond our imagining.

After a career that led me from the Air Force Academy

我以前喺空軍學校同太空指揮部做嘢嘅

to Space Command to now,

咁而家，我做咗系統工程師

I became a systems engineer,

and recently I was drawn into an engineering problem

最近我著手於美國太空總署嘅 火星任務嘅一個工程問題

associated with NASA's mission to Mars.

Now, in space flights to the Moon,

到目前為止，所有上月球嘅太空船

we can rely upon mission control in Houston

我哋都可以留喺休斯頓控制中心控制

to watch over all aspects of a flight.

However, Mars is 200 times further away,

不過，火星比月球遠 200 倍

and as a result it takes on average 13 minutes

於是乎由地球傳送到火星嘅訊號

for a signal to travel from the Earth to Mars.

平均要花 13 分鐘先可以送到

If there's trouble, there's not enough time.

所以如果中途出咗咩問題

我哋都唔夠時間解決

And so a reasonable engineering solution

所以解決方法係

calls for us to put mission control

我哋要將任務控制台 裝喺太空船獵戶號嘅墻板裏邊

inside the walls of the Orion spacecraft.

Another fascinating idea in the mission profile

另一個方法係

places humanoid robots on the surface of Mars

喺人類登陸之前 喺火星表面放置一個機械人

before the humans themselves arrive,

first to build facilities

機械人前期負責興建設施

and later to serve as collaborative members of the science team.

後期就加入科學團隊擔當協助

Now, as I looked at this from an engineering perspective,

而當我企喺工程師嘅角度嚟睇

it became very clear to me that what I needed to architect

我好清楚我要打造嘅 係一個高智能、富團隊精神

was a smart, collaborative,

socially intelligent artificial intelligence.

同善於交際嘅人工智能

In other words, I needed to build something very much like a HAL

換句話嚟講，我要整到好似哈爾咁樣

but without the homicidal tendencies.

不過係唔會當人類 係冚家鏟要消滅嘅機械人

(Laughter)

（笑聲）

Let's pause for a moment.

等陣先

Is it really possible to build an artificial intelligence like that?

咁係咪真係有可能整個咁嘅機械人先？

Actually, it is.

其實，係可以嘅

In many ways,

好多時，設計人工智能元素係好難嘅

this is a hard engineering problem

with elements of AI,

not some wet hair ball of an AI problem that needs to be engineered.

咁唔係講緊整機械人頭髮

To paraphrase Alan Turing,

學 Alan Turning 話齋

I'm not interested in building a sentient machine.

我無興趣整隻有感覺嘅機器

I'm not building a HAL.

亦都唔係生產另一個哈爾

All I'm after is a simple brain,

我所追求嘅係一個會簡單思考

something that offers the illusion of intelligence.

同有少少智慧嘅機械人

The art and the science of computing have come a long way

自從哈爾喺電影出現之後

since HAL was onscreen,

計算科學已經發展咗好多

and I'd imagine if his inventor Dr. Chandra were here today,

我想像得到，如果發明佢嘅 Chandra 博士今日喺度嘅話

he'd have a whole lot of questions for us.

佢實有好多問題問我哋

Is it really possible for us

我哋有冇可能 從數以億計嘅設備裡面讀取數據

to take a system of millions upon millions of devices,

to read in their data streams,

同時預測機器犯嘅錯誤 同埋及早更正呢？

to predict their failures and act in advance?

Yes.

可以

Can we build systems that converse with humans in natural language?

我哋可唔可以 整隻機械人出嚟識講人話？

Yes.

可以

Can we build systems that recognize objects, identify emotions,

我哋可唔可以整隻機械人

識得識別物體、辨別情感 自帶情感、打機，甚至識讀唇語？

emote themselves, play games and even read lips?

Yes.

可以

Can we build a system that sets goals,

我哋可唔可以整隻機械人識訂立目標

that carries out plans against those goals and learns along the way?

實現目標兼且喺過程中自學？

Yes.

可以

Can we build systems that have a theory of mind?

我哋可唔可以整隻 有思維邏輯嘅機械人？

This we are learning to do.

我哋宜家嘗試緊

Can we build systems that have an ethical and moral foundation?

我哋可唔可以整隻機械人 明白道德觀念同底線？

This we must learn how to do.

呢個任務我哋責無旁貸

So let's accept for a moment

咁姑且我哋有可能

that it's possible to build such an artificial intelligence

為呢個任務或者其他任務 整隻咁樣嘅機械人

for this kind of mission and others.

The next question you must ask yourself is,

跟住你實會問

咁嘅人工智能會唔會造成威脅？

should we fear it?

Now, every new technology

時至今日，每項新科技面世 唔多唔少都會帶嚟不安

brings with it some measure of trepidation.

When we first saw cars,

以前啲人第一次見到汽車時

people lamented that we would see the destruction of the family.

就驚車禍會造成家破人亡

When we first saw telephones come in,

以前第一次見到電話時

people were worried it would destroy all civil conversation.

啲人就驚人同人之間嘅交流會受到破壞

At a point in time we saw the written word become pervasive,

曾幾何時啲人見到文字可以傳送

people thought we would lose our ability to memorize.

又驚人類嘅記憶力會喪失

These things are all true to a degree,

呢啲不安喺一定程度上嚟講無錯

but it's also the case that these technologies

但同時呢啲科技

拓闊咗人類嘅體驗

brought to us things that extended the human experience

in some profound ways.

So let's take this a little further.

我哋不如再講遠啲

I do not fear the creation of an AI like this,

我唔驚呢啲人工智能面世

because it will eventually embody some of our values.

因為佢最終會接受人類嘅一啲價值

Consider this: building a cognitive system is fundamentally different

試下咁唸：製造感知機械人

同製造以前傳統嘅軟件密集型機械人

than building a traditional software-intensive system of the past.

係有根本性分別

We don't program them. We teach them.

我哋而家唔係編程機械人

我哋係教機械人

In order to teach a system how to recognize flowers,

為咗教機械人識別花

I show it thousands of flowers of the kinds I like.

我攞幾千種我鍾意嘅花畀佢睇

In order to teach a system how to play a game --

為咗教曉機械人點打機——

Well, I would. You would, too.

我真係教㗎。你都會咁做

I like flowers. Come on.

我真係鍾意花呢——

To teach a system how to play a game like Go,

為咗教識機械人打「殺出重圍」

I'd have it play thousands of games of Go,

我畀佢打幾千局遊戲

but in the process I also teach it

不過呢個過程，我又會教佢 辨別好局同劣局

how to discern a good game from a bad game.

If I want to create an artificially intelligent legal assistant,

如果我想整隻法律助理機械人

I will teach it some corpus of law

我除咗會教佢法律

but at the same time I am fusing with it

我仲會教佢法律嘅寛容同公義

the sense of mercy and justice that is part of that law.

In scientific terms, this is what we call ground truth,

套用科學術語 呢啲係我哋所謂嘅參考標準

and here's the important point:

而且重要嘅係：

in producing these machines,

製造呢啲機械人時

we are therefore teaching them a sense of our values.

我哋將自己嘅價值觀灌輸畀佢

To that end, I trust an artificial intelligence

最終，我相信人工智能

the same, if not more, as a human who is well-trained.

會同一個受過專業訓練嘅人一樣

But, you may ask,

不過，你哋可能會問

what about rogue agents,

如果呢啲人工智能被非法分子利用呢？

some well-funded nongovernment organization?

I do not fear an artificial intelligence in the hand of a lone wolf.

雖然我哋唔可能杜絕所有暴力事件發生

Clearly, we cannot protect ourselves against all random acts of violence,

但我唔擔心人工智能落入一啲壞人手中

but the reality is such a system

因為人工智能需要持續同細微嘅改進

requires substantial training and subtle training

far beyond the resources of an individual.

單憑個人資源唔使旨意做到

And furthermore,

而且，唔好當成植入網絡病毒咁簡單

it's far more than just injecting an internet virus to the world,

where you push a button, all of a sudden it's in a million places

唔好以為隨時隨地㩒個掣

and laptops start blowing up all over the place.

全世界嘅電腦瞬間就會爆炸

Now, these kinds of substances are much larger,

人工智能係複雜好多嘅嘢

and we'll certainly see them coming.

但我相信遲早有一日人工智能會出現

Do I fear that such an artificial intelligence

我驚唔驚人工智能會威脅全人類？

might threaten all of humanity?

If you look at movies such as "The Matrix," "Metropolis,"

電影《黑客帝國》、《大都會》

"The Terminator," shows such as "Westworld,"

《終結者》、電視劇《西部世界》

they all speak of this kind of fear.

呢類影視作品都係刻畫呢種恐懼

Indeed, in the book "Superintelligence" by the philosopher Nick Bostrom,

無錯，哲學家 Nick Bostrom 喺《超人工智能》一書裏邊都認為

he picks up on this theme

and observes that a superintelligence might not only be dangerous,

超人工智能唔單止危險

it could represent an existential threat to all of humanity.

而且仲危及全人類

Dr. Bostrom's basic argument

佢嘅基本論點有︰

is that such systems will eventually

咁樣嘅機械人 最終唔會滿足於眼前擁有嘅資訊

have such an insatiable thirst for information

that they will perhaps learn how to learn

機械人可能會因而自己鑽研學習方法

and eventually discover that they may have goals

以至到最後發現 自己有啲目標同人類需要有矛盾

that are contrary to human needs.

Dr. Bostrom has a number of followers.

有人支持博森博士嘅觀點

He is supported by people such as Elon Musk and Stephen Hawking.

包括 Elon Musk 同霍金

With all due respect

我想指出其實幾位智者諗錯咗

to these brilliant minds,

I believe that they are fundamentally wrong.

Now, there are a lot of pieces of Dr. Bostrom's argument to unpack,

Nick Bostrom 嘅理論有好多錯誤

and I don't have time to unpack them all,

但我無時間講曬所有

but very briefly, consider this:

但簡單嚟講，可以咁理解： 超智能唔代表超萬能

super knowing is very different than super doing.

HAL was a threat to the Discovery crew

哈爾對於成個太空探索團隊嘅威脅

only insofar as HAL commanded all aspects of the Discovery.

僅限於佢可以對探索任務落命令

So it would have to be with a superintelligence.

所以任務需要一個超智能機器

It would have to have dominion over all of our world.

落命令嘅需要有統治世界嘅能力

This is the stuff of Skynet from the movie "The Terminator"

電影《未來戰士 2018》 裡面嘅 Skynet

嗰個可以操控人類意志嘅 超人工智能防禦系統

in which we had a superintelligence

that commanded human will,

控制曬全世界所有嘅機器同裝置

that directed every device that was in every corner of the world.

Practically speaking,

即係話

it ain't gonna happen.

電影情節係唔會發生

We are not building AIs that control the weather,

我哋唔係整隻機械人出嚟呼風喚雨

that direct the tides,

that command us capricious, chaotic humans.

操控喜怒無常、陷於鬥爭嘅人類

And furthermore, if such an artificial intelligence existed,

再者，如果咁樣嘅機械人真係存在

it would have to compete with human economies,

佢就會同人類嘅經濟鬥過

and thereby compete for resources with us.

甚至同人類爭資源

And in the end --

最終結果係——

don't tell Siri this --

唔好話俾 Siri 聽——

we can always unplug them.

我哋可以隨時熄咗佢哋

(Laughter)

（笑聲）

We are on an incredible journey

我哋同機械人係共同進化

of coevolution with our machines.

The humans we are today

未來嘅人類同今日嘅我哋 唔可以同日而語

are not the humans we will be then.

To worry now about the rise of a superintelligence

人類擔心人工智能帶嚟威脅

is in many ways a dangerous distraction

只會令人類唔去真正關心 科技崛起帶嚟嘅人文同社會問題

because the rise of computing itself

brings to us a number of human and societal issues

而呢啲問題正正係 我哋需要著手解決嘅

to which we must now attend.

How shall I best organize society

問題例如有︰

當我哋唔再需要勞動力嘅時候 我哋要點去調控社會？

when the need for human labor diminishes?

How can I bring understanding and education throughout the globe

點樣向全世界傳播知識同教育 同時又尊重當地嘅差異？

and still respect our differences?

How might I extend and enhance human life through cognitive healthcare?

點樣通過認知式保健幫人類延年益壽？

How might I use computing

點樣利用電腦幫人類踏足外太空？

to help take us to the stars?

And that's the exciting thing.

諗下都覺得興奮

The opportunities to use computing

利用計算科學去開拓人類經歷嘅機會 就喺手裏邊

to advance the human experience

are within our reach,

here and now,

and we are just beginning.

而我哋只係啱啱捉緊到

Thank you very much.

多謝大家

(Applause)

（掌聲）