Don't fear superintelligent AI | Grady Booch

翻译人员: Weidi Liu 校对人员: Jiawei Ni

When I was a kid, I was the quintessential nerd.

在我还是孩子的时候 ， 我是一个典型的书呆子。

I think some of you were, too.

我猜你们的一部分人和我一样。

(Laughter)

（笑声）

And you, sir, who laughed the loudest, you probably still are.

还有你，那位先生，笑得最大声， 说不定你现在还是呢。

(Laughter)

（笑声）

I grew up in a small town in the dusty plains of north Texas,

我成长在德克萨斯州北部 荒凉平原上的一个小镇。

the son of a sheriff who was the son of a pastor.

我的爸爸是警长，爷爷是一位牧师，

Getting into trouble was not an option.

所以自找麻烦从来不是一个好的选项。

And so I started reading calculus books for fun.

所以我开始看有关 微积分的书来打发时间。

(Laughter)

（笑声）

You did, too.

你也是这样。

That led me to building a laser and a computer and model rockets,

于是我学着制作了一个激光器， 一台电脑，和一个火箭模型，

and that led me to making rocket fuel in my bedroom.

并且在自己的卧室制取火箭燃料。

Now, in scientific terms,

现在，从科学的角度而言，

we call this a very bad idea.

我们把这个叫做 一个糟糕透顶的主意。

(Laughter)

（笑声）

Around that same time,

在差不多同一时间，

Stanley Kubrick's "2001: A Space Odyssey" came to the theaters,

Stanley Kubrick的 “2001：太空漫游”上映了，

and my life was forever changed.

我的生活从此改变。

I loved everything about that movie,

我喜欢关于那部电影的一切，

especially the HAL 9000.

特别是 HAL 9000。

Now, HAL was a sentient computer

HAL是一台有情感的电脑，

designed to guide the Discovery spacecraft

为引导“发现一号”飞船从地球

from the Earth to Jupiter.

前往木星而设计出来。

HAL was also a flawed character,

HAL 也是一个有缺陷的角色，

for in the end he chose to value the mission over human life.

因为在结尾他将任务的 价值置于生命之上。

Now, HAL was a fictional character,

HAL 是一个虚构的角色，

but nonetheless he speaks to our fears,

但尽管如此，他挑战了我们的恐惧，

our fears of being subjugated

被一些冷漠无情的

by some unfeeling, artificial intelligence

人工智能（AI）

who is indifferent to our humanity.

所统治的恐惧。

I believe that such fears are unfounded.

但我相信这些恐惧只是杞人忧天。

Indeed, we stand at a remarkable time

的确，我们正处于人类历史上

一个值得铭记的时间点，

in human history,

where, driven by refusal to accept the limits of our bodies and our minds,

不甘被自身肉体和头脑所局限，

we are building machines

我们正在制造

of exquisite, beautiful complexity and grace

那些可以通过我们无法想象的方式

that will extend the human experience

来拓展人类体验的机器，

in ways beyond our imagining.

它们精美，复杂，而且优雅。

After a career that led me from the Air Force Academy

我在空军学院

to Space Command to now,

和航天司令部工作过，

I became a systems engineer,

现在是一个系统工程师。

and recently I was drawn into an engineering problem

最近我碰到了一个和 NASA火星任务相关的

associated with NASA's mission to Mars.

工程问题。

Now, in space flights to the Moon,

当前，前往月球的宇宙飞行，

we can rely upon mission control in Houston

我们可以依靠休斯顿的指挥中心

to watch over all aspects of a flight.

来密切关注飞行的各个方面。

However, Mars is 200 times further away,

但是，火星相比而言 多出了200倍的距离，

and as a result it takes on average 13 minutes

这使得一个信号从地球到火星

for a signal to travel from the Earth to Mars.

平均要花费13分钟。

If there's trouble, there's not enough time.

如果有了麻烦， 我们并没有足够的时间来解决。

And so a reasonable engineering solution

所以一个可靠的工程解决方案

calls for us to put mission control

促使我们把一个指挥中枢

inside the walls of the Orion spacecraft.

放在“猎户星”飞船之中。

Another fascinating idea in the mission profile

在任务档案中的另一个创意

places humanoid robots on the surface of Mars

是在人类抵达火星之前，把人形机器人

before the humans themselves arrive,

先一步放在火星表面，

first to build facilities

它们可以先建造据点，

and later to serve as collaborative members of the science team.

然后作为科学团队的合作伙伴驻扎。

Now, as I looked at this from an engineering perspective,

当我从工程师的角度看待这个想法，

it became very clear to me that what I needed to architect

对于建造一个聪明，懂得合作，

was a smart, collaborative,

擅长社交的人工智能的需求

socially intelligent artificial intelligence.

是十分明显的。

In other words, I needed to build something very much like a HAL

换句话说，我需要建造一个和HAL一样，

but without the homicidal tendencies.

但是没有谋杀倾向的机器。

(Laughter)

（笑声）

Let's pause for a moment.

待会儿再回到这个话题。

Is it really possible to build an artificial intelligence like that?

真的有可能打造一个 类似的人工智能吗？

Actually, it is.

是的，当然可以。

In many ways,

在许多方面，

this is a hard engineering problem

一个困难的工程问题来自于

with elements of AI,

AI的各个零件，

not some wet hair ball of an AI problem that needs to be engineered.

而不是什么琐碎的AI问题。

To paraphrase Alan Turing,

借用Alan Turing的话来说，

I'm not interested in building a sentient machine.

我没有兴趣建造一台有情感的机器。

I'm not building a HAL.

我不是要建造HAL。

All I'm after is a simple brain,

我只想要一个简单的大脑，

something that offers the illusion of intelligence.

一个能让你以为它有智能的东西。

The art and the science of computing have come a long way

自从HAL登上荧幕， 关于编程的技术和艺术

since HAL was onscreen,

已经发展了许多，

and I'd imagine if his inventor Dr. Chandra were here today,

我能想象如果它的发明者 Chandra博士今天在这里的话，

he'd have a whole lot of questions for us.

他会有很多的问题问我们。

Is it really possible for us

我们真的可能

to take a system of millions upon millions of devices,

用一个连接了无数设备的系统，

to read in their data streams,

通过读取数据流，

to predict their failures and act in advance?

来预测它们的失败并提前行动吗？

是的。

Yes.

Can we build systems that converse with humans in natural language?

我们能建造一个和人类 用语言交流的系统吗？

Yes.

能。

Can we build systems that recognize objects, identify emotions,

我们能够打造一个能 辨识目标，鉴别情绪，

emote themselves, play games and even read lips?

表现自身情感，打游戏， 甚至读唇的系统吗？

Yes.

可以。

Can we build a system that sets goals,

我们可以建造一个能设定目标，

that carries out plans against those goals and learns along the way?

通过各种达成目的的 方法来学习的系统吗？

Yes.

也可以。

Can we build systems that have a theory of mind?

我们可以建造一个类似人脑的系统吗？

This we are learning to do.

这是我们正在努力做的。

Can we build systems that have an ethical and moral foundation?

我们可以建造一个有道德 和感情基础的系统吗？

This we must learn how to do.

这是我们必须要学习的。

So let's accept for a moment

总而言之，

that it's possible to build such an artificial intelligence

建造一个类似的用于这类任务的

for this kind of mission and others.

人工智能是可行的。

The next question you must ask yourself is,

另一个你必须扪心自问的问题是，

should we fear it?

我们应该害怕它吗？

Now, every new technology

毕竟，每一种新技术

brings with it some measure of trepidation.

都给我们带来某种程度的不安。

When we first saw cars,

我们第一次看见汽车的时候，

people lamented that we would see the destruction of the family.

人们悲叹我们会看到家庭的毁灭。

When we first saw telephones come in,

我们第一次看见电话的时候，

people were worried it would destroy all civil conversation.

人们担忧这会破坏所有的文明交流。

在某个时间点我们看到书写文字的蔓延，

At a point in time we saw the written word become pervasive,

people thought we would lose our ability to memorize.

人们认为我们会丧失记忆的能力。

These things are all true to a degree,

这些在某个程度上是合理的，

but it's also the case that these technologies

但也正是这些技术

给人类的生活在某些方面

brought to us things that extended the human experience

in some profound ways.

带来了前所未有的体验。

So let's take this a little further.

我们再稍稍扩展一下。

I do not fear the creation of an AI like this,

我并不畏惧这类人工智能的诞生，

because it will eventually embody some of our values.

因为它最终会融入我们的部分价值观。

Consider this: building a cognitive system is fundamentally different

想想这个：建造一个认知系统

than building a traditional software-intensive system of the past.

与建造一个以传统的软件为主的 系统有着本质的不同。

We don't program them. We teach them.

我们不编译它们。我们教导它们。

In order to teach a system how to recognize flowers,

为了教会系统如何识别花朵，

I show it thousands of flowers of the kinds I like.

我给它看了上千种我喜欢的花。

In order to teach a system how to play a game --

为了教会系统怎么打游戏——

Well, I would. You would, too.

当然，我会。你们也会。

I like flowers. Come on.

我喜欢花。这没什么。

To teach a system how to play a game like Go,

为了教会系统如何玩像围棋一样的游戏，

I'd have it play thousands of games of Go,

我玩了许多围棋的游戏，

but in the process I also teach it

但是在这个过程中

how to discern a good game from a bad game.

我也教会它如何分别差游戏和好游戏。

If I want to create an artificially intelligent legal assistant,

如果我想要一个人工智能法律助手，

I will teach it some corpus of law

我会给它一些法律文集，

but at the same time I am fusing with it

但同时我会将 怜悯和正义

the sense of mercy and justice that is part of that law.

也是法律的一部分 这种观点融入其中。

In scientific terms, this is what we call ground truth,

用一个术语来解释， 就是我们所说的真相，

and here's the important point:

而关键在于：

in producing these machines,

为了制造这些机器，

we are therefore teaching them a sense of our values.

我们正教给它们我们的价值观。

To that end, I trust an artificial intelligence

正因如此，我相信一个人工智能

the same, if not more, as a human who is well-trained.

绝不逊于一个经过良好训练的人类。

But, you may ask,

但是，你或许会问，

what about rogue agents,

要是流氓组织，

some well-funded nongovernment organization?

和资金充沛的无政府组织 也在利用它们呢？

I do not fear an artificial intelligence in the hand of a lone wolf.

我并不害怕独狼掌控的人工智能。

Clearly, we cannot protect ourselves against all random acts of violence,

很明显，我们无法从 随机的暴力行为中保护自己，

but the reality is such a system

但是现实是，制造这样一个系统

requires substantial training and subtle training

超越了个人所拥有资源的极限，

far beyond the resources of an individual.

因为它需要踏实细致的训练和培养。

And furthermore,

还有，

it's far more than just injecting an internet virus to the world,

这远比向世界散播一个网络病毒，

where you push a button, all of a sudden it's in a million places

比如你按下一个按钮， 瞬间全世界都被感染，

and laptops start blowing up all over the place.

并且在各处的笔记本电脑中 开始爆发来的复杂。

Now, these kinds of substances are much larger,

这类东西正在越来越强大，

and we'll certainly see them coming.

我们必然会看到它们的来临。

Do I fear that such an artificial intelligence

我会害怕一个有可能威胁所有人类的

might threaten all of humanity?

人工智能吗？

If you look at movies such as "The Matrix," "Metropolis,"

如果你看过《黑客帝国》，《大都会》，

"The Terminator," shows such as "Westworld,"

《终结者》，或者 《西部世界》这类电视剧，

they all speak of this kind of fear.

它们都在表达这种恐惧。

Indeed, in the book "Superintelligence" by the philosopher Nick Bostrom,

的确，在哲学家Nick Bostrom 写的《超级智能》中，

he picks up on this theme

他选择了这个主题，

and observes that a superintelligence might not only be dangerous,

并且观察到超级智能不仅仅危险，

it could represent an existential threat to all of humanity.

它还对所有人类的存在造成了威胁。

Dr. Bostrom's basic argument

Bostrom博士的基础观点认为，

is that such systems will eventually

这样的系统迟早会

have such an insatiable thirst for information

产生对信息的无止境渴求，

that they will perhaps learn how to learn

也许它们会开始学习，

and eventually discover that they may have goals

并且最终发现它们的目的

that are contrary to human needs.

和人类的需求背道而驰。

Dr. Bostrom has a number of followers.

Bostrom博士有一群粉丝。

He is supported by people such as Elon Musk and Stephen Hawking.

Elon Musk和Stephen Hawking也支持他。

With all due respect

虽然要向这些伟大的头脑

to these brilliant minds,

致以崇高的敬意，

但我还是相信他们从一开始就错了。

I believe that they are fundamentally wrong.

Now, there are a lot of pieces of Dr. Bostrom's argument to unpack,

Bostrom博士的观点 有很多地方可以细细体会，

and I don't have time to unpack them all,

但现在我没有时间一一解读，

but very briefly, consider this:

简要而言，请考虑这句话：

super knowing is very different than super doing.

全知并非全能。

HAL was a threat to the Discovery crew

HAL成为了对发现一号成员的威胁，

only insofar as HAL commanded all aspects of the Discovery.

只是因为它控制了 发现一号的各个方面。

So it would have to be with a superintelligence.

正因如此它才需要是一个人工智能。

它需要对于我们世界的完全掌控。

It would have to have dominion over all of our world.

This is the stuff of Skynet from the movie "The Terminator"

这就是《终结者》中的天网，

in which we had a superintelligence

一个控制了人们意志，

that commanded human will,

控制了世界各处

that directed every device that was in every corner of the world.

各个机器的超级智能。

Practically speaking,

说实在的，

这完全是杞人忧天。

it ain't gonna happen.

We are not building AIs that control the weather,

我们不是在制造可以控制天气，

that direct the tides,

引导潮水，

that command us capricious, chaotic humans.

指挥我们这些 多变，混乱的人类的人工智能。

And furthermore, if such an artificial intelligence existed,

另外，即使这类人工智能存在，

it would have to compete with human economies,

它需要和人类的经济竞争，

and thereby compete for resources with us.

进而和我们拥有的资源竞争。

And in the end --

最后——

don't tell Siri this --

不要告诉Siri——

we can always unplug them.

我们可以随时拔掉电源。

(Laughter)

（笑声）

We are on an incredible journey

我们正处于和机器共同演化的

of coevolution with our machines.

奇妙旅途之中。

The humans we are today

未来的人类

are not the humans we will be then.

将与今天的人类大相径庭。

To worry now about the rise of a superintelligence

当前对人工智能崛起的担忧，

is in many ways a dangerous distraction

从各方面来说都是 一个危险的错误指引，

because the rise of computing itself

因为电脑的崛起

brings to us a number of human and societal issues

带给了我们许多必须参与的

to which we must now attend.

关乎人类和社会的问题。

How shall I best organize society

我应该如何管理社会

来应对人类劳工需求量的降低？

when the need for human labor diminishes?

How can I bring understanding and education throughout the globe

我应该怎样在进行全球化 交流和教育的同时，

and still respect our differences?

依旧尊重彼此的不同？

我应该如何通过可认知医疗 来延长并强化人类的生命？

How might I extend and enhance human life through cognitive healthcare?

How might I use computing

我应该如何通过计算机

to help take us to the stars?

来帮助我们前往其他星球？

And that's the exciting thing.

这些都很令人兴奋。

The opportunities to use computing

通过计算机来升级

to advance the human experience

人类体验的机会

are within our reach,

就在我们手中，

here and now,

就在此时此刻，

and we are just beginning.

我们的旅途才刚刚开始。

Thank you very much.

谢谢大家。

(Applause)

（掌声）