What Is an AI Anyway? | Mustafa Suleyman | TED

翻译人员: Yip Yan Yeung 校对人员: Lening Xu

I want to tell you what I see coming.

我想告诉你们 我预见接下来会发生一些什么。

I've been lucky enough to be working on AI for almost 15 years now.

我有幸能在人工智能领域 工作了将近 15 年。

Back when I started, to describe it as fringe would be an understatement.

回到我刚开始的时候， 称之为“前沿”还是轻描淡写了。

Researchers would say, “No, no, we’re only working on machine learning.”

研究人员说：“不，不， 我们只是在研究机器学习。”

Because working on AI was seen as way too out there.

因为人们觉得 研究 AI 还差得远呢。

In 2010, just the very mention of the phrase “AGI,”

2010 年， 只要提到 “AGI” 这个词语，

artificial general intelligence,

即“人工通用智能”，

would get you some seriously strange looks

就能收获一些相当异样的眼神，

and even a cold shoulder.

甚至是不屑一顾。

"You're actually building AGI?" people would say.

“你真的在做 AGI？” 人们会这么说。

"Isn't that something out of science fiction?"

“那不是科幻小说里的东西吗？”

People thought it was 50 years away or 100 years away,

人们觉得还要 50 年、100 年呢，

if it was even possible at all.

还建立在它真的是可能的前提上。

Talk of AI was, I guess, kind of embarrassing.

我觉得谈论 AI 有点尴尬。

People generally thought we were weird.

人们普遍觉得我们很奇怪。

And I guess in some ways we kind of were.

我觉得在某种程度上确实如此。

It wasn't long, though, before AI started beating humans

但是不久之后， AI 就开始战胜人类，

at a whole range of tasks

胜任人们以前以为 遥不可及的各种任务。

that people previously thought were way out of reach.

Understanding images,

理解图像、

translating languages,

翻译语言、

transcribing speech,

抄录演说、

playing Go and chess

下围棋、下象棋，

and even diagnosing diseases.

甚至诊断疾病。

People started waking up to the fact

人们开始意识到

that AI was going to have an enormous impact,

AI 将产生巨大的影响，

and they were rightly asking technologists like me

于是他们理所当然地 向我这样的技术专家提出了

some pretty tough questions.

一些非常棘手的问题。

Is it true that AI is going to solve the climate crisis?

AI 真的能解决气候危机吗？

Will it make personalized education available to everyone?

它能为所有人提供个性化教育吗？

Does it mean we'll all get universal basic income

是不是意味着 我们都将获得统一的基本收入，

and we won't have to work anymore?

不必再工作了？

Should I be afraid?

我应该害怕吗？

What does it mean for weapons and war?

这对武器和战争意味着什么？

And of course, will China win?

当然还有：中国会赢吗？

Are we in a race?

我们在比赛吗？

Are we headed for a mass misinformation apocalypse?

我们是在走向 大规模错误信息的世界末日吗？

All good questions.

都是好问题。

But it was actually a simpler

但其实有一个更简单、

and much more kind of fundamental question that left me puzzled.

更基础的问题让我感到困惑。

One that actually gets to the very heart of my work every day.

它才是我日常工作的核心。

One morning over breakfast,

有一天早上吃早餐时，

my six-year-old nephew Caspian was playing with Pi,

我六岁的侄子凯斯宾 （Caspian） 正在玩 Pi，

the AI I created at my last company, Inflection.

是我在上一家公司 Inflection 搭建的。

With a mouthful of scrambled eggs,

他塞满了一嘴炒蛋，

he looked at me plain in the face and said,

直视着我的脸说：

"But Mustafa, what is an AI anyway?"

“但是穆斯塔法， AI 到底是什么？”

He's such a sincere and curious and optimistic little guy.

他可真是个真诚、 好奇、乐观的小伙子。

He'd been talking to Pi about how cool it would be if one day in the future,

他一直在和 Pi 说， 要是未来有一天

he could visit dinosaurs at the zoo.

他能去动物园看恐龙， 那会有多酷。

And how he could make infinite amounts of chocolate at home.

他怎么能在家里 制作无限量的巧克力。

And why Pi couldn’t yet play I Spy.

还有为什么 Pi 还不会玩 《I SPY 视觉大发现》。

"Well," I said, "it's a clever piece of software

“好吧，” 我说， “它是一款聪明的软件，

that's read most of the text on the open internet,

能阅读开放互联网上的大多数文本，

and it can talk to you about anything you want."

和你谈论任何你想谈论的内容。”

"Right.

“对。

So like a person then?"

像人那样吗？”

I was stumped.

我一时语塞。

Genuinely left scratching my head.

着实让我陷入了沉思。

All my boring stock answers came rushing through my mind.

我的脑海里闪现了各种 无聊又老套的答案，

"No, but AI is just another general-purpose technology,

“不，AI 只是一项新的通用技术，

like printing or steam."

就像印刷或蒸汽机那样。

It will be a tool that will augment us

它会是一个提升我们、

and make us smarter and more productive.

让我们更聪明、更高效的工具。

And when it gets better over time,

随着时间的推移， 它改进得越来越好，

it'll be like an all-knowing oracle

就会像一个无所不知的预言机一样，

that will help us solve grand scientific challenges."

帮助我们解决重大的科学挑战。”

You know, all of these responses started to feel, I guess,

我感觉这些回答 有点太冠冕堂皇了。

a little bit defensive.

And actually better suited to a policy seminar

更适用于一场政策研讨会，

than breakfast with a no-nonsense six-year-old.

而不是与天真无邪的 六岁孩童的早餐桌上。

"Why am I hesitating?" I thought to myself.

“我为什么要犹豫？”我心想。

You know, let's be honest.

说实话。

My nephew was asking me a simple question

我的侄子问了我一个简单的问题，

that those of us in AI just don't confront often enough.

简单到我们这些从事 AI 行业的人 都不太常遇到这样的问题。

What is it that we are actually creating?

我们到底在创造什么？

What does it mean to make something totally new,

创造一个史无前例、

fundamentally different to any invention that we have known before?

与我们从前的认知 截然不同的东西，意味着什么？

It is clear that we are at an inflection point

很显然我们正处于人类历史的转折点。

in the history of humanity.

On our current trajectory,

按照我们目前的发展轨迹，

we're headed towards the emergence of something

我们的眼前会出现

that we are all struggling to describe,

一个我们都难以描述的东西，

and yet we cannot control what we don't understand.

然而，我们无法控制 我们不能理解的东西。

And so the metaphors,

因此，比喻、

the mental models,

心理模型、

the names, these all matter

名称，这些都很重要，

if we’re to get the most out of AI whilst limiting its potential downsides.

如果我们要限制其潜在的负面影响， 发挥它最大的价值的话。

As someone who embraces the possibilities of this technology,

作为一个接受这项技术可能性，

but who's also always cared deeply about its ethics,

但也一直非常关心其伦理的人，

we should, I think,

我认为我们

be able to easily describe what it is we are building.

理应轻而易举地 描述出我们在创造的是什么。

And that includes the six-year-olds.

也包括六岁的孩子。

So it's in that spirit that I offer up today the following metaphor

秉承这一精神， 我今天要提出以下比喻，

for helping us to try to grapple with what this moment really is.

帮助我们努力理解 眼下到底是一个怎样的时刻。

I think AI should best be understood

我认为最好将 AI

as something like a new digital species.

当作一种新的数字物种。

Now, don't take this too literally,

不要太注重书面含义，

but I predict that we'll come to see them as digital companions,

但我预计我们最终会 将它们视作我们一生中的

new partners in the journeys of all our lives.

数字伴侣和新搭档。

Whether you think we’re on a 10-, 20- or 30-year path here,

不管你认为我们走在这条 10 年、 20 年或 30 年长的道路上，

this is, in my view, the most accurate and most fundamentally honest way

在我看来，这就是最准确、 最诚实地描述

of describing what's actually coming.

未来所见的方式。

And above all, it enables everybody to prepare for

最重要的是，这让所有人 都能为接下来要发生的事

and shape what comes next.

做好准备，动手塑造。

Now I totally get, this is a strong claim,

我完全理解 这是一个强势的观点，

and I'm going to explain to everyone as best I can why I'm making it.

我将尽我所能向各位解释 为什么我会这么说。

But first, let me just try to set the context.

但首先，让我描述一下背景。

From the very first microscopic organisms,

从最初的微观生物开始，

life on Earth stretches back billions of years.

地球上的生命 可以追溯到数十亿年前。

Over that time, life evolved and diversified.

在那段时间里， 生命不断演变、多样化。

Then a few million years ago, something began to shift.

然后在几百万年前， 情况开始发生变化。

After countless cycles of growth and adaptation,

在经历了无数次的成长 和适应周期之后，

one of life’s branches began using tools, and that branch grew into us.

生命中的一个分支开始使用工具， 这个分支发展成了我们。

We went on to produce a mesmerizing variety of tools,

我们接着创造了琳琅满目的工具，

at first slowly and then with astonishing speed,

起初是缓慢的，后来势如破竹，

we went from stone axes and fire

我们从石斧和火转向了

to language, writing and eventually industrial technologies.

语言、写作， 最终出现了工业技术。

One invention unleashed a thousand more.

一项发明引出了上千项发明。

And in time, we became homo technologicus.

随着时间的推移， 我们变成了单一技术的狂热粉丝。

Around 80 years ago,

大约 80 年前，

another new branch of technology began.

一个新的技术分支出现了。

With the invention of computers,

随着计算机的发明，

we quickly jumped from the first mainframes and transistors

我们迅速从最初的大型机和晶体管

to today's smartphones and virtual-reality headsets.

飞跃到了如今的智能手机 和虚拟现实头显。

Information, knowledge, communication, computation.

信息、知识、通信、计算。

In this revolution,

在这场革命中，

creation has exploded like never before.

创造以前所未有的方式涌现。

And now a new wave is upon us.

眼下新的浪潮正在袭来。

Artificial intelligence.

人工智能。

These waves of history are clearly speeding up,

这些历史浪潮显然正在加速，

as each one is amplified and accelerated by the last.

因为每一波浪潮都会 受到上一波的增强和加速。

And if you look back,

回头看，

it's clear that we are in the fastest

很显然，我们正处于有史以来最快、

and most consequential wave ever.

影响最大的浪潮中。

The journeys of humanity and technology are now deeply intertwined.

人类与科技的旅程 深深地交织在一起。

In just 18 months,

在短短 18 个月内，

over a billion people have used large language models.

超过 10 亿人使用了大语言模型。

We've witnessed one landmark event after another.

我们目睹了一个又一个 具有里程碑意义的事件。

Just a few years ago, people said that AI would never be creative.

就在几年前， 人们说 AI 永远不会有创造力。

And yet AI now feels like an endless river of creativity,

然而，现在的 AI 感觉可以无限地思如泉涌，

making poetry and images and music and video that stretch the imagination.

创作诗歌、图像、音乐和视频， 以扩展想象力。

People said it would never be empathetic.

人们说 AI 永远不会有同理心。

And yet today, millions of people enjoy meaningful conversations with AIs,

然而今天，数百万人享受着 与 AI 的深刻对话，

talking about their hopes and dreams

谈论他们的希望和梦想，

and helping them work through difficult emotional challenges.

帮助他们应对艰难的情感挑战。

AIs can now drive cars,

AI 现在可以驾驶汽车、

manage energy grids

管理能源网，

and even invent new molecules.

甚至发明新分子。

Just a few years ago, each of these was impossible.

就在几年前，这一切都是不可能的。

And all of this is turbocharged by spiraling exponentials of data

这一切都是由螺旋式增长的数据 和计算所推动的。

and computation.

Last year, Inflection 2.5, our last model,

去年，我们最新的模型 Inflection 2.5

used five billion times more computation

使用的计算量

than the DeepMind AI that beat the old-school Atari games

比 10 多年前通关 雅达利游戏的 DeepMind AI

just over 10 years ago.

多 50 亿倍。

That's nine orders of magnitude more computation.

计算量要多出九个数量级。

10x per year,

每年翻 10 倍，

every year for almost a decade.

近十年来每年翻一次。

Over the same time, the size of these models has grown

同时，这些模型的规模

from first tens of millions of parameters to then billions of parameters,

从最初的数千万个参数增长 到了后来的数十亿个参数，

and very soon, tens of trillions of parameters.

很快又增长到了数万亿个参数。

If someone did nothing but read 24 hours a day for their entire life,

如果有人一辈子 每天 24 小时都在读书，

they'd consume eight billion words.

他/她将读到 80 亿个单词。

And of course, that's a lot of words.

当然，这是大量的单词。

But today, the most advanced AIs consume more than eight trillion words

但如今，最先进的 AI 能在一个月的训练中

in a single month of training.

阅读超过八万亿个单词。

And all of this is set to continue.

这是大势所趋。

The long arc of technological history is now in an extraordinary new phase.

科技的历史长河 现在到了一个非同寻常的新阶段。

So what does this mean in practice?

在实践中意味着什么呢？

Well, just as the internet gave us the browser

正如互联网给了我们浏览器，

and the smartphone gave us apps,

智能手机给了我们应用程序一样，

the cloud-based supercomputer is ushering in a new era

基于云的超级计算机正在开创一个

of ubiquitous AIs.

AI 无处不在的新时代。

Everything will soon be represented by a conversational interface.

很快，一切都会以对话界面的形式呈现。

Or, to put it another way, a personal AI.

或者换句话说，个人 AI。

And these AIs will be infinitely knowledgeable,

而且这些 AI 将拥有无限的知识，

and soon they'll be factually accurate and reliable.

很快它们就会掌握准确和可靠的事实。

They'll have near-perfect IQ.

它们的智商将接近完美。

They’ll also have exceptional EQ.

它们还将拥有出色的情商。

They’ll be kind, supportive, empathetic.

它们友善、助人、具备同理心。

These elements on their own would be transformational.

这些要素本身将是变革性的。

Just imagine if everybody had a personalized tutor in their pocket

试想一下，如果每个人的口袋里 都有一个个性化的导师，

and access to low-cost medical advice.

可以获得低成本的医疗建议。

A lawyer and a doctor,

律师、医生、

a business strategist and coach --

商业策略师和教练，

all in your pocket 24 hours a day.

每天 24 小时都在你的口袋里。

But things really start to change when they develop what I call AQ,

但当它们发展出我口中的 “AQ”， 即“行动商”（actions quotient）时，

their “actions quotient.”

情况才真正开始发生变化。

This is their ability to actually get stuff done

就是它们在数字和现实世界中

in the digital and physical world.

真正完成任务的能力。

And before long, it won't just be people that have AIs.

不久之后， 拥有 AI 的将不仅仅是人类。

Strange as it may sound, every organization,

尽管听起来很奇怪， 但每个组织，

from small business to nonprofit to national government,

从小型企业 到非营利组织再到国家政府，

each will have their own.

都会有自己的 AI。

Every town, building and object

每个城镇、建筑物和物体

will be represented by a unique interactive persona.

都将由独特的互动角色呈现。

And these won't just be mechanistic assistants.

不仅仅是机械助手。

They'll be companions, confidants,

它们将成为同伴、知己、

colleagues, friends and partners,

同事、朋友和伙伴，

as varied and unique as we all are.

就像我们所有人一样 多种多样、独一无二。

At this point, AIs will convincingly imitate humans at most tasks.

到那时，AI 将在大多数任务中 可靠地模仿人类。

And we'll feel this at the most intimate of scales.

我们将以最亲密的程度 感觉到这一点。

An AI organizing a community get-together for an elderly neighbor.

AI 为年长邻居组织一场邻里聚会。

A sympathetic expert helping you make sense of a difficult diagnosis.

一位富有同情心的专家 帮助您理解复杂的诊断。

But we'll also feel it at the largest scales.

但我们也会以最大的规模感受到它。

Accelerating scientific discovery,

加速科学发现，

autonomous cars on the roads,

自动驾驶汽车上路，

drones in the skies.

无人机飞向空中。

They'll both order the takeout and run the power station.

它们同时具备点外卖 和运营发电厂的能力。

They’ll interact with us and, of course, with each other.

它们将与我们互动， 当然也会互相互动。

They'll speak every language,

它们会说每种语言，

take in every pattern of sensor data,

采集传感器数据、

sights, sounds,

视觉、声音、

streams and streams of information,

源源不断的信息流中的每一种模式，

far surpassing what any one of us could consume in a thousand lifetimes.

远远超过我们任何人 在一千次人生中所能吸收的水平。

So what is this?

那它是什么呢？

What are these AIs?

这些 AI 是什么？

If we are to prioritize safety above all else,

如果我们将安全置于首位，

to ensure that this new wave always serves and amplifies humanity,

确保这一新浪潮始终 为人类服务且增强人类，

then we need to find the right metaphors for what this might become.

我们需要找到正确的比喻来描述 它可能会变成什么样子。

For years, we in the AI community, and I specifically,

多年来，我们在 AI 社区中， 尤其是我，

have had a tendency to refer to this as just tools.

一直倾向于将其称为“工具”。

But that doesn't really capture what's actually happening here.

但这并不能真正反映 实际发生的情况。

AIs are clearly more dynamic,

显然，AI 比单纯的工具 更变化多端、

more ambiguous, more integrated

更神秘莫测、更完整、

and more emergent than mere tools,

更新兴，

which are entirely subject to human control.

因为工具完全受到人类的控制。

So to contain this wave,

为了应对这一浪潮，

to put human agency at its center

以人类的能动性为中心，

and to mitigate the inevitable unintended consequences

减轻不可避免的意外结果，

that are likely to arise,

很有可能会出现这样的结果，

we should start to think about them as we might a new kind of digital species.

我们应该开始将它们 视作一种新的数字物种。

Now it's just an analogy,

这只是一个类比，

it's not a literal description, and it's not perfect.

不是字面描述，也不是完美的。

For a start, they clearly aren't biological in any traditional sense,

首先，从任何传统意义上讲， 它们显然不是生物，

but just pause for a moment

但只要暂停一下，

and really think about what they already do.

认真思考它们已经做了些什么。

They communicate in our languages.

它们用我们的语言交流。

They see what we see.

它们看到了我们所看到的。

They consume unimaginably large amounts of information.

它们摄取了难以想象的大量信息。

They have memory.

它们有记忆。

They have personality.

它们有性格。

They have creativity.

它们有创造力。

They can even reason to some extent and formulate rudimentary plans.

它们甚至可以在某种程度上 推理并制定基本的计划。

They can act autonomously if we allow them.

如果我们允许，它们可以自主行动。

And they do all this at levels of sophistication

它们完成这些任务的高超水平

that is far beyond anything that we've ever known from a mere tool.

远超我们见过 单纯的工具所能展现的水平。

And so saying AI is mainly about the math or the code

因此，要是说 AI 大致就是数学或代码，

is like saying we humans are mainly about carbon and water.

就像是说 我们人类大致就是碳和水一样。

It's true, but it completely misses the point.

说得没错，但只是以偏概全。

And yes, I get it, this is a super arresting thought

我知道，这是一个 非常引人注目的想法，

but I honestly think this frame helps sharpen our focus on the critical issues.

但老实说，我认为这个框架 有助于我们更加聚焦于关键问题。

What are the risks?

有哪些风险？

What are the boundaries that we need to impose?

我们需要设定的边界是什么？

What kind of AI do we want to build or allow to be built?

我们想要创造 或允许创造什么样的 AI？

This is a story that's still unfolding.

故事还没写完。

Nothing should be accepted as a given.

任何事情都不应被视为理所当然。

We all must choose what we create.

我们都必须选择我们创造的东西。

What AIs we bring into the world, or not.

我们要给世界带来什么样的 AI， 或者没有带来什么样的 AI。

These are the questions for all of us here today,

这些是我们今天在座的所有人

and all of us alive at this moment.

以及此刻活着的所有人 面临的问题。

For me, the benefits of this technology are stunningly obvious,

对我来说， 这项技术的好处是显而易见的,

and they inspire my life's work every single day.

它们每一天都在激励着我， 成为奋斗一生的工作。

But quite frankly, they'll speak for themselves.

但坦率地说， 它们会为自己代言。

Over the years, I've never shied away from highlighting risks

多年来，我从不避讳强调风险、

and talking about downsides.

谈论缺点。

Thinking in this way helps us focus on the huge challenges

以这种方式思考有助于我们 关注所有人面临的巨大挑战。

that lie ahead for all of us.

But let's be clear.

但是，我们来说清楚。

There is no path to progress

如果我们抛下技术， 就没有向前进步的道路。

where we leave technology behind.

The prize for all of civilization is immense.

整个文明的回报是巨大的。

We need solutions in health care and education, to our climate crisis.

我们需要医疗保健和教育方面的 解决方案应对气候危机。

And if AI delivers just a fraction of its potential,

AI 只需发挥一小部分潜力，

the next decade is going to be the most productive in human history.

未来的十年就会是人类历史上 生产力最高的十年。

Here's another way to think about it.

我们可以换种思路思考。

In the past,

过去，

unlocking economic growth often came with huge downsides.

激发经济增长 往往会带来严重的不利影响。

The economy expanded as people discovered new continents

随着人们发现新大陆、 开拓新前沿，经济不断增长。

and opened up new frontiers.

But they colonized populations at the same time.

但是他们同时殖民了人民。

We built factories,

我们建造了工厂，

but they were grim and dangerous places to work.

但它们是阴森又危险的工作场所。

We struck oil,

我们开采了石油，

but we polluted the planet.

但我们污染了地球。

Now because we are still designing and building AI,

由于我们仍在设计和打造 AI，

we have the potential and opportunity to do it better,

我们有潜力和机会把它做得更好、

radically better.

好得多。

And today, we're not discovering a new continent

眼下，我们并没有 发现一个新的大陆

and plundering its resources.

并掠夺其资源。

We're building one from scratch.

我们正在从头开始建造一个。

Sometimes people say that data or chips are the 21st century’s new oil,

有时人们说数据或芯片 是 21 世纪的新型石油，

but that's totally the wrong image.

但这是完全错误的形象。

AI is to the mind

AI 之于思想

what nuclear fusion is to energy.

就如同核聚变之于能源。

Limitless, abundant,

无限、丰富、

world-changing.

改变世界。

And AI really is different,

而且 AI 真的不一样，

and that means we have to think about it creatively and honestly.

意味着我们必须大开脑洞地、 诚实地看待它。

We have to push our analogies and our metaphors

我们必须将我们的类比和隐喻

to the very limits

发挥到极致，

to be able to grapple with what's coming.

才能应对即将发生的事情。

Because this is not just another invention.

因为这不只是一项新发明。

AI is itself an infinite inventor.

AI 本身就是一个 没有上限的发明家。

And yes, this is exciting and promising and concerning

没错，这令人兴奋 又充满希望，既令人担忧

and intriguing all at once.

又让人着迷。

To be quite honest, it's pretty surreal.

说实话，太不可思议了。

But step back,

但是往前回溯，

see it on the long view of glacial time,

从冰川时代的长远角度来看，

and these really are the very most appropriate metaphors that we have today.

这些确实是我们现在 能找到最恰当的比喻。

Since the beginning of life on Earth,

自地球生命开始以来，

we've been evolving, changing

我们一直在进化、改变，

and then creating everything around us in our human world today.

然后在当今人类世界中 创造我们周围的一切。

And AI isn't something outside of this story.

AI 并不是这个故事的局外人。

In fact, it's the very opposite.

实际情况恰恰相反。

It's the whole of everything that we have created,

它是我们创造的一切

distilled down into something that we can all interact with

凝聚成我们所有人 都可以与之互动

and benefit from.

并从中受益的东西。

It's a reflection of humanity across time,

它是人类跨越时空的写照，

and in this sense,

从这个层面来说，

it isn't a new species at all.

它根本不是一个新物种。

This is where the metaphors end.

比喻到此结束。

Here's what I'll tell Caspian next time he asks.

凯斯宾下次问， 我就会这么告诉他。

AI isn't separate.

AI 不是独立的。

AI isn't even in some senses, new.

从某种意义上说， AI 甚至不是新的。

AI is us.

AI 就是我们。

It's all of us.

是我们所有人。

And this is perhaps the most promising and vital thing of all

这也许是连六岁孩子都能理解的

that even a six-year-old can get a sense for.

最有希望、最关键的一点。

As we build out AI,

在我们创造 AI 的过程中，

we can and must reflect all that is good,

我们可以，也必须 反映出所有美好的、

all that we love,

所有我们热爱的、

all that is special about humanity:

所有人类的特别之处：

our empathy, our kindness,

我们的同理心、我们的善良、

our curiosity and our creativity.

我们的好奇心和我们的创造力。

This, I would argue, is the greatest challenge of the 21st century,

我认为，这是 21 世纪最大的挑战，

but also the most wonderful,

也是我们所有人最美妙、

inspiring and hopeful opportunity for all of us.

最振奋人心、最有希望的机会。

Thank you.

谢谢。

(Applause)

（掌声）

Chris Anderson: Thank you Mustafa.

克里斯·安德森（Chris Anderson）： 谢谢穆斯塔法。

It's an amazing vision and a super powerful metaphor.

这是一个美好的愿景， 也是一个非常有力的比喻。

You're in an amazing position right now.

你现在身处一个了不起的位置。

I mean, you were connected at the hip

你与 OpenAI 正在进行的 令人叹为观止的工作紧密相关。

to the amazing work happening at OpenAI.

You’re going to have resources made available,

你能接触到资源，

there are reports of these giant new data centers,

有报道这些新的大型数据中心，

100 billion dollars invested and so forth.

投资了 1000 亿美元等等。

And a new species can emerge from it.

从中会产生一个新物种。

I mean, in your book,

你在书里

you did, as well as painting an incredible optimistic vision,

除了描绘相当乐观的愿景外，

you were super eloquent on the dangers of AI.

还在 AI 的危险上耗费了大量笔墨。

And I'm just curious, from the view that you have now,

我很好奇， 从你现在的角度来看，

what is it that most keeps you up at night?

最让你彻夜难眠的是什么？

Mustafa Suleyman: I think the great risk is that we get stuck

穆斯塔法·苏莱曼（Mustafa Suleyman）： 我认为最大的风险

in what I call the pessimism aversion trap.

是我们陷入了我所谓的 “悲观情绪厌恶陷阱”。

You know, we have to have the courage to confront

我们必须有勇气面对 黑暗情形的出现，

the potential of dark scenarios

in order to get the most out of all the benefits that we see.

才能最大限度地利用 我们所看到的各种益处。

So the good news is that if you look at the last two or three years,

好消息是， 如果你回顾过去的两三年，

there have been very, very few downsides, right?

负面结果非常非常少，对吧？

It’s very hard to say explicitly what harm an LLM has caused.

很难言之凿凿 LLM 造成了什么伤害。

But that doesn’t mean that that’s what the trajectory is going to be

但这并不意味着这就是

未来 10 年的发展轨迹。

over the next 10 years.

So I think if you pay attention to a few specific capabilities,

所以我认为， 如果你留意一些特定的能力，

take for example, autonomy.

例如自主性。

Autonomy is very obviously a threshold

自主性显然是一个 我们增加社会风险的门槛。

over which we increase risk in our society.

And it's something that we should step towards very, very closely.

这是我们要尽可能靠近的一个目标。

The other would be something like recursive self-improvement.

另一个能力是递归自我改进。

If you allow the model to independently self-improve,

如果你让模型独立自我改进，

update its own code,

更新自己的代码，

explore an environment without oversight, and, you know,

在不受监督的情况下探索环境，

without a human in control to change how it operates,

在没有人类控制的情况下 改变其运行方式，

that would obviously be more dangerous.

显然会更加危险。

But I think that we're still some way away from that.

但我认为， 我们距此还有一定距离。

I think it's still a good five to 10 years before we have to really confront that.

我认为我们离真正面对这个问题 还有 5 到 10 年的时间。

But it's time to start talking about it now.

但现在是时候开始讨论这个问题了。

CA：与任何生物物种不同， 数字物种不是在九个月内复制，

CA: A digital species, unlike any biological species,

can replicate not in nine months,

but in nine nanoseconds,

而是在九纳秒内复制，

and produce an indefinite number of copies of itself,

并产生无限个自身复制体，

all of which have more power than we have in many ways.

所有复制体在许多方面 都比我们拥有更强大的力量。

I mean, the possibility for unintended consequences seems pretty immense.

我想说，出现意想不到的 后果的可能性似乎非常大。

And isn't it true that if a problem happens,

如果确实如此， 如果问题要出现了，

it could happen in an hour?

是不是在一个小时内就会出现呢？

MS: No.

MS：不是。

That is really not true.

这确实不是真的。

I think there's no evidence to suggest that.

我认为没有证据表明这一点。

And I think that, you know,

而且我认为，

that’s often referred to as the “intelligence explosion.”

这通常被称为“智能爆炸”。

And I think it is a theoretical, hypothetical maybe

我认为它也许是 理论上的、假想的，

that we're all kind of curious to explore,

我们都好奇地去探索，

but there's no evidence that we're anywhere near anything like that.

但是没有证据表明它确实要发生了。

And I think it's very important that we choose our words super carefully.

而且我认为 非常谨慎地斟酌用词非常重要。

Because you're right, that's one of the weaknesses of the species framing,

因为你说得对， 这是物种定义的缺陷之一，

that we will design the capability for self-replication into it

如果人们想这么做的话，

我们就能把自我复制的能力加进去。

if people choose to do that.

And I would actually argue that we should not,

其实我会说我们不该这么做，

that would be one of the dangerous capabilities

这应该是我们要远离的 最危险的一种能力，对吧？

that we should step back from, right?

因此，它不可能意外“出现”。

So there's no chance that this will "emerge" accidentally.

I really think that's a very low probability.

我真的认为这种可能性很低。

It will happen if engineers deliberately design those capabilities in.

如果工程师故意设计出这些功能， 才会发生这种情况。

And if they don't take enough efforts to deliberately design them out.

而且是在他们没有尽全力 想方设法把它排除出去的情况下。

所以这正是 在尽早考虑“安全”的过程中

And so this is the point of being explicit

and transparent about trying to introduce safety by design very early on.

保证公开透明的意义。

CA: Thank you, your vision of humanity injecting into this new thing

CA：谢谢，你的愿景 是人类将自己最美好的部分

the best parts of ourselves,

注入这个新事物，

avoiding all those weird, biological, freaky,

避免我们在某些情况下可能出现的所有 奇怪的、生物的、怪异的、可怕的情形，

horrible tendencies that we can have in certain circumstances,

I mean, that is a very inspiring vision.

这是一个非常鼓舞人心的愿景。

And thank you so much for coming here and sharing it at TED.

非常感谢你来到这里， 在 TED 作分享。

Thank you, good luck.

谢谢，祝你好运。

(Applause)

（掌声）