How to Keep AI Under Control | Max Tegmark | TED

翻译人员: Yip Yan Yeung 校对人员: Yanyan Hong

Five years ago,

五年前，

I stood on the TED stage

我站在 TED 舞台上，

and warned about the dangers of superintelligence.

预警了超级智能的危险。

I was wrong.

我错了。

It went even worse than I thought.

情况比我想象的还要糟糕。

(Laughter)

（笑声）

I never thought governments would let AI companies get this far

我从没想过政府会 在没有任何有效的监管下

without any meaningful regulation.

让人工智能（AI）公司发展成这样，

And the progress of AI went even faster than I predicted.

而且 AI 的进步比我预期的还要快。

Look, I showed this abstract landscape of tasks

这是一幅任务的抽象图，

where the elevation represented how hard it was for AI

其中海拔代表了 AI 要 以人类水平完成每项任务的难度。

to do each task at human level.

And the sea level represented what AI could be back then.

而海平面代表了 AI 曾经的水平。

And boy or boy, has the sea been rising fast ever since.

从那以后，海平面一直在快速提高。

But a lot of these tasks have already gone blub blub blub blub blub blub.

有很多任务已经被水淹没。

And the water is on track to submerge all land,

海水势必淹没所有的土地，

matching human intelligence at all cognitive tasks.

在认知任务中与人类智慧媲美。

This is a definition of artificial general intelligence, AGI,

这就是通用人工智能 （AGI）的定义，

which is the stated goal of companies like OpenAI,

也是 OpenAI、谷歌 DeepMind 和 Anthropic 等公司的既定目标。

Google DeepMind and Anthropic.

And these companies are also trying to build superintelligence,

这些公司也在努力打造超级智能，

leaving human intelligence far behind.

远超人类智能。

And many think it'll only be a few years, maybe, from AGI to superintelligence.

许多人认为，从 AGI 到超级智能， 可能只需要几年的时间。

So when are we going to get AGI?

我们什么时候能做出 AGI 呢？

Well, until recently, most AI researchers thought it was at least decades away.

直到最近，大多数人工智能研究人员 还认为至少还有几十年的时间。

And now Microsoft is saying, "Oh, it's almost here."

微软说：“哦，差不多了。”

We're seeing sparks of AGI in ChatGPT-4,

我们在 ChatGPT-4 中 看到了 AGI 的苗头，

and the Metaculus betting site is showing the time left to AGI

而 Metaculus 博彩网站显示， 在过去的 18 个月中，

plummeting from 20 years away to three years away

距离 AGI 时间 从 20 年后骤降至 3 年后。

in the last 18 months.

And leading industry people are now predicting

行业领袖现在预测

that we have maybe two or three years left until we get outsmarted.

我们距离被人工智能超越 也许还有两三年的时间。

So you better stop talking about AGI as a long-term risk,

所以你还是不要再说 AGI 是一种长期风险了，

or someone might call you a dinosaur stuck in the past.

不然有人会认为 你是头被困在过去的恐龙。

It's really remarkable how AI has progressed recently.

AI 最近的进步真是太了不起了。

Not long ago, robots moved like this.

不久前，机器人是这样移动的。

(Music)

（音乐）

Now they can dance.

现在它们可以跳舞了。

(Music)

（音乐）

Just last year, Midjourney produced this image.

就在去年，Midjourney 生成了这张照片。

This year, the exact same prompt produces this.

今年，相同的指示生成了这样的结果。

Deepfakes are getting really convincing.

深度伪造以假乱真。

(Video) Deepfake Tom Cruise: I’m going to show you some magic.

（视频）深度伪造的汤姆·克鲁斯： 我要变个魔术。

It's the real thing.

这是真的。

(Laughs)

（笑）

I mean ...

我的意思是……

It's all ...

全部……

the real ...

都是……

thing.

真的。

Max Tegmark: Or is it?

迈克斯·泰格马克 （Max Tegmark）：是吗？

And Yoshua Bengio now argues

约书亚·本吉奥（Yoshua Bengio）

现在认为，大语言模型 已经掌握了语言

that large language models have mastered language

and knowledge to the point that they pass the Turing test.

和知识，水平高到 它们可以通过图灵测试。

I know some skeptics are saying,

我知道一些质疑者会说：

"Nah, they're just overhyped stochastic parrots

“不，它们只是过度炒作的随机鹦鹉，

that lack a model of the world,"

没有这个世界的模型，”

but they clearly have a representation of the world.

但它们显然有着世界的表征。

In fact, we recently found that Llama-2 even has a literal map of the world in it.

我们最近发现 Llama-2 甚至有一张真实的世界地图。

And AI also builds

AI 还构建了

geometric representations of more abstract concepts

更抽象概念的几何表现形式，

like what it thinks is true and false.

比如它对是非的认知。

So what's going to happen if we get AGI and superintelligence?

如果我们有了 AGI 和超级智能，会发生什么？

If you only remember one thing from my talk, let it be this.

如果你只会记得演讲中的一点， 那就记住这一点吧。

AI godfather, Alan Turing predicted

人工智能教父 艾伦·图灵（Alan Turing）预测

that the default outcome is the machines take control.

默认的结果就是计算机掌控一切。

The machines take control.

计算机掌控一切。

I know this sounds like science fiction,

我知道这听起来像科幻小说，

but, you know, having AI as smart as GPT-4

但是，像 GPT-4 这样聪明的 人工智能在不久前

also sounded like science fiction not long ago.

听起来也像科幻小说。

And if you think of AI,

如果你将 AI，

if you think of superintelligence in particular, as just another technology,

尤其是超级智能， 视为一项司空见惯的技术，

like electricity,

比如电力，

you're probably not very worried.

你可能并不会十分担心。

But you see,

但是你看，

Turing thinks of superintelligence more like a new species.

图灵认为超级智能 更像是一个新物种。

Think of it,

想一想，

we are building creepy, super capable,

我们正在培养可怕的、能力逆天的、

amoral psychopaths

不道德的心理变态，

that don't sleep and think much faster than us,

它们不需要睡觉， 脑子转得比我们快得多，

can make copies of themselves

可以复制，

and have nothing human about them at all.

没有丝毫人性。

So what could possibly go wrong?

可能会出什么问题呢？

(Laughter)

（笑声）

And it's not just Turing.

不只是图灵。

OpenAI CEO Sam Altman, who gave us ChatGPT,

为我们带来 ChatGPT 的 OpenAI CEO 山姆·阿尔特曼（Sam Altman）

recently warned that it could be "lights out for all of us."

最近警告说，这可能会是 “全人类的黑暗”。

Anthropic CEO, Dario Amodei, even put a number on this risk:

Anthropic CEO 达里奥·阿莫迪（Dario Amodei）

甚至给这种风险定了一个数字：

10-25 percent.

10-25%。

And it's not just them.

不只是他们。

Human extinction from AI went mainstream in May

AI 造成的人类灭绝 在五月份甚嚣尘上，

when all the AGI CEOs and who's who of AI researchers

当时，所有 AGI 首席执行官 和 AI 领域有头有脸的研究人员

came on and warned about it.

站出来，发出了警告。

And last month, even the number one of the European Union

上个月，连欧盟委员会主席

warned about human extinction by AI.

都警告了人工智能会导致人类灭绝。

So let me summarize everything I've said so far

让我用一页猫咪表情包 总结一下我刚说的内容。

in just one slide of cat memes.

Three years ago,

三年前，

people were saying it's inevitable, superintelligence,

人们说这是不可避免的，超级智能，

it'll be fine,

没事儿。

it's decades away.

还有几十年的时间。

Last year it was more like,

去年我们这么说，

It's inevitable, it'll be fine.

这是不可避免的，没事儿。

Now it's more like,

现在我们说，

It's inevitable.

这是不可避免的。

(Laughter)

（笑声）

But let's take a deep breath and try to raise our spirits

来深呼吸，打起精神，

and cheer ourselves up,

振作起来，

because the rest of my talk is going to be about the good news,

因为我接下来要讲的都是好消息，

that it's not inevitable, and we can absolutely do better,

这不是不可避免的， 我们绝对还有努力的空间，

alright?

好吗？

(Applause)

（掌声）

So ...

所以……

The real problem is that we lack a convincing plan for AI safety.

真正的问题是我们没有 针对 AI 安全的明确计划。

People are working hard on evals

人们正在努力进行评估，

looking for risky AI behavior, and that's good,

寻找危险的 AI 行为，这很好，

but clearly not good enough.

但显然还不够好。

They're basically training AI to not say bad things

他们都是在训练人工智能 不要“说”不好的内容，

rather than not do bad things.

而不是不去“做”坏事。

Moreover, evals and debugging are really just necessary,

此外，评估和调试其实 只是 AI 安全的必要

not sufficient, conditions for safety.

而不是充分条件。

In other words,

换句话说，

they can prove the presence of risk,

它们可以证明风险的存在，

not the absence of risk.

而非无风险。

So let's up our game, alright?

我们来玩一玩吧，好吗？

Try to see how we can make provably safe AI that we can control.

看看我们如何才能制造出 可控的、“可证明安全”的 AI。

Guardrails try to physically limit harm.

护栏可以试着从物理上控制伤害。

But if your adversary is superintelligence

但是，如果你的对手是超级智能，

or a human using superintelligence against you, right,

或者是使用超级智能 对付你的人类，

trying is just not enough.

“试着获胜”是不够的。

You need to succeed.

你必须得成功。

Harm needs to be impossible.

伤害必须是不存在的。

So we need provably safe systems.

我们需要可证明安全的系统。

Provable, not in the weak sense of convincing some judge,

“可证明”，不是局限于 说服法官的单薄含义，

but in the strong sense of there being something that's impossible

而是彻彻底底说明根据物理定律， 有些事情是不可能的。

according to the laws of physics.

Because no matter how smart an AI is,

因为无论 AI 有多么聪明，

it can't violate the laws of physics and do what's provably impossible.

它都无法违反物理定律， 做“可证明”不可能的事情。

Steve Omohundro and I wrote a paper about this,

我和史蒂夫·奥莫洪德罗 （Steve Omohundro）

关于这点写了一篇论文，

and we're optimistic that this vision can really work.

我们乐观地认为 这个愿景能够真正奏效。

So let me tell you a little bit about how.

我简单说一说要如何做到。

There's a venerable field called formal verification,

有一个神圣的领域， 叫做“形式验证”，

which proves stuff about code.

可以证明有关代码的东西。

And I'm optimistic that AI will revolutionize automatic proving business

我乐观地认为，AI 将彻底改变自动证明任务，

and also revolutionize program synthesis,

还将彻底改变程序合成，

the ability to automatically write really good code.

即自动编写非常好的代码的能力。

So here is how our vision works.

因此，我们的愿景是这样的。

You, the human, write a specification

作为人类，你要写一份

that your AI tool must obey,

你的 AI 工具必须遵守的规范，

that it's impossible to log in to your laptop

比如，如果没有正确的密码， 它就不可能登录你的电脑，

without the correct password,

or that a DNA printer cannot synthesize dangerous viruses.

或者 DNA 打印机 无法合成危险病毒。

Then a very powerful AI creates both your AI tool

然后，非常强大的 AI 既要创建你的 AI 工具，

and a proof that your tool meets your spec.

又要创建可以证明你的工具 遵守你的规范的证据。

Machine learning is uniquely good at learning algorithms,

机器学习尤其擅长学习算法，

but once the algorithm has been learned,

一旦它学习了算法，

you can re-implement it in a different computational architecture

你就可以在另一种更易于验证的 计算架构中重新实现它。

that's easier to verify.

Now you might worry,

你可能会担心，

how on earth am I going to understand this powerful AI

我到底该如何理解这个强大的 AI、

and the powerful AI tool it built

它构建的强大 AI 工具和证据，

and the proof,

if they're all too complicated for any human to grasp?

如果它们对于所有人类都 过于复杂，难以理解呢？

Here is the really great news.

以下就是真正的好消息。

You don't have to understand any of that stuff,

你不必了解任何东西，

because it's much easier to verify a proof than to discover it.

因为验证证据比找证据要容易得多。

So you only have to understand or trust your proof-checking code,

因此，你只需要理解 或信任你的校验代码，

which could be just a few hundred lines long.

它可能只有几百行长。

And Steve and I envision

史蒂夫和我设想

that such proof checkers get built into all our compute hardware,

这样的校验器会内置于 我们所有的计算机硬件中，

so it just becomes impossible to run very unsafe code.

因此绝无可能运行非常不安全的代码。

What if the AI, though, isn't able to write that AI tool for you?

但是，如果 AI 无法 为你编写那个 AI 工具呢？

Then there's another possibility.

那就还有另一种可能性。

You train an AI to first just learn to do what you want

你训练 AI 先学会做你想做的事，

and then you use a different AI

然后再换一个 AI

to extract out the learned algorithm and knowledge for you,

为你提取出所学的算法和知识，

like an AI neuroscientist.

就像一个 AI 神经科学家。

This is in the spirit of the field of mechanistic interpretability,

这就是机械可解释性领域的精髓，

which is making really impressive rapid progress.

该领域正在取得惊艳的快速进步。

Provably safe systems are clearly not impossible.

可证明安全的系统 显然不是不可能的。

Let's look at a simple example

我们来看一个简单的例子，

of where we first machine-learn an algorithm from data

首先，基于数据， 用计算机学习一个算法，

and then distill it out in the form of code

然后以可证明符合规范的 代码形式提炼出算法。

that provably meets spec, OK?

Let’s do it with an algorithm that you probably learned in first grade,

我们就拿你一年级或许 就学会的算法为例，

addition,

加法，

where you loop over the digits from right to left,

你从右到左遍历数位，

and sometimes you do a carry.

有时还会进位。

We'll do it in binary,

我们用二进制来做，

as if you were counting on two fingers instead of ten.

如同用两根手指， 而不是十个手指数数。

And we first train a recurrent neural network,

我们训练了一个循环神经网络，

never mind the details,

不用管细节， 来完成这个任务。

to nail the task.

So now you have this algorithm that you don't understand how it works

你现在有了这个算法，你也不知道 它在黑箱中是怎么运作的，

in a black box

defined by a bunch of tables of numbers that we, in nerd speak,

黑箱由一大堆数字定义， 用“技术宅”的话来说，

call parameters.

就是“参数”。

Then we use an AI tool we built to automatically distill out from this

然后，我们用构建的 AI 工具

以 Python 程序的形式 自动从中提炼出所学算法。

the learned algorithm in the form of a Python program.

And then we use the formal verification tool known as Daphne

然后，我们使用 名为 Daphne 的形式验证工具

to prove that this program correctly adds up any numbers,

证明该程序可以 正确地将任意数字相加，

not just the numbers that were in your training data.

而不仅仅是训练数据中的数字。

So in summary,

总而言之，

provably safe AI, I'm convinced is possible,

可证明安全的 AI， 我坚信这是可能的，

but it's going to take time and work.

但这需要时间和努力。

And in the meantime,

同时，

let's remember that all the AI benefits

请记住，各种 AI 的益处，

that most people are excited about

很多人为之兴奋的益处，

actually don't require superintelligence.

其实并不需要超级智能。

We can have a long and amazing future with AI.

我们可以和 AI 共同 拥有漫长且奇妙的未来。

So let's not pause AI.

所以请不要暂停 AI。

Let's just pause the reckless race to superintelligence.

而是暂停追求超级智能的无脑竞争。

Let's stop obsessively training ever-larger models

停止执着于训练越来越大

that we don't understand.

又无法理解的模型。

Let's heed the warning from ancient Greece

让我们听从古希腊的警告，

and not get hubris, like in the story of Icarus.

不要像伊卡洛斯的故事那样自负。

Because artificial intelligence

AI 为我们

is giving us incredible intellectual wings

插上了神奇的智慧之翼，

with which we can do things beyond our wildest dreams

如果我们不再执迷于飞向太阳，

if we stop obsessively trying to fly to the sun.

我们就能用它天马行空地飞翔。

Thank you.

谢谢。

(Applause)

（掌声）