How to Keep AI Under Control | Max Tegmark | TED

譯者: Lilian Chiu 審譯者: Shelley Tsang 曾雯海

Five years ago,

五年前，

I stood on the TED stage

我站在 TED 舞台上，警告大家

and warned about the dangers of superintelligence.

超智慧的危險性。

I was wrong.

我錯了。

It went even worse than I thought.

情況比我想的還更糟。

(Laughter)

（笑聲）

I never thought governments would let AI companies get this far

我從來沒有想到政府 會讓人工智慧公司

在毫無任何有意義規範的 情況下發展到這個地步，

without any meaningful regulation.

And the progress of AI went even faster than I predicted.

而人工智慧的進步速度 比我預測的還快。

Look, I showed this abstract landscape of tasks

我先前給大家看過這張 抽象的工作任務地景圖，

where the elevation represented how hard it was for AI

高度代表人工智慧進行該工作 任務並做到人類水平的難度。

to do each task at human level.

And the sea level represented what AI could be back then.

海平面代表當時人工智慧 可能是什麼樣子的。

And boy or boy, has the sea been rising fast ever since.

而，天哪，海平面從那時起 就一直快速上升。

But a lot of these tasks have already gone blub blub blub blub blub blub.

但這些工作任務有很多 都已經淹到海裡去了。

And the water is on track to submerge all land,

水正在迅速淹沒所有陸地，

matching human intelligence at all cognitive tasks.

在所有認知類的工作任務上 都能與人類智慧匹敵。

This is a definition of artificial general intelligence, AGI,

這就是通用人工智慧的一個定義，

簡稱 AGI，

which is the stated goal of companies like OpenAI,

AGI 是許多公司宣稱的 目標，如 OpenAI、

Google DeepMind and Anthropic.

Google DeepMind，和 Anthropic 。

And these companies are also trying to build superintelligence,

而這些公司也在試圖打造超智慧，

leaving human intelligence far behind.

把人類智慧遠遠拋在後頭。

And many think it'll only be a few years, maybe, from AGI to superintelligence.

許多人認為，從 AGI 到超智慧 只需要幾年的時間。

So when are we going to get AGI?

那麼我們何時會有 AGI？

Well, until recently, most AI researchers thought it was at least decades away.

目前為止，大多數 人工智慧研究者都認為

至少還要數十年。

And now Microsoft is saying, "Oh, it's almost here."

現在微軟說：「喔， 它就快問世了。」

We're seeing sparks of AGI in ChatGPT-4,

在 ChatGPT-4 中我們 可以看到 AGI 的火花，

and the Metaculus betting site is showing the time left to AGI

Metaculus 投注網站顯示，

在過去十八個月間， 實現 AGI 的時間

plummeting from 20 years away to three years away

從二十年後縮短到三年後。

in the last 18 months.

And leading industry people are now predicting

而業界領頭羊現在預測

大概兩到三年後， 我們就不是最聰明的了。

that we have maybe two or three years left until we get outsmarted.

So you better stop talking about AGI as a long-term risk,

因此，你最好不要再把 AGI 當作長期風險來談，

or someone might call you a dinosaur stuck in the past.

否則你可能會被認為 是活在過去的恐龍。

It's really remarkable how AI has progressed recently.

最近人工智慧的發展 真的很令人驚訝。

Not long ago, robots moved like this.

不久前，機器人的動作 還是像這樣子的。

(Music)

（音樂）

Now they can dance.

現在它們還可以跳舞呢。

(Music)

（音樂）

Just last year, Midjourney produced this image.

就在去年，MidJourney 製作了這張影像。

This year, the exact same prompt produces this.

今年，用完全相同的提示， 產生出來的結果是這樣。

Deepfakes are getting really convincing.

深偽已經變得非常有說服力。

(Video) Deepfake Tom Cruise: I’m going to show you some magic.

（影片）深偽的湯姆‧克魯斯： 我變個魔術給大家看。

It's the real thing.

這是真的東西。

(Laughs)

（笑）

I mean ...

我的意思是……

It's all ...

這一切都是……

the real ...

真的……東西。

thing.

Max Tegmark: Or is it?

講者：是嗎？

And Yoshua Bengio now argues

約書亞‧班吉歐現在主張

that large language models have mastered language

大型語言模型精通語言和知識已經 到了可以通過圖靈測試的程度。

and knowledge to the point that they pass the Turing test.

I know some skeptics are saying,

我知道有些懷疑論者會說：「不， 它們只是被過度吹捧的隨機鸚鵡，

"Nah, they're just overhyped stochastic parrots

that lack a model of the world,"

缺乏世界觀模型。」

but they clearly have a representation of the world.

但它們很顯然能夠表達出世界。

In fact, we recently found that Llama-2 even has a literal map of the world in it.

事實上，我們最近發現

Llama-2 裡面甚至 還有一張真的世界地圖。

And AI also builds

人工智慧還可以針對

geometric representations of more abstract concepts

抽象的概念構建出 幾何的表示方式，比如

like what it thinks is true and false.

它認為的是真的和假的。

So what's going to happen if we get AGI and superintelligence?

那麼，當我們有了 AGI 和超智慧時會發生什麼事？

If you only remember one thing from my talk, let it be this.

如果這場演說你只能記住 一點，那請記住這點：

AI godfather, Alan Turing predicted

人工智慧教父艾倫‧圖靈

that the default outcome is the machines take control.

預測預設的結果是機器取得掌控權。

The machines take control.

機器取得掌控權。

I know this sounds like science fiction,

我知道這聽起來像科幻小說，

but, you know, having AI as smart as GPT-4

但，有像 GPT-4 這麼聰明的人工智慧，

also sounded like science fiction not long ago.

在不久前也會覺得 聽起來像是科幻小說。

And if you think of AI,

如果你把人工智慧，

if you think of superintelligence in particular, as just another technology,

特別是，如果你把超智慧視為

不過是另一種技術，就像電力，

like electricity,

you're probably not very worried.

你可能不會太擔心。

But you see,

但，要知道，在圖靈眼中， 超智慧更像是一個新物種。

Turing thinks of superintelligence more like a new species.

Think of it,

想想看，我們是在打造

we are building creepy, super capable,

令人毛骨悚然、能力超強、

amoral psychopaths

沒道德的精神病患，不用睡覺，

that don't sleep and think much faster than us,

思考速度比我們快很多， 可以自我複製，且毫無人性。

can make copies of themselves

and have nothing human about them at all.

So what could possibly go wrong?

所以，怎麼可能會出錯呢？

(Laughter)

（笑聲）

And it's not just Turing.

且不只是圖靈。

OpenAI CEO Sam Altman, who gave us ChatGPT,

帶給我們 ChatGPT 的 OpenAI 執行長山姆‧奧特曼

recently warned that it could be "lights out for all of us."

近期警告說，它可能會是 「對我們所有人而言的熄燈時刻」。

Anthropic CEO, Dario Amodei, even put a number on this risk:

Anthropic 執行長達里奧‧阿莫迪 甚至給了這個風險一個數字：

10-25 percent.

10-25%。

And it's not just them.

且不僅是他們。

Human extinction from AI went mainstream in May

五月時，人工智慧造成 人類滅絕成了主流話題，

when all the AGI CEOs and who's who of AI researchers

因為那時所有的 AGI 執行長 和人工智慧的重要研究者

came on and warned about it.

都跳出來做警告。

上個月，連歐盟的老大也警告

And last month, even the number one of the European Union

warned about human extinction by AI.

要小心人工智慧造成人類滅絕。

So let me summarize everything I've said so far

讓我用一張貓咪迷因投影片 來總結目前為止我所說的一切。

in just one slide of cat memes.

Three years ago,

三年前，

people were saying it's inevitable, superintelligence,

大家都說這是不可避免的，超智慧，

it'll be fine,

一切都會很好的，

it's decades away.

還有幾十年。

Last year it was more like,

去年的狀況比較像是：

It's inevitable, it'll be fine.

這是不可避免的，一切都會很好的。

Now it's more like,

現在的狀況更像是：

It's inevitable.

這是不可避免的。

(Laughter)

（笑聲）

But let's take a deep breath and try to raise our spirits

但，咱們深呼吸一下，

試著提振精神，讓自己開心點，

and cheer ourselves up,

because the rest of my talk is going to be about the good news,

因為這場演說後續的 部分都會是好消息，

that it's not inevitable, and we can absolutely do better,

說明這並非不可避免的， 且我們絕對可以做得更好，好嗎？

alright?

(Applause)

（掌聲）

So ...

所以……

The real problem is that we lack a convincing plan for AI safety.

真正的問題在於

針對人工智慧安全， 我們缺乏有說服力的計畫。

People are working hard on evals

大家很努力在做評估，

尋找人工智慧有哪些行為 會造成風險，這是好事，

looking for risky AI behavior, and that's good,

but clearly not good enough.

但顯然不夠好。

They're basically training AI to not say bad things

基本上，他們是在訓練 人工智慧不要說不好的事，

rather than not do bad things.

而不是不要做不好的事。

Moreover, evals and debugging are really just necessary,

此外，

評估和除錯

對於安全而言， 是必要條件但還不足夠。

not sufficient, conditions for safety.

In other words,

換句話說，

they can prove the presence of risk,

他們可以證明有風險存在，

not the absence of risk.

而不是證明沒有風險。

So let's up our game, alright?

所以，咱們再努力提升點水平吧。

Try to see how we can make provably safe AI that we can control.

試看看我們要如何做出安全性可以 被證明且我們能控制的人工智慧。

Guardrails try to physically limit harm.

護欄是以實體的方式嘗試限制傷害。

But if your adversary is superintelligence

但如果你的敵人是超智慧， 或使用超智慧對付你的人，

or a human using superintelligence against you, right,

trying is just not enough.

光嘗試還不足夠。

You need to succeed.

你得要成功。

Harm needs to be impossible.

必須要讓傷害不可能發生。

So we need provably safe systems.

我們需要安全性可以被證明的系統。

Provable, not in the weak sense of convincing some judge,

證明指的並不是說服某位 法官的那種證明，那太弱了，

but in the strong sense of there being something that's impossible

而是要強到比如根據物理定律 就是不可能發生的程度，

according to the laws of physics.

Because no matter how smart an AI is,

因為不論人工智慧有多聰明，

it can't violate the laws of physics and do what's provably impossible.

也不可能違反物理定律， 做出已被證明不可能的事。

Steve Omohundro and I wrote a paper about this,

我和史帝夫‧奧莫亨卓

寫了一篇相關論文， 我們對此抱持樂觀，

and we're optimistic that this vision can really work.

認為這個願景是行得通的。

So let me tell you a little bit about how.

讓我來談談怎麼行得通。

There's a venerable field called formal verification,

有個很讓人敬佩的領域， 叫做正式驗證，

which proves stuff about code.

用來證明和程式碼相關的東西。

And I'm optimistic that AI will revolutionize automatic proving business

我很樂觀認為，人工智慧

會為「自動證明」事業帶來革命，

and also revolutionize program synthesis,

也會為程式合成帶來革命，

the ability to automatically write really good code.

也就是自動寫出 很好的程式碼的能力。

So here is how our vision works.

我們的願景是這麼運作的： 當身為人類的你

You, the human, write a specification

撰寫一份規格，

that your AI tool must obey,

你的人工智慧工具 必須要遵守這份規格，

that it's impossible to log in to your laptop

內容是，沒有正確的密碼 就不可能登入你的筆電，

without the correct password,

or that a DNA printer cannot synthesize dangerous viruses.

或者 DNA 印表機不可以 合成出危險的病毒。

Then a very powerful AI creates both your AI tool

接著，一個非常強大的人工智慧 不但創建了你的人工智慧工具，

and a proof that your tool meets your spec.

也證明了你的工具符合你的規格。

Machine learning is uniquely good at learning algorithms,

機器學習在學習演算法上 有獨特的優勢，

but once the algorithm has been learned,

但一旦演算法已經學好了，

you can re-implement it in a different computational architecture

你就可以把它重新導入到 不同的計算架構中，

that's easier to verify.

更容易驗證的架構中。

Now you might worry,

現在你可能會擔心我到底要怎麼了解

how on earth am I going to understand this powerful AI

這個強大的人工智慧、它建造的 強大人工智慧工具，及證據。

and the powerful AI tool it built

and the proof,

畢竟它們可能複雜到人類無法理解？

if they're all too complicated for any human to grasp?

Here is the really great news.

我有個超棒的消息： 這些你通通不需要了解，

You don't have to understand any of that stuff,

because it's much easier to verify a proof than to discover it.

因為驗證證據 比發現證據更容易許多。

So you only have to understand or trust your proof-checking code,

你只需要了解或信任

你用來檢查證據的程式碼，

which could be just a few hundred lines long.

它的長度可能只有幾百行。

And Steve and I envision

史帝夫和我期望未來

that such proof checkers get built into all our compute hardware,

這種證據檢查程式都會被 內建在我們所有的電腦硬體中，

so it just becomes impossible to run very unsafe code.

所以根本就不可能 執行不安全的程式碼。

What if the AI, though, isn't able to write that AI tool for you?

但若人工智慧無法為你寫出 那個人工智慧工具，怎麼辦？

那麼，還有另一種可能性。

Then there's another possibility.

You train an AI to first just learn to do what you want

你先訓練一個人工智慧 來做你想要做的事，

and then you use a different AI

接著，使用一個不同的人工智慧

to extract out the learned algorithm and knowledge for you,

為你取出已經學好的演算法和知識，

like an AI neuroscientist.

就像人工智慧神經科學家一樣。

This is in the spirit of the field of mechanistic interpretability,

這完全符合機制 可解釋性領域的精神，

which is making really impressive rapid progress.

這個領域的進展快得驚人。

Provably safe systems are clearly not impossible.

很顯然，安全性可以 被證明的系統不是不可能的。

Let's look at a simple example

咱們來看個簡單的例子：首先，我們

of where we first machine-learn an algorithm from data

用機器學習方法， 從資料中學習演算法，

and then distill it out in the form of code

接著，把它提取出來，

以程式碼的形式呈現， 且可證明是符合規格的。

that provably meets spec, OK?

Let’s do it with an algorithm that you probably learned in first grade,

咱們用個大家可能在一年級 就學到的演算法來當例子，

addition,

加法，

從最右的位數到最左， 進行迴圈，有時要進位。

where you loop over the digits from right to left,

and sometimes you do a carry.

We'll do it in binary,

我們用二進位來做。

as if you were counting on two fingers instead of ten.

就像把用十隻手指數數 改為用兩隻手指。

And we first train a recurrent neural network,

我們首先訓練一個 循環神經網路，不用管細節，

never mind the details,

to nail the task.

用來完成任務。

So now you have this algorithm that you don't understand how it works

現在你有這個演算法， 不過你不知道它怎麼運作，

它是黑箱作業，

in a black box

defined by a bunch of tables of numbers that we, in nerd speak,

用一堆數字表格來定義的黑箱，

用阿宅語言來說就是「參數」。

call parameters.

Then we use an AI tool we built to automatically distill out from this

接著，我們使用我們建造的 工具從黑箱中提取出

the learned algorithm in the form of a Python program.

學好的演算法，並以 Python 程式的形式呈現，

And then we use the formal verification tool known as Daphne

接著，

我們使用正式的驗證工具， 即大家熟知的 Daphne，來證明

to prove that this program correctly adds up any numbers,

這個程式可以將任何數字 正確相加起來，

not just the numbers that were in your training data.

不僅是在訓練資料集中的數字。

So in summary,

總結來說，

provably safe AI, I'm convinced is possible,

安全性可以被證明的人工智慧， 我相信是有可能的，

but it's going to take time and work.

但需要投入時間和努力。

And in the meantime,

與此同時，別忘記

let's remember that all the AI benefits

讓大多數人感到興奮的 各種人工智慧益處

that most people are excited about

actually don't require superintelligence.

其實並不需要超智慧。

We can have a long and amazing future with AI.

我們可以和人工智慧共存， 創造長久且不凡的未來。

So let's not pause AI.

所以，咱們別暫停人工智慧。

Let's just pause the reckless race to superintelligence.

咱們要暫停的是魯莾的超智慧競賽。

Let's stop obsessively training ever-larger models

咱們別再沉迷於訓練

我們都還不了解的更大模型。

that we don't understand.

Let's heed the warning from ancient Greece

咱們要聽從古希臘的警告，

and not get hubris, like in the story of Icarus.

別像伊卡洛斯的故事一樣變得傲慢。

Because artificial intelligence

因為人工智慧

is giving us incredible intellectual wings

能給予我們很棒的智慧之翼，

with which we can do things beyond our wildest dreams

讓我們能超越我們最狂野的夢想，

前提是我們不能再繼續 沉迷於飛向太陽。

if we stop obsessively trying to fly to the sun.

Thank you.

謝謝。

(Applause)

（掌聲）