What Is an AI Anyway? | Mustafa Suleyman | TED

譯者: 麗玲 辛

I want to tell you what I see coming.

我想告訴各位我預見的未來。

I've been lucky enough to be working on AI for almost 15 years now.

我很幸運能夠從事 人工智慧工作近 15 年。

Back when I started, to describe it as fringe would be an understatement.

當我剛入門時，將其描述為邊緣 其實還算是輕描淡寫。

Researchers would say, “No, no, we’re only working on machine learning.”

研究人員會說：「沒有， 我們只是在研究機器學習。」

Because working on AI was seen as way too out there.

因為人們認為研發 AI 太誇張。

In 2010, just the very mention of the phrase “AGI,”

2010 年，只要一提到 “AGI”

artificial general intelligence,

（通用人工智慧）這個詞，

would get you some seriously strange looks

你就會招來別人異樣的眼光，

and even a cold shoulder.

甚至遭到冷落。

"You're actually building AGI?" people would say.

「你真的在構建 AGI ？!」 人們會說。

"Isn't that something out of science fiction?"

「這不是科幻小說裡的東西嗎？」

People thought it was 50 years away or 100 years away,

人們認為，就算有可能，

if it was even possible at all.

也還要 50 或 100 年才能實現。

Talk of AI was, I guess, kind of embarrassing.

我猜，那時候，談 AI 有點尷尬，　

People generally thought we were weird.

人們一般認為我們很怪異。

And I guess in some ways we kind of were.

我想某些方面，確實如此。

It wasn't long, though, before AI started beating humans

然而，沒多久，AI 就開始

at a whole range of tasks

在一連串人們認為 遙不可及的任務上擊敗人類。

that people previously thought were way out of reach.

Understanding images,

理解圖像，

translating languages,

翻譯語言，

transcribing speech,

轉錄語音，

playing Go and chess

下圍棋、西洋棋，

and even diagnosing diseases.

甚至診斷疾病。

People started waking up to the fact

人們開始意識到

that AI was going to have an enormous impact,

AI 將帶來巨大衝擊，

and they were rightly asking technologists like me

他們合理地向像我這樣的科技專家

some pretty tough questions.

提出了一些非常困難的問題。

Is it true that AI is going to solve the climate crisis?

AI 真的能解決氣候危機？

Will it make personalized education available to everyone?

它能讓每個人都獲得個人化教育嗎？

Does it mean we'll all get universal basic income

這是否意味著我們 都會有全民基本收入，

and we won't have to work anymore?

不再需要工作？

Should I be afraid?

我應該擔心嗎？

What does it mean for weapons and war?

這在武器和戰爭方面意味著什麼改變？

And of course, will China win?

還有，中國會贏嗎？

Are we in a race?

我們在比賽嗎？

Are we headed for a mass misinformation apocalypse?

我們是否正在走向 大規模的錯誤資訊末日？

All good questions.

都是好問題。

But it was actually a simpler

但其實有個更單純、更基本的問題，

and much more kind of fundamental question that left me puzzled.

讓我苦思不已。

這問題其實是我每天工作的核心。

One that actually gets to the very heart of my work every day.

One morning over breakfast,

有天早上吃早餐時，

my six-year-old nephew Caspian was playing with Pi,

我六歲的侄子凱斯賓正在玩 Pi，

the AI I created at my last company, Inflection.

這是我在上一家公司 Inflection 開發的 AI。

With a mouthful of scrambled eggs,

他一邊嚼著炒蛋，

he looked at me plain in the face and said,

平淡地看著我，說：

"But Mustafa, what is an AI anyway?"

「穆斯塔法，AI 到底是什麼呢？」

He's such a sincere and curious and optimistic little guy.

他是個真誠、好奇、樂觀的小傢伙。

He'd been talking to Pi about how cool it would be if one day in the future,

他一直在和 Pi 聊

如果將來有一天他能去動物園 看到恐龍，那會有多酷。

he could visit dinosaurs at the zoo.

And how he could make infinite amounts of chocolate at home.

還有他如何在家裡 製作無限量的巧克力，

And why Pi couldn’t yet play I Spy.

以及為什麼 Pi 還不會玩 《我是小間諜》遊戲。

"Well," I said, "it's a clever piece of software

「嗯，」我說，「這是個聰明的軟體，

它可以閱讀公開網路上的大部分文本，

that's read most of the text on the open internet,

and it can talk to you about anything you want."

也可以與你談論任何你想談的內容。」

"Right.

「對。

So like a person then?"

那麼，就像一個人囉？」

I was stumped.

我被難住了。

Genuinely left scratching my head.

真的讓我搔頭苦思。

All my boring stock answers came rushing through my mind.

所有無聊俗套的答案 在我的腦海中閃過。

"No, but AI is just another general-purpose technology,

「不，AI 只是另一種通用科技，

like printing or steam."

就像印刷術或蒸汽機。

It will be a tool that will augment us

它將成為一種工具， 可以增強我們的能力，

and make us smarter and more productive.

讓我們變得更聰明、更有生產力。

And when it gets better over time,

隨著時間的推移，當它更完善，

it'll be like an all-knowing oracle

它會像一位無所不知的專家，

that will help us solve grand scientific challenges."

幫助我們解決重大的科學難題。」

You know, all of these responses started to feel, I guess,

你知道，這些回答都讓我開始覺得

a little bit defensive.

有防備戒心。

And actually better suited to a policy seminar

這種談話其實更適合政策研討會，

than breakfast with a no-nonsense six-year-old.

而不是與一個六歲孩子 不講廢話的早餐時光。

"Why am I hesitating?" I thought to myself.

「我為什麼會猶豫？」我自問。

You know, let's be honest.

坦白說，

My nephew was asking me a simple question

我的侄子問了我一個簡單的問題，

只是我們這些研究 AI 的人 不常正視這問題。

that those of us in AI just don't confront often enough.

What is it that we are actually creating?

我們實際上正在創造什麼？

What does it mean to make something totally new,

創造某個全新的東西，

fundamentally different to any invention that we have known before?

本質上與我們已知的 任何發明都不同，這意味著什麼？

It is clear that we are at an inflection point

很明顯，我們處於 人類歷史上的一個轉折點。

in the history of humanity.

On our current trajectory,

在目前的軌跡上，

we're headed towards the emergence of something

我們正迎向某個東西的出現，

that we are all struggling to describe,

某個我們都無法描述的東西，

and yet we cannot control what we don't understand.

但我們不理解的，我們就無法控制。

And so the metaphors,

因此，比喻、心智模型、 名稱等等都是必要的，

the mental models,

the names, these all matter

if we’re to get the most out of AI whilst limiting its potential downsides.

如果我們想要充分利用 AI， 同時抑制其潛在的缺點。

As someone who embraces the possibilities of this technology,

我欣然接受這項科技的各種可能性，

but who's also always cared deeply about its ethics,

也始終深切關注其道德規範，

we should, I think,

我認為，我們應該

be able to easily describe what it is we are building.

能夠輕易地描述我們正在建構的東西，

And that includes the six-year-olds.

包括向六歲的孩子。

So it's in that spirit that I offer up today the following metaphor

因此，正是本著這種精神， 我今天提出了以下比喻，

for helping us to try to grapple with what this moment really is.

以幫助我們應對此時此刻的真實情況。

I think AI should best be understood

我認為 AI 最好被理解為

as something like a new digital species.

一種新的數位物種。

Now, don't take this too literally,

不要太從字面上理解這一點，

but I predict that we'll come to see them as digital companions,

但我預測人們將把它們視為數位伴侶，

new partners in the journeys of all our lives.

我們生命旅程中的新夥伴。

Whether you think we’re on a 10-, 20- or 30-year path here,

無論你認為這條路要走 10 年、20 年還是 30 年，

this is, in my view, the most accurate and most fundamentally honest way

在我看來，這都是描述未來實際情況

of describing what's actually coming.

最準確、最可信的方式。

And above all, it enables everybody to prepare for

最重要的是，這讓每個人都能夠

為接下來的發展 做好準備，並塑造未來。

and shape what comes next.

Now I totally get, this is a strong claim,

我完全明白，這是個強烈的主張，

and I'm going to explain to everyone as best I can why I'm making it.

我將盡我所能，向大家解釋 我為什麼這樣比喻。

But first, let me just try to set the context.

但首先，容我試著說明情境。

From the very first microscopic organisms,

從最早的微生物開始，

地球上的生命可以追溯到數十億年前。

life on Earth stretches back billions of years.

Over that time, life evolved and diversified.

隨著時間的推移， 生命不斷進化，發展出多樣性。

Then a few million years ago, something began to shift.

幾百萬年前，開始有些變動。

After countless cycles of growth and adaptation,

經過無數的成長和適應週期後，

one of life’s branches began using tools, and that branch grew into us.

生物的某個分支開始使用工具，

那個分支變成了我們。

We went on to produce a mesmerizing variety of tools,

我們繼續製作各種令人驚嘆的工具，

at first slowly and then with astonishing speed,

一開始很慢，然後以驚人的速度，

we went from stone axes and fire

從石斧和火

to language, writing and eventually industrial technologies.

到語言、書寫，最後到工業技術。

One invention unleashed a thousand more.

一項發明催生了更多發明。

And in time, we became homo technologicus.

隨著時間的推移， 我們變成了科技人類。

Around 80 years ago,

大約 80 年前，

another new branch of technology began.

另一個新的科技分支開始。

With the invention of computers,

隨著電腦的發明，

we quickly jumped from the first mainframes and transistors

我們迅速從最初的大型主機和電晶體

躍升至今日的智慧型手機 和虛擬實境耳機。

to today's smartphones and virtual-reality headsets.

Information, knowledge, communication, computation.

資訊、知識、通訊、演算。

In this revolution,

在這場革命中，

創造發明前所未有地爆增。

creation has exploded like never before.

And now a new wave is upon us.

現在，新的浪潮正向我們襲來，

Artificial intelligence.

人工智慧。

These waves of history are clearly speeding up,

歷史浪潮顯然正在加速，

as each one is amplified and accelerated by the last.

因為每一波都比上一波 幅度更大、速度更快。

And if you look back,

如果你回顧過去，

it's clear that we are in the fastest

很明顯我們正處於有史以來最快、

and most consequential wave ever.

影響最深遠的浪潮中。

The journeys of humanity and technology are now deeply intertwined.

人類和科技的旅程 現在已經深深地交織在一起。

In just 18 months,

在短短 18 個月內，

超過 10 億人使用了 大型語言模型。

over a billion people have used large language models.

We've witnessed one landmark event after another.

我們見證了一個又一個里程碑事件。

Just a few years ago, people said that AI would never be creative.

就在幾年前，人們還說 AI 不可能有創意。

And yet AI now feels like an endless river of creativity,

然而現在，AI 似乎就像 一條無盡的創造力之河，

making poetry and images and music and video that stretch the imagination.

創造出擴展想像力的詩歌、 圖像、音樂和影片。

People said it would never be empathetic.

人們說它永遠不會有同理心。

And yet today, millions of people enjoy meaningful conversations with AIs,

然而今日，數百萬人喜歡 與 AI 進行有意義的對話，

talking about their hopes and dreams

談論他們的希望和夢想，

and helping them work through difficult emotional challenges.

並幫助他們應對困難的情緒問題。

AIs can now drive cars,

AI 現在會駕駛汽車，

manage energy grids

管理能源網絡，

甚至發明新分子。

and even invent new molecules.

Just a few years ago, each of these was impossible.

就在幾年前，這些都是不可能的。

And all of this is turbocharged by spiraling exponentials of data

這一切都是由數據和演算力的 螺旋式指數增長所推動的。

and computation.

Last year, Inflection 2.5, our last model,

去年，我們上一個模型 Inflection 2.5

used five billion times more computation

使用的計算量

比 DeepMind AI 十多年前 擊敗雅達利老式電子遊戲時

than the DeepMind AI that beat the old-school Atari games

多了 50 億倍。

just over 10 years ago.

That's nine orders of magnitude more computation.

計算量增加了九個數量級。

10x per year,

每年 10 倍，

every year for almost a decade.

近十年，每年如此。

Over the same time, the size of these models has grown

同時，這些模型的規模

from first tens of millions of parameters to then billions of parameters,

從最初的數千萬個參數 成長到後來的數十億個參數，

and very soon, tens of trillions of parameters.

很快就達到數十億個參數。

If someone did nothing but read 24 hours a day for their entire life,

如果有個人一生什麼都不做， 只是每天 24 小時閱讀，

they'd consume eight billion words.

他能讀完 80 億字。

And of course, that's a lot of words.

當然，這很多。

But today, the most advanced AIs consume more than eight trillion words

但如今，最先進的 AI

在一個月的訓練中 就能閱讀超過八兆個字。

in a single month of training.

And all of this is set to continue.

這一切都還會持續進行。

The long arc of technological history is now in an extraordinary new phase.

漫長的技術史現在 正處於一個非凡的新階段。

So what does this mean in practice?

現實上，這意味著什麼？

Well, just as the internet gave us the browser

正如網路為我們提供瀏覽網站、

and the smartphone gave us apps,

智慧型手機為我們 提供應用程式一樣，

the cloud-based supercomputer is ushering in a new era

奠基於雲端的超級電腦開啟了 AI 無所不在的新時代。

of ubiquitous AIs.

Everything will soon be represented by a conversational interface.

一切很快將由對話式介面來呈現。

或者，換個名稱，個人 AI。

Or, to put it another way, a personal AI.

And these AIs will be infinitely knowledgeable,

這些 AI 將擁有無限的知識，

and soon they'll be factually accurate and reliable.

很快地，它們會變得準確可靠，

擁有幾乎完美的智商，

They'll have near-perfect IQ.

They’ll also have exceptional EQ.

也會有出色的情緒智商。

They’ll be kind, supportive, empathetic.

它們友善，給予鼓勵，具同情心。

These elements on their own would be transformational.

這些元素本身就具有變革性。

Just imagine if everybody had a personalized tutor in their pocket

想像一下，每個人的口袋裡 都有一個私人教師,

and access to low-cost medical advice.

並能取得低成本的醫療建議。

A lawyer and a doctor,

律師和醫生、

a business strategist and coach --

商業策略師和教練——

all in your pocket 24 hours a day.

每天 24 小時都在你的口袋裡。

But things really start to change when they develop what I call AQ,

當它們發展出我所說的 AQ（「行動商數」）時，

their “actions quotient.”

事情就會真正開始改變。

This is their ability to actually get stuff done

這指的是它們在數位和實體世界中 實際完成工作的能力。

in the digital and physical world.

And before long, it won't just be people that have AIs.

很快地，擁有 AI 的 將不只是人類。

Strange as it may sound, every organization,

聽起來可能很奇怪，

每個組織，從小型企業到 非營利組織、再到國家政府，

from small business to nonprofit to national government,

each will have their own.

都會有自己的 AI。

Every town, building and object

每個城鎮、建築、物品，

will be represented by a unique interactive persona.

都將由獨特的互動角色代表。

And these won't just be mechanistic assistants.

這些不僅僅是機械助理。

They'll be companions, confidants,

它們將成為同伴、知己、

同事、朋友和合作夥伴，

colleagues, friends and partners,

as varied and unique as we all are.

就像我們一樣多樣化和獨特。

At this point, AIs will convincingly imitate humans at most tasks.

此時，AI 將在大多數任務中 令人信服地模仿人類。

And we'll feel this at the most intimate of scales.

我們將在最切身的方面感受到這一點。

An AI organizing a community get-together for an elderly neighbor.

AI 為年長鄰居舉辦社區聚會。

A sympathetic expert helping you make sense of a difficult diagnosis.

富有同情心的專家幫助你 理解一個困難的診斷。

But we'll also feel it at the largest scales.

但我們也會大規模地感受到它的存在。

加快科學上的發現，

Accelerating scientific discovery,

autonomous cars on the roads,

道路上的自動駕駛汽車，

drones in the skies.

天空中的無人機。

They'll both order the takeout and run the power station.

它們會訂購外賣，也會經營發電站。

They’ll interact with us and, of course, with each other.

它們會與我們互動， 當然，也會彼此互動。

They'll speak every language,

它們會說各種語言，

take in every pattern of sensor data,

接收各種模式的感測資料、

sights, sounds,

景象、聲音、

streams and streams of information,

各式各樣的資訊流，

far surpassing what any one of us could consume in a thousand lifetimes.

遠遠超過我們任何人 一千輩子所能吸收的內容。

So what is this?

那麼，這是什麼？

What are these AIs?

這些 AI 是什麼？

If we are to prioritize safety above all else,

如果我們要將安全放在優先考量，

以確保這股新浪潮始終 為人類服務，並增進人類福祉，

to ensure that this new wave always serves and amplifies humanity,

then we need to find the right metaphors for what this might become.

那麼我們需要為未來可能的發展 找到正確的比喻。

For years, we in the AI community, and I specifically,

多年來，在 AI 領域中的 我們，尤其是我，

have had a tendency to refer to this as just tools.

一直傾向於將它視為工具。

但這並沒有真正捕捉到 現今實際發展的情況。

But that doesn't really capture what's actually happening here.

AIs are clearly more dynamic,

AI 顯然更加動態、

more ambiguous, more integrated

更模糊、更整合，

and more emergent than mere tools,

而且比完全受人類控制的 單純工具更突出。

which are entirely subject to human control.

So to contain this wave,

因此，為了遏制這一浪潮，

to put human agency at its center

將人類的能動性置於核心，

and to mitigate the inevitable unintended consequences

並減輕可能出現的 不可避免的意外後果，

that are likely to arise,

we should start to think about them as we might a new kind of digital species.

我們應該開始想像 AI， 如同想像一種新的數位物種。

Now it's just an analogy,

現在這只是一個比喻，

it's not a literal description, and it's not perfect.

不是字面描述，也不完美。

首先，它們顯然不是 任何傳統意義上的生物，

For a start, they clearly aren't biological in any traditional sense,

but just pause for a moment

但請暫停一下，

and really think about what they already do.

認真想一想它們已經會做哪些事。

They communicate in our languages.

它們會用我們的語言溝通，

They see what we see.

它們看到我們眼見的。

They consume unimaginably large amounts of information.

它們能吸收難以想像的大量訊息。

They have memory.

它們有記憶，

They have personality.

有個性，

They have creativity.

它們有創造力。

它們甚至可以進行一定程度的推理 並制定基本計劃。

They can even reason to some extent and formulate rudimentary plans.

They can act autonomously if we allow them.

如果我們允許，它們能自主行動。

And they do all this at levels of sophistication

所有它們能達成的複雜程度

that is far beyond anything that we've ever known from a mere tool.

遠遠超出了我們先前 所了解的單純工具。

And so saying AI is mainly about the math or the code

如果說 AI 只是數學或程式碼，

is like saying we humans are mainly about carbon and water.

就如同說我們人類 只是碳和水組成一樣。

It's true, but it completely misses the point.

這是事實，但這完全沒有抓住重點。

And yes, I get it, this is a super arresting thought

我知道，這是一個 非常引人注目的想法，

but I honestly think this frame helps sharpen our focus on the critical issues.

但老實說，我認為這框架有助於 增強我們對這個關鍵問題的關注。

What are the risks?

有哪些風險？

What are the boundaries that we need to impose?

我們需要施加哪些界線？

What kind of AI do we want to build or allow to be built?

我們想要建構或允許建構 什麼樣的 AI？

This is a story that's still unfolding.

這是一個仍在展開的故事。

Nothing should be accepted as a given.

任何事情都不應該被視為理所當然。

We all must choose what we create.

對於所創造的東西，我們必須有選擇。

What AIs we bring into the world, or not.

我們要為世界帶來了 什麼樣的 AI，或是選擇不要。

These are the questions for all of us here today,

這些是我們今天在座的所有人

and all of us alive at this moment.

以及此時此刻還活著的所有人的問題。

For me, the benefits of this technology are stunningly obvious,

對我來說，這項科技的好處 是非常明顯的，

and they inspire my life's work every single day.

這激勵著我每天投身這個畢生事業。

But quite frankly, they'll speak for themselves.

但坦白說，這不言自明。

Over the years, I've never shied away from highlighting risks

多年來，我從不迴避 強調風險，並談論缺點。

and talking about downsides.

Thinking in this way helps us focus on the huge challenges

以這種方式思考幫助我們專注於

所有人面臨的巨大挑戰。

that lie ahead for all of us.

But let's be clear.

但明確地說，

There is no path to progress

如果我們撇下科技， 就沒有進步的道路。

where we leave technology behind.

The prize for all of civilization is immense.

文明進步得到的獎賞是無限的。

我們需要各種解決方案， 醫療保健、教育方面、氣候危機。

We need solutions in health care and education, to our climate crisis.

And if AI delivers just a fraction of its potential,

只要 AI 發揮其潛力的一小部分，

the next decade is going to be the most productive in human history.

未來十年將是人類歷史上 生產力最高的十年。

Here's another way to think about it.

以下是另一種思考方式。

In the past,

過去，

經濟成長往往會帶來巨大的負面影響。

unlocking economic growth often came with huge downsides.

The economy expanded as people discovered new continents

人們發現新大陸、開闢新疆域，

and opened up new frontiers.

經濟隨之不斷擴張。

But they colonized populations at the same time.

人們同時也進行殖民。

We built factories,

我們建造了工廠，

but they were grim and dangerous places to work.

但這些工作場所醜陋又危險。

我們開採了石油，

We struck oil,

but we polluted the planet.

但污染了地球。

Now because we are still designing and building AI,

現在，因為我們仍在 設計和建造 AI，

we have the potential and opportunity to do it better,

所以我們有潛力和機會做得更好，

radically better.

徹底更好。

現在，我們並不是發現新大陸

And today, we're not discovering a new continent

and plundering its resources.

並掠奪其資源，

We're building one from scratch.

我們正在從頭開始建造一個新大陸。

Sometimes people say that data or chips are the 21st century’s new oil,

有時人們會說數據或晶片 是 21 世紀的新石油，

but that's totally the wrong image.

但這個意象完全錯了。

AI is to the mind

AI 之於大腦

what nuclear fusion is to energy.

就像核融合之於能源，

Limitless, abundant,

無限，豐沛，能改變世界。

world-changing.

And AI really is different,

AI 的確完全不同，

and that means we have to think about it creatively and honestly.

這意味著我們必須 創造性地、如實地想像它。

We have to push our analogies and our metaphors

我們必須將我們的類比 和隱喻推向極限，

to the very limits

以便能夠應對即將發生的事情。

to be able to grapple with what's coming.

Because this is not just another invention.

因為這不僅僅是另一項發明。

AI is itself an infinite inventor.

AI 本身就是無限發明家。

And yes, this is exciting and promising and concerning

是的，這令人興奮、充滿希望、

令人擔憂、同時也引人入勝。

and intriguing all at once.

To be quite honest, it's pretty surreal.

老實說，這很超現實。

But step back,

但退一步，

see it on the long view of glacial time,

從冰河期的長遠角度來看，

and these really are the very most appropriate metaphors that we have today.

這確實是我們現今 所擁有的最恰當的比喻。

Since the beginning of life on Earth,

自從地球上出現生命以來，

we've been evolving, changing

我們一直在進化、改變，

and then creating everything around us in our human world today.

創造了今日人類世界， 我們周遭的一切。

And AI isn't something outside of this story.

AI 並不是在這個歷程之外。

In fact, it's the very opposite.

事實上，恰恰相反。

It's the whole of everything that we have created,

它是我們所創造的一切的整合，

distilled down into something that we can all interact with

被提煉成人人都可以 與之互動的東西，並從中受益。

and benefit from.

It's a reflection of humanity across time,

它是人類跨越時間的映像，

and in this sense,

從這個意義上說，

it isn't a new species at all.

它根本不是新物種。

This is where the metaphors end.

比喻到這裡就結束了。

Here's what I'll tell Caspian next time he asks.

下次凱斯賓問起時，我會告訴他，

AI isn't separate.

AI 並非孤立的。

AI isn't even in some senses, new.

從某種意義上說， AI 甚至不是新的東西。

AI is us.

AI 就是我們，

It's all of us.

是我們所有人。

這也許是連六歲孩子都能感受到的

And this is perhaps the most promising and vital thing of all

that even a six-year-old can get a sense for.

最有希望、最重大的事。

As we build out AI,

當我們建構 AI 時，

we can and must reflect all that is good,

我們可以、而且必須 映照出人類所有的良善、

all that we love,

珍愛與特殊之處：

all that is special about humanity:

our empathy, our kindness,

我們的同理心、善良、

our curiosity and our creativity.

好奇心和創造力。

This, I would argue, is the greatest challenge of the 21st century,

我認為，這是 21 世紀最大的挑戰，

but also the most wonderful,

但對我們所有人來說也是最美妙、

inspiring and hopeful opportunity for all of us.

最鼓舞人心、最充滿希望的機會。

Thank you.

謝謝。

(Applause)

（掌聲）

Chris Anderson: Thank you Mustafa.

克里斯·安德森： 謝謝穆斯塔法。

It's an amazing vision and a super powerful metaphor.

這是令人驚嘆的願景 和超級強大的隱喻。

你正處於一個很棒的位置。

You're in an amazing position right now.

你對 OpenAI 所進行的 令人驚嘆的工作有著密切的了解。

I mean, you were connected at the hip

to the amazing work happening at OpenAI.

You’re going to have resources made available,

你擁有可用的資源，

there are reports of these giant new data centers,

這些新的巨型資料中心的相關報告，

100 billion dollars invested and so forth.

1000 億美元的投資等等。

一種新物種可以從中出現。

And a new species can emerge from it.

I mean, in your book,

我的意思是，在你的書中，

you did, as well as painting an incredible optimistic vision,

你做到了，除了描繪 令人難以置信的樂觀願景之外，

you were super eloquent on the dangers of AI.

你也很有說服力地 闡述了AI 的危險。

And I'm just curious, from the view that you have now,

我只是好奇，從你現在的角度來看，

what is it that most keeps you up at night?

最讓你徹夜難眠的是什麼？

Mustafa Suleyman: I think the great risk is that we get stuck

穆斯塔法·蘇萊曼： 我認為最大的風險是

我們陷入我所謂的悲觀厭惡陷阱。

in what I call the pessimism aversion trap.

You know, we have to have the courage to confront

你知道，我們必須有勇氣 面對潛在的黑暗腳本，

the potential of dark scenarios

in order to get the most out of all the benefits that we see.

才能充分利用我們眼見的所有好處。

So the good news is that if you look at the last two or three years,

好消息是，如果你看看過去兩三年，

there have been very, very few downsides, right?

你會發現幾乎沒有什麼缺點，對吧？

It’s very hard to say explicitly what harm an LLM has caused.

很難明確說出大型語言模型 造成了什麼危害。

But that doesn’t mean that that’s what the trajectory is going to be

但這並不意味著這就會是 未來十年的發展軌跡。

over the next 10 years.

我認為，如果你專注於 一些具體的能力，

So I think if you pay attention to a few specific capabilities,

take for example, autonomy.

例如，自主性。

自主權顯然是個門檻， 如果我們跨越，將會增加社會風險。

Autonomy is very obviously a threshold

over which we increase risk in our society.

And it's something that we should step towards very, very closely.

這是我們應該非常密切地 關注發展的點。

The other would be something like recursive self-improvement.

另一個是類似會不斷自我修正的程式。

If you allow the model to independently self-improve,

如果你允許模型獨立地自我改進，

update its own code,

更新自己的程式碼，

explore an environment without oversight, and, you know,

在沒有監督的情況下探索環境，

而沒有人可以控制改變它的運作方式，

without a human in control to change how it operates,

that would obviously be more dangerous.

那顯然會更加危險。

But I think that we're still some way away from that.

但我認為我們離這點還有一段距離。

我認為我們還需要五到十年的時間 才必須面對這個問題。

I think it's still a good five to 10 years before we have to really confront that.

但現在該開始談論它了。

But it's time to start talking about it now.

CA：與任何生物物種不同，

CA: A digital species, unlike any biological species,

數位物種可以在九奈秒內 而不是九個月內複製，

can replicate not in nine months,

but in nine nanoseconds,

and produce an indefinite number of copies of itself,

並產生無限數量的自身副本，

all of which have more power than we have in many ways.

這些副本在許多方面 都比我們擁有更多的能力。

I mean, the possibility for unintended consequences seems pretty immense.

出現意外後果的可能性似乎相當大。

And isn't it true that if a problem happens,

如果出現問題， 可能會在一小時內發生，

it could happen in an hour?

這是真的嗎？

MS: No.

穆斯塔法·蘇萊曼：不，

That is really not true.

這確實不是真的。

I think there's no evidence to suggest that.

我認為沒有證據表明這一點。

And I think that, you know,

我認為，

that’s often referred to as the “intelligence explosion.”

這通常被稱為「智慧爆炸」。

And I think it is a theoretical, hypothetical maybe

我認為這是一個理論上的假設，

也許我們都很好奇去探索，

that we're all kind of curious to explore,

but there's no evidence that we're anywhere near anything like that.

但沒有證據表明 我們已經接近這樣的狀況。

And I think it's very important that we choose our words super carefully.

我認為我們必須非常謹慎地 選擇我們的措辭。

Because you're right, that's one of the weaknesses of the species framing,

因為你說的對， 這是物種框架的弱點之一，

that we will design the capability for self-replication into it

如果人們選擇這樣做， 我們將為其設計自我複製的能力。

if people choose to do that.

And I would actually argue that we should not,

實際上，我認為我們不應該這樣做，

這是我們該避開的 危險能力之一，是吧？

that would be one of the dangerous capabilities

that we should step back from, right?

So there's no chance that this will "emerge" accidentally.

因此，這不可能意外地「出現」。

I really think that's a very low probability.

我確實認為這種可能性非常低。

It will happen if engineers deliberately design those capabilities in.

如果工程師故意設計這些功能，

And if they don't take enough efforts to deliberately design them out.

卻沒有付出足夠的努力來避免風險， 那麼這種情況就會發生。

因此，這就是為什麼要一開始

And so this is the point of being explicit

就明確、透明地透過設計引入安全性。

and transparent about trying to introduce safety by design very early on.

CA：謝謝你，你的願景是 人類在這個新事物中，

CA: Thank you, your vision of humanity injecting into this new thing

注入我們自己最好的部分，

the best parts of ourselves,

avoiding all those weird, biological, freaky,

避免我們在某些​​情況下可能出現的

horrible tendencies that we can have in certain circumstances,

所有奇怪的、生物的、 怪異的、可怕的傾向，

I mean, that is a very inspiring vision.

這是非常鼓舞人心的願景。

And thank you so much for coming here and sharing it at TED.

非常感謝你來到這裡， 並在 TED 上分享。

Thank you, good luck.

謝謝，祝好運。

(Applause)

（掌聲）