How to get empowered, not overpowered, by AI | Max Tegmark

譯者: Lilian Chiu 審譯者: Melody Tang

After 13.8 billion years of cosmic history,

在 138 億年的宇宙歷史之後，

our universe has woken up

我們的宇宙終於覺醒了，

and become aware of itself.

開始意識到自己。

From a small blue planet,

從一顆藍色的小星球，

tiny, conscious parts of our universe have begun gazing out into the cosmos

我們宇宙中微小、有意識的部分

開始用望遠鏡窺視宇宙，

with telescopes,

discovering something humbling.

有了讓人謙卑的發現。

We've discovered that our universe is vastly grander

我們發現宇宙比我們祖先所想像的

than our ancestors imagined

要大很多很多，

and that life seems to be an almost imperceptibly small perturbation

而且生命似乎是非常小的擾動， 小到幾乎無法感知到，

on an otherwise dead universe.

若沒有它，宇宙就是死寂的了。

But we've also discovered something inspiring,

但我們也有振奮人心的發現：

which is that the technology we're developing has the potential

那就是我們所開發的技術，

有潛能可以協助生命 變得前所未有的繁盛，

to help life flourish like never before,

not just for centuries but for billions of years,

不是只有數世紀， 而是能持續數十億年。

and not just on earth but throughout much of this amazing cosmos.

不只是在地球上， 還是在這整個不可思議的宇宙中。

I think of the earliest life as "Life 1.0"

我把最早的生命 視為是「生命 1.0」，

because it was really dumb,

因為它其實很蠢，

like bacteria, unable to learn anything during its lifetime.

像細菌，它在一生中 無法學習任何東西。

I think of us humans as "Life 2.0" because we can learn,

我把人類視為是「生命 2.0」， 因為我們可以學習，

which we in nerdy, geek speak,

用很宅的方式來說，

might think of as installing new software into our brains,

可以視為是把新軟體 安裝到我們的大腦中，

like languages and job skills.

就像語言以及工作技能。

"Life 3.0," which can design not only its software but also its hardware

「生命 3.0」不只能設計 它的軟體，還能設計硬體，

of course doesn't exist yet.

當然，它還不存在。

But perhaps our technology has already made us "Life 2.1,"

但，也許我們的科技已經 讓我們成為「生命 2.1」了，

with our artificial knees, pacemakers and cochlear implants.

因為我們現在有人工膝蓋、 心律調節器，以及耳蝸植入。

So let's take a closer look at our relationship with technology, OK?

咱們來更進一步談談 我們與科技的關係，好嗎？

As an example,

舉個例子，

the Apollo 11 moon mission was both successful and inspiring,

阿波羅 11 月球任務 很成功也很鼓舞人心，

showing that when we humans use technology wisely,

展示出當我們人類聰明地使用科技時，

we can accomplish things that our ancestors could only dream of.

我們能達成祖先只能夢想的事情。

But there's an even more inspiring journey

但還有一趟更鼓舞人心的旅程，

propelled by something more powerful than rocket engines,

由比火箭引擎 更強大的東西所推動，

where the passengers aren't just three astronauts

乘客也不只是三個太空人，

but all of humanity.

而是全人類。

Let's talk about our collective journey into the future

咱們來談談我們全體與人工智慧

with artificial intelligence.

一起前往未來的旅程，

My friend Jaan Tallinn likes to point out that just as with rocketry,

我的朋友楊·塔林常說， 就像火箭學一樣，

it's not enough to make our technology powerful.

光是讓我們的科技 有強大的力量是不足夠的。

We also have to figure out, if we're going to be really ambitious,

若我們真的很有野心的話， 我們還得要想出

how to steer it

如何操控它，

and where we want to go with it.

及我們要和它一起去到哪裡。

So let's talk about all three for artificial intelligence:

所以，咱們針對人工智慧 來談談這三點：

the power, the steering and the destination.

力量、操控，以及目的地。

Let's start with the power.

咱們先從力量談起。

I define intelligence very inclusively --

我對於人工智慧的定義非常全面，

simply as our ability to accomplish complex goals,

就是我們能夠完成 複雜目標的能力，

because I want to include both biological and artificial intelligence.

因為我想要把生物智慧 和人工智慧都包含進去。

And I want to avoid the silly carbon-chauvinism idea

且我想要排除愚蠢的碳沙文主義，

that you can only be smart if you're made of meat.

它認為若你很聰明， 你就一定是肉做的。

It's really amazing how the power of AI has grown recently.

人工智慧的力量 在近期的成長十分驚人。

Just think about it.

試想看看。

Not long ago, robots couldn't walk.

沒多久以前，機器人還無法走路。

Now, they can do backflips.

現在，它們還會後翻。

Not long ago,

沒多久以前，

we didn't have self-driving cars.

我們還沒有自動駕駛的汽車。

Now, we have self-flying rockets.

現在，我們有自動飛行的火箭。

Not long ago,

沒多久以前，

AI couldn't do face recognition.

人工智慧還無法做臉孔辨識。

Now, AI can generate fake faces

現在，人工智慧能產生出假臉孔，

and simulate your face saying stuff that you never said.

並模擬你的臉孔， 說出你從來沒有說過的話。

Not long ago,

沒多久以前，

AI couldn't beat us at the game of Go.

人工智慧無法在圍棋中打敗人類。

Then, Google DeepMind's AlphaZero AI took 3,000 years of human Go games

接著， Google DeepMind AlphaZero 的人工智慧

拿來人類三千年的圍棋譜和圍棋智慧，

and Go wisdom,

ignored it all and became the world's best player by just playing against itself.

全部擺在一邊，

透過和自己比賽的練習， 變成了世界上最厲害的圍棋手。

And the most impressive feat here wasn't that it crushed human gamers,

這裡最讓人印象深刻的功績 並不是它擊垮了人類的棋手，

but that it crushed human AI researchers

而是它擊垮了人類的 人工智慧研究者，

who had spent decades handcrafting game-playing software.

這些研究者花了數十年 手工打造下棋軟體。

And AlphaZero crushed human AI researchers not just in Go but even at chess,

除了圍棋，AlphaZero 也在西洋棋 擊垮了人類的人工智慧研究者，

which we have been working on since 1950.

西洋棋從 1950 年起就被研究著。

So all this amazing recent progress in AI really begs the question:

所以，近期這些驚人的 人工智慧進步，讓大家想問：

How far will it go?

它能做到什麼程度？

I like to think about this question

我在思考這個問題時，

in terms of this abstract landscape of tasks,

想從工作任務的抽象地景來切入，

where the elevation represents how hard it is for AI to do each task

高度表示人工智慧要把每一項工作

做到人類水平的難度，

at human level,

and the sea level represents what AI can do today.

海平面高度表示現今的 人工智慧能做什麼。

The sea level is rising as AI improves,

海平面隨著人工智慧的改進而上升，

so there's a kind of global warming going on here in the task landscape.

所以，在這工作任務地景上， 有類似全球暖化的現象發生。

And the obvious takeaway is to avoid careers at the waterfront --

很顯然的結論是： 避免從事在海濱的職業——

(Laughter)

（笑聲）

which will soon be automated and disrupted.

這些工作很快就會被自動化所取代。

But there's a much bigger question as well.

但這裡還有一個更大的問題。

How high will the water end up rising?

水面最後會升到多高？

Will it eventually rise to flood everything,

它最後是否會升高到淹沒一切，

matching human intelligence at all tasks.

在所有工作任務上 都能和人類的智慧匹敵？

This is the definition of artificial general intelligence --

這是「強人工智慧」的定義，

AGI,

縮寫 AGI，

which has been the holy grail of AI research since its inception.

打從它的一開始， 它就是人工智慧研究的聖杯。

By this definition, people who say,

依這個定義，當有人說：

"Ah, there will always be jobs that humans can do better than machines,"

「啊，總是會有些工作， 人類能做得比機器好。」

are simply saying that we'll never get AGI.

他們說的只是 「我們永遠不會有 AGI。」

Sure, we might still choose to have some human jobs

當然，我們仍可選擇 保留一些人類的工作，

or to give humans income and purpose with our jobs,

或是用工作讓人類保有收入和目的，

but AGI will in any case transform life as we know it

但不論如何，AGI 都會 轉變我們所認知的生活，

with humans no longer being the most intelligent.

人類將不再是最有智慧的。

Now, if the water level does reach AGI,

如果海平面真的升到 AGI 的高度，

then further AI progress will be driven mainly not by humans but by AI,

那麼進一步的人工智慧進步將會 由人工智慧來主導，而非人類，

which means that there's a possibility

意思是，有可能進一步的 人工智慧進展會非常快，

that further AI progress could be way faster

than the typical human research and development timescale of years,

超越用「年」來計算 典型人類研究和發展的時間，

raising the controversial possibility of an intelligence explosion

提高很受爭議的智慧噴發的可能性，

where recursively self-improving AI

即，不斷遞迴的 自我改進的人工智慧

rapidly leaves human intelligence far behind,

很快就會遠遠超越人類的智慧，

creating what's known as superintelligence.

創造出所謂的超級人工智慧。

Alright, reality check:

好了，回歸現實：

Are we going to get AGI any time soon?

我們很快就會有 AGI 嗎？

Some famous AI researchers, like Rodney Brooks,

有些知名的人工智慧研究者， 像羅德尼·布魯克斯，

think it won't happen for hundreds of years.

認為在數百年內還不會發生。

But others, like Google DeepMind founder Demis Hassabis,

但其他人，像 Google DeepMind 的 創辦人傑米斯·哈薩比斯，

are more optimistic

就比較樂觀，

and are working to try to make it happen much sooner.

且努力想要讓它早點發生。

And recent surveys have shown that most AI researchers

近期的調查顯示， 大部分的人工智慧研究者

actually share Demis's optimism,

其實和傑米斯一樣樂觀，

expecting that we will get AGI within decades,

預期我們會在數十年內就有 AGI，

so within the lifetime of many of us,

所以，許多人 在有生之年就能看到，

which begs the question -- and then what?

這就讓人不禁想問：接下來呢？

What do we want the role of humans to be

我們希望人類扮演什麼角色，

if machines can do everything better and cheaper than us?

如果機器每件事都做得 比人類好、成本又更低的話？

The way I see it, we face a choice.

依我所見，我們面臨一個選擇。

選項之一是不假思索的滿足。

One option is to be complacent.

We can say, "Oh, let's just build machines that can do everything we can do

我們可以說：「咱們來打造機器， 讓它們做所有我們能做的事，

and not worry about the consequences.

不要擔心結果。

Come on, if we build technology that makes all humans obsolete,

拜託，如果我們能打造科技， 讓全人類變成過時，

what could possibly go wrong?"

怎可能會出錯的？」

(Laughter)

（笑聲）

But I think that would be embarrassingly lame.

但我覺得那樣是差勁到令人難堪。

I think we should be more ambitious -- in the spirit of TED.

我認為我們該更有野心—— 抱持 TED 精神。

Let's envision a truly inspiring high-tech future

咱們來想像一下 一個真正鼓舞人心的高科技未來，

and try to steer towards it.

並試著朝它邁進。

This brings us to the second part of our rocket metaphor: the steering.

這就帶我們來到了 火箭比喻的第二部分：操控。

We're making AI more powerful,

我們讓人工智慧的力量更強大，

but how can we steer towards a future

但我們要如何將人工智慧導向

協助人類繁盛而非掙扎無助的未來？

where AI helps humanity flourish rather than flounder?

To help with this,

為了協助做到這點，

I cofounded the Future of Life Institute.

我共同創辦了「生命未來研究所」。

It's a small nonprofit promoting beneficial technology use,

它是個小型的非營利機構， 旨在促進有益的科技使用，

and our goal is simply for the future of life to exist

我們的目標很簡單： 希望生命的未來能夠存在，

and to be as inspiring as possible.

且越是鼓舞人心越好。

You know, I love technology.

你們知道的，我很愛科技。

Technology is why today is better than the Stone Age.

現今之所以比石器時代更好， 就是因為科技。

And I'm optimistic that we can create a really inspiring high-tech future ...

我很樂觀地認為我們能創造出 真的很鼓舞人心的高科技未來……

if -- and this is a big if --

如果——這個「如果」很重要——

if we win the wisdom race --

如果我們能贏得這場智慧賽跑——

the race between the growing power of our technology

這場賽跑的兩位競爭者是

不斷成長的科技力量 和不斷成長的管理科技智慧。

and the growing wisdom with which we manage it.

But this is going to require a change of strategy

這會需要策略的改變，

because our old strategy has been learning from mistakes.

因為我們的舊策略 是從錯誤中學習。

We invented fire,

我們發明了火，

screwed up a bunch of times --

搞砸了很多次——

invented the fire extinguisher.

發明了滅火器。

(Laughter)

（笑聲）

We invented the car, screwed up a bunch of times --

我們發明了汽車，搞砸了很多次——

invented the traffic light, the seat belt and the airbag,

發明了紅綠燈、 安全帶，和安全氣囊，

but with more powerful technology like nuclear weapons and AGI,

但對於更強大的科技， 比如核子武器和 AGI，

learning from mistakes is a lousy strategy,

從錯誤中學習是很糟的策略，

don't you think?

對吧？（笑聲）

(Laughter)

事前的主動比事後的反應更好；

It's much better to be proactive rather than reactive;

plan ahead and get things right the first time

先計畫好，第一次就把事情做對，

because that might be the only time we'll get.

因為我們可能只有一次機會。

But it is funny because sometimes people tell me,

好笑的是，有時人們告訴我：

"Max, shhh, don't talk like that.

「麥克斯，噓，別那樣說話。

That's Luddite scaremongering."

那是危言聳聽。」

But it's not scaremongering.

但那並非危言聳聽。

It's what we at MIT call safety engineering.

我們在麻省理工學院 稱之為安全工程。

Think about it:

想想看：

before NASA launched the Apollo 11 mission,

在美國太空總署的 阿波羅 11 任務之前，

they systematically thought through everything that could go wrong

他們系統性地設想過 所有可能出錯的狀況，

when you put people on top of explosive fuel tanks

畢竟是要把人類 放在易爆燃料槽上，

and launch them somewhere where no one could help them.

再將他們發射到 沒有人能協助他們的地方。

And there was a lot that could go wrong.

可能會出錯的狀況非常多。

Was that scaremongering?

那是危言聳聽嗎？

No.

不。

That's was precisely the safety engineering

那正是安全工程，

用來確保任務能夠成功。

that ensured the success of the mission,

and that is precisely the strategy I think we should take with AGI.

那正是我認為處理 AGI 時 應該採用的策略。

Think through what can go wrong to make sure it goes right.

想清楚有什麼可能會出錯， 確保它能不要出錯。

So in this spirit, we've organized conferences,

基於這種精神，我們辦了一些會議，

bringing together leading AI researchers and other thinkers

集合了最領先的人工智慧 研究者和其他思想家，

to discuss how to grow this wisdom we need to keep AI beneficial.

討論要如何發展這項必要的智慧， 確保人工智慧是有益的。

Our last conference was in Asilomar, California last year

我們最近一次會議是去年 在加州的阿西洛馬會議中舉辦，

and produced this list of 23 principles

得出了這 23 條原則。

which have since been signed by over 1,000 AI researchers

從那之後，已經有 超過一千名人工智慧研究者

and key industry leaders,

與重要產業領導者簽署。

and I want to tell you about three of these principles.

我想要和各位談其中三條原則。

One is that we should avoid an arms race and lethal autonomous weapons.

其一是我們應該要避免 軍備競賽以及自主的致命武器。

The idea here is that any science can be used for new ways of helping people

想法是，任何科學都能被 用作助人或傷人的新方法。

or new ways of harming people.

For example, biology and chemistry are much more likely to be used

比如，生物和化學更有可能會被用來

for new medicines or new cures than for new ways of killing people,

做新的藥物或治療方法， 而不是殺人的新方法，

because biologists and chemists pushed hard --

因為生物學家和化學家 很努力推動——

and successfully --

且很成功——

for bans on biological and chemical weapons.

禁止生物及化學武器的禁令。

And in the same spirit,

基於同樣的精神，

most AI researchers want to stigmatize and ban lethal autonomous weapons.

大部分的人工智慧研究者 想要譴責和禁用自主的致命武器。

Another Asilomar AI principle

另一條阿西洛馬會議人工智慧原則

is that we should mitigate AI-fueled income inequality.

是我們應該要減輕 由人工智慧引起的收入不平等。

I think that if we can grow the economic pie dramatically with AI

我認為如果能用人工智慧 讓經濟大餅大幅地成長，

and we still can't figure out how to divide this pie

而仍無法弄清楚如何分割這塊餅

so that everyone is better off,

來讓每個人都過得更好，

then shame on us.

那我們真該感到羞恥。

(Applause)

（掌聲）

Alright, now raise your hand if your computer has ever crashed.

好，如果你的電腦曾當過機，請舉手。

(Laughter)

（笑聲）

Wow, that's a lot of hands.

哇，好多人舉手。

Well, then you'll appreciate this principle

那麼你們就會欣賞這條原則：

that we should invest much more in AI safety research,

我們應該要更投入 人工智慧安全的研究，

because as we put AI in charge of even more decisions and infrastructure,

因為當我們讓人工智慧來主導 更多決策和基礎設施時。

we need to figure out how to transform today's buggy and hackable computers

我們得要想出該如何將現今 有程式錯誤且可能被駭入的電腦，

into robust AI systems that we can really trust,

轉變成我們能真正信任的 穩定的人工智慧系統。

because otherwise,

all this awesome new technology can malfunction and harm us,

要不然，所有這些了不起的新科技 都可能故障、傷害我們，

or get hacked and be turned against us.

或被駭入而轉成對抗我們。

And this AI safety work has to include work on AI value alignment,

這項人工智慧安全性的工作必須要 包含人工智慧價值校準的工作，

because the real threat from AGI isn't malice,

因為 AGI 帶來的 真正威脅不是惡意——

like in silly Hollywood movies,

像愚蠢的好萊塢電影裡那樣——

but competence --

而是能力——

AGI accomplishing goals that just aren't aligned with ours.

AGI 要完成的目標就是 與我們的目標不一致。

For example, when we humans drove the West African black rhino extinct,

比如，當我們人類 讓西非的黑犀牛濱臨絕種時，

we didn't do it because we were a bunch of evil rhinoceros haters, did we?

並不是因為我們邪惡、痛恨犀牛 才這麼做的，對吧？

We did it because we were smarter than them

會這麼做，是因為我們比牠們聰明，

and our goals weren't aligned with theirs.

我們的目標和牠們的目標不一致。

But AGI is by definition smarter than us,

但 AGI 在定義上 就是比我們聰明的，

so to make sure that we don't put ourselves in the position of those rhinos

所以要確保我們在 創造了 AGI 之後，

if we create AGI,

不會淪落到那些犀牛的處境，

we need to figure out how to make machines understand our goals,

我們就得要想出如何 讓機器了解我們的目標，

adopt our goals and retain our goals.

採用並且保持我們的目標。

And whose goals should these be, anyway?

不過，這些目標該是誰的目標？

Which goals should they be?

這些目標該是哪些目標？

This brings us to the third part of our rocket metaphor: the destination.

這就帶我們到了火箭比喻的 第三部分：目的地。

We're making AI more powerful,

我們要讓人工智慧的力量更強大，

trying to figure out how to steer it,

試圖想辦法來操控它，

but where do we want to go with it?

但我們如何帶它到哪裡？

This is the elephant in the room that almost nobody talks about --

這是幾乎沒人談論的房中大象 （顯而易見又被忽略）——

not even here at TED --

即使在 TED 也沒人在談——

because we're so fixated on short-term AI challenges.

因為我們都把目光放在 短期的人工智慧挑戰。

Look, our species is trying to build AGI,

聽著，我們人類正試著建造 AGI，

motivated by curiosity and economics,

動機是好奇心和經濟，

but what sort of future society are we hoping for if we succeed?

但如果成功了，我們希望 創造出什麼樣的未來社會？

We did an opinion poll on this recently,

最近我們針對這點做了意見調查，

and I was struck to see

結果讓我很驚訝，

that most people actually want us to build superintelligence:

大部分的人其實希望 我們建造超級人工智慧：

AI that's vastly smarter than us in all ways.

全面比我們聰明的人工智慧。

What there was the greatest agreement on was that we should be ambitious

大家最一致的意見， 就是我們應該要有野心，

and help life spread into the cosmos,

並協助生命在宇宙中散播，

but there was much less agreement about who or what should be in charge.

但對於該由誰或由什麼來主導， 大家的意見就不那麼一致了。

And I was actually quite amused

有件事讓我覺得很有趣的

to see that there's some some people who want it to be just machines.

是我看到有些人希望就由機器主導。

(Laughter)

（笑聲）

And there was total disagreement about what the role of humans should be,

至於人類該扮演什麼角色， 意見就完全不一致了，

even at the most basic level,

即使在最基礎的層級也一樣。

so let's take a closer look at possible futures

讓咱們來近看

我們或許會選擇的可能未來，好嗎？

that we might choose to steer toward, alright?

So don't get me wrong here.

別誤會我的意思，

I'm not talking about space travel,

我並不是在談太空旅行，

只是要談人類進入未來的比喻之旅。

merely about humanity's metaphorical journey into the future.

So one option that some of my AI colleagues like

我的一些人工智慧同事 很喜歡的一個選擇是

is to build superintelligence and keep it under human control,

建造超級人工智慧， 並保持讓它被人類控制，

like an enslaved god,

就像能被奴役的神一樣，

disconnected from the internet

和網路沒有連結，

and used to create unimaginable technology and wealth

用來創造無法想像的科技和財富，

for whoever controls it.

全交給控制它的人。

But Lord Acton warned us

阿克頓男爵警告我們：

that power corrupts, and absolute power corrupts absolutely,

「權力會產生腐敗， 絕對權力絕對腐敗。」

so you might worry that maybe we humans just aren't smart enough,

你可能會擔心 也許我們人類就是不夠聰明，

or wise enough rather,

或是沒有足夠的智慧

to handle this much power.

來操作這麼多權力。

Also, aside from any moral qualms you might have

除了奴役更優秀的智慧的 任何道德疑慮之外，

about enslaving superior minds,

you might worry that maybe the superintelligence could outsmart us,

你也許會擔心

或許超級人工智慧會智勝我們，

break out and take over.

衝破籓籬和掌管。

But I also have colleagues who are fine with AI taking over

但我也有些同事覺得 讓人工智慧接管也沒不好，

and even causing human extinction,

甚至造成人類絕種也無妨，

as long as we feel the the AIs are our worthy descendants,

只要我們覺得人工智慧 配得上做我們的後裔就好，

like our children.

就像我們的孩子一樣。

But how would we know that the AIs have adopted our best values

但，我們要如何確知人工智慧 已經採用了我們最好的價值觀，

and aren't just unconscious zombies tricking us into anthropomorphizing them?

而不只是有意識的殭屍， 騙我們將人性賦予它們？

Also, shouldn't those people who don't want human extinction

此外，不希望人類絕種的那些人

have a say in the matter, too?

也應被容許對此事表達意見吧？

Now, if you didn't like either of those two high-tech options,

如果這兩個高科技選項 都不合你們的意，

it's important to remember that low-tech is suicide

很重要的是要記得 從宇宙的觀點來看低科技是自殺，

from a cosmic perspective,

because if we don't go far beyond today's technology,

因為如果我們不遠遠超越 現今的科技，

the question isn't whether humanity is going to go extinct,

問題就不是人類是否會絕種，

merely whether we're going to get taken out

而是讓我們絕種的會是下一次的

by the next killer asteroid, supervolcano

巨型慧星撞擊、超級火山爆發，

or some other problem that better technology could have solved.

或是更優的科技 本可解決的其他問題？

So, how about having our cake and eating it ...

所以，何不接受和吃下這蛋糕……

with AGI that's not enslaved

這個不是被奴役的 AGI，

but treats us well because its values are aligned with ours?

而是價值觀和我們一致， 善待我們的 AGI 呢？

This is the gist of what Eliezer Yudkowsky has called "friendly AI,"

那就是亞里艾瑟·尤考斯基 所謂的「友善的人工智慧」，

and if we can do this, it could be awesome.

若我們能做到這點，可能會很棒。

It could not only eliminate negative experiences like disease, poverty,

它可能不只會除去負面的遭遇，

比如疾病、貧困、

crime and other suffering,

犯罪，和其他苦難，

but it could also give us the freedom to choose

它也可能會給予我們自由，

讓我們從各式各樣 新的正面經驗中做選擇——

from a fantastic new diversity of positive experiences --

basically making us the masters of our own destiny.

基本上，就是讓我們 成為自己命運的主宰。

So in summary,

所以，總結一下，

our situation with technology is complicated,

我們在科技方面的情況很複雜，

but the big picture is rather simple.

但整體來看是很簡單的。

Most AI researchers expect AGI within decades,

多數人工智慧研究者預期 AGI 會在數十年內出現，

and if we just bumble into this unprepared,

如果我們沒有先準備好面對它，

it will probably be the biggest mistake in human history --

那可能會成為人類史上最大的錯誤。

let's face it.

讓我們正視事實吧，

It could enable brutal, global dictatorship

它會讓殘酷的 全球獨裁主義成為可能，

with unprecedented inequality, surveillance and suffering,

造成前所未有的不平等、 監控，以及苦難，

and maybe even human extinction.

甚至讓人類絕種。

But if we steer carefully,

但如果我們小心地操控，

we could end up in a fantastic future where everybody's better off:

我們可能會有個美好的未來， 人人都過得更好：

the poor are richer, the rich are richer,

貧窮的人有錢，有錢的人更有錢，

everybody is healthy and free to live out their dreams.

每個人都健康，能自由自在地 去實現他們的夢想。

Now, hang on.

等等，別急。

Do you folks want the future that's politically right or left?

你們希望未來在政治上 是右派還是左派？

Do you want the pious society with strict moral rules,

你們想要一個有著嚴格 道德規則的虔誠社會，

or do you an hedonistic free-for-all,

或一個人人可參與的享樂主義社會，

more like Burning Man 24/7?

就像全年無休的燃燒人節慶？

Do you want beautiful beaches, forests and lakes,

你們想要美麗的海灘、森林和湖泊，

or would you prefer to rearrange some of those atoms with the computers,

或是偏好用電腦重新排列原子，

enabling virtual experiences?

產生出虛擬經驗？

With friendly AI, we could simply build all of these societies

有了友善的人工智慧， 我們就能建立出所有這些社會，

and give people the freedom to choose which one they want to live in

並讓大家有自由去選擇 他們想要住在哪個社會中，

因為我們不會再受智慧的限制，

because we would no longer be limited by our intelligence,

唯一的限制是物理法則。

merely by the laws of physics.

So the resources and space for this would be astronomical --

所以，資源和空間會非常龐大 ——

literally.

天文級的龐大。

So here's our choice.

我們的選擇如下：

We can either be complacent about our future,

我們可以對未來感到滿足，

taking as an article of blind faith

帶著盲目的信念，

that any new technology is guaranteed to be beneficial,

相信任何新科技都必然有益，

and just repeat that to ourselves as a mantra over and over and over again

當作真言，不斷對自己 一次又一次地重述，

as we drift like a rudderless ship towards our own obsolescence.

而我們像無舵的船漂向淘汰。

Or we can be ambitious --

或是我們可以有野心，

thinking hard about how to steer our technology

努力去想出如何操控我們的科技，

and where we want to go with it

以及我們想要去的目的地，

to create the age of amazement.

創造出驚奇的時代。

We're all here to celebrate the age of amazement,

我們在這裡讚頌驚奇的時代，

and I feel that its essence should lie in becoming not overpowered

我覺得精髓應該在於不受控於科技，

but empowered by our technology.

而是讓它賦予我們力量。

Thank you.

謝謝。

(Applause)

（掌聲）