The Dark Side of Competition in AI | Liv Boeree | TED

譯者: Jack Liu 審譯者: Shelley Tsang 曾雯海

Competition.

競爭，

It's a fundamental part of human nature.

是人類的天性

I was a professional poker player for 10 years,

我當了十年的專業撲克玩家

so I've very much seen all the good, bad and ugly ways it can manifest.

見識到人性各種善良、邪惡，及醜陋的體現

When it's done right,

競爭，可以是助力

it can drive us to incredible feats in sports and innovation,

使我們在運動和創新方面達成壯舉

like when car companies compete over who can build the safest cars

比如汽車公司， 競相製造出最安全的汽車

or the most efficient solar panels.

或最有效率的太陽能板。

Those are all examples of healthy competition,

這些都是良性競爭的例子，

because even though individual companies might come and go,

因為即使公司倒了

這種良性競爭還是能留下雙贏局面

in the long run,

the game between them creates win-win outcomes

where everyone benefits in the end.

最後對大家都有好處

But sometimes competition is not so great

但並不是所有競爭都是良性的

and can create lose-lose outcomes where everyone's worse off than before.

有時反而會造成雙輸局面， 讓大家過得更慘

Take these AI beauty filters, for example.

以這些 AI 美容濾鏡為例

As you can see, they're a very impressive technology.

如你所見，這科技非常厲害

They can salvage almost any picture.

幾乎能挽救任何照片

They can even make Angelina and Margot more beautiful.

甚至能讓安潔麗娜和瑪格更加美麗

So they're very handy,

相當實用的科技

especially for influencers who, now, at the click of a button,

尤其是對網紅來說， 只要按個鈕

can transform into the most beautiful Hollywood versions of themselves.

就可把自己變成跟好萊塢明星一樣美

But handy doesn't always mean healthy.

但方便並不一定代表健康。

And I've personally noticed how quickly these things can train you

我開始注意到

這些科技會快速的讓你討厭自己天生的樣貌。

to hate your natural face.

And there's growing evidence that they're creating issues

而且有越來越多的證據指出

like body dysmorphia, especially in young people.

這些科技增長了身體臆形症等心理問題， 尤其在年輕族群

Nonetheless, these things are now endemic to social media

儘管如此， 這些科技已經在社交媒體上氾濫

because the nature of the game demands it.

因為如果想要在現有制度下成功， 你不能不用它

The platforms are incentivized to provide them

社群平台主動提供這些濾鏡

because hotter pictures means more hijacked limbic systems,

因為有更多性感照，就能綁架更多注意力

which means more scrolling and thus more ad revenue.

而更多的注意力，代表更高的廣告收益。

And users are incentivized to use them

而用戶也被鼓勵去使用這些濾鏡

because hotter pictures get you more followers.

因為性感的照片能帶來更多追隨者

But this is a trap,

但這是一個陷阱，

because once you start using these things,

因為這些科技你只要一用

it's really hard to go back.

就很難不再去用它

Plus, you don't even get a competitive advantage from them anymore

而且如果不繼續用， 你會因此處於劣勢

because everyone else is already using them too.

因為就算你不用，別人照樣會用

So influencers are stuck using these things

因此，被迫使用這些工具的網紅

承受其帶來的負面影響， 卻只換來微薄的收益

with all the downsides

and very little upside.

A lose-lose game.

全盤皆輸

A similar kind of trap is playing out in our news media right now,

同樣的狀況也發生於新聞媒體

but with much worse consequences.

但造成的影響更加嚴重

You'd think since the internet came along

照理來說，網路的出現

that the increased competition between news outlets

能促使新聞媒體之間的良性競爭

would create a sort of positive spiral,

like a race to the top of nuanced, impartial, accurate journalism.

比誰能產出更多元、 有益且公正的新聞

Instead, we're seeing a race to the bottom of clickbait and polarization,

但事與願違， 我們只看到新聞騙取流量，煽動爭議

where even respectable papers are increasingly leaning

連有威望的報社

into these kind of low-brow partisan tactics.

也開始靠挑動黨派對立來爭取銷量

Again, this is due to crappy incentives.

跟前面一樣，這是不良誘因造成的後果

Today, we no longer just read our news.

在現代，我們不只會閱讀新聞

We interact with it by sharing and commenting.

還會藉由分享、留言與其互動

And headlines that trigger emotions like fear or anger

而能引起恐慌或怒氣的聳動頭條

are far more likely to go viral than neutral or positive ones.

比起中立或正向報導，更能激起討論熱度

新聞編輯陷入和網紅相似的困境

So in many ways, news editors are in a similar kind of trap

as the influencers,

where, the more their competitors lean into clickbaity tactics,

當對手用釣魚標題獲得更多流量時

the more they have to as well.

自己也被迫照做

Otherwise, their stories just get lost in the noise.

否則報導就會被埋沒

But this is terrible for everybody,

但這對每個人都有壞處

because now the media get less trust from the public,

因為不只是新聞媒體失去了公信力

but also it becomes harder and harder for anyone to discern truth from fiction,

也讓我們更難辨視事實真假

which is a really big problem for democracy.

在民主社會中，是很嚴重的問題

Now, this process of competition gone wrong

這種失序競爭的情況

is actually the driving force behind so many of our biggest issues.

就是許多當代問題的最大主因

Plastic pollution,

塑料污染

deforestation,

森林砍伐

antibiotic overuse in farming,

養殖業抗生素濫用

arms races,

軍武競賽

greenhouse gas emissions.

溫室氣體排放

These are all a result of crappy incentives,

這些都是負面競爭的結果

of poorly designed games that push their players --

設計不良的制度

逼得不管是民眾、企業或政府

be them people, companies or governments --

into taking strategies and tactics that defer costs and harms to the future.

都必須已危害未來的方法做事， 讓後人承擔後果

And what's so ridiculous is that most of the time,

但荒唐的是

大部分的人也不想這麼做

these guys don't even want to be doing this.

You know, it's not like packaging companies

包裝公司也不想用塑料填滿海洋

want to fill the oceans with plastic

or farmers want to worsen antibiotic resistance.

養殖者也不想用抗生素

But they’re all stuck in the same dilemma of:

但他們都陷入相同的困境：

"If I don't use this tactic,

「如果我不採用這種手法

I’ll get outcompeted by all the others who do.

我會輸給其他這麼做的競爭者

So I have to do it, too.”

所以我也必須這樣做」

This is the mechanism we need to fix as a civilization.

導正這個狀況，是我們文明人的義務

And I know what you're probably all thinking, "So it's capitalism."

你們可能心想:「啊! 萬惡的資本主義!」

不，它不是原因

No, it's not capitalism.

Which, yes, can cause problems,

它的確會製造問題

but it can also solve them and has been fantastic in general.

但同時也能解決問題，整體來說很棒

It's something much deeper.

問題出在更深層的地方

It's a force of misaligned incentives of game theory itself.

出自於不鼓勵良性競爭的環境

幾年前，我退出撲克圈

So a few years ago, I retired from poker,

in part because I wanted to understand this mechanism better.

一部分是因為我想深入了解這種失衡機制。

Because it takes many different forms, and it goes by many different names.

它有各種不同的面相，不同的稱呼

這些只是其中幾個名字 ［協調問題］［負和賭局］

These are just some of those names.

這些只是其中幾個名字 ［多方角逐］［共有地悲劇］

如你所見，都是些很抽象生硬的名詞，對吧? ［逐底競爭］［多方囚犯困境］

You can see they're a little bit abstract and clunky, right?

They don't exactly roll off the tongue.

十分拗口 ［社會難題］［不當平衡］

And given how insidious and connected all of these problems are,

這些課題難以被察覺又互相關聯

it helps to have a more visceral way of recognizing them.

所以我們需要個極端的例子，來了解、辨別它們

這次集會應該只有我這場會提起聖經

So this is probably the only time

you're going to hear about the Bible at this conference.

但我想說個小故事，

But I want to tell you a quick story from it,

because allegedly, back in the Canaanite days,

因為據稱， 在迦南時代，

there was a cult who wanted money and power so badly,

有個非常渴望金錢與權力的邪教

they were willing to sacrifice their literal children for it.

甚至願意將親生血肉綁在木像上

活活燒死，想以此換取財富與權力

And they did this by burning them alive in an effigy

of a God that they believed would then reward them

相信他們的神會獎勵這種終極犧牲

for this ultimate sacrifice.

And the name of this god was Moloch.

這位神叫“摩洛”

Bit of a bummer, as stories go.

這故事滿負面的

但也是一個恰當的比喻

But you can see why it's an apt metaphor,

because sometimes we get so lost in winning the game right in front of us,

因為有時我們會為了搶得先機

we lose sight of the bigger picture

而犧牲未來

and sacrifice too much in our pursuit of victory.

不知不覺的付出我們無法承受的代價

So just like these guys were sacrificing their children for power,

就像為了權力獻祭親生骨肉的邪教徒

those influencers are sacrificing their happiness for likes.

網紅為了按讚數， 犧牲自己的快樂

Those news editors are sacrificing their integrity for clicks,

新聞編輯為了流量， 犧牲自己的信用

and polluters are sacrificing the biosphere for profit.

而污染者為了獲利， 犧牲了大自然

In all these examples,

以上的例子， 都是為了短利的負面競爭造成的

the short-term incentives of the games themselves are pushing,

they're tempting their players

誘惑每個人犧牲越來越多的未來

to sacrifice more and more of their future,

trapping them in a death spiral where they all lose in the end.

把大家拉進負面漩渦中， 人人都不好過

That's Moloch's trap.

陷入摩洛的圈套

The mechanism of unhealthy competition.

鼓勵負面競爭的機制

而相同的故事， 也在AI產業上演中

And the same is now happening in the AI industry.

We're all aware of the race that's heating up

我們都知道各個公司在互相競爭

between companies right now

over who can score the most compute,

比誰能獲得最高分的運算評比

who can get the biggest funding round or get the top talent.

誰能獲得最多的經費、 最頂尖的人才

Well, as more and more companies enter this race,

當競爭對手逐漸增加

the greater the pressure for everyone to go as fast as possible

公司必須超越對手的壓力也越來越大

and sacrifice other important stuff like safety testing.

驅使公司犧牲重要的堅持， 比如安全測試

This has all the hallmarks of a Moloch trap.

摩洛圈套的特點在此展露無遺

Because, like, imagine you're a CEO who, you know, in your heart of hearts,

想像你是一位執行長

真心認為你的團隊有能力做出安全又強大的AI

believes that your team is the best

to be able to safely build extremely powerful AI.

Well, if you go too slowly, then you run the risk of other,

但如果你動作太慢，

就可能被其他沒那麼嚴謹的團隊超越

much less cautious teams getting there first

搶先部屬它們的系統

and deploying their systems before you can.

So that in turn pushes you to be more reckless yourself.

這種風險反而使你更加魯莽

And given how many different experts and researchers,

己經有不少學者與專家

both within these companies

不管是來自職場還是獨立的

but also completely independent ones,

have been warning us about the extreme risks of rushed AI,

都開始警告我們草率推出AI的風險

this approach is absolutely mad.

是非常不負責任、瘋狂的行為

Plus, almost all AI companies

況且，幾乎所有 AI 公司

are beholden to satisfying their investors,

都有必須滿足的投資者、股東

a short-term incentive which, over time, will inevitably start to conflict

這種必須短期內達成的目標

久而久之， 必定會與善良的本意衝突

with any benevolent mission.

And this wouldn't be a big deal

如果今天只是在討論烤麵包機， 那就沒什麼大不了

if this was really just toasters we're talking about here.

But AI, and especially AGI,

但人工智慧， 尤其是通用人工智慧

is set to be a bigger paradigm shift

將會徹底改變社會型態

than the agricultural or industrial revolutions.

影響大於農業或工業革命

A moment in time so pivotal,

處在轉捩點的我們

it's deserving of reverence and reflection,

必須認真反思這個議題

not something to be reduced to a corporate rat race

別以為這只是企業在互相比較

of who can score the most daily active users.

比誰能搶到更多用戶

我不知道如何在進步與安全中取得平衡

I'm not saying I know

what the right trade-off between acceleration and safety is,

but I do know that we'll never find out what that right trade-off is

但我知道， 如果我們不脫離摩洛圈套

if we let Moloch dictate it for us.

我們永遠不會找到正確的平衡點

So what can we do?

那我們該如何改善現況呢?

Well, the good news is we have managed to coordinate

好消息是， 我們以經在某些部分達成共識

to escape some of Moloch's traps before.

躲掉了一些摩洛的陷阱

We managed to save the ozone layer from CFCs

在蒙特婁議定書的幫助下，

成功的保護臭氧層， 不被氟氯碳化物破壞

with the help of the Montreal Protocol.

We managed to reduce the number of nuclear weapons on Earth

1991年的削減戰略武器條約

by 80 percent,

with the help of the Strategic Arms Reduction Treaty in 1991.

成功減少了八成的核武

So smart regulation may certainly help with AI too,

同理，我們也能用相同的方式規範人工智慧

but ultimately,

但說到底

it's the players within the game who have the most influence on it.

對此事最有影響力的， 就是在AI產業裡的人

So we need AI leaders to show us

因此， 我們必須要求他們證明

that they're not only aware of the risks their technologies pose,

他們不只了解開發AI的潛在風險

but also the destructive nature of the incentives

也知道負面競爭的現況， 對未來百害無一利

that they're currently beholden to.

As their technological capabilities reach towards the power of gods,

如果他們要發展那麼神的科技

they're going to need the godlike wisdom to know how to wield them.

那他們也必須展現神一般的智慧， 才有資格掌控它

So it doesn't fill me with encouragement

因此， 當我聽到某大公司執行長說出:

when I see a CEO of a very major company saying something like,

「我們可把競爭對手逼到跳腳啦!」 這種話時

"I want people to know we made our competitor dance."

That is not the type of mindset we need here.

我並不感到興奮， 那不是我們現在需要的心態

We need leaders who are willing to flip Moloch's playbook,

我們需要能翻轉摩洛圈套的領導者

who are willing to sacrifice their own individual chance of winning

願意犧牲自己勝利的機會

for the good of the whole.

來換得全體福祉

Now, fortunately, the three leading labs are showing some signs of doing this.

幸運的是， 現在有三個頂尖實驗室展現此意

Anthropic recently announced their responsible scaling policy,

Anthropic 最近宣布了他們的責任分級政策 【模型能力越強，安全標準越高】

which pledges to only increase capabilities

該政策承諾，只在符合安全標準時 【模型能力越強，安全標準越高】

once certain security criteria have been met.

才會提升人工智慧的能力 【模型能力越強，安全標準越高】

OpenAI have recently pledged

OpenAI 最近承諾把兩成的運算資源

to dedicate 20 percent of their compute purely to alignment research.

用於確保AI是往對人類有益的方向發展

And DeepMind have shown a decade-long focus

而 DeepMind透過十年的堅持

of science ahead of commerce,

證明比起商業利益，他們更注重科學精神

比如他們將自己開發的AlphaFold

like their development of AlphaFold,

which they gave away to the science community for free.

免費開放給科學社群使用

These are all steps in the right direction,

這些方向是對的

but they are still nowhere close to being enough.

但單靠這些方法仍然不夠。

I mean, most of these are currently just words,

畢竟，這些都只是紙上談兵

they're not even proven actions.

還沒被實際執行

因此，我們需要明確的規則， 確保AI的發展為良性競爭

So we need a clear way to turn the AI race into a definitive race to the top.

Perhaps companies can start competing over who can be within these metrics,

促使公司自主遵守規範

over who can develop the best security criteria.

或誰制定的安全標準最完善。

A race of who can dedicate the most compute to alignment.

比誰能投入最多的運算力於AI對齊上

Now that would truly flip the middle finger to Moloch.

如果成功的話， 就等於是對摩洛豎起了大大的中指

Competition can be an amazing tool,

競爭，可以是一種美好的工具

provided we wield it wisely.

前提是我們明智的運用它

And we're going to need to do that

而我們也必須如此

because the stakes we are playing for are astronomical.

因為這場競爭的風險， 大的難以想像

If we get AI, and especially AGI, wrong,

如果我們錯用 AI， 尤其是通用人工智慧，

it could lead to unimaginable catastrophe.

很有可能鑄下難以彌補的大錯

But if we get it right,

但相反的，如果我們做得正確，

it could be our path out of many of these Moloch traps

AI能成為脫離摩洛圈套的途徑

that I've mentioned today.

接下來這幾年， 情勢大概只會更加瘋狂

And as things get crazier over the coming years,

which they're probably going to,

it's going to be more important than ever

所以我們更需要意識到:

that we remember that it is the real enemy here, Moloch.

真正的敵人， 是摩洛和祂的圈套

Not any individual CEO or company, and certainly not one another.

不是某個企業， 或哪個執行長，更不是你我彼此

So don't hate the players,

所以不要仇視對手

change the game.

要改寫規則

(Applause)

（掌聲）