The Dark Side of Competition in AI | Liv Boeree | TED

翻译人员: Yip Yan Yeung 校对人员: suya f.

Competition.

竞争。

It's a fundamental part of human nature.

它是人性的基本组成部分。

I was a professional poker player for 10 years,

我从事职业扑克牌手已有 10 年了，

so I've very much seen all the good, bad and ugly ways it can manifest.

所以我已经亲眼目睹了它所展现的 所有好、坏和丑恶一面。

When it's done right,

如果用在正道，

it can drive us to incredible feats in sports and innovation,

它可以推动我们在体育和 创新方面取得丰功伟绩，

like when car companies compete over who can build the safest cars

就像汽车公司 竞争制造最安全的汽车

or the most efficient solar panels.

或最高效的太阳能电池板。

Those are all examples of healthy competition,

这些都是良性竞争的例子，

because even though individual companies might come and go,

因为尽管个别公司可能会是过客，

in the long run,

但从长远来看，

the game between them creates win-win outcomes

它们之间的博弈会创造双赢的结果，

where everyone benefits in the end.

最终让各方受益。

But sometimes competition is not so great

但有时候竞争并不是那么有益，

and can create lose-lose outcomes where everyone's worse off than before.

可能会造成双输的结果， 每个人的境况都比以前更糟。

Take these AI beauty filters, for example.

以这些 AI 美颜滤镜为例。

As you can see, they're a very impressive technology.

如你所见， 这是一项非常厉害的技术。

They can salvage almost any picture.

它几乎可以拯救任何照片。

They can even make Angelina and Margot more beautiful.

它甚至可以让安吉丽娜 和玛格特更加美丽。

So they're very handy,

它非常方便，

especially for influencers who, now, at the click of a button,

尤其是对网红来说，一键

can transform into the most beautiful Hollywood versions of themselves.

就可以变成自己 最漂亮的好莱坞版本。

But handy doesn't always mean healthy.

但是方便并不总是代表健康。

And I've personally noticed how quickly these things can train you

我亲身体验了这些东西 能有多快训练你

to hate your natural face.

讨厌你的天然长相。

And there's growing evidence that they're creating issues

越来越多的证据表明， 它们正在造成

like body dysmorphia, especially in young people.

容貌焦虑等问题， 尤其是在年轻人中。

Nonetheless, these things are now endemic to social media

尽管如此，这些东西 目前仅限于社交媒体中，

because the nature of the game demands it.

出于竞争的本质。

The platforms are incentivized to provide them

这些平台兴致勃勃地 提供这样的服务，

because hotter pictures means more hijacked limbic systems,

因为更火爆的图片意味着 更能抓住你的兴趣、

which means more scrolling and thus more ad revenue.

更多的滑动屏幕、 更多的广告收入。

And users are incentivized to use them

用户兴致勃勃地使用它们，

because hotter pictures get you more followers.

因为更火爆的图片可以 让你获取更多的粉丝。

But this is a trap,

但这是一个陷阱，

because once you start using these things,

因为你一旦开始使用这些东西，

it's really hard to go back.

就很难回头。

Plus, you don't even get a competitive advantage from them anymore

而且你甚至无法再从中 获得竞争优势，

because everyone else is already using them too.

因为其他人也在使用它们。

So influencers are stuck using these things

网红离不开这些滤镜，

with all the downsides

全是负面影响，

and very little upside.

几乎没有正面影响。

A lose-lose game.

这是一场双输的游戏。

A similar kind of trap is playing out in our news media right now,

我们的新闻媒体 正在落入类似的陷阱，

but with much worse consequences.

但后果要恶劣得多。

You'd think since the internet came along

你可能会认为 自互联网问世以来，

that the increased competition between news outlets

新闻媒体之间竞争的加剧

would create a sort of positive spiral,

将形成一种积极的螺旋式上升，

like a race to the top of nuanced, impartial, accurate journalism.

比如争相向细致、公正、 准确的新闻报道的顶峰进发。

Instead, we're seeing a race to the bottom of clickbait and polarization,

相反，我们看到的是争相 向标题党和“二极管”的低谷进发，

where even respectable papers are increasingly leaning

连颇有名望的报纸都逐渐

into these kind of low-brow partisan tactics.

采取这种令人不齿的 带有倾向性的策略。

Again, this is due to crappy incentives.

同样，这是因为激励措施不当。

Today, we no longer just read our news.

我们现在不只是“看”新闻。

We interact with it by sharing and commenting.

我们还会通过分享、评论与它互动。

And headlines that trigger emotions like fear or anger

引发恐惧或愤怒等情绪的头条新闻

are far more likely to go viral than neutral or positive ones.

比中立或正面的头条新闻 更有可能会被广泛传播。

So in many ways, news editors are in a similar kind of trap

某种程度新闻编辑陷入的陷阱

as the influencers,

和网红差不多，

where, the more their competitors lean into clickbaity tactics,

他们的竞争对手 越采取标题党策略，

the more they have to as well.

他们也不得不这么做。

Otherwise, their stories just get lost in the noise.

否则，他们的故事 就会被纷纷扰扰淹没。

But this is terrible for everybody,

但这对每个人来说都很糟糕，

because now the media get less trust from the public,

因为现在公众对媒体的信任越来越少，

but also it becomes harder and harder for anyone to discern truth from fiction,

而且所有人都越来越难以分辨真伪，

which is a really big problem for democracy.

这对民主来说是一个非常大的问题。

Now, this process of competition gone wrong

竞争过程出错

is actually the driving force behind so many of our biggest issues.

其实在背后推动着 我们的许多重大问题。

Plastic pollution,

塑料污染、

deforestation,

森林砍伐、

antibiotic overuse in farming,

农业过度使用抗生素、

arms races,

军备竞赛、

greenhouse gas emissions.

温室气体排放。

These are all a result of crappy incentives,

这些都是激励措施不当的后果，

of poorly designed games that push their players --

竞争游戏设计不当， 迫使玩家——

be them people, companies or governments --

个人、公司或政府，

into taking strategies and tactics that defer costs and harms to the future.

采取策略和战术 把成本和伤害推迟到以后再还。

And what's so ridiculous is that most of the time,

可笑的是，大多数时候，

these guys don't even want to be doing this.

连这些人自己都不想这样做。

You know, it's not like packaging companies

并不是包装公司 想用塑料填满海洋，

want to fill the oceans with plastic

or farmers want to worsen antibiotic resistance.

也不是农民想加剧抗生素耐药性。

But they’re all stuck in the same dilemma of:

但是他们都陷入了同样的困境：

"If I don't use this tactic,

“如果我不采取这种策略，

I’ll get outcompeted by all the others who do.

我就会被所有这么干的人击败。

So I have to do it, too.”

所以我也必须这样做。”

This is the mechanism we need to fix as a civilization.

这是我们整个文明需要修复的机制。

And I know what you're probably all thinking, "So it's capitalism."

我知道你们可能在想什么， “这就是资本主义。”

No, it's not capitalism.

不，这不是资本主义。

Which, yes, can cause problems,

是的，这可能会导致问题，

but it can also solve them and has been fantastic in general.

但它也可以解决问题， 总体来说很不错。

It's something much deeper.

这是更深层次的东西。

It's a force of misaligned incentives of game theory itself.

这是博弈论本身的不当激励。

So a few years ago, I retired from poker,

几年前，我从扑克届退役，

in part because I wanted to understand this mechanism better.

部分原因是我想更好地理解这种机制。

Because it takes many different forms, and it goes by many different names.

因为它有许多不同的形式， 还有许多不同的名称。

These are just some of those names.

这只是其中的一些名字。

You can see they're a little bit abstract and clunky, right?

你可以看出它们 有点抽象、笨重，对吧？

They don't exactly roll off the tongue.

它们并不完全是胡说八道。

And given how insidious and connected all of these problems are,

由于这些问题的隐藏性和关联性，

it helps to have a more visceral way of recognizing them.

用一种更直观的方式 理解它们会更好一些。

So this is probably the only time

这可能是

you're going to hear about the Bible at this conference.

你在这场会议中 唯一一次听到圣经。

But I want to tell you a quick story from it,

但我想简短地讲一个故事，

because allegedly, back in the Canaanite days,

因为据称，在迦南时代，

there was a cult who wanted money and power so badly,

有一个邪教组织非常渴望金钱和权力，

they were willing to sacrifice their literal children for it.

他们愿意为此牺牲自己的子女。

And they did this by burning them alive in an effigy

为此，他们在神像前 活活烧死自己的孩子，

of a God that they believed would then reward them

他们相信神会奖励他们做出

for this ultimate sacrifice.

这一终极牺牲。

And the name of this god was Moloch.

这尊神的名字叫摩洛克（Moloch）。

Bit of a bummer, as stories go.

故事讲下去就有点扫兴了。

But you can see why it's an apt metaphor,

但是你会明白 为什么这是一个恰当的比喻，

because sometimes we get so lost in winning the game right in front of us,

因为有时我们太过沉迷于 赢得眼前的比赛，

we lose sight of the bigger picture

而在追求胜利时 忽视了大局，牺牲了太多。

and sacrifice too much in our pursuit of victory.

So just like these guys were sacrificing their children for power,

就像这些人为了权力牺牲孩子一样，

those influencers are sacrificing their happiness for likes.

那些网红为了点赞牺牲了快乐。

Those news editors are sacrificing their integrity for clicks,

那些新闻编辑为了点击量 牺牲了自己的诚信，

and polluters are sacrificing the biosphere for profit.

污染者则为了利润 牺牲了生物圈。

In all these examples,

在这些例子中，

the short-term incentives of the games themselves are pushing,

游戏本身的短期激励措施是在鼓动，

they're tempting their players

引诱玩家牺牲越来越多的未来，

to sacrifice more and more of their future,

trapping them in a death spiral where they all lose in the end.

陷入死亡螺旋中，最终全盘皆输。

That's Moloch's trap.

那是摩洛克的陷阱。

The mechanism of unhealthy competition.

不良竞争的机制。

And the same is now happening in the AI industry.

AI 行业中也正在上演。

We're all aware of the race that's heating up

我们都知道公司间的竞争正热火朝天，

between companies right now

over who can score the most compute,

竞争最高算力、

who can get the biggest funding round or get the top talent.

最高融资、最优人才。

Well, as more and more companies enter this race,

随着越来越多公司加入竞争，

the greater the pressure for everyone to go as fast as possible

每个人都面临着更大的压力， 迫使他们尽可能快地动起来，

and sacrifice other important stuff like safety testing.

牺牲安全测试等重要事项。

This has all the hallmarks of a Moloch trap.

这就是摩洛克陷阱的所有特征。

Because, like, imagine you're a CEO who, you know, in your heart of hearts,

因为想象一下你是一位 CEO， 你内心深处

believes that your team is the best

相信你的团队是

to be able to safely build extremely powerful AI.

能够安全地打造 极其强大的 AI 的最佳人选。

Well, if you go too slowly, then you run the risk of other,

如果你动作太慢，那你就会面临其他

much less cautious teams getting there first

不那么谨慎的团队先行一步，

and deploying their systems before you can.

在你之前部署他们的系统。

So that in turn pushes you to be more reckless yourself.

因此，这反过来又会 促使你自己更冲动行事。

And given how many different experts and researchers,

有多少专家和研究人员，

both within these companies

包括这些公司的职员，

but also completely independent ones,

还有完全独立的人士，

have been warning us about the extreme risks of rushed AI,

一直在警告我们警惕 仓促发展的 AI 带来的极端风险，

this approach is absolutely mad.

这种做法简直太疯狂了。

Plus, almost all AI companies

此外，几乎所有 AI 公司

are beholden to satisfying their investors,

都必须满足投资者的要求，

a short-term incentive which, over time, will inevitably start to conflict

这种短期激励措施随着时间的推移， 将不可避免地开始

with any benevolent mission.

与任何“圣母”使命发生冲突。

And this wouldn't be a big deal

如果我们在这里谈论的 真的只是面包机，

if this was really just toasters we're talking about here.

那也没什么大不了的。

But AI, and especially AGI,

但是，AI，尤其是 AGI （通用人工智能），

is set to be a bigger paradigm shift

必将带来比农业 或工业革命更大的范式转变。

than the agricultural or industrial revolutions.

A moment in time so pivotal,

这是个关键时刻，

it's deserving of reverence and reflection,

需要敬意和反思，

not something to be reduced to a corporate rat race

不能简化成企业间你死我活的竞争，

of who can score the most daily active users.

比拼谁能获取更多的日活跃用户。

I'm not saying I know

我并不是说我知道

what the right trade-off between acceleration and safety is,

加速发展和安全之间 权衡取舍的正确答案，

but I do know that we'll never find out what that right trade-off is

但我确实知道， 如果我们受制于摩洛克，

if we let Moloch dictate it for us.

我们将永远找不到 正确的权衡取舍。

So what can we do?

那我们能做些什么？

Well, the good news is we have managed to coordinate

好消息是我们之前合力成功

to escape some of Moloch's traps before.

逃脱了一些摩洛克陷阱。

We managed to save the ozone layer from CFCs

通过《蒙特利尔议定书》，

with the help of the Montreal Protocol.

我们从氯氟碳化物中拯救了臭氧层。

We managed to reduce the number of nuclear weapons on Earth

通过 1991 年的《削减和 限制进攻性战略武器条约》

by 80 percent,

我们将地球上的核武器数量 减少了 80％。

with the help of the Strategic Arms Reduction Treaty in 1991.

So smart regulation may certainly help with AI too,

明智的监管措施当然 也可能会对 AI 有所帮助，

but ultimately,

但归根结底，

it's the players within the game who have the most influence on it.

对 AI 影响最大的是 游戏中的玩家。

So we need AI leaders to show us

因此，我们需要 AI 领袖向我们表明

that they're not only aware of the risks their technologies pose,

他们不仅意识到了 其技术构成的风险，

but also the destructive nature of the incentives

还有他们目前所面临的 激励措施的破坏性。

that they're currently beholden to.

As their technological capabilities reach towards the power of gods,

当他们的技术能力 匹敌神明的力量时，

they're going to need the godlike wisdom to know how to wield them.

他们需要神一般的智慧 学习如何驾驭它们。

So it doesn't fill me with encouragement

因此，当我看到一家 大型公司的 CEO 说

when I see a CEO of a very major company saying something like,

“我想让人们知道对手对我们言听计从” 之类的话时，我并不觉得信心倍增。

"I want people to know we made our competitor dance."

That is not the type of mindset we need here.

这不是我们需要的那种心态。

We need leaders who are willing to flip Moloch's playbook,

我们需要愿意 推翻摩洛克故事的领导者，

who are willing to sacrifice their own individual chance of winning

他们愿意为了整体利益 牺牲自己的获胜机会。

for the good of the whole.

Now, fortunately, the three leading labs are showing some signs of doing this.

所幸有三家头部实验室 都或多或少展现了这种迹象。

Anthropic recently announced their responsible scaling policy,

Anthropic 最近宣布了 负责任的扩展政策，

which pledges to only increase capabilities

该政策承诺只有在满足 特定安全标准后才会提升性能。

once certain security criteria have been met.

OpenAI have recently pledged

OpenAI 最近承诺

to dedicate 20 percent of their compute purely to alignment research.

将其 20% 的算力 完全用于“对齐”研究。

And DeepMind have shown a decade-long focus

DeepMind 已经在过去十年中表现出

of science ahead of commerce,

对科学的关注优先于商业，

like their development of AlphaFold,

例如他们开发的 AlphaFold，

which they gave away to the science community for free.

将其免费赠与科学界。

These are all steps in the right direction,

这些都是朝着正确方向迈出的步伐，

but they are still nowhere close to being enough.

但还远远不够。

I mean, most of these are currently just words,

我想说目前大都只是言语，

they're not even proven actions.

甚至不是经过验证的行动。

So we need a clear way to turn the AI race into a definitive race to the top.

我们需要一种明确的方法

将 AI 竞赛转变为明确向上的竞争。

Perhaps companies can start competing over who can be within these metrics,

也许各公司可以开始 争夺谁可以达到这些指标，

over who can develop the best security criteria.

谁可以制定出最佳的安全标准。

A race of who can dedicate the most compute to alignment.

竞赛谁能将最多的算力投入到对齐上。

Now that would truly flip the middle finger to Moloch.

这确实是与摩洛克陷阱 背道而驰的做法。

Competition can be an amazing tool,

如果我们明智地运用竞争， 它会是一个强大的工具。

provided we wield it wisely.

And we're going to need to do that

我们之所以需要这样做，

because the stakes we are playing for are astronomical.

是因为我们手上的赌注是天文数字。

If we get AI, and especially AGI, wrong,

如果我们让 AI， 尤其是 AGI，跑偏了，

it could lead to unimaginable catastrophe.

可能会导致难以想象的灾难。

But if we get it right,

但如果我们做对了，

it could be our path out of many of these Moloch traps

可能就会是我们摆脱多次提及的 摩洛克陷阱的出路。

that I've mentioned today.

And as things get crazier over the coming years,

接下来几年， 情况会越来越疯狂，

which they're probably going to,

确实可能如此，

it's going to be more important than ever

我们必须记住摩洛克 才是我们真正的敌人。

that we remember that it is the real enemy here, Moloch.

Not any individual CEO or company, and certainly not one another.

不是哪个 CEO，哪家公司， 当然也不是彼此。

So don't hate the players,

不要憎恨玩家，

change the game.

去改变游戏吧。

(Applause)

（掌声）