Can a robot pass a university entrance exam? | Noriko Arai

翻译人员: heng gao 校对人员: Jack Zhang

Today, I'm going to talk about AI and us.

今天我将和大家聊聊人工智能和我们

AI researchers have always said

曾经，人工智能方面的研究人员 总是告诉我们，

that we humans do not need to worry,

我们人类不需要担心，

because only menial jobs will be taken over by machines.

因为机器只能从事体力劳动，

Is that really true?

可真的是那样吗？

They have also said that AI will create new jobs,

他们还说人工智能可以创造 新的工作机会，

so those who lose their jobs will find a new one.

因此，那些因其失去工作的人 可以找到新的工作。

Of course.

这当然对的。

But the real question is:

但是，真正的问题是：

How many of those who may lose their jobs to AI

那些因为人工智能出现 而失去工作的人中

will be able to land a new one,

有多少人能够找到一份新的工作？

especially when AI is smart enough to learn better than most of us?

尤其是，当人工智足够聪明到 比大多数人类做得更好时。

Let me ask you a question:

我想问大家一个问题：

How many of you think

你们中有多少人认为

that AI will pass the entrance examination of a top university by 2020?

到2020年，人工智能可以通过 顶尖大学的入学考试。

Oh, so many. OK.

哦，好多。好的

So some of you may say, "Of course, yes!"

你们中有人可能会说： "当然了，这是肯定的！“

Now singularity is the issue.

现在的问题是什么时候 会到达这个科技奇点。

And some others may say, "Maybe,

其他一些人会说：”可能吧，

because AI already won against a top Go player."

因为人工智能已经赢了 一个顶级的围棋选手。“

And others may say, "No, never. Uh-uh."

然后其他人可能说： ”不会的，这不可能发生。“

That means we do not know the answer yet, right?

这就说明我们现在其实并不知道答案 对不对？

So that was the reason why I started Todai Robot Project,

这就是我为什么发起了 “东大”机器人项目，

making an AI which passes the entrance examination

创造一个可以通过日本顶尖大学

of the University of Tokyo,

（东京大学）的入学考试

the top university in Japan.

的机器人。

This is our Todai Robot.

这就是我们的“东大”机器人。

And, of course, the brain of the robot is working in the remote server.

当然，机器人的大脑是在远程服务器中工作。

It is now writing a 600-word essay

图中他正在写一篇600字的短文，

on maritime trade in the 17th century.

主题是17世纪的海运贸易。

How does that sound?

这听起来怎么样？

Why did I take the entrance exam as its benchmark?

为什么我要选择入学考试作为 他的衡量标准呢？

Because I thought we had to study the performance of AI

因为，我认为我们要研究 人工智能的表现，

in comparison to humans,

就一定要将它与人类进行对比

especially on the skills and expertise

尤其是在那些我们认为只有人类可以

which are believed to be acquired only by humans

而且只能通过教育，

and only through education.

才能获得的技能和知识。

To enter Todai, the University of Tokyo,

为了进入东大——东京大学

you have to pass two different types of exams.

你需要通过两个不同类型的考试。

The first one is a national standardized test

第一个是全国标准测试，

in multiple-choice style.

考试是以多项选择的形式。

You have to take seven subjects

你需要考7个科目，

and achieve a high score --

并且获得一个较高的分数。

I would say like an 85 percent or more accuracy rate --

我估计大概有84%的人， 准确的说或者有更多，

to be allowed to take the second stage written test

能有机会参加第二阶段

prepared by Todai.

由东大准备的笔试。

So let me first explain how modern AI works,

首先，让我向大家解释一下 现代人工智能是如何工作的，

taking the "Jeopardy!" challenge as an example.

就以”危险边缘！“挑战游戏作例子。

Here is a typical "Jeopardy!" question:

这有一个典型的”危险边缘！" 游戏问题：

"Mozart's last symphony shares its name with this planet."

“莫扎特的最后一部交响曲与这颗星球 有相同的名字。”

Interestingly, a "Jeopardy!" question always asks,

有趣的是，“危险边缘！”游戏

总是以提问的方式出现，而且总是以 “这个”什么东西作为结尾。

always ends with "this" something:

"this" planet, "this" country,

这颗星球，这个国家，

"this" rock musician, and so on.

这个摇滚明星，等等。

In other words, "Jeopardy!" doesn't ask many different types of questions,

换而言之，“危险边缘！"游戏 并不会问不同类型的问题，

but a single type,

而是只有一种形式，

which we call "factoid questions."

我们称之为“事实问答”。

By the way, do you know the answer?

对了，你们知道这个问题的答案吗？

If you do not know the answer and if you want to know the answer,

如果你们不知道这个问题的答案， 但是如果你们想知道答案，

what would you do?

你们会怎么做？

You Google, right? Of course.

你们会用谷歌搜索，对不对？ 当然是这样。

Why not?

为什么不呢？

But you have to pick appropriate keywords

但是，你必须选择合适的关键字，

like "Mozart," "last" and "symphony" to search.

比如“莫扎特”，“最后”， “交响曲”来进行搜索。

The machine basically does the same.

人工智能基本上也是这样做的。

Then this Wikipedia page will be ranked top.

然后关于这个的维基百科页面 会被排在第一个位置。

Then the machine reads the page.

接下来，机器将阅读这个页面，

No, uh-uh.

不，不是这样。

Unfortunately, none of the modern AIs,

不幸的是，没有任何现存的人工智能

including Watson, Siri and Todai Robot,

包括 Watson（IBM创造的人工智能）, Siri 和东大机器人，

is able to read.

能够进行阅读。

But they are very good at searching and optimizing.

但是，他们非常擅长搜索和 寻找最优解。

It will recognize

他们可以识别到一个位置，

that the keywords "Mozart," "last" and "symphony"

这些关键字 “莫扎特”，“最后”和 “交响曲”

are appearing heavily around here.

在这个位置附近出现的频率非常高。

So if it can find a word which is a planet

因此，如果他可以在这个位置附近

and which is co-occurring with these keywords,

找到一个代表星球的单词，

that must be the answer.

那么这个词就一定是答案了。

This is how Watson finds the answer "Jupiter," in this case.

因此，Watson 找到了答案是“木星”。

Our Todai Robot works similarly, but a bit smarter

我们的东大机器人工作方式与这个相似，但稍稍聪明点，

in answering history yes-no questions,

对于回答历史方面的“是-否”问题时，

like, "'Charlemagne repelled the Magyars.' Is this sentence true or false?"

比如，“查理曼击退了马扎尔人。” 这句话是真的还是假的？

Our robot starts producing a factoid question,

我们的机器人自己首先 会提出一个事实问题，

like: "Charlemagne repelled [this person type]" by itself.

比如：“查理曼击退【……人】”

Then, "Avars" but not "Magyars" is ranked top.

然后，如果是“阿尔瓦人”而不是 “马扎儿人”被排在了第一个。

This sentence is likely to be false.

这句话就很可能就是错误的。

Our robot does not read, does not understand,

我们的机器人并不会阅读， 也无法理解，

but it is statistically correct in many cases.

但是在统计意义上 大多数情况下是对的。

For the second stage written test,

对于第二阶段的笔试，

it is required to write a 600-word essay like this one:

要求考试者写一篇600字的短文，比如这个：

[Discuss the rise and fall of the maritime trade

【讨论一下十七世纪东亚和东南亚

海运贸易的兴起和衰落……】

正如我之前展示的，

in East and Southeast Asia in the 17th century ...]

and as I have shown earlier,

我们的机器人从课本和维基百科上 摘抄句子，

our robot took the sentences from the textbooks and Wikipedia,

combined them together,

并将它们组合在一起，

and optimized it to produce an essay

然后以最优解的方式生成一篇短文，

without understanding a thing.

尽管他自己完全不理解 自己写的是啥。

(Laughter)

（笑声）

But surprisingly, it wrote a better essay

但令人吃惊的是，他比大多数学生

than most of the students.

写的文章还要好。

(Laughter)

（笑声）

How about mathematics?

那把人格智能应用到数学方面会怎么样呢？

A fully automatic math-solving machine

自从“人工智能”这个词诞生开始，

has been a dream

完全自动的解决数学问题的机器，

since the birth of the word "artificial intelligence,"

就一直是人们梦寐以求的。

but it has stayed at the level of arithmetic for a long, long time.

但是他已经停留在只能做算术的水平 很长时间了。

Last year, we finally succeeded in developing a system

去年，我们终于成功开发了一个系统

which solved pre-university-level problems from end to end,

它可以从头到尾求解

like this one.

像这样的大学入学水准的题目。

This is the original problem written in Japanese,

这道题是由日语写的，

and we had to teach it 2,000 mathematical axioms

我们需要教它2000个数学公理，

and 8,000 Japanese words

和8000日文单词，

to make it accept the problems written in natural language.

它才能识别用我们用日文写的问题。

And it is now translating the original problems

现在他正将原始的问题

into machine-readable formulas.

翻译成机器可识别的公式。

Weird, but it is now ready to solve it, I think.

这听起来有点怪， 但我想它现在可以解这道题了。

Go and solve it.

开始解这道题。

Yes! It is now executing symbolic computation.

好的，现在它在正在进行符号计算。

Even more weird,

这可能看起来可能更奇怪。

but probably this is the most fun part for the machine.

但对于这台机器而言， 这可能是最有趣的部分。

(Laughter)

（笑声）

Now it outputs a perfect answer,

现在它输出了一份完美的答案。

though its proof is impossible to read, even for mathematicians.

尽管他的证明过程对数学家来说 也是不可能读懂的。

Anyway, last year our robot was among the top one percent

但不管怎么说，我们的机器人去年在 第二阶段的数学笔试中，

in the second stage written exam in mathematics.

排名前1%。

(Applause)

（鼓掌）

Thank you.

谢谢！

So, did it enter Todai?

那么，他考入东大了吗？

No, not as I expected.

没有，并不象我期望的那样。

Why?

为什么呢？

Because it doesn't understand any meaning.

因为它无法理解任何意义。

Let me show you a typical error it made in the English test.

让我给你们展示一个它在英文考试中 犯的一个典型错误。

[Nate: We're almost at the bookstore. Just a few more minutes.

【Nate：我们再过几分钟就要到书店了。

Sunil: Wait. ______ . Nate: Thank you! That always happens ...]

Sunil: 等一下。_______. Nate: 谢谢！ 这经常发生……。】

Two people are talking.

两个人在谈话。

For us, who can understand the situation --

对于我们这些能够理解语境的人，

[1. "We walked for a long time." 2. "We're almost there."

A.”我们走了很长时间了。“ B. ”我们马上到那了。“

3. "Your shoes look expensive." 4. "Your shoelace is untied."]

C.”你的鞋看起来很贵。“ D.”你的鞋带开了。“

it is obvious number four is the correct answer, right?

很明显D是正确答案，对吧？

But Todai Robot chose number two,

但是，东大机器人选了B答案。

even after learning 15 billion English sentences

而且还是在使用深度学习技术

using deep learning technologies.

学习了150亿个英文句子之后。

OK, so now you might understand what I said:

好的， 你们现在应该能理解我之前说的：

modern AIs do not read,

现代人工智能不能阅读，

do not understand.

无法理解意义。

They only disguise as if they do.

他们只是装作他们好像能这样做。

This is the distribution graph

这是一张50万学生成绩的分布图，

of half a million students who took the same exam as Todai Robot.

他们参加了和东大机器人一样的考试。

Now our Todai Robot is among the top 20 percent,

我们的东大机器人排在前20%，

and it was capable to pass

而且他现在可以通过

more than 60 percent of the universities in Japan --

超过60%的日本大学的考试，

but not Todai.

除了东京大学。

But see how it is beyond the volume zone

但是，我们可以看到 他处在能够成为白领的人群比例

of to-be white-collar workers.

之上。

You might think I was delighted.

你们可能认为我应该感到高兴。

After all, my robot was surpassing students everywhere.

毕竟我的机器人 超过了多有地方的学生。

Instead, I was alarmed.

但是相反地，我感到恐慌。

How on earth could this unintelligent machine outperform students --

这个不智能的机器到底为什么 比我们的学生，我们的孩子，

our children?

表现的还好？

Right?

是不是？

I decided to investigate what was going on in the human world.

我决定调查一下我们这到底是为什么。

I took hundreds of sentences from high school textbooks

我从高中的课本中摘了几百段句子，

and made easy multiple-choice quizzes,

制作了简单的多项选择问卷，

and asked thousands of high school students to answer.

然后让上千名学生回答。

Here is an example:

这是有一个例子：

[Buddhism spread to ... , Christianity to ... and Oceania,

[佛教传播到....， 基督教传播到....和大洋洲，

and Islam to ...]

伊斯兰教传播到....，]

Of course, the original problems are written in Japanese,

当然，问题都是用日文写的，

their mother tongue.

他们的母语。

[ ______ has spread to Oceania.

[ ________传播到了大洋洲。

1. Hinduism 2. Christianity 3. Islam 4. Buddhism ]

A.印度教 B. 基督教 C.伊斯兰教 D. 佛教】

Obviously, Christianity is the answer, isn't it?

很明显，答案是基督教，对不对？

It's written!

上边写着呢。

And Todai Robot chose the correct answer, too.

而东大机器人也选择了正确的答案。

But one-third of junior high school students

但是有1/3的初中学生

failed to answer this question.

并没有回答对。

Do you think it is only the case in Japan?

你认为这种情况只发生在日本吗？

I do not think so,

我不这么认为，

because Japan is always ranked among the top in OECD PISA tests,

因为日本人总是在经合组织的 国际学生评估测试中名列前茅。

measuring 15-year-old students' performance in mathematics,

这个项目每3年

science and reading

会评估15岁学生

every three years.

在数学、科学和阅读方面的表现。

We have been believing

我们一直相信

that everybody can learn

每个人都能够学习

and learn well,

而且能够学好，

as long as we provide good learning materials

只要我们在网上提供

free on the web

免费的，好的学习材料，

so that they can access through the internet.

这样他们就可以通过网络 获取这些资料。

But such wonderful materials may benefit only those who can read well,

但是这些优质的学习资料可能 只会使那些理解能力强的人受益，

and the percentage of those who can read well

而那些理解能力强的人所占的比例

may be much less than we expected.

可能远远比我们认为的要少。

How we humans will coexist with AI

基于真实的证据，

我们将如何与人工智能和平共存?

is something we have to think about carefully,

是我们人类需要认真思考一个问题。

based on solid evidence.

At the same time, we have to think in a hurry

同时，我们要赶快思考这个问题，

because time is running out.

因为，留给我们的时间十分有限。

Thank you.

谢谢！

(Applause)

（鼓掌）

Chris Anderson: Noriko, thank you.

Chris Anderson: Noriko, 谢谢你。

Noriko Arai: Thank you.

Noriko Arai: 谢谢！

CA: In your talk, you so beautifully give us a sense of how AIs think,

CA: 你的出色的演讲， 让我们感受到了人工智能是如何思考

what they can do amazingly

它们能够做什么令人吃惊的事，

and what they can't do.

以及什么事情是它们做不了。

But -- do I read you right,

但是，我不知道我理解的对不对

that you think we really need quite an urgent revolution in education

你认为我们在教育方面急切地 需要进行改革，

to help kids do the things that humans can do better than AIs?

以帮助孩子们从事那些人类能够比 人工智能做的更好的工作。

NA: Yes, yes, yes.

NA:是， 是的。

Because we humans can understand the meaning.

因为我们人类能够理解其中的意义。

That is something which is very, very lacking in AI.

这对人工智能来说非常缺乏的。

But most of the students just pack the knowledge

但是，大多数学生仅仅是 被动的接受知识（填鸭式教育），

without understanding the meaning of the knowledge,

而并没有理解知识背后的意义。

so that is not knowledge, that is just memorizing,

因此，那些根本不是知识， 那些只是记忆而已。

and AI can do the same thing.

而人工智能也能做同样的事。

So we have to think about a new type of education.

因此，我们要思考一种 新的教育方式。

CA: A shift from knowledge, rote knowledge, to meaning.

CA: 从死记硬背转向理解意义。

NA: Mm-hmm.

NA: 嗯嗯。

CA: Well, there's a challenge for the educators. Thank you so much.

CA: 好的，这是对现场教育者 的一个挑战。非常感谢你。

NA: Thank you very much. Thank you.

NA： 谢谢你们。

(Applause)