How AI Is Decoding Ancient Scrolls | Julian Schilliger and Youssef Nader | TED

翻译人员: Lening Xu 校对人员: Bruce Wang

We always think about the potential of AI changing the future.

我们一直思考AI改变未来的潜力。

But what about the potential of AI changing the past?

但AI改变过去的潜力呢？

My name is Youssef Nader.

我是优素福·纳德。

I'm an Egyptian AI researcher

是一名埃及人工智能研究员，

and a PhD student at the Free University in Berlin,

也是柏林自由大学的博士生，

and last year, I led the Vesuvius Grand Prize winning team

去年，我带领维苏威火山奖团队

on exploring this very question.

探索了这个问题。

You see, the story starts almost 2,000 years ago.

你看，故事开始于将近两千年前。

A Greek philosopher that we believe was Philodemus of Gadara

一位希腊哲学家, 我们认为是加达拉的菲洛德默斯

sat in one of the many rooms of the Villa dei Papiri.

坐在纸莎草别墅众多房间的一间里。

He talked about music, he talked about pleasure,

谈论音乐，谈论快乐，

he talked about what makes things enjoyable,

他谈到什么让事情变得愉快，

questions that still plague us until today.

直到今天这些问题仍然困扰着我们。

One of his scribes wrote down his thoughts on sheets of papyrus.

他的一位抄写员在纸莎草纸上 写下他的想法。

The sheets were rolled and stowed away for later generations.

这些纸被卷起来存放供后代使用。

Fast-forward 150 years, ...

快进 150 年，

Mount Vesuvius erupts,

维苏威火山爆发，

burying Herculaneum, the villa and the words of the philosopher

将赫库兰尼姆、别墅和哲学家的话

under a sea of hot mud and ashes.

埋在充满热泥和灰烬的海洋下。

Now fast-forward again, to the 17th century.

快进到 17 世纪。

People are excavating around the area.

人们正在该地区四处挖掘。

They found beautiful statues, breathtaking frescoes

他们发现了漂亮的雕像、 令人叹为观止的壁画

and some weird-looking pieces of charcoal,

和一些看上去很奇怪的木炭，

like you see in this picture.

就像你们在这张照片中看到的那样。

This is when the first scrolls were discovered,

这是发现第一卷轴的时候，

and people were racing to excavate more of these.

人们竞相挖掘更多的卷轴。

What knowledge is included that is not known to us now?

卷轴中有哪些 我们现在不知道的知识？

What things should we know about these scrolls?

关于这些卷轴， 我们应该知道什么？

My name is Julian, and I am a digital archaeologist.

我叫朱利安， 是一名数字考古学者。

When the pyroclastic flow hit the scrolls, it had a destructive effect.

当火山灰撞击卷轴时， 产生了破坏性影响。

It tore into them, shredded off pieces, and it charred them badly.

它们被撕成碎片， 然后被烧焦。

Even the deformation that you can see happened at that point.

你看到的变形也是那个时候发生的。

People, 250-something years ago,

大约在 250 年前，

were curious what's lying inside those scrolls,

人们好奇那些卷轴里面隐藏的， 不可见的到底是什么？

hidden and not accessible anymore.

Because of a lack of technology,

由于缺乏技术，

they had to resort to physically unrolling

他们不得不采用物理方式展开，

and thereby destroying most of the scrolls.

从而摧毁大部分卷轴。

To this day,

时至今日，

only the most damaged and deformed scrolls

只有损坏和变形最严重的卷轴

remain in their initial, rolled-up configuration.

保留着初始的卷起状态。

Fast-forwarding a little bit, the computer age arrives.

稍微快一点， 计算机时代就到来了。

Youssef and I are born.

我和优素福出生了。

We are going on and getting our education --

我们正在接受教育——

(Laughter)

（笑声）

and at the same time, Brent Seales, a researcher and professor,

同时，研究员兼教授布伦特·西尔斯

had the idea to use CT scan technology to actually digitize the scrolls,

有了使用CT扫描技术对卷轴 进行数字化的想法，

with the hope of, one day, digitally unrolling them.

希望有一天能以数字方式展开它们。

Behind me, you can see a video of such a CT scan,

在我身后，你可以看到这样 一个CT扫描的视频，

and it goes through the CT scan 3D volume,

它逐层通过CT扫描3D卷轴。

layer by layer.

The papyrus is visible as a spiral,

纸莎草纸呈螺旋状，

and you can see it's tightly wound-up,

你可以看到它紧紧缠绕，

sometimes touching each other, flaying off.

有时会互相粘连，然后脱落。

It's a difficult question, how to unroll this digitally.

这是一个很难回答的问题， 如何以数字方式开启这个卷轴。

Nat Friedman, a Silicon Valley investor,

硅谷投资者纳特·弗里德曼

also saw this research, and he wanted to help.

也看到了这项研究， 他想提供帮助。

That was in 2022.

那是 2022 年。

He reached out, and together with Brent Seales,

他出手与布伦特·西尔斯

they created the Vesuvius Challenge,

一起创建了维苏威挑战赛，

with the goal to motivate nerds all over the world

目标是激励全世界的书呆子

to solve this problem.

来解决这个问题。

(Laughter)

（笑声）

They created a grand prize,

他们设立了大奖，

promising eternal glory and monetary incentives

承诺向任何有能力的人 提供永恒荣耀和金钱激励。

to anyone who could do that.

(Laughter)

（笑声）

I myself saw that on the internet

我本人在网上看到这个消息时，

while writing my master's thesis at ETH Zurich, in robotics,

正在苏黎世联邦理工学院 撰写机器人专业研究生论文，

and I was instantly happy to solve it --

我立刻对解决这个问题产生兴趣——

or at least try, why not, you know?

或者至少尝试一下，为什么不呢？

And I went on, joined the Discord community

我加入了Discord社区，

where all the people that were also contestants

and playing with the scroll data

就卷轴数据交流想法，

were exchanging ideas,

and I joined there and started working on it.

我加入并开始研究这个问题。

Also there, on Discord, I met Youssef and Luke [Farritor],

同样在那里，在 Discord 上， 我遇到了优素福和卢克，

who would become my teammates,

他们将成为我的队友，

and with whom I would actually win the grand prize.

我和他们一起赢得大奖。

Surprisingly, it went on, and made global headline news.

令人惊讶的是， 它将成为全球头条新闻。

It even got into the British tabloids.

甚至登上了英国的小报。

(Laughter)

（笑声）

So when we started,

我们刚开始的时候，

there were two main problems still remaining.

面临两个主要问题。

One, you had to unroll the scroll.

第一，你必须展开卷轴。

And two, you then had to make the ink visible.

第二，然后你必须让墨水可见。

Youssef will tell you more about that part.

优素福会告诉你 更多关于这部分的信息。

For me, the most exciting thing

对我来说，最令人兴奋的是

was the computer-vision problem of unrolling those scrolls virtually.

虚拟地展开这些卷轴的计算机视觉问题。

I decided to iterate on a tool

我决定对肯塔基大学研究员 创建的工具进行迭代，

that was created by the Kentucky researchers,

and make it faster, less prone to errors

让它变得更快，更不容易出错，

and just iterate on it and make it better.

然后对其进行迭代使其变得更好。

The Vesuvius Challenge team saw that

维苏威挑战赛团队看到了这一点，

and also implemented a team of 10 people that would use my tool.

还组建了一个由 10 人组成的团队 来使用我的工具。

They would annotate scroll data, like you see in this video,

他们会为卷轴数据添加注释， 就像你在这段视频中看到的那样，

where they created a red line where the surface would lie.

他们在表面上画了一条红线。

The algorithm then would take it into 3D space,

算法会将其带入三维空间，

creating a three-dimensional representation of the surface.

创建表面的三维展示。

Computer algorithms

计算机算法会将其扁平化 并创建分区。

would then flatten it and create a segment.

This all would be called “segmentation”

在卷轴展开讨论社区中，

in the space of the scrolling and unrolling community.

这被称为 “分割”。

(Laughter)

（笑声）

So I created open-source commits to this tool

我为这个工具创建了开源代码，

and implemented new algorithms from my studies, like Optical Flow,

并根据我的研究实现了新的算法， 比如Optical Flow，

to better track the sheets through the volume,

以更好地通过模型跟踪卷轴，

and we end up with something like what you see behind me.

最后我们得到的结果 就像你在我身后看到的那样。

First off, those were really small segments,

首先，这些都是很小的片段，

and I added improvement,

我进行了改进，

made the code faster

使代码更快，

and had lots of feedback from the community.

并收到了来自社区的大量反馈。

They were really happy, and I was happy getting lots of feedback.

他们真的很高兴， 我也很高兴收到很多反馈。

It was a really positive environment.

这是一个非常积极的环境。

So in the end,

因此，最终，

I could track the performance of the algorithms,

我可以跟踪算法的性能，

how the segmentation team performed,

分割部分的运行，

and I could see that my improvements, from start to finish,

我从始至终可以看到改进，

would be around a 10,000-fold improvement over the initial version.

最终版比初始版本提高约 10,000 倍。

This algorithm was then also used to unroll all the area

该算法还被用来展开所有区域， 你可以在我们提交的内容中看到。

that you can see in our submission.

All the sheets were generated with these methods.

所有表单都是用这些方法生成的。

In December, I was looking for teammates.

十二月时，我一直在寻找队友。

I made a blog post,

我写了一篇博客，

and I showcased my newest algorithms,

展示了我最新的算法，

reaching out to anyone that was willing to team up.

接触任何愿意合作的人。

Youssef and Luke got into contact with me.

优素福和卢克联系了我。

They were happy to team up, and I was happy as well.

他们很高兴组队， 我也很高兴。

(Laughter)

（笑声）

So after the virtual unwrapping, the words still are not visible.

虚拟展开后， 这些文字仍然不可见。

The main problem is that the ink that was used at the time

主要问题是， 当时使用的墨水是碳基墨水，

was a carbon-based ink,

and carbon-based ink on carbon-based papyrus in a CT scan

而在CT扫描中， 碳基纸莎草纸上的碳基墨水

isn’t visible, or at least [not] to the naked eye.

是不可见的，

或者至少肉眼看不见。

So the same team at the University of Kentucky

因此，肯塔基大学的同一个研究小组

decided to test whether the ink was present at all

决定测试CT扫描中是否存在墨水。

in the CT scans.

For this, they took some of the pieces that people broke off the scrolls,

为此，他们采集了 人们从卷轴上拆下的一些碎片，

and they fed them into the same pipeline of the X-ray CT scanning,

然后将它们送入同一个X射线 CT扫描通道中，

and this gives us the 3D data that we were working with.

这为我们提供了 正在处理的三维数据。

Because you can see the ink and it’s an exposed surface,

因为你可以看到墨水， 而且它是裸露的表面，

you can even improve it with infrared imaging.

所以你甚至可以通过红外成像 对其进行改善。

And this gives you a ground truth

这为想找到的字母， 提供了基本的数据。

of what letters you're actually trying to find.

And then from there,

然后，

you can train a machine-learning model to try to find these letters.

你可以训练一个机器学习模型 来尝试找到这些字母。

The way this works

其工作原理是，

is that the model looks at very small cubes at a single time

模型一次只能查看非常小的立方体，

and tries to decide whether there is ink present in this area or not.

并尝试确定该区域是否存在墨水。

And then, when you keep moving this cube all around,

然后，当你继续移动这个立方体时，

the model gets to see different data samples

模型会看到不同的数据样本，

and then tries to understand what ink actually is.

然后尝试了解墨水到底是什么。

So this is how it looks while the model is training.

所以这就是模型训练时的样子。

It's not perfect, but you can see that, especially around the middle,

它并不完美，但你可以看到， 尤其是在中间，

the model is starting to see the letters perfectly.

模型已经开始完美地看到字母了。

So the data is there.

所以数据在那里。

The ink is there.

墨水也在那里。

But it’s just very hard to find and see.

但是很难找到和看见。

Looking at the CT scan raw data on the left here,

看看左边的CT扫描的原始数据，

you can see the fibers,

你可以看到纤维，

you can see the structure of the papyrus,

你可以看到纸莎草纸的结构，

but the letters are very, very faint.

但是字母非常非常微弱。

The letters from the right image are very, very faint in the CT scans.

右侧图像中的字母 在CT扫描中非常非常浅显。

And they're actually, in this special case,

实际上，在这种特殊情况下，

characterized by a difference of contrast

它们的特征是对比度不同，

and some speckles, freckles, features that are very hard to see.

还有一些斑点、雀斑和很难看清的特征。

So what happens if we try to take a look

那么，如果我们试着

at the segment that Julian was just showing?

看看朱利安刚才放映的片段 会发生什么？

So this is the data that we were working with.

这是我们正在使用的数据。

And I'm going to give you 10 seconds to try to find the letters yourself.

我会给你 10 秒钟来尝试 自己找到字母。

(Laughter)

（笑声）

And as a hint,

作为提示，

I'll tell you that there are three letters in this image.

我要告诉你， 这张照片中有三个字母。

Believe me.

相信我。

Try to find some pattern, some crackle patterns,

试着找到一些图案， 一些裂纹图案，

some cracks in there.

里面有一些裂缝。

If you were able to identify this pattern of these three letters --

如果你能识别出这三个字母的图案--

(Laughter)

（笑声）

then congratulations.

恭喜你。

One year ago, you may have won 40,000 dollars.

一年前，你可能赢了 40,000 美元。

(Laughter)

（笑声）

However, if you're like me, and you couldn't make sense of this,

但是，如果你像我一样， 你无法理解这一点，

there's a different way that you can find this ink --

那么你可以通过另一种方式 找到这种墨水——

one that actually scales very, very well.

一种可以很好地缩放的墨水。

So this is where my journey begins with the Vesuvius Challenge.

因此，我的旅程从 维苏威火山挑战赛开始。

There is this neat idea in computer-vision literature

计算机视觉文献中有这样一个好主意，

where if you don't actually have labels,

if you don't have the goal that you want your AI model to reach,

如果没有你想要的人工智能模型 实现的目标，

you can pick an intermediary goal along the way.

你可以选择一个中间目标。

So, looking at these two pairs of images,

因此，看着这两对图像，

our eyes can identify that these are the same images,

我们的眼睛可以识别出 它们是相同的图像，

just flipped.

只需翻转即可。

And we can do that because we understand

我们之所以能做到这一点，

the structures that are present in the images.

是因为我们了解图像中存在的结构。

We can see this little triangle, and it's flipped,

我们可以看到这个小三角形， 它被翻转了，

so we know this is the same triangle.

所以我们知道这是同一个三角形。

Our eyes already have this feature,

我们的眼睛已经有了这个特征，

but neural networks don't.

但是神经网络没有。

When they see these images,

当他们看到这些图像时，

they can't [tell] that these are the same image.

他们不能分辨出 它们是同一张图片。

So one idea, just to let it know about the structures

因此，一个想法是让它知道结构

and familiarize it with the data,

并熟悉数据，

is to show it different views of the same image

就是向它展示同一张图像的不同视图，

and tell it that these are the same images.

并告诉它这些是相同的图像。

And after that, you take this model

之后，你采用这个模型

and you train it like the previous models that the University of Kentucky did.

然后像肯塔基大学以前的模型一样 对其进行训练。

And while the approach doesn't fully work, it also doesn't fully not work.

虽然这种方法并不完全有效， 它也不是完全不起作用。

And this was the first image that was produced by the model.

这是该模型生成的第一张图像。

And there was some very faint signal in there.

里面有一些非常微弱的信号。

It seemed like the model was catching on something,

看来模型正在捕捉一些东西，

but it wasn't clear, exactly, what the model was catching on.

但尚不清楚该模型到底在捕捉什么。

So I decided to take these predictions and create a new ground truth,

因此，我决定根据这些预测 来创造一个新的数据，

asking the model, "Hey, I think these might be letters.

问模型：“嘿，我想这些可能是字母。

I think there's something in there. Try to find more of this."

我想里面有东西。 试着找到更多这样的东西。”

And my ground truth, actually,

实际上，我的真实情况

has four correct letters and four other delusions.

有四个正确的字母和另外四个错觉。

But that was OK.

但那没关系。

So training a new model with this data,

因此，使用这些数据训练一个新模型，

the model started to find more ink, find more letters,

模型开始寻找更多的墨水， 找到更多的字母，

and the lines even looked complete.

甚至线条看起来很完整。

So I thought, "What are the chances that if I do this again,

于是我想，“如果我再这样做，

the models keep improving?"

模型不断改进的可能性有多大？”

And this was the core behind our grand prize-winning solution.

这是我们获奖方案 背后的核心。

Repeating this process over and over, the models kept improving.

一遍又一遍地重复这个过程， 模型不断改进。

The main trick was you needed to prevent the models from memorizing

主要的技巧是 你需要防止模型记住

what the previous models have learned.

以前的模型学到的东西。

You're essentially asking the model to learn

你本质上是在要求模型学习 另一个模型学到了什么。

what the other model has learned.

So overfitting was a serious problem that required a lot of experiments.

因此，过度拟合是一个严重的问题， 需要进行大量实验。

But in the end, getting the recipe right,

但最终，有了正确的配方，

we were able to predict all of these letters

我们得以在模型看不到的情况下 预测所有这些字母。

without the models ever seeing them.

These were the first 10 letters.

这是前 10 个字母。

There are, like, 20 in there,

这里面大概有 20 个，

but this was the first coherent word read from an unopened papyrus sheet.

但这是从一张未展开的纸莎草纸上 读出来的第一个连贯的单词。

From there, scaling the process,

从那以后，

within weeks, we had, now, columns of text,

在几周之内，我们就有了数列文本，

even special characters

模型甚至能够找到， 纸莎学家认为非常有趣的特殊字符，

that papyrologists found very interesting that the model was able to find.

The approach was open-sourced,

这种方法是开源的，

and the data and the code were out there,

数据和代码都在那里，

and the race for the grand prize was on.

大奖的争夺正在进行中。

Recovering four paragraphs at an 85-percent clarity.

恢复四个段落， 清晰度为 85% 。

And the key to our success was perfecting the data and the model

而我们成功的关键 是通过如此多的迭代

with so many iterations and so many experiments.

和大量的实验来完善数据和模型。

In the end, we were able to recover

最终，我们得以恢复

more than 14 columns of text, and 2,000 letters.

超过 14 列文本和 2,000 个字母。

(Applause)

（掌声）

2,000 characters safely stored away two millennia ago.

2,000 个字符安全地存放了两千年。

In just nine months, we discovered them again.

在短短九个月内， 我们再次发现了它们。

AI helped us, in large portions,

人工智能在很大程度上帮助我们

writing better code and even being part in our algorithms.

编写了更好的代码， 甚至参与了我们的算法。

It opened a window into the past.

它打开了一扇通往过去的窗口。

What's next?

下一步是什么？

Let's open this window more.

让我们更多地打开这个窗口。

AI will help us access information that was so far safely locked away.

人工智能将帮助我们访问迄今 为止被安全封锁的信息。

In the words of the author,

用作者的话说：

"We do not refrain from questioning nor understanding,

“我们不回避质疑或理解，

and may it be evident to say true things as they appear."

愿事情真相大白。”

(Applause)

（掌声）