See invisible motion, hear silent sounds. Cool? Creepy? We can't decide | Michael Rubinstein

翻译人员: dahong zhang 校对人员: Donghua Lin

So over the past few centuries, microscopes have revolutionized our world.

过去的几个世纪，显微镜改变了世界。

They revealed to us a tiny world of objects, life and structures

它揭示了肉眼难以看到的

that are too small for us to see with our naked eyes.

物体，生命和结构的微观世界。

They are a tremendous contribution to science and technology.

它对科技进步做出了巨大的贡献。

Today I'd like to introduce you to a new type of microscope,

今天我要向您介绍一类新型显微镜，

a microscope for changes.

观察变化的显微镜。

It doesn't use optics like a regular microscope

它不是通常所说的光学显微镜

to make small objects bigger,

可以放大微小物体，

but instead it uses a video camera and image processing

它利用视频摄像机和图像处理

to reveal to us the tiniest motions and color changes in objects and people,

来揭示物体和人的微小的颜色和运动变化，

changes that are impossible for us to see with our naked eyes.

这些变化是光凭肉眼难以分辨的。

And it lets us look at our world in a completely new way.

它可以让我们以全新角度看世界。

So what do I mean by color changes?

我所说的颜色变化是什么意思呢？

Our skin, for example, changes its color very slightly

举例来说，我们的皮肤，当血液流动时

when the blood flows under it.

皮肤表面的颜色有微弱变化。

That change is incredibly subtle,

变化十分微小，

which is why, when you look at other people,

小到当你观察坐在

when you look at the person sitting next to you,

你身边的人的时候，

you don't see their skin or their face changing color.

你不能看到他脸上皮肤的颜色变化。

When we look at this video of Steve here, it appears to us like a static picture,

我们看看史蒂夫的这段视频，看上去好像是静态图片，

but once we look at this video through our new, special microscope,

但是一旦我们通过我们新的、特别的显微镜看这段视频，

suddenly we see a completely different image.

我们会看到完全不同的影像。

What you see here are small changes in the color of Steve's skin,

您在这里看到的是史蒂夫皮肤颜色微小变化

magnified 100 times so that they become visible.

放大100倍之后的肉眼可见的效果。

We can actually see a human pulse.

我们可以明确的看到人体的脉搏。

We can see how fast Steve's heart is beating,

我们可以看到史蒂夫心脏跳动得多快，

but we can also see the actual way that the blood flows in his face.

我们还可以明确地看到血液是如何流经史蒂夫脸部的。

And we can do that not just to visualize the pulse,

我们不仅可以看到脉搏，

but also to actually recover our heart rates,

还可以推测心跳速率，

and measure our heart rates.

得到我们心跳速率。

And we can do it with regular cameras and without touching the patients.

我们可以通过常用摄像机，不需碰触病人就可以实现。

So here you see the pulse and heart rate we extracted from a neonatal baby

这里您可以看到我们提取的新生儿宝宝的脉搏和心跳速率

from a video we took with a regular DSLR camera,

这是从常规数码单反相机视频中提取的，

and the heart rate measurement we get

我们测出的心率

is as accurate as the one you'd get with a standard monitor in a hospital.

和医院监测器测出的一样精确。

And it doesn't even have to be a video we recorded.

而且我们的软件甚至不需要我们自己专门拍摄的视频。

We can do it essentially with other videos as well.

我们可以对其他视频作同样处理。

So I just took a short clip from "Batman Begins" here

这是我截取《蝙蝠侠：侠影之谜》（Batman Begins）的一小段片断

just to show Christian Bale's pulse.

显示演员克里斯蒂安·贝尔（Christian Bale）的脉搏。

(Laughter)

（笑声）

And you know, presumably he's wearing makeup,

你知道，他应该是化了妆了，

the lighting here is kind of challenging,

灯光也是挑战，

but still, just from the video, we're able to extract his pulse

但是我们同样从这段视频中提取到了他的脉搏

and show it quite well.

演示效果不错。

So how do we do all that?

我们是如何做到的呢？

We basically analyze the changes in the light that are recorded

首先我们分析了录像中每个像素

at every pixel in the video over time,

的光线随时间发生的变化，

and then we crank up those changes.

我们放大这些变化。

We make them bigger so that we can see them.

经过放大使我们肉眼可以看见这些变化。

The tricky part is that those signals,

棘手的部分是，那些信号，

those changes that we're after, are extremely subtle,

那些我们要放大的信号十分微小，

so we have to be very careful when you try to separate them

所以我们必须把它们和视频中存在的噪音

from noise that always exists in videos.

区分出来。

So we use some clever image processing techniques

所以我们应用了一些聪明的影像处理技巧

to get a very accurate measurement of the color at each pixel in the video,

精确的测量每个像素中的颜色，

and then the way the color changes over time,

然后得出颜色随时间的变化，

and then we amplify those changes.

再然后我们放大这些变化。

We make them bigger to create those types of enhanced videos, or magnified videos,

我们可以放大这些变化，增强视频、或放大视频，

that actually show us those changes.

这样就可以演示出变化。

But it turns out we can do that not just to show tiny changes in color,

事实上，我们不仅可以展示颜色的细小变化，

but also tiny motions,

同样也可以展示微小的运动，

and that's because the light that gets recorded in our cameras

这是因为我们的摄像机

will change not only if the color of the object changes,

不仅记录物体的颜色变化，

but also if the object moves.

同样也记录了物体的运动。

So this is my daughter when she was about two months old.

这是我女儿，那时她才两个月大。

It's a video I recorded about three years ago.

这是我三年前拍的。

And as new parents, we all want to make sure our babies are healthy,

作为初为人父母，我们要确保孩子的健康，

that they're breathing, that they're alive, of course.

当然是，确保呼吸，确保活着。

So I too got one of those baby monitors

所以我也买了个婴儿监控器

so that I could see my daughter when she was asleep.

这也我可以在她睡着时看到她。

And this is pretty much what you'll see with a standard baby monitor.

这是标准婴儿监控器所看到的。

You can see the baby's sleeping, but there's not too much information there.

您可以看到孩子睡着了，但没有其他信息了。

There's not too much we can see.

我们看不到更多东西。

Wouldn't it be better, or more informative, or more useful,

可不可以更好点儿，有更多信息，更有用，

if instead we could look at the view like this.

就像这样。

So here I took the motions and I magnified them 30 times,

这里我放大了运动30倍，

and then I could clearly see that my daughter was indeed alive and breathing.

这样我就可以真切地看到我女儿还活着，在呼吸。

(Laughter)

（笑声）

Here is a side-by-side comparison.

这是效果对比。

So again, in the source video, in the original video,

同样，在原始视频上，

there's not too much we can see,

我们看不到什么，

but once we magnify the motions, the breathing becomes much more visible.

但是一旦我放大了运动，呼吸变得肉眼可见了 。

And it turns out, there's a lot of phenomena

事实上，我们用我们的“运动显微镜”

we can reveal and magnify with our new motion microscope.

可以揭示很多现象。

We can see how our veins and arteries are pulsing in our bodies.

我们可以看到体内静脉和动脉的脉动。

We can see that our eyes are constantly moving

我们可以看到我们的眼睛在不断运动

in this wobbly motion.

在晃动中。

And that's actually my eye,

这实际上是我的眼睛，

and again this video was taken right after my daughter was born,

同样，这段视频拍摄于我女儿出生不久之后，

so you can see I wasn't getting too much sleep. (Laughter)

所以你可以看到我睡得不多。（血丝）（笑声）

Even when a person is sitting still,

即使当人在静止坐着，

there's a lot of information we can extract

我们也可以提取许多信息

about their breathing patterns, small facial expressions.

如呼吸节律，面部表情微小变化。

Maybe we could use those motions

也许这可以

to tell us something about our thoughts or our emotions.

告诉我们，我们的想法或情绪。

We can also magnify small mechanical movements,

我们同样也可以方法微小的机械运动，

like vibrations in engines,

如引擎的震动，

that can help engineers detect and diagnose machinery problems,

这可以帮助工程师诊断机械问题，

or see how our buildings and structures sway in the wind and react to forces.

或者看到建筑或结构的随风摇动。

Those are all things that our society knows how to measure in various ways,

我们知道这些变化可以通过其他方法测量，

but measuring those motions is one thing,

但是测量是一回事，

and actually seeing those motions as they happen

肉眼看到的又是

is a whole different thing.

另一回事。

And ever since we discovered this new technology,

自从我们开发者这项新技术，

we made our code available online so that others could use and experiment with it.

我们便把代码放到了网上，这样其他人可以使用试验它。

It's very simple to use.

用起来很简单。

It can work on your own videos.

可以处理你自己拍的视频。

Our collaborators at Quanta Research even created this nice website

我们的合作伙伴，量研科技(Quanta Research)，甚至建了个网站

where you can upload your videos and process them online,

这里可以上传视频并在线处理，

so even if you don't have any experience in computer science or programming,

这样即使你没有计算机科学或编程经验

you can still very easily experiment with this new microscope.

也可以试验这种新型“显微镜”。

And I'd like to show you just a couple of examples

我要演示几个其他的例子

of what others have done with it.

别人用它做了什么。

So this video was made by a YouTube user called Tamez85.

这段视频是一个叫Tamez85的YouTube用户作的

I don't know who that user is,

我不知道他是谁，

but he, or she, used our code

但是他(她)用我们的代码

to magnify small belly movements during pregnancy.

方法了孕妇腹部的运动。

It's kind of creepy.

令人毛骨悚然。

(Laughter)

（笑声）

People have used it to magnify pulsing veins in their hands.

人们用它放在自己的腕部脉搏。

And you know it's not real science unless you use guinea pigs,

你知道要能被称为科学必须用到豚鼠，

and apparently this guinea pig is called Tiffany,

显然这只豚鼠叫蒂芙妮，

and this YouTube user claims it is the first rodent on Earth

这位YouTube用户声称

that was motion-magnified.

这是地球上首个运动被放大的啮齿类动物。

You can also do some art with it.

你也可以用它进行艺术创作。

So this video was sent to me by a design student at Yale.

这是耶鲁大学设计系学生发给我的视频。

She wanted to see if there's any difference

她想看看

in the way her classmates move.

她同学运动的方式有何不同。

She made them all stand still, and then magnified their motions.

她让他们静止站立然后放大他们的运动。

It's like seeing still pictures come to life.

者看上去像是图片有了生命。

And the nice thing with all those examples

这些例子的有意思的地方是

is that we had nothing to do with them.

我们没有进行干预。

We just provided this new tool, a new way to look at the world,

我们只是提供了新工具，一种看世界的新方法，

and then people find other interesting, new and creative ways of using it.

然后人们就找到其他有意思、新的创造性地方法使用这个工具。

But we didn't stop there.

我们不满足于此。

This tool not only allows us to look at the world in a new way,

这个工具不仅使我们有了看世界的新方法，

it also redefines what we can do

同样它还重新定义了

and pushes the limits of what we can do with our cameras.

摄像机的功能限制。

So as scientists, we started wondering,

作为科学家，我们开始思考，

what other types of physical phenomena produce tiny motions

其他物理现象造成的微小振动

that we could now use our cameras to measure?

现在我们可以用摄像机来测量？

And one such phenomenon that we focused on recently is sound.

其中一种现象就是，声音。

Sound, as we all know, is basically changes

声音，我们知道

in air pressure that travel through the air.

声音是一种压力波，依靠空气的压缩变化在空气中传播。

Those pressure waves hit objects and they create small vibrations in them,

压力波碰到物体，就会引起物体本身的微小震动。

which is how we hear and how we record sound.

这就是我们听到了录音的原理。

But it turns out that sound also produces visual motions.

这样，声音也会造成物体的视觉运动。

Those are motions that are not visible to us

这些运动，肉眼难于分辨

but are visible to a camera with the right processing.

但是通过处理摄像机却可见。

So here are two examples.

这里有两个例子。

This is me demonstrating my great singing skills.

这里我展示我伟大的歌唱技巧。

(Singing)

（歌唱）

(Laughter)

（笑声）

And I took a high-speed video of my throat while I was humming.

我哼唱时，我录了一段高速视频。

Again, if you stare at that video,

如果您盯着视频看的话，

there's not too much you'll be able to see,

仅凭肉眼您是看不出什么端倪的，

but once we magnify the motions 100 times, we can see all the motions and ripples

但是当我把运动方法100倍后我们就可以看到颈部的运动和波纹

in the neck that are involved in producing the sound.

这是由于声音震动造成的。

That signal is there in that video.

信号就藏在这段视频中。

We also know that singers can break a wine glass

我们知道，歌唱家可以震碎酒杯

if they hit the correct note.

如果他们发出正确的音符。

So here, we're going to play a note

我们播放一个音符

that's in the resonance frequency of that glass

玻璃杯旁边音箱发出

through a loudspeaker that's next to it.

的共振频率。

Once we play that note and magnify the motions 250 times,

我们播放音符，然后方法运动250倍，

we can very clearly see how the glass vibrates

我们可以清楚的看到

and resonates in response to the sound.

玻璃杯是如何响应声音共振的振动的。

It's not something you're used to seeing every day.

这是我们日常见不到的。

But this made us think. It gave us this crazy idea.

但是这激发了我们思考。给我们一个疯狂的主意。

Can we actually invert this process and recover sound from video

我们能不能通过分析声音在物体上造成的微小振动

by analyzing the tiny vibrations that sound waves create in objects,

从视频中逆向重构出声音来呢，

and essentially convert those back into the sounds that produced them.

重构出原来的声音呢？

In this way, we can turn everyday objects into microphones.

依靠这种方法，我们可以把任何物体变成麦克风。

So that's exactly what we did.

我们真的照做了。

So here's an empty bag of chips that was lying on a table,

这是一个空薯片袋，放在桌上，

and we're going to turn that bag of chips into a microphone

我们要把这个薯片袋变为一个麦克风

by filming it with a video camera

用摄像机拍摄视频

and analyzing the tiny motions that sound waves create in it.

然后分析视频中声音造成的微小振动。

So here's the sound that we played in the room.

这是我们在房间中播放的声音。

(Music: "Mary Had a Little Lamb")

（音乐：“Mary Had a Little Lamb”）

And this is a high-speed video we recorded of that bag of chips.

这是我们摄制的薯片袋的高速视频。

Again it's playing.

同样，

There's no chance you'll be able to see anything going on in that video

您凭肉眼

just by looking at it,

是看不出来什么的，

but here's the sound we were able to recover just by analyzing

但是我们可以通过分析视频中微小的振动

the tiny motions in that video.

恢复出原有的声音。

(Music: "Mary Had a Little Lamb")

（音乐：“Mary Had a Little Lamb”）

I call it -- Thank you.

我叫它 —— 谢谢。

(Applause)

（掌声）

I call it the visual microphone.

我叫它视觉麦克风。

We actually extract audio signals from video signals.

实际上我们从视频信号中提取了音频信号。

And just to give you a sense of the scale of the motions here,

为了让您了解运动的大小，

a pretty loud sound will cause that bag of chips to move less than a micrometer.

一个特别大的声音可以导致薯片袋一微米的变化。

That's one thousandth of a millimeter.

也就是千分之一毫米。

That's how tiny the motions are that we are now able to pull out

这就是我们能够提取出来的微小运动

just by observing how light bounces off objects

仅仅是通过摄像机录制的视频

and gets recorded by our cameras.

观察光线在物体上的变化。

We can recover sounds from other objects, like plants.

我们可以从物体上重构出原声音，如植物。

(Music: "Mary Had a Little Lamb")

（音乐：“Mary Had a Little Lamb”）

And we can recover speech as well.

我们也可以重构出讲话。

So here's a person speaking in a room.

这是一个人在房间中讲话。

Voice: Mary had a little lamb whose fleece was white as snow,

（声音：Mary had a little lamb whose fleece was white as snow,）

and everywhere that Mary went, that lamb was sure to go.

and everywhere that Mary went, that lamb was sure to go.

Michael Rubinstein: And here's that speech again recovered

迈克尔·鲁宾斯坦：这是一个讲话

just from this video of that same bag of chips.

这段视频同样是由那个薯片袋重构出声音。

Voice: Mary had a little lamb whose fleece was white as snow,

声音：Mary had a little lamb whose fleece was white as snow,

and everywhere that Mary went, that lamb was sure to go.

and everywhere that Mary went, that lamb was sure to go.

MR: We used "Mary Had a Little Lamb"

迈克尔·鲁宾斯坦：我们用儿歌“Mary Had a Little Lamb”

because those are said to be the first words

因为托马斯 爱迪生在1877年

that Thomas Edison spoke into his phonograph in 1877.

第一次录制在留声机上的也是这首儿歌。

It was one of the first sound recording devices in history.

这是历史上第一个录音设备。

It basically directed the sounds onto a diaphragm

原理是把声音录在薄膜上

that vibrated a needle that essentially engraved the sound on tinfoil

声音通过振动的针头刻录在锡箔上

that was wrapped around the cylinder.

锡箔被包裹在圆柱体上。

Here's a demonstration of recording and replaying sound with Edison's phonograph.

这是演示爱迪生留声机的录制和回放。

(Video) Voice: Testing, testing, one two three.

（视频）声音：Testing, testing, one two three.

Mary had a little lamb whose fleece was white as snow,

Mary had a little lamb whose fleece was white as snow,

and everywhere that Mary went, the lamb was sure to go.

and everywhere that Mary went, the lamb was sure to go.

Testing, testing, one two three.

Testing, testing, one two three.

Mary had a little lamb whose fleece was white as snow,

Mary had a little lamb whose fleece was white as snow,

and everywhere that Mary went, the lamb was sure to go.

and everywhere that Mary went, the lamb was sure to go.

MR: And now, 137 years later,

迈克尔·鲁宾斯坦：137年过去了，

we're able to get sound in pretty much similar quality

我们可以重构出同样质量的声音

but by just watching objects vibrate to sound with cameras,

却是通过观察摄像机拍摄的物体振动实现的，

and we can even do that when the camera

我们甚至可以把摄像机

is 15 feet away from the object, behind soundproof glass.

放到15英尺外隔音玻璃后边。

So this is the sound that we were able to recover in that case.

这是我们在这种情况下还原的声音。

Voice: Mary had a little lamb whose fleece was white as snow,

声音：Mary had a little lamb whose fleece was white as snow,

and everywhere that Mary went, the lamb was sure to go.

and everywhere that Mary went, the lamb was sure to go.

MR: And of course, surveillance is the first application that comes to mind.

迈克尔·鲁宾斯坦：当然，监控是我们想到的第一个应用。

(Laughter)

（笑声）

But it might actually be useful for other things as well.

但是它应该还有其他用处。

Maybe in the future, we'll be able to use it, for example,

可能，未来我们可以用它

to recover sound across space,

太空中还原声音，

because sound can't travel in space, but light can.

因为声音不可以在太空中传播，但光可以。

We've only just begun exploring

我们仅仅是刚刚开始探索

other possible uses for this new technology.

这项新技术的可能用途。

It lets us see physical processes that we know are there

这让我们熟知的物理过程

but that we've never been able to see with our own eyes until now.

变得肉眼可见了。

This is our team.

这是我们的团队。

Everything I showed you today is a result of a collaboration

今天我展示的一切

with this great group of people you see here,

都是这群伟大的人的协作成果。

and I encourage you and welcome you to check out our website,

我鼓励您，欢迎您访问我们的网站，

try it out yourself,

亲身体验，

and join us in exploring this world of tiny motions.

加入我们，一同探索微小振动的世界。

Thank you.

谢谢。

(Applause)

（掌声）