See invisible motion, hear silent sounds. Cool? Creepy? We can't decide | Michael Rubinstein

譯者: 豪 袞桑 審譯者: Adrienne Lin

So over the past few centuries, microscopes have revolutionized our world.

幾世紀以來， 顯微鏡對世界產生革命性的影響

They revealed to us a tiny world of objects, life and structures

它為我們揭開了微小世界的 物體、生命和結構

that are too small for us to see with our naked eyes.

那些因為太過微小而沒有辦法用肉眼觀察

They are a tremendous contribution to science and technology.

對於科學和科技是個巨大的貢獻

Today I'd like to introduce you to a new type of microscope,

今天我想介紹大家一種新的顯微鏡

a microscope for changes.

一種觀察變化的顯微鏡

It doesn't use optics like a regular microscope

不像一般顯微鏡使用光線

to make small objects bigger,

放大微小的物體

but instead it uses a video camera and image processing

而是使用攝影機和圖像處理

to reveal to us the tiniest motions and color changes in objects and people,

來讓我們看見 人與物的微小運動和顏色變化

changes that are impossible for us to see with our naked eyes.

那些變化是我們沒有辦法用肉眼觀察的

And it lets us look at our world in a completely new way.

讓我們以全新的方式去看世界

So what do I mean by color changes?

而我所說的顏色變化到底是甚麼?

Our skin, for example, changes its color very slightly

舉例來說:皮膚 我們的皮膚會產生極細微的顏色變化

when the blood flows under it.

當我們的血液流過

That change is incredibly subtle,

這個變化是極度細微的

which is why, when you look at other people,

這也是為什麼當大家看向別人時

when you look at the person sitting next to you,

當你看著坐在你身旁的人

you don't see their skin or their face changing color.

你不會看到他臉部皮膚的顏色變化

When we look at this video of Steve here, it appears to us like a static picture,

當我們看到影片中的史蒂夫， 顯示出的是一張靜態的圖片

but once we look at this video through our new, special microscope,

但當我們以新的、特殊的顯微鏡

suddenly we see a completely different image.

突然間看見一個全然不同的影像

What you see here are small changes in the color of Steve's skin,

這裡你們看見的是 史蒂夫臉部皮膚的微小顏色變化

放大一百倍後

magnified 100 times so that they become visible.

變得肉眼可見

We can actually see a human pulse.

我們可以真的看見

人類的脈搏

We can see how fast Steve's heart is beating,

我們可以看見史蒂夫的心跳 是多麼的快速

而且我們也看見 血液實際上如何流過他的臉部

but we can also see the actual way that the blood flows in his face.

And we can do that not just to visualize the pulse,

而且我們不只是將脈搏變得可見

but also to actually recover our heart rates,

還能夠真的獲取我們的心跳

and measure our heart rates.

和測量我們的心跳速率

And we can do it with regular cameras and without touching the patients.

並且我們可以使用普通的鏡頭 而不需要觸碰病人

So here you see the pulse and heart rate we extracted from a neonatal baby

因此在這裡你們可以看見

得自新生嬰兒的脈搏和心跳速率

from a video we took with a regular DSLR camera,

透過我們攝自普通的數位單眼相機

and the heart rate measurement we get

我們測量出心跳速率

is as accurate as the one you'd get with a standard monitor in a hospital.

是與醫院中標準監測設備一樣準確的

And it doesn't even have to be a video we recorded.

而且甚至不需要使用我們拍攝的影片

We can do it essentially with other videos as well.

我們可以實質上使用在其他人拍攝的影片

So I just took a short clip from "Batman Begins" here

因此我擷取了 "蝙蝠俠：開戰時刻"裡的一個片段

just to show Christian Bale's pulse.

來顯示克里斯汀·貝爾的脈搏

(Laughter)

(大笑)

And you know, presumably he's wearing makeup,

而你們知道，他大概有化妝

the lighting here is kind of challenging,

這樣的明暗變化有點挑戰性

but still, just from the video, we're able to extract his pulse

但還是一樣，在這個影片 我們仍能夠取得他的脈搏

and show it quite well.

而且還蠻明顯的

So how do we do all that?

然而我們到底是怎麼做到的?

We basically analyze the changes in the light that are recorded

基本上，我們分析記錄到的光線變化

at every pixel in the video over time,

每一個畫面中的像素

and then we crank up those changes.

然後我們處理這些變化

We make them bigger so that we can see them.

我們放大這些變化讓它們可以被看見

The tricky part is that those signals,

其中最難處理的是那些訊號

those changes that we're after, are extremely subtle,

那些我們想要尋找的變化， 是非常細微的

so we have to be very careful when you try to separate them

所以當我們分離出變化時必須非常細心

from noise that always exists in videos.

因為干擾會一直出現在影像中

So we use some clever image processing techniques

因此我們使用巧妙的圖像處理技巧

to get a very accurate measurement of the color at each pixel in the video,

自影像裡每個像素取得 非常精確的顏色辨識

and then the way the color changes over time,

和隨著時間產生的顏色變化

and then we amplify those changes.

然後我們放大這些變化

We make them bigger to create those types of enhanced videos, or magnified videos,

我們讓變化增大制出改進的影像 或是放大的影像

that actually show us those changes.

而確實地向我們顯示出變化來

But it turns out we can do that not just to show tiny changes in color,

[運動顯微鏡] 但後來我們不僅能展示微小的顏色變化

but also tiny motions,

還能表現出細微的動作

and that's because the light that gets recorded in our cameras

那是因為我們鏡頭紀錄的光線

will change not only if the color of the object changes,

不只有物體顏色上的變化

but also if the object moves.

還有物體的運動

So this is my daughter when she was about two months old.

這是我女兒，當時兩個月大

It's a video I recorded about three years ago.

這是我三年前拍攝的影片

And as new parents, we all want to make sure our babies are healthy,

而且像所有的新手父母親一樣， 我希望我們的小孩健康

that they're breathing, that they're alive, of course.

他們有沒有呼吸，還活著，想當然

So I too got one of those baby monitors

所以我也買了嬰兒監視器

so that I could see my daughter when she was asleep.

讓我能夠在我女兒睡覺時看見她

And this is pretty much what you'll see with a standard baby monitor.

一般嬰兒監視器拍到的 差不多就像這樣

You can see the baby's sleeping, but there's not too much information there.

你知道的，你可以看到嬰兒的睡眠 但資訊顯現的不多

There's not too much we can see.

我們能看見的不多

Wouldn't it be better, or more informative, or more useful,

如果換成這種觀看的視角

if instead we could look at the view like this.

更多的訊息、更有幫助會好很多

So here I took the motions and I magnified them 30 times,

所以我拍攝了這些動作並且放大30倍

and then I could clearly see that my daughter was indeed alive and breathing.

然後我可以清楚看見我的女兒 的確活著而且有呼吸

(Laughter)

(大笑)

Here is a side-by-side comparison.

這裡並列的是比較圖

So again, in the source video, in the original video,

在原始的影像裡

there's not too much we can see,

並沒有太多可以觀察的

but once we magnify the motions, the breathing becomes much more visible.

而一旦經過我們放大動作， 呼吸變得更容易觀察

And it turns out, there's a lot of phenomena

而這表示，許多現象的出現

we can reveal and magnify with our new motion microscope.

透過新式運動顯微鏡我們可以找且放大

We can see how our veins and arteries are pulsing in our bodies.

我們可以觀察我們的動脈和靜脈 如何在我們身體裡脈動

We can see that our eyes are constantly moving

可以看見我們的眼睛是不斷的移動

in this wobbly motion.

不穩定的擺動

And that's actually my eye,

並且那實際上是我的眼睛

and again this video was taken right after my daughter was born,

再次，這個影片拍攝就在我女兒出生之後

so you can see I wasn't getting too much sleep. (Laughter)

所以你看得出我那時候沒甚麼睡 (大笑)

Even when a person is sitting still,

就算一個人坐著

there's a lot of information we can extract

還是有許多的訊息我們能夠獲取

about their breathing patterns, small facial expressions.

他們呼吸的模式、細微的臉部表情

Maybe we could use those motions

說不定我們可以運用這些運動

to tell us something about our thoughts or our emotions.

從而得知我們的想法或情緒

We can also magnify small mechanical movements,

我們可以放大細微的機械運動

like vibrations in engines,

像是引擎的顫動

that can help engineers detect and diagnose machinery problems,

這可以幫助工程師檢測和診斷出機械問題

or see how our buildings and structures sway in the wind and react to forces.

或是我們的大樓和建築物 在風吹、地震影響下的晃動

Those are all things that our society knows how to measure in various ways,

在這個社會中知道 各種不同測量物體的方式

but measuring those motions is one thing,

但這種測量是另一回事

and actually seeing those motions as they happen

真的看見這些運動的發生

is a whole different thing.

是完全不同的事情

And ever since we discovered this new technology,

自我們發現這個新科技後

we made our code available online so that others could use and experiment with it.

我們就將編碼放在網路上 讓其他人能夠使用並測試它

It's very simple to use.

它是非常容易上手的

It can work on your own videos.

它能夠運用在你的影片上

Our collaborators at Quanta Research even created this nice website

我們量子研究的合作人 還架設了一個很好的網站

where you can upload your videos and process them online,

可以讓你上傳影片和在網路上進行處理

so even if you don't have any experience in computer science or programming,

所以即使你沒有任何電腦科學 和寫軟體的經驗

you can still very easily experiment with this new microscope.

還是很容易就能測試新式顯微鏡

And I'd like to show you just a couple of examples

並且在這裡我很樂意秀幾個例子給你們

of what others have done with it.

其他人如何去使用它

So this video was made by a YouTube user called Tamez85.

而這是 YouTube 使用者 Tamez85 所做的影片

I don't know who that user is,

我並不知道使用者是誰

but he, or she, used our code

但他或她，使用了編碼

to magnify small belly movements during pregnancy.

放大懷孕中腹部的細微運動

It's kind of creepy.

是有點詭異

(Laughter)

(大笑)

People have used it to magnify pulsing veins in their hands.

有人使用它去放大手部的靜脈脈博

And you know it's not real science unless you use guinea pigs,

你們也知道只有用天竺鼠測試過 才叫真的科學

and apparently this guinea pig is called Tiffany,

這隻天竺鼠叫做蒂芬妮

and this YouTube user claims it is the first rodent on Earth

這位YouTube使用者表示 這是世界上第一隻囓齒動物

that was motion-magnified.

使用動作放大化

You can also do some art with it.

你還可以加入藝術

So this video was sent to me by a design student at Yale.

這個影片是一位 耶魯大學設計系的學生寄給我的

She wanted to see if there's any difference

她想要看到 她同學的移動方式是否有所差異

in the way her classmates move.

She made them all stand still, and then magnified their motions.

她讓同學們都站直不動 然後放大他們的動作

It's like seeing still pictures come to life.

像把靜止的相片變得有生命

And the nice thing with all those examples

而這些不錯的例子

is that we had nothing to do with them.

完全不是我們做出的

We just provided this new tool, a new way to look at the world,

我們只提供新工具， 一種新的方式去觀察世界

and then people find other interesting, new and creative ways of using it.

然後有人發現一些 有趣、有創意的新方式去使用

But we didn't stop there.

而我們沒有就停在這裡

This tool not only allows us to look at the world in a new way,

這個工具並非只讓我們用新的方式看世界

it also redefines what we can do

它還重新定義我們能夠做的

and pushes the limits of what we can do with our cameras.

並且將相機的使用推向極限

So as scientists, we started wondering,

所以身為科學家，我們開始想

what other types of physical phenomena produce tiny motions

有哪些物理現象會產生細微的運動

that we could now use our cameras to measure?

可以讓我們用相機去測量

And one such phenomenon that we focused on recently is sound.

最近我們專注於聲音這個現象

Sound, as we all know, is basically changes

聲音就我們所知是一種

in air pressure that travel through the air.

大氣壓力的變化通過空氣

Those pressure waves hit objects and they create small vibrations in them,

壓力波動打在物體上使其產生微小的顫動

which is how we hear and how we record sound.

而我們透過它聽見和紀錄聲音

But it turns out that sound also produces visual motions.

結果我們還發現聲音會產生可見的運動

Those are motions that are not visible to us

那些不能用肉眼觀察

but are visible to a camera with the right processing.

但相機可以看見且處理的

So here are two examples.

而這裡是兩個例子

This is me demonstrating my great singing skills.

這裡展示著我厲害的歌唱能力

(Singing)

(唱歌)

(Laughter)

(大笑)

And I took a high-speed video of my throat while I was humming.

而我拍攝了一段當我在唱歌時 我喉嚨的高速影片

Again, if you stare at that video,

再次，如果你注意這個影片

there's not too much you'll be able to see,

並沒有太多東西你能看到的

but once we magnify the motions 100 times, we can see all the motions and ripples

但一旦以 100 倍放大那些動作

我們可以看見運動和波紋

in the neck that are involved in producing the sound.

在脖子裡面，參與聲音的製造

That signal is there in that video.

這個聲音訊息就在這部影片中

We also know that singers can break a wine glass

我們還知道歌手可以唱破酒杯

if they hit the correct note.

假設他們抓到了對的音

So here, we're going to play a note

所以這裡我們要來播放一個音

that's in the resonance frequency of that glass

在玻璃的共振頻率範圍內

through a loudspeaker that's next to it.

透過旁邊的擴音器

Once we play that note and magnify the motions 250 times,

一旦我們放出並且放大動作 250 倍

we can very clearly see how the glass vibrates

可以清晰地看到玻璃是如何顫動

and resonates in response to the sound.

和與聲音的共振

It's not something you're used to seeing every day.

這並非我們平日能夠所見

But this made us think. It gave us this crazy idea.

但這讓我們思考。 給我們一個瘋狂的想法

Can we actually invert this process and recover sound from video

我們能夠反過來處理 並恢復影像中的聲音嗎?

by analyzing the tiny vibrations that sound waves create in objects,

透過分析這些由聲音 影響物體產生的微小顫動

and essentially convert those back into the sounds that produced them.

然後實際上還原出源頭的聲音

In this way, we can turn everyday objects into microphones.

透過這個方式，我們可以 讓所有東西都變成麥克風

So that's exactly what we did.

而這裡是我們實際做的

So here's an empty bag of chips that was lying on a table,

一個空的洋芋片包裝放在桌上

and we're going to turn that bag of chips into a microphone

然後我們要將那包洋芋片 藉由拍攝將它變成我們的麥克風

by filming it with a video camera

and analyzing the tiny motions that sound waves create in it.

並且分析它因為聲音影響 而生的細微運動

So here's the sound that we played in the room.

這是我們在房間裡撥放的聲音

(Music: "Mary Had a Little Lamb")

(音樂：Mary Had a Little Lamb)

And this is a high-speed video we recorded of that bag of chips.

這裡是我們拍攝包裝袋的高速影像

Again it's playing.

聲音在撥放

There's no chance you'll be able to see anything going on in that video

若只是用看的 你無法看見影片中發生甚麼事

just by looking at it,

but here's the sound we were able to recover just by analyzing

但這裡我們能還原聲音

就只依靠分析影片中的細微運動

the tiny motions in that video.

(Music: "Mary Had a Little Lamb")

(音樂：Mary Had a Little Lamb)

I call it -- Thank you.

我稱呼它 -- 謝謝

(Applause)

(掌聲)

I call it the visual microphone.

我稱呼它為視覺麥克風

We actually extract audio signals from video signals.

我們真的從影像訊號中擷取聲音訊號

And just to give you a sense of the scale of the motions here,

而這裡給你了解運動的規模

a pretty loud sound will cause that bag of chips to move less than a micrometer.

響亮的聲音只會讓 洋芋片包裝袋移動不到一微米

That's one thousandth of a millimeter.

那是一千分之一毫米

That's how tiny the motions are that we are now able to pull out

這就是我們現在找出的動作，如此細微

just by observing how light bounces off objects

只透過反射自物體的光線

and gets recorded by our cameras.

並且用我們的相機記錄

We can recover sounds from other objects, like plants.

我們可以從物體上還原聲音，像植物

(Music: "Mary Had a Little Lamb")

(音樂：Mary Had a Little Lamb)

And we can recover speech as well.

還可以還原言語

So here's a person speaking in a room.

使用喇叭，這是一個人在房間裡講話

Voice: Mary had a little lamb whose fleece was white as snow,

聲音：瑪莉有一隻小羊， 牠的毛和雪一樣白

and everywhere that Mary went, that lamb was sure to go.

而不論瑪莉去哪裡，都會帶著小羊

Michael Rubinstein: And here's that speech again recovered

麥可·魯賓斯坦： 這裡是那段話被再次還原

just from this video of that same bag of chips.

正是從這影片中同樣的洋芋片包裝袋

Voice: Mary had a little lamb whose fleece was white as snow,

聲音：瑪莉有一隻小羊， 牠的毛和雪一樣白

and everywhere that Mary went, that lamb was sure to go.

而不論瑪莉去哪裡，都會帶著小羊

MR: We used "Mary Had a Little Lamb"

我們使用 "Mary Had a Little Lamb"

because those are said to be the first words

是因為這據說是愛迪生

that Thomas Edison spoke into his phonograph in 1877.

在 1877 年，第一次對他的留聲機所留

It was one of the first sound recording devices in history.

這是歷史上第一個聲音紀錄裝置

It basically directed the sounds onto a diaphragm

它基本上在一個隔膜上接受聲音

that vibrated a needle that essentially engraved the sound on tinfoil

而隔膜震動了針而雕刻錫板

that was wrapped around the cylinder.

錫板是包裹在圓柱體上

Here's a demonstration of recording and replaying sound with Edison's phonograph.

此處展示的是愛迪生的留聲機 如何錄入和重放

(Video) Voice: Testing, testing, one two three.

(影片)聲音：測試，測試，一二三

Mary had a little lamb whose fleece was white as snow,

瑪莉有隻小羊 羊毛像雪一般白

and everywhere that Mary went, the lamb was sure to go.

而且瑪莉到哪裡，都會帶著小羊

Testing, testing, one two three.

測試，測試，一二三

Mary had a little lamb whose fleece was white as snow,

瑪莉有隻小羊 羊毛像雪一般白

and everywhere that Mary went, the lamb was sure to go.

而且瑪莉到哪裡，都會帶著小羊

MR: And now, 137 years later,

而現在，137 年後的現在

we're able to get sound in pretty much similar quality

我們能夠擷取聲音到非常相似的程度

but by just watching objects vibrate to sound with cameras,

卻只是藉由相機觀察物體對聲音的顫動

and we can even do that when the camera

我們還能做到這件事在相機

is 15 feet away from the object, behind soundproof glass.

離物體 15 英呎

在隔音玻璃外

So this is the sound that we were able to recover in that case.

而這是我們在這個情況下得到的聲音

Voice: Mary had a little lamb whose fleece was white as snow,

聲音：瑪莉有隻小羊 羊毛像雪一般白

and everywhere that Mary went, the lamb was sure to go.

而且瑪莉到哪裡，都會帶著小羊

MR: And of course, surveillance is the first application that comes to mind.

想當然，監控裝置 是第一個被想到的應用方式

(Laughter)

（大笑）

But it might actually be useful for other things as well.

但這可能真的對其他事情很有用

Maybe in the future, we'll be able to use it, for example,

可能在未來我們能夠使用它，舉例來說

to recover sound across space,

在外太空還原聲音

because sound can't travel in space, but light can.

因為聲音不能在太空中傳播而光線可以

We've only just begun exploring

我們才剛開始探索

other possible uses for this new technology.

這個裝置其他的運用可能

It lets us see physical processes that we know are there

它讓看見我們實際知道存在的物理運作

but that we've never been able to see with our own eyes until now.

但直到現在我們才能用肉眼觀察

This is our team.

這是我的團隊

Everything I showed you today is a result of a collaboration

每一個在今天展示出的東西是團隊成果

with this great group of people you see here,

和你在這裡看到的一群偉大的人

and I encourage you and welcome you to check out our website,

而我鼓勵各位並且 歡迎各位看看我們的網站

try it out yourself,

自己試試看

and join us in exploring this world of tiny motions.

並且加入我們一起探索世界中的微小運動

Thank you.

謝謝大家

(Applause)

（掌聲）