See invisible motion, hear silent sounds. Cool? Creepy? We can't decide | Michael Rubinstein

Prevoditelj: Anja Kolobarić Recezent: Ivan Stamenković

So over the past few centuries, microscopes have revolutionized our world.

Tijekom posljednjih stoljeća mikroskopi su promijenili naš svijet.

They revealed to us a tiny world of objects, life and structures

Otkrili su nam maleni svijet predmeta, života i struktura

that are too small for us to see with our naked eyes.

koje su premale da bismo ih vidjeli golim okom.

They are a tremendous contribution to science and technology.

One su ogroman doprinos znanosti i tehnologiji.

Today I'd like to introduce you to a new type of microscope,

Danas vam želim predstaviti novu vrstu mikroskopa -

a microscope for changes.

mikroskop za promjene.

It doesn't use optics like a regular microscope

Ne koristi optiku kao običan mikroskop

to make small objects bigger,

koji uvećava manje objekte,

but instead it uses a video camera and image processing

već koristi video-kameru i prerađuje sliku

to reveal to us the tiniest motions and color changes in objects and people,

ne bi li nam otkrio najmanje pokrete i promjene boje predmeta i ljudi,

changes that are impossible for us to see with our naked eyes.

pokrete koje je nemoguće vidjeti golim okom.

And it lets us look at our world in a completely new way.

to nam omogućava da svijet promotrimo na potpuno nov način.

So what do I mean by color changes?

Na što mislim pod "promjene u boji"?

Our skin, for example, changes its color very slightly

Npr. naša koža gotovo nezamjetno mijenja boju

when the blood flows under it.

dok krv teče ispod nje.

That change is incredibly subtle,

To je izuzetno neprimjetna promjena,

which is why, when you look at other people,

a upravo zato kada gledate druge ljude,

when you look at the person sitting next to you,

kada gledate osobu koja sjedi do vas,

you don't see their skin or their face changing color.

ne vidite kako njezina koža ili njezino lice mijenja boju.

When we look at this video of Steve here, it appears to us like a static picture,

Kada gledamo ovaj video o Steveu, izgleda nam kao statična slika,

but once we look at this video through our new, special microscope,

ali nakon što ga pogledamo kroz naš nov, poseban mikroskop,

suddenly we see a completely different image.

odjednom vidimo sasvim drugačiju sliku.

What you see here are small changes in the color of Steve's skin,

Ovdje vidite male promjene u Steveovoj boji kože

magnified 100 times so that they become visible.

koja je povećana 100 puta ne bi li te promjene postale vidljive.

We can actually see a human pulse.

Možemo zaista vidjeti čovjekov puls.

We can see how fast Steve's heart is beating,

Možemo vidjeti koliko brzo kuca Steveovo srce,

but we can also see the actual way that the blood flows in his face.

ali možemo vidjeti i kako mu točno krv teče u lice.

And we can do that not just to visualize the pulse,

To možemo učiniti ne samo za vizualizaciju pulsa

but also to actually recover our heart rates,

nego i za umirivanje otkucaja srca

and measure our heart rates.

i mjerenje brzine njegova otkucaja.

And we can do it with regular cameras and without touching the patients.

To možemo učiniti pomoću običnih kamera, bez da uopće dotaknemo pacijenta.

So here you see the pulse and heart rate we extracted from a neonatal baby

Ovdje vidite puls i otkucaje srca neonatalne bebe izvučenog

from a video we took with a regular DSLR camera,

iz videa snimljenog običnom DSLR-kamerom,

and the heart rate measurement we get

a broj otkucaja srca jednako je

is as accurate as the one you'd get with a standard monitor in a hospital.

precizan kao i onaj kojeg dobijete klasičnim pregledom u bolnici.

And it doesn't even have to be a video we recorded.

To čak ne mora biti ni video koji smo mi osobno snimili.

We can do it essentially with other videos as well.

Možemo to učiniti i s drugim videima.

So I just took a short clip from "Batman Begins" here

Ovdje imam kratki isječak iz filma "Batman: Početak",

just to show Christian Bale's pulse.

samo da vam pokažem Baleov puls.

(Laughter)

(Smijeh)

And you know, presumably he's wearing makeup,

Samo da znate, vjerojatno je našminkan,

the lighting here is kind of challenging,

osvjetljenje je pravi izazov.

but still, just from the video, we're able to extract his pulse

a ipak, samo na osnovu snimke možemo izvući njegov puls

and show it quite well.

i prilično ga precizno prikazati.

So how do we do all that?

Kako sad to?

We basically analyze the changes in the light that are recorded

Analiziramo snimljene promjene u svjetlu

at every pixel in the video over time,

na svakom snimljenom pikselu

and then we crank up those changes.

i onda ih dodatno pojačamo.

We make them bigger so that we can see them.

Povećamo ih kako bismo ih mogli vidjeti.

The tricky part is that those signals,

Problematično je to što su ti signali,

those changes that we're after, are extremely subtle,

te promjene koje tražimo iznimno suptilni

so we have to be very careful when you try to separate them

i moramo biti jako oprezni kad ih pokušavamo razdvojiti

from noise that always exists in videos.

od buke koje uvijek ima na snimkama.

So we use some clever image processing techniques

Služimo se pametnim tehnikama računalne obrade slike

to get a very accurate measurement of the color at each pixel in the video,

ne bismo li dobili iznimno točna mjerenja boje na svakom pikselu,

and then the way the color changes over time,

a onda i mjerenja postupnih promjena boje

and then we amplify those changes.

koje zatim pojačamo.

We make them bigger to create those types of enhanced videos, or magnified videos,

Povećamo ih ne bismo li stvorili onakve povećane videe

that actually show us those changes.

koji uistinu pokazuju te promjene,

But it turns out we can do that not just to show tiny changes in color,

ali ispada da to nije dobro samo za malene promjene u boji

but also tiny motions,

već i za malene pokrete,

and that's because the light that gets recorded in our cameras

a to je zato što će se svjetlo koje naše kamere snimaju

will change not only if the color of the object changes,

promijeniti ne samo ako se boja predmeta promijeni

but also if the object moves.

već i ako se predmet pokrene.

So this is my daughter when she was about two months old.

Ovo je moja kćer s otprilike dva mjeseca.

It's a video I recorded about three years ago.

Video je snimljen prije neke tri godine.

And as new parents, we all want to make sure our babies are healthy,

Kao mladi roditelji svi mi želimo da nam djeca budu zdrava,

that they're breathing, that they're alive, of course.

da dišu i, naravno, da su živa.

So I too got one of those baby monitors

Stoga sam i ja nabavio "baby monitor"

so that I could see my daughter when she was asleep.

ne bih li pazio na kćer i dok spava.

And this is pretty much what you'll see with a standard baby monitor.

Sve to imate i na klasičnom baby monitoru.

You can see the baby's sleeping, but there's not too much information there.

Vidite da dijete spava, ali nemate nikakvih dodatnih informacija.

There's not too much we can see.

Nema se baš što vidjeti.

Wouldn't it be better, or more informative, or more useful,

Ne bi li bilo bolje, informativnije i korisnije

if instead we could look at the view like this.

kad bismo mogli vidjeti ovo?

So here I took the motions and I magnified them 30 times,

Pokrete sam uvećao trideset puta.

and then I could clearly see that my daughter was indeed alive and breathing.

Po njima sam jasno vidio da mi je kćer živa i da diše.

(Laughter)

(Smijeh)

Here is a side-by-side comparison.

Ovdje imamo usporedni prikaz.

So again, in the source video, in the original video,

Kao što sam već rekao, u originalnom videu

there's not too much we can see,

nema se baš što vidjeti,

but once we magnify the motions, the breathing becomes much more visible.

no kada uvećamo pokrete, disanje postaje uočljivije.

And it turns out, there's a lot of phenomena

Izgleda da ima mnogo fenomena

we can reveal and magnify with our new motion microscope.

koje možemo otkriti i povećati našim novim mikroskopom za pokret.

We can see how our veins and arteries are pulsing in our bodies.

Možemo vidjeti kako nam vene i arterije pulsiraju tijelom.

We can see that our eyes are constantly moving

Možemo vidjeti kako nam se oči

in this wobbly motion.

ljuljuškaju.

And that's actually my eye,

Ovo je moje oko.

and again this video was taken right after my daughter was born,

I video je snimljen odmah nakon rođenja kćeri,

so you can see I wasn't getting too much sleep. (Laughter)

dakle nisam baš puno spavao. (Smijeh)

Even when a person is sitting still,

Čak i kad osoba mirno sjedi,

there's a lot of information we can extract

mnogo informacija možemo izvući

about their breathing patterns, small facial expressions.

iz njezinog disanja i mikrofacijalnih ekspresija.

Maybe we could use those motions

Možda ih možemo iskoristiti

to tell us something about our thoughts or our emotions.

kako bismo saznali nešto o našim mislima ili emocijama.

We can also magnify small mechanical movements,

Možemo povećati i male mehaničke pokrete

like vibrations in engines,

poput vibracija motora koje bi

that can help engineers detect and diagnose machinery problems,

inženjerima pomogle u pronalaženju i dijagnosticiranju mehaničkih problema

or see how our buildings and structures sway in the wind and react to forces.

ili u izučavanju kako se građevine njišu na vjetru i reagiraju na sile.

Those are all things that our society knows how to measure in various ways,

Sve su to stvari koje čovječanstvo može izmjeriti na razne načine,

but measuring those motions is one thing,

ali njihovo je mjerenje jedna stvar,

and actually seeing those motions as they happen

a njihovo promatranje dok se zaista događaju

is a whole different thing.

potpuno je druga stvar.

And ever since we discovered this new technology,

Otkad smo otkrili tu novu tehnologiju,

we made our code available online so that others could use and experiment with it.

objavili smo taj kod online ne bi li ga i drugi mogli isprobati.

It's very simple to use.

Vrlo je jednostavan za uporabu.

It can work on your own videos.

Možete ga koristiti na vlastitim snimkama.

Our collaborators at Quanta Research even created this nice website

Naši suradnici u Quanta Researchu stvorili su ovu zgodnu stranicu

where you can upload your videos and process them online,

gdje možete učitati svoje snimke i tamo ih obrađivati,

so even if you don't have any experience in computer science or programming,

pa čak i ako nemate nikakvog iskustva s računarstvom i programiranjem,

you can still very easily experiment with this new microscope.

možete vrlo lako eksperimentirati s ovim novim mikroskopom.

And I'd like to show you just a couple of examples

Htio bih vam pokazati tek nekoliko primjera

of what others have done with it.

tuđih eksperimenata.

So this video was made by a YouTube user called Tamez85.

Ovaj je video napravio korisnik YT-a pod korisničkim imenom "Tamez85".

I don't know who that user is,

Ne znam tko je to,

but he, or she, used our code

ali on/a je iskoristio naš kod

to magnify small belly movements during pregnancy.

ne bi li uvećao male pokrete trbuha tijekom trudnoće.

It's kind of creepy.

Malo je jezivo.

(Laughter)

(Smijeh)

People have used it to magnify pulsing veins in their hands.

Ljudi su ga koristili ne bi li povećali pulsirajuće vene na rukama.

And you know it's not real science unless you use guinea pigs,

To ne bi bila prava znanost da se nije koristio zamorac,

and apparently this guinea pig is called Tiffany,

ovo je zamorac Tiffany,

and this YouTube user claims it is the first rodent on Earth

a ovaj korisnik YT-a tvrdi da je to prvi glodavac na Zemlji

that was motion-magnified.

čije su kretnje uvećane.

You can also do some art with it.

Možete ga koristiti i u umjetničke svrhe.

So this video was sent to me by a design student at Yale.

Ovo mi je poslala studentica dizajna na Yaleu.

She wanted to see if there's any difference

Htjela je vidjeti postoji li ikakva razlika

in the way her classmates move.

u načinu kretanja njenih kolega.

She made them all stand still, and then magnified their motions.

Natjerala ih je da mirno stoje, a onda je uvećala njihove pokrete.

It's like seeing still pictures come to life.

Kao da gledate kako nepomične slike oživljavaju.

And the nice thing with all those examples

Ono što je dobro sa svim tim primjerima

is that we had nothing to do with them.

jest to da mi nismo imali nikakve veze s njima.

We just provided this new tool, a new way to look at the world,

Mi smo ljudima samo dali taj novi alat, nov način gledanja na svijet,

and then people find other interesting, new and creative ways of using it.

a onda su oni našli nove, zanimljive i kreativne načine njegova korištenja.

But we didn't stop there.

No nismo se zaustavili samo na tome.

This tool not only allows us to look at the world in a new way,

Taj alat ne samo da nam omogućava da svijet promatramo na drugi način

it also redefines what we can do

već i redefinira što možemo raditi

and pushes the limits of what we can do with our cameras.

i pomiče granice toga što možemo raditi s našim kamerama.

So as scientists, we started wondering,

Kao znanstvenici počeli smo se pitati

what other types of physical phenomena produce tiny motions

koji još fizički fenomeni proizvode malene pokrete

that we could now use our cameras to measure?

koje bismo mogli izmjeriti kamerama.

And one such phenomenon that we focused on recently is sound.

Jedan takav fenomen na koji smo se nedavno usredotočili jest zvuk.

Sound, as we all know, is basically changes

Zvuk, kao što već znamo, predstavlja promjene

in air pressure that travel through the air.

u tlaku zraka koje putuju zrakom.

Those pressure waves hit objects and they create small vibrations in them,

Ti tlačni valovi udaraju o predmete i u njima stvaraju malene vibracije,

which is how we hear and how we record sound.

a tako čujemo i snimamo zvukove.

But it turns out that sound also produces visual motions.

No ispada da zvuk proizvodi i vizualne pokrete.

Those are motions that are not visible to us

To su pokreti koji nama nisu vidljivi,

but are visible to a camera with the right processing.

ali jesu vidljivi kameri koja je za to opremljena.

So here are two examples.

Evo dva primjera.

This is me demonstrating my great singing skills.

Tu ja demonstriram svoje sjajne pjevačke sposobnosti.

(Singing)

(Pjevanje)

(Laughter)

(Smijeh)

And I took a high-speed video of my throat while I was humming.

Ubrzao sam video svog grla dok sam pjevušio.

Again, if you stare at that video,

Ako se zagledate u video,

there's not too much you'll be able to see,

nemate tu što vidjeti,

but once we magnify the motions 100 times, we can see all the motions and ripples

ali kad pokrete uvećamo 100 puta, vidimo sve pokrete i mreškanja

in the neck that are involved in producing the sound.

u vratu koja su zadužena za proizvodnju zvuka.

That signal is there in that video.

Taj je signal u videu.

We also know that singers can break a wine glass

Znamo da pjevači mogu razbiti čaše za vino

if they hit the correct note.

ako pogode pravi ton.

So here, we're going to play a note

Sada ćemo odsvirati ton

that's in the resonance frequency of that glass

koja je u istoj frekvenciji kao i čaša

through a loudspeaker that's next to it.

kroz zvučnik pored nje.

Once we play that note and magnify the motions 250 times,

Kad odsviramo taj ton i povećamo pokrete 250 puta,

we can very clearly see how the glass vibrates

jasno vidimo kako čaša vibrira

and resonates in response to the sound.

i odjekuje kao reakcija na zvuk.

It's not something you're used to seeing every day.

To se ne vidi baš svaki dan,

But this made us think. It gave us this crazy idea.

što nas je natjeralo na razmišljanje. Došli smo na jednu ludu ideju.

Can we actually invert this process and recover sound from video

Mogli bismo preokrenuti taj proces i povratiti zvuk iz videa

by analyzing the tiny vibrations that sound waves create in objects,

analizirajući malene vibracije koje zvukovi stvaraju na predmetima

and essentially convert those back into the sounds that produced them.

i ponovno ih pretvoriti u zvukove koji su ih i proizveli.

In this way, we can turn everyday objects into microphones.

Na taj način obične predmete možemo pretvoriti u mikrofone.

So that's exactly what we did.

I učinili smo upravo to.

So here's an empty bag of chips that was lying on a table,

Ovo je prazna vrećica čipsa koja je bila na stolu

and we're going to turn that bag of chips into a microphone

i pretvorit ćemo je u mikrofon

by filming it with a video camera

snimajući je kamerom

and analyzing the tiny motions that sound waves create in it.

i analizirajući majušne pokrete koje stvaraju zvučni valovi.

So here's the sound that we played in the room.

Ovo je sviralo u sobi.

(Music: "Mary Had a Little Lamb")

(Glazba: "Mary Had a Little Lamb")

And this is a high-speed video we recorded of that bag of chips.

Ovo je ubrzani video snimljene vrećice čipsa.

Again it's playing.

Ponovno puštamo.

There's no chance you'll be able to see anything going on in that video

Nema šanse da možete vidjeti da se na tom videu nešto događa

just by looking at it,

samo gledajući ga,

but here's the sound we were able to recover just by analyzing

ali evo zvuka koji smo uspjeli dobiti samo analizirajući

the tiny motions in that video.

malene pokrete na toj snimci.

(Music: "Mary Had a Little Lamb")

(Glazba: "Mary Had a Little Lamb")

I call it -- Thank you.

Ja to nazivam -- hvala vam.

(Applause)

(Pljesak)

I call it the visual microphone.

Nazivam to vizualnim mikrofonom.

We actually extract audio signals from video signals.

Izdvajamo zvučne signale iz videosignala.

And just to give you a sense of the scale of the motions here,

Samo da vam malo približim tu skalu pokreta --

a pretty loud sound will cause that bag of chips to move less than a micrometer.

prilično glasan zvuk pomaknut će vrećicu čipsa za manje od mikrometra.

That's one thousandth of a millimeter.

To je tisućina milimetra.

That's how tiny the motions are that we are now able to pull out

Toliko su maleni pokreti koje sada možemo izvući

just by observing how light bounces off objects

samo promatranjem kako svjetlo odskakuje od predmeta

and gets recorded by our cameras.

i kako ga naše kamere hvataju.

We can recover sounds from other objects, like plants.

Možemo povratiti zvukove i iz drugih predmeta, poput biljaka.

(Music: "Mary Had a Little Lamb")

(Glazba: "Mary Had a Little Lamb")

And we can recover speech as well.

A možemo povratiti i govor.

So here's a person speaking in a room.

Ovdje osoba govori u prostoriji.

Voice: Mary had a little lamb whose fleece was white as snow,

Glas: Marija je imala malo janje čija vuna bijaše bijela poput snijega

and everywhere that Mary went, that lamb was sure to go.

i gdjegod da bi Marija pošla, pošlo bi i janje s njom.

Michael Rubinstein: And here's that speech again recovered

Michael Rubinstein: A evo i tog govora koji smo povratili

just from this video of that same bag of chips.

samo iz videa one iste vrećice čipsa.

Voice: Mary had a little lamb whose fleece was white as snow,

Glas: Marija je imala malo janje čija vuna bijaše bijela poput snijega

and everywhere that Mary went, that lamb was sure to go.

i gdjegod da bi Marija pošla, pošlo bi i janje s njom.

MR: We used "Mary Had a Little Lamb"

MR: Koristili smo "Mary Had a Little Lamb"

because those are said to be the first words

zato što su to prve riječi

that Thomas Edison spoke into his phonograph in 1877.

koje je Thomas Edison 1877. godine izgovorio u svoj fonograf.

It was one of the first sound recording devices in history.

Bio je to jedan od prvih uređaja za snimanje glasa u povijesti.

It basically directed the sounds onto a diaphragm

Uređaj je zvukove preusmjeravao u dijafragmu

that vibrated a needle that essentially engraved the sound on tinfoil

od koje je vibrirala iglica koja je gravirala zvuk u alu-foliju

that was wrapped around the cylinder.

koja je bila omotana oko cilindra.

Here's a demonstration of recording and replaying sound with Edison's phonograph.

Evo demonstracije snimanja i reprodukcije zvuka Edisonovim gramofonom.

(Video) Voice: Testing, testing, one two three.

(Video) Glas: Test, test, jedan, dva, tri.

Mary had a little lamb whose fleece was white as snow,

Marija je imala malo janje čija vuna bijaše bijela poput snijega

and everywhere that Mary went, the lamb was sure to go.

i gdjegod da bi Marija pošla, pošlo bi i janje s njom.

Testing, testing, one two three.

Test, test, jedan, dva, tri.

Mary had a little lamb whose fleece was white as snow,

Marija je imala malo janje čija vuna bijaše bijela poput snijega

and everywhere that Mary went, the lamb was sure to go.

i gdjegod da bi Marija pošla, pošlo bi i janje s njom.

MR: And now, 137 years later,

MR: A sada, 137 godina kasnije

we're able to get sound in pretty much similar quality

možemo dobiti zvuk slične kvalitete,

but by just watching objects vibrate to sound with cameras,

ali putem kamera - promatranjem kako predmeti vibriraju na zvukove,

and we can even do that when the camera

a možemo to raditi i onda kad je kamera

is 15 feet away from the object, behind soundproof glass.

udaljena od predmeta pet metara, iza zvučno izoliranog stakla.

So this is the sound that we were able to recover in that case.

Ovo je zvuk koji smo uspjeli povratiti u jednom takvom slučaju.

Voice: Mary had a little lamb whose fleece was white as snow,

Marija je imala malo janje čija vuna bijaše bila bijela poput snijega

and everywhere that Mary went, the lamb was sure to go.

i gdjegod da bi Marija pošla, pošlo bi i janje s njom.

MR: And of course, surveillance is the first application that comes to mind.

MR: Naravno, odmah se sjetite mogućnosti nadzora.

(Laughter)

(Smijeh)

But it might actually be useful for other things as well.

Ali može biti korisno i za neke druge stvari.

Maybe in the future, we'll be able to use it, for example,

U budućnosti ćemo ga možda moći koristiti i za, npr.

to recover sound across space,

reprodukciju zvukova iz svemira

because sound can't travel in space, but light can.

jer zvuk ne može putovati svemirom, ali svjetlo može.

We've only just begun exploring

Tek smo počeli istraživati

other possible uses for this new technology.

druge mogućnosti korištenja te nove tehnologije.

It lets us see physical processes that we know are there

Daje nam da vidimo fizikalne procese koji znamo da postoje,

but that we've never been able to see with our own eyes until now.

ali koje nismo mogli vidjeti vlastitim očima - sve dosad.

This is our team.

Ovo je naš tim.

Everything I showed you today is a result of a collaboration

Sve što sam vam danas pokazao rezultat je suradnje

with this great group of people you see here,

s ovom sjajnom grupom ljudi koju tu vidite,

and I encourage you and welcome you to check out our website,

a potičem vas da posjetite našu web-stranicu,

try it out yourself,

sami isprobate

and join us in exploring this world of tiny motions.

i pridružite nam se u otkrivanju svijeta mini-pokreta.

Thank you.

Hvala vam.

(Applause)

(Pljesak)