How we teach computers to understand pictures | Fei Fei Li

Prevoditelj: Senzos Osijek Recezent: Mislav Ante Omazić - EFZG

Let me show you something.

Dopustite da vam pokažem nešto

(Video) Girl: Okay, that's a cat sitting in a bed.

[Video] Djevojka: Dobro, to je mačka koja sjedi na krevetu.

The boy is petting the elephant.

Dječak mazi slona.

Those are people that are going on an airplane.

Ovo su ljudi koji idu u avion.

That's a big airplane.

To je veliki avion.

Fei-Fei Li: This is a three-year-old child

Fei-Fei Li: Ovo je trogodišnje dijete

describing what she sees in a series of photos.

koje opisuje što vidi na ovim slikama.

She might still have a lot to learn about this world,

Iako ima još dosta toga što mora naučiti o svijetu

but she's already an expert at one very important task:

već je ekspert u nečemu važnom:

to make sense of what she sees.

razumije što vidi.

Our society is more technologically advanced than ever.

Naše društvo je tehnološki naprednije no ikada.

We send people to the moon, we make phones that talk to us

Šaljemo ljude na mjesec, izrađujemo telefone koji pričaju s nama

or customize radio stations that can play only music we like.

i prilagođene radio stanice koje puštaju samo glazbu koju volimo.

Yet, our most advanced machines and computers

Ipak, naši najnapredniji uređaj i računala

still struggle at this task.

imaju poteškoća s ovim zadatkom.

So I'm here today to give you a progress report

Ovdje sam danas kako bi vas izvijestila

on the latest advances in our research in computer vision,

o najnovijim dostignućima u istraživanju računalnog vida,

one of the most frontier and potentially revolutionary

jednoj od glavnih i potencijalno revolucionarnih

technologies in computer science.

tehnologija računarstva.

Yes, we have prototyped cars that can drive by themselves,

Imamo prototipe auta koji se sami voze,

but without smart vision, they cannot really tell the difference

ali bez pametnog vida, ne mogu zapravo vidjeti razliku

between a crumpled paper bag on the road, which can be run over,

između zgužvane papirnate vrećice na putu, koju mogu pregaziti,

and a rock that size, which should be avoided.

i kamena te veličine koji treba izbjeći.

We have made fabulous megapixel cameras,

imamo odlične megapikselne kamere,

but we have not delivered sight to the blind.

ali nismo dali vid slijepima.

Drones can fly over massive land,

Dronovi mogu letjeti vrlo daleko

but don't have enough vision technology

ali nemaju dovoljno tehnologije vida

to help us to track the changes of the rainforests.

da nam pomognu pratiti promjene u kišnim šumama

Security cameras are everywhere,

Sigurnosne kamere su svugdje,

but they do not alert us when a child is drowning in a swimming pool.

ali ne upozoravaju nas kada se dijete utaplja u bazenu.

Photos and videos are becoming an integral part of global life.

Slike i videi postaju integralni dio globalnog života.

They're being generated at a pace that's far beyond what any human,

Stvaraju se brzinom koja je daleko veća od od one koji bi čovjek

or teams of humans, could hope to view,

ili timovi ljudi željeli vidjeti,

and you and I are contributing to that at this TED.

a vi i ja pridonosimo tome ovdje na TED-u.

Yet our most advanced software is still struggling at understanding

Ipak naš najnapredniji softver se i dalje muči oko razumjevanja

and managing this enormous content.

i upravljanja tog ogromnog sadržaja.

So in other words, collectively as a society,

Drugim riječima, zajedno kao društvo,

we're very much blind,

poprilično smo slijepi,

because our smartest machines are still blind.

jer su naši najpametniji uređaji i dalje slijepi.

"Why is this so hard?" you may ask.

"Zašto je to tako teško?", možda se pitate.

Cameras can take pictures like this one

Kamere mogu fotografirati slike poput ove

pretvarajući svjetlost u dvodimenzionalne redove brojeva

by converting lights into a two-dimensional array of numbers

poznate kao pikseli,

known as pixels,

but these are just lifeless numbers.

ali to su samo beživotni brojevi.

They do not carry meaning in themselves.

Ne nose smisao u sebi.

Just like to hear is not the same as to listen,

Jednako kao što slušati ne znači isto što i čuti,

to take pictures is not the same as to see,

fotografirati sliku nije isto što i vidjeti,

and by seeing, we really mean understanding.

a pod vidjeti mislimo na razumijevanje.

In fact, it took Mother Nature 540 million years of hard work

Zapravo, prirodi je bilo potrebno 540 milijuna godina teškog posla

to do this task,

da to uspije,

and much of that effort

a većina tog posla

went into developing the visual processing apparatus of our brains,

otišla je u razvijanje uređaja za obradu vida u našem mozgu,

not the eyes themselves.

ne u samim očima.

So vision begins with the eyes,

Vid započinje s očima,

but it truly takes place in the brain.

ali zapravo se sve događa u mozgu.

So for 15 years now, starting from my Ph.D. at Caltech

Već 15 godina, započevši od mog doktorata u Caltech-u

and then leading Stanford's Vision Lab,

i zatim vodeći Stanfordov laboratorij za vid,

I've been working with my mentors, collaborators and students

radila sam s mentorima, suradnicima i studentima

to teach computers to see.

kako bi naučili računala da vide.

Our research field is called computer vision and machine learning.

Naše polje se zove računarni vid i strojno učenje.

It's part of the general field of artificial intelligence.

Dio je većeg polja umjetne inteligencije.

So ultimately, we want to teach the machines to see just like we do:

Naposljetku, želimo naučiti uređaje da vide kao što mi vidimo:

naming objects, identifying people, inferring 3D geometry of things,

imenovanje objekata, prepoznavanje ljudi, razumjevanje trodimenzionalnosti objekata,

understanding relations, emotions, actions and intentions.

razumjevanje odnosa, emocija akcija i namjera.

You and I weave together entire stories of people, places and things

Vi i ja vidimo cijele priče ljudi, mjesta i stvari

the moment we lay our gaze on them.

u trenutku kada ih pogledamo.

The first step towards this goal is to teach a computer to see objects,

Prvi korak do ovog cilja je naučiti računala da vide objekte,

the building block of the visual world.

građevne jedinice vizualnog svijeta.

In its simplest terms, imagine this teaching process

U svom najjednostavnijem obliku, zamislite ovaj proces učenja

as showing the computers some training images

kao pokazivanje računalu raznih prizora za trening

of a particular object, let's say cats,

određenog objekta, recimo mačaka,

and designing a model that learns from these training images.

i dizajniranje modela koji uči iz ovih prikaza za .

How hard can this be?

Koliko teško to može biti?

After all, a cat is just a collection of shapes and colors,

Nakon svega, mačka je samo skup oblika i boja,

and this is what we did in the early days of object modeling.

i ovo je ono što smo radili u početcima modeliranja objekta.

We'd tell the computer algorithm in a mathematical language

Napisali bi računalu algoritme u matematičkom jeziku

that a cat has a round face, a chubby body,

da mačka ima okruglo lice, debeljuškasto tijelo,

two pointy ears, and a long tail,

dva šiljata uha i dugačak rep,

and that looked all fine.

i da izgleda lijepo.

But what about this cat?

ali što je s ovom mačkom?

(Laughter)

(Smijeh)

It's all curled up.

Sva je izvijena.

Now you have to add another shape and viewpoint to the object model.

Sad morate dodati drugi oblik i pogled modelnom objektu.

But what if cats are hidden?

Što ako su mačke skrivene?

What about these silly cats?

Što je sa smiješnim mačkama?

Now you get my point.

Sad vidite što želim reći.

Even something as simple as a household pet

Čak i nešto jednostavno poput kućnog ljubimca

can present an infinite number of variations to the object model,

može imati beskonačan broj varijacija modelnog objekta,

and that's just one object.

i to je samo jedan objekt.

So about eight years ago,

Prije osam godina,

a very simple and profound observation changed my thinking.

vrlo jednostavno i duboko zapažanje promjenilo mi je razmišljanje.

No one tells a child how to see,

Nitko ne govori djetetu kako da vidi,

especially in the early years.

posebno u ranijim godinama.

They learn this through real-world experiences and examples.

Oni to uče kroz iskustvo i primjere iz stvarnog svijeta.

If you consider a child's eyes

Ako smatrate dječje oči

as a pair of biological cameras,

parom bioloških kamera,

they take one picture about every 200 milliseconds,

one fotografiraju svakih 200 milisekundi,

the average time an eye movement is made.

prosječno vrijeme koliko je potrebno za pokret oka.

So by age three, a child would have seen hundreds of millions of pictures

Do svoje treće godine, dijete bi vidjelo stotine milijuna slika

of the real world.

stvarnog svijeta.

That's a lot of training examples.

To je puno primjera za vježbu.

So instead of focusing solely on better and better algorithms,

Umjesto fokusiranja samo na sve bolje i bolje algoritme,

my insight was to give the algorithms the kind of training data

mislila sam dati algoritmima nekakakve podatke za vježbu

that a child was given through experiences

koje je dijete dobijalo kroz iskustva

in both quantity and quality.

i to kvantitativno i kvalitativno.

Once we know this,

Jednom kada znamo ovo,

we knew we needed to collect a data set

znali smo da moramo skupiti skup podataka

that has far more images than we have ever had before,

koji ima puno više prikaza no što smo mi imali ikad prije,

perhaps thousands of times more,

možda i tisuću puta više,

and together with Professor Kai Li at Princeton University,

i zajedno s profesorom Kai Li na sveučilištu Princeton,

we launched the ImageNet project in 2007.

2007. lansirali smo ImageNet projekt.

Luckily, we didn't have to mount a camera on our head

Sva sreća nismo morali montirati kamere na naše glave

and wait for many years.

i čekati godinama.

We went to the Internet,

Otišli smo na Internet,

the biggest treasure trove of pictures that humans have ever created.

najveću riznicu slika koju je čovječanstvo stvorilo.

We downloaded nearly a billion images

skinuli smo skoro milijardu slika i

and used crowdsourcing technology like the Amazon Mechanical Turk platform

koristili crowdsourcing tehnologiju poput platforme Amazon Mechanical Turk

to help us to label these images.

da označimo te prikaze.

At its peak, ImageNet was one of the biggest employers

Kako je raslo, ImageNet je bio jedan od najvećih poslodavaca

of the Amazon Mechanical Turk workers:

radnika Amazon Mechanical Turk-a:

together, almost 50,000 workers

zajedno, skoro 50.000 radnika

from 167 countries around the world

iz 167 država svijeta

helped us to clean, sort and label

pomoglo nam je da očistimo, sortiramo i označimo

nearly a billion candidate images.

skoro milijardu korisnih prikaza.

That was how much effort it took

Toliko truda je trebalo

to capture even a fraction of the imagery

da se uhvati dio prikaza

a child's mind takes in in the early developmental years.

koje djetetov um uhvati u ranim godinama razvoja.

In hindsight, this idea of using big data

Na očigled, ova ideja korištenja mnogo podataka

to train computer algorithms may seem obvious now,

da se istreniraju računalni algoritmi se možda sada čini očiglednim,

but back in 2007, it was not so obvious.

ali 2007., nije bilo tako očigledno.

We were fairly alone on this journey for quite a while.

Prilično dugo bili smo poprilično sami na tom putu.

Some very friendly colleagues advised me to do something more useful for my tenure,

Neke prijateljski nastrojene kolege su me savjetovale da radim nešto korisnije,

and we were constantly struggling for research funding.

i cijelo vrijeme smo se borili za financiranje istraživanja.

Once, I even joked to my graduate students

Jednom, sam se čak našalila sa studentima

that I would just reopen my dry cleaner's shop to fund ImageNet.

da ću ponovno otvoriti kemijsku čistionicu kako bih mogla financirati ImageNet.

After all, that's how I funded my college years.

Naposljetku, tako sam financirala svoj studij.

So we carried on.

Nastavili smo dalje.

In 2009, the ImageNet project delivered

2009. ImageNet je dosegao

a database of 15 million images

bazu podataka od 15 milijuna prikaza

across 22,000 classes of objects and things

preko 22.000 klasa objekata i stvari

organized by everyday English words.

organiziranih u svakodnevne engleske riječi.

In both quantity and quality,

I po kvantiteti i po kvaliteti

this was an unprecedented scale.

ovo je dosad nedostignuta skala.

As an example, in the case of cats,

Kao primjer, u slučaju mačaka,

we have more than 62,000 cats

imamo više od 62.000 mačaka

of all kinds of looks and poses

u svim oblicima i pozama,

and across all species of domestic and wild cats.

i različitih vrsta domaćih i divljih mačaka.

We were thrilled to have put together ImageNet,

Bili smo oduševljeni što smo sastavili ImageNet,

and we wanted the whole research world to benefit from it,

i htjeli smo da cijeli znanstveni svijet ima koristi od njega,

so in the TED fashion, we opened up the entire data set

tako da smo po modi TED-a otvorili cijeli skup podataka

to the worldwide research community for free.

svim istraživačkim zajednicama, besplatno.

(Pljesak)

(Applause)

Now that we have the data to nourish our computer brain,

Sad kad imamo podatke da opskrbimo mozgove naših računala,

we're ready to come back to the algorithms themselves.

spremni smo vratiti se na same algoritme.

As it turned out, the wealth of information provided by ImageNet

Ispalo je kako je bogatstvo informacija s ImageNet-a

was a perfect match to a particular class of machine learning algorithms

savršeno za određene vrste algoritama za strojno učenje

called convolutional neural network,

koji se zovu konvolucijske neuronske mreže

pioneered by Kunihiko Fukushima, Geoff Hinton, and Yann LeCun

osmišljene od strane Kunihiko Fukushime, Geoff Hintona i Yann LeCuna

back in the 1970s and '80s.

davnih 1970-ih i 1980-ih.

Just like the brain consists of billions of highly connected neurons,

Upravo kako se mozak sastoji od milijardu vrlo povezanih neurona,

a basic operating unit in a neural network

osnovna operacijska jedinica neuronskih mreža

is a neuron-like node.

jest čvor sličan neuronu.

It takes input from other nodes

Prima podatke od drugih čvorova

and sends output to others.

i šalje ih drugima.

Moreover, these hundreds of thousands or even millions of nodes

Ove stotine tisuća ili čak milijuni čvorova

are organized in hierarchical layers,

su organizirani po hijerarhijskim slojevima

also similar to the brain.

sličnim onima u mozgu.

In a typical neural network we use to train our object recognition model,

U tipičnoj neuralnoj mreži koju koristimo u učenju prepoznavanja modela,

it has 24 million nodes,

ima 24 milijuna čvorova,

140 million parameters,

140 milijuna parametara,

and 15 billion connections.

i 15 milijardi veza.

That's an enormous model.

To je ogroman model.

Powered by the massive data from ImageNet

Upogonjen je s mnoštvom podataka s ImageNet-a

and the modern CPUs and GPUs to train such a humongous model,

te modernih CPJ-a i GPJ-a kako bi istrenirao ove ogrome modele,

the convolutional neural network

skupna neuronska mreža

blossomed in a way that no one expected.

je procvala na način koji nitko nije očekivao.

It became the winning architecture

Postala je ključna struktura

to generate exciting new results in object recognition.

koja je dovodila do novih uzbudljivih rezultata u prepoznavanju objekata.

This is a computer telling us

Ovo je računalo koje nam govori

this picture contains a cat

da je na slici mačka

and where the cat is.

i gdje je mačka.

Of course there are more things than cats,

Naravno ne radi se samo o mački,

so here's a computer algorithm telling us

ovdje nam računalni algoritam govori

the picture contains a boy and a teddy bear;

da slika sadrži dječaka i medvjedića;

a dog, a person, and a small kite in the background;

psa, osobu i malog zmaja u pozadini;

or a picture of very busy things

ili slika vrlo zbrkanih stvari

like a man, a skateboard, railings, a lampost, and so on.

poput čovjeka, skateboarda, ograde, lampe itd.

Sometimes, when the computer is not so confident about what it sees,

Ponekad kada računalo nije sigurno što vidi,

we have taught it to be smart enough

moramo ga naučiti da bude dovoljno pametno

to give us a safe answer instead of committing too much,

da nam pruži siguran odgovor,

just like we would do,

kao što bismo mi odgovorili,

but other times our computer algorithm is remarkable at telling us

ali u drugim slučajevima računalni alogoritam nam besprijekorno kaže

what exactly the objects are,

što su točno ti objekti,

like the make, model, year of the cars.

poput materijala, modela, godine auta.

We applied this algorithm to millions of Google Street View images

Primjenili smo ovaj algoritam na milijune Google Street View prikaza

across hundreds of American cities,

u stotinama američkih gradova,

and we have learned something really interesting:

i spoznali smo nešto vrlo zanimljivo:

first, it confirmed our common wisdom

prvo, potvrdilo se staro pravilo

that car prices correlate very well

da cijene auta dobro koreliraju

with household incomes.

s kućnim primanjima.

But surprisingly, car prices also correlate well

Ali isto tako cijene auta koreliraju također sa

with crime rates in cities,

stopom kriminala u gradovima,

or voting patterns by zip codes.

ili načina glasanja po poštanskom broju.

So wait a minute. Is that it?

Čekajte. Je li to, to?

Has the computer already matched or even surpassed human capabilities?

Je li nas računalo već sustigao ili čak prestiglo u našim sposobnostima?

Not so fast.

Ne tako brzo.

So far, we have just taught the computer to see objects.

Zasad smo samo naučili računalo da vidi objekte.

This is like a small child learning to utter a few nouns.

To je kao da malo dijete učite reći nekoliko imenica.

It's an incredible accomplishment,

To je ogromno postignuće,

but it's only the first step.

ali je to tek prvi korak.

Soon, another developmental milestone will be hit,

Uskoro će drugo razvojno postignuće biti dosegnuto,

and children begin to communicate in sentences.

i djeca počinju komunicirati u rečenicama.

So instead of saying this is a cat in the picture,

Stoga umjesto govorenja kako je mačka na slici,

you already heard the little girl telling us this is a cat lying on a bed.

već ste čuli malu djevojčicu koja govori da mačka leži na krevetu.

So to teach a computer to see a picture and generate sentences,

Kako bi naučili računalo da vidi sliku i stvori rečenice,

the marriage between big data and machine learning algorithm

brak između velikih podataka i algoritama strojnog učenja

has to take another step.

mora ići korak dalje.

Now, the computer has to learn from both pictures

Računalo mora naučiti učiti i iz slika

as well as natural language sentences

i iz prirodnih jezičnih rečenica

generated by humans.

stvorenih od strane ljudi.

Just like the brain integrates vision and language,

Upravo kako mozak integrira vid i jezik,

we developed a model that connects parts of visual things

razvili smo model koji spaja vidljive dijelove

like visual snippets

poput vidnih komada

with words and phrases in sentences.

s riječima i frazama u rečenicama.

About four months ago,

Otprilike prije četiri mjeseca,

we finally tied all this together

konačno smo uspjelo sve povezati

and produced one of the first computer vision models

i proizveli smo jedan od prvih modela računalnog vida

that is capable of generating a human-like sentence

koji je sposoban stvoriti rečenicu sličnu ljudskoj

when it sees a picture for the first time.

kada vidi sliku po prvi puta.

Now, I'm ready to show you what the computer says

Pokazat ću vam što računalo kaže

when it sees the picture

kada vidi slike

that the little girl saw at the beginning of this talk.

koje je mala djevojčica vidjela na početku govora.

(Video) Computer: A man is standing next to an elephant.

(Video) Računalo: Čovjek stoji pored slona.

A large airplane sitting on top of an airport runway.

Veliki avion sjedi na vrhu avionske piste.

FFL: Of course, we're still working hard to improve our algorithms,

FFL: Naravno, i dalje se trudimo unaprijediti naše algoritme,

and it still has a lot to learn.

i još puno toga mora naučiti.

(Applause)

(Pljesak)

And the computer still makes mistakes.

I računalo i dalje pravi greške.

(Video) Computer: A cat lying on a bed in a blanket.

(Video) Računalo: Mačka leži na krevetu u deci.

FFL: Naravno, kada vidi previše mačaka,

FFL: So of course, when it sees too many cats,

it thinks everything might look like a cat.

misli da bi sve moglo izgledati kao mačka.

(Video) Computer: A young boy is holding a baseball bat.

(Video) Računalo: Dječak drži bejzbolsku palicu.

(Laughter)

(Smijeh)

FFL: Or, if it hasn't seen a toothbrush, it confuses it with a baseball bat.

FFL: Ili, ako nije vidio četkicu za zube, pomiješat će je s bejzbolskom palicom.

(Video) Computer: A man riding a horse down a street next to a building.

(Video) Računalo: Čovjek jaše konja niz ulicu pored zgrade.

(Laughter)

(Smijeh)

FFL: We haven't taught Art 101 to the computers.

FFL: Nismo računalo naučili neke osnove umjetnosti.

(Video) Computer: A zebra standing in a field of grass.

(Video) Računalo: Zebra stoji u polju trave.

FFL: And it hasn't learned to appreciate the stunning beauty of nature

FFL: I nije naučio diviti se prekrasnoj ljepoti prirode

like you and I do.

kao vi i ja.

So it has been a long journey.

Bilo je to dugo putovanje.

To get from age zero to three was hard.

Od rođenja do treće godine je bilo teško.

The real challenge is to go from three to 13 and far beyond.

Pravi izazov je doći od treće do trinaeste godine, i dalje.

Let me remind you with this picture of the boy and the cake again.

Podsjetit ću vas s opet s ovom slikom dječaka i kolača.

So far, we have taught the computer to see objects

Dosad smo naučili računalo da vidi objekte

or even tell us a simple story when seeing a picture.

ili čak nam kaže jednostavnu priču onoga što je na slici.

(Video) Computer: A person sitting at a table with a cake.

(Video) Računalo: Osoba sjedi za stolom s kolačem.

FFL: But there's so much more to this picture

FFL: Ali postoji puno više na ovoj slici

nego samo osoba i kolač.

than just a person and a cake.

What the computer doesn't see is that this is a special Italian cake

Što računalo ne vidi jest da je to poseban talijanski kolač

that's only served during Easter time.

koji se jedino servira za vrijeme Uskrsa.

The boy is wearing his favorite t-shirt

Dječak nosi svoju omiljenu majicu

given to him as a gift by his father after a trip to Sydney,

koju je dobio od oca nakon putovanja u Sidney,

and you and I can all tell how happy he is

i vi i ja možemo reći da je jako stretan

and what's exactly on his mind at that moment.

i što je na njegovom umu u ovom trenu.

This is my son Leo.

To je moj sin Leo.

On my quest for visual intelligence,

Na mom pohodu na vidnu inteligenciju,

I think of Leo constantly

razmišljam o Leu konstantno

and the future world he will live in.

i budućnosti u kojoj će živjeti.

When machines can see,

Kada uređaji vide,

doctors and nurses will have extra pairs of tireless eyes

doktori i sestre će imati dodatan par neumornih očiju

to help them to diagnose and take care of patients.

koje im pomažu dijagnosticirati i pobrinuti se za pacijenta.

Cars will run smarter and safer on the road.

Auti će voziti pametnije i sigurnije na putu.

Robots, not just humans,

Roboti, ne samo ljudi,

will help us to brave the disaster zones to save the trapped and wounded.

će pomoći u opasnim situacijama kako bi spasili zatočene i ozljeđene.

We will discover new species, better materials,

Otkrit ćemo nove vrste, bolje materijale,

and explore unseen frontiers with the help of the machines.

i istražiti neviđene granice uz pomoć uređaja.

Little by little, we're giving sight to the machines.

Malo po malo, dajemo vid uređajima.

First, we teach them to see.

Prvo, smo ih naučili da vide.

Then, they help us to see better.

Onda nam oni pomažu vidjeti bolje.

For the first time, human eyes won't be the only ones

Po prvi put, ljudsko oko neće biti jedino

pondering and exploring our world.

koje gleda i istražuje svijet.

We will not only use the machines for their intelligence,

Nećemo koristiti uređaje zbog njihove inteligencije,

we will also collaborate with them in ways that we cannot even imagine.

surađivat ćemo s njima na načine koje ne možemo zamisliti.

This is my quest:

Ovo je moj pothvat:

to give computers visual intelligence

dati računalima vidnu inteligenciju

and to create a better future for Leo and for the world.

i stvoriti bolje sutra za Lea i za svijet.

Thank you.

Hvala vam.

(Pljesak)

(Applause)