3 ways to spot a bad statistic | Mona Chalabi

Prevodilac: Ivana Krivokuća Lektor: Tijana Mihajlović

I'm going to be talking about statistics today.

Danas ću govoriti o statistici.

If that makes you immediately feel a little bit wary, that's OK,

Ako se zbog toga odmah osećate pomalo obazrivo, to je u redu;

that doesn't make you some kind of crazy conspiracy theorist,

to vas ne čini ludim teoretičarem zavere,

it makes you skeptical.

već vas čini skeptičnim.

And when it comes to numbers, especially now, you should be skeptical.

A kada se radi o brojevima, pogotovo sada, treba da budete skeptični.

But you should also be able to tell which numbers are reliable

Ali, takođe bi trebalo da možete da prepoznate

koji brojevi su pouzdani, a koji nisu.

and which ones aren't.

So today I want to try to give you some tools to be able to do that.

Danas želim da probam da vam pružim izvesna pomagala da biste to umeli.

But before I do,

Pre nego što to uradim,

I just want to clarify which numbers I'm talking about here.

želim samo da razjasnim o kojim brojevima ovde govorim.

I'm not talking about claims like,

Ne govorim o tvrdnjama poput:

"9 out of 10 women recommend this anti-aging cream."

„Devet od deset žena preporučuje ovu kremu protiv bora.“

I think a lot of us always roll our eyes at numbers like that.

Mislim da veliki broj nas uvek prevrće očima na takve brojeve.

What's different now is people are questioning statistics like,

Ono što je sada drugačije je što ljudi dovode u pitanje podatke poput:

"The US unemployment rate is five percent."

„Nezaposlenost u SAD je pet procenata.“

What makes this claim different is it doesn't come from a private company,

Ono po čemu je ova tvrdnja drugačija je to što ne proističe iz privatne firme,

it comes from the government.

već iz vlade.

Oko četiri od deset Amerikanaca ne veruje ekonomskim podacima

About 4 out of 10 Americans distrust the economic data

that gets reported by government.

o kojima izveštava vlast.

Among supporters of President Trump it's even higher;

Među pristalicama predsednika Trampa, taj broj je još veći; oko sedam od deset.

it's about 7 out of 10.

I don't need to tell anyone here

Ne treba nikome ovde da pričam

that there are a lot of dividing lines in our society right now,

da u našem društvu danas postoji mnogo linija razdvajanja,

and a lot of them start to make sense,

a mnoge počinju da imaju smisla

kada razumete odnos ljudi prema tim vladinim brojevima.

once you understand people's relationships with these government numbers.

On the one hand, there are those who say these statistics are crucial,

Sa jedne strane, tu su oni koji kažu da su ovi podaci od ključnog značaja,

that we need them to make sense of society as a whole

da su nam potrebni da bi nam društvo kao celina imalo smisla,

in order to move beyond emotional anecdotes

kako bismo prevazišli emocionale anegdote

and measure progress in an [objective] way.

i merili napredak na objektivan način.

And then there are the others,

Zatim, tu su oni drugi,

koji kažu da su ovi podaci elitistički,

who say that these statistics are elitist,

maybe even rigged;

možda čak i namešteni;

they don't make sense and they don't really reflect

da nemaju smisla i ne odražavaju zaista

what's happening in people's everyday lives.

ono što se dešava u svakodnevnom životu ljudi.

It kind of feels like that second group is winning the argument right now.

Nekako deluje da druga grupa trenutno pobeđuje u raspravi.

We're living in a world of alternative facts,

Živimo u svetu alternativnih činjenica,

where people don't find statistics this kind of common ground,

gde ljudi ne smatraju statističke podatke nekom vrstom zajedničke osnove,

this starting point for debate.

početnom tačkom za debatu.

This is a problem.

To je problem.

There are actually moves in the US right now

Trenutno zapravo postoje pokreti u SAD

to get rid of some government statistics altogether.

da se potpuno otarasimo nekih vladinih statističkih podataka.

Right now there's a bill in congress about measuring racial inequality.

Baš sada postoji predlog zakona u kongresu o merenju rasne nejednakosti.

The draft law says that government money should not be used

Nacrt zakona kaže da novac vlade ne treba koristiti

za prikupljanje podataka o rasnoj segregaciji.

to collect data on racial segregation.

This is a total disaster.

To je potpuna katastrofa.

If we don't have this data,

Ako nemamo ove podatke,

how can we observe discrimination,

kako možemo da posmatramo diskriminaciju,

let alone fix it?

a kamoli da je popravimo?

In other words:

Drugim rečima,

How can a government create fair policies

kako vlast može da stvara pravednu politiku

ako ne može da izmeri trenutni nivo nepravednosti?

if they can't measure current levels of unfairness?

This isn't just about discrimination,

Ovde se ne radi samo o diskriminaciji, već o svemu; razmislite o tome.

it's everything -- think about it.

Kako možemo donositi zakone o zdravstvenoj zaštiti

How can we legislate on health care

if we don't have good data on health or poverty?

ako nemamo dobre podatke o zdravlju ili siromaštvu?

How can we have public debate about immigration

Kako možemo javno debatovati o imigraciji ako se ne možemo makar složiti

if we can't at least agree

on how many people are entering and leaving the country?

oko toga koliko ljudi ulazi u zemlju i izlazi iz nje?

Statistics come from the state; that's where they got their name.

Statistički podaci proističu iz države; tako su dobili svoje ime.

The point was to better measure the population

Poenta je bila da se dobiju bolje mere stanovništva

in order to better serve it.

kako bi mu se bolje služilo.

So we need these government numbers,

Dakle, potrebni su nam ti vladini brojevi,

but we also have to move beyond either blindly accepting

ali treba i da prevaziđemo njihovo slepo prihvatanje,

or blindly rejecting them.

kao i slepo odbacivanje.

We need to learn the skills to be able to spot bad statistics.

Potrebne su nam veštine da bismo mogli da uočimo loše podatke.

I started to learn some of these

Počela sam da ih stičem

when I was working in a statistical department

kada sam radila na odeljenju za statistiku

that's part of the United Nations.

koje je deo Ujedinjenih nacija.

Our job was to find out how many Iraqis had been forced from their homes

Naš posao je bio da saznamo koliko Iračana je proterano iz svojih domova

as a result of the war,

kao posledica rata

and what they needed.

i šta im je potrebno.

To je bio zaista važan posao, ali, takođe, neverovatno težak.

It was really important work, but it was also incredibly difficult.

Svakoga dana smo donosili odluke

Every single day, we were making decisions

koje su uticale na tačnost naših brojeva -

that affected the accuracy of our numbers --

decisions like which parts of the country we should go to,

odluke poput toga u koje delove zemlje bi trebalo da idemo,

who we should speak to,

sa kim treba da razgovaramo, koja pitanja treba da postavljamo.

which questions we should ask.

And I started to feel really disillusioned with our work,

Počela sam da se osećam zaista razočarano u vezi sa našim radom,

because we thought we were doing a really good job,

jer smo mislili da zaista dobro obavljamo posao,

but the one group of people who could really tell us were the Iraqis,

ali ona grupa ljudi koja bi stvarno mogla da nam ispriča stvari bili su Iračani,

and they rarely got the chance to find our analysis, let alone question it.

a oni su retko imali priliku da naiđu na našu analizu,

a kamoli da je preispituju.

So I started to feel really determined

Zato sam postala zaista odlučna

that the one way to make numbers more accurate

da je jedini način da postignemo da brojevi budu tačniji

is to have as many people as possible be able to question them.

da omogućimo da što više ljudi može da ih preispituje.

So I became a data journalist.

Tako sam postala novinarka koja se bavi podacima.

My job is finding these data sets and sharing them with the public.

Moj posao je da pronađem skupove podataka i podelim ih sa javnošću.

Anyone can do this, you don't have to be a geek or a nerd.

Bilo ko to može, ne morate biti štreber ili bubalica.

Možete zanemariti te reči; koriste ih ljudi

You can ignore those words; they're used by people

koji pokušavaju da kažu da su pametni dok se pretvaraju da su skromni.

trying to say they're smart while pretending they're humble.

Apsolutno svako to može.

Absolutely anyone can do this.

I want to give you guys three questions

Htela bih da vam postavim tri pitanja

that will help you be able to spot some bad statistics.

koja će vam pomoći da možete da primetite loše statističke podatke.

So, question number one is: Can you see uncertainty?

Dakle, pitanje broj jedan glasi: možete li da uočite nepouzdanost?

One of things that's really changed people's relationship with numbers,

Jedna od stvari koja je zaista promenila odnos ljudi prema brojevima,

and even their trust in the media,

pa čak i njihovo poverenje u medije,

has been the use of political polls.

bilo je korišćenje političkih anketa.

I personally have a lot of issues with political polls

Lično imam mnogo problema sa političkim anketama

because I think the role of journalists is actually to report the facts

jer smatram da je uloga novinara da izveštava o činjenicama,

and not attempt to predict them,

a ne da pokušava da ih predvidi,

especially when those predictions can actually damage democracy

naročito kada ta previđanja mogu da naškode demokratiji

davanjem signala ljudima da se ne trude da glasaju za nekog

by signaling to people: don't bother to vote for that guy,

jer nema šanse.

he doesn't have a chance.

Stavimo to po strani za sada i popričajmo o preciznosti ovog nastojanja.

Let's set that aside for now and talk about the accuracy of this endeavor.

Na osnovu državnih izbora

Based on national elections in the UK, Italy, Israel

u Ujedinjenom Kraljevstvu, Italiji, Izraelu

and of course, the most recent US presidential election,

i, naravno, najskorijih predsedničkih izbora u SAD,

using polls to predict electoral outcomes

korišćenje anketa za predviđanje ishoda izbora

is about as accurate as using the moon to predict hospital admissions.

otprilike je tačno kao korišćenje meseca za predviđanje prijema u bolnice.

No, seriously, I used actual data from an academic study to draw this.

Ne, ozbiljno, koristila sam stvarne podatke iz akademske studije

da bih ovo nacrtala.

There are a lot of reasons why polling has become so inaccurate.

Postoji mnogo razloga zašto je anketiranje postalo tako netačno.

Our societies have become really diverse,

Naša društva su postala veoma raznolika,

which makes it difficult for pollsters to get a really nice representative sample

što otežava anketarima da dobiju fini reprezentativni uzorak stanovništva

of the population for their polls.

za svoje ankete.

People are really reluctant to answer their phones to pollsters,

Ljudi se nerado javljaju na telefon anketarima,

and also, shockingly enough, people might lie.

a, takođe, ko bi rekao, ljudi mogu da slažu.

But you wouldn't necessarily know that to look at the media.

Međutim, to nećete nužno znati pogledavši medije.

For one thing, the probability of a Hillary Clinton win

Između ostalog, o verovatnoći pobede Hilari Klinton

was communicated with decimal places.

izveštavano je pomoću decimalnih brojeva.

We don't use decimal places to describe the temperature.

Ne koristimo decimale da izrazimo temperaturu.

How on earth can predicting the behavior of 230 million voters in this country

Kako, pobogu, predviđanje ponašanja 230 miliona glasača u ovoj zemlji

be that precise?

može biti tako precizno?

And then there were those sleek charts.

Zatim, tu su bili oni doterani grafikoni.

See, a lot of data visualizations will overstate certainty, and it works --

Vidite, mnogo vizualizacije podataka će prenaglasiti sigurnost, i to deluje -

these charts can numb our brains to criticism.

ovi grafikoni mogu da otupe naš mozak za kriticizam.

When you hear a statistic, you might feel skeptical.

Kada čujete podatak, možda ćete biti skeptični.

As soon as it's buried in a chart,

Čim je upakovan u grafikon,

it feels like some kind of objective science,

čini se kao nekakva objektivna nauka,

and it's not.

a nije.

So I was trying to find ways to better communicate this to people,

Zato sam pokušavala da pronađem načine da ovo bolje prenesem ljudima,

to show people the uncertainty in our numbers.

da im pokažem nesigurnost u našim brojevima.

What I did was I started taking real data sets,

Počela sam da uzimam stvarne skupove podataka

and turning them into hand-drawn visualizations,

i pretvaram ih u vizualizacije nacrtane rukom,

so that people can see how imprecise the data is;

tako da ljudi mogu da vide koliko su podaci neprecizni;

tako da mogu da vide da je to uradilo ljudsko biće,

so people can see that a human did this,

a human found the data and visualized it.

čovek je našao podatke i vizualizovao ih.

For example, instead of finding out the probability

Na primer, umesto saznavanja verovatnoće

of getting the flu in any given month,

da dobijete grip u bilo kom mesecu,

you can see the rough distribution of flu season.

možete videti grubu raspodelu sezone gripa.

This is --

Ovo je -

(Laughter)

(Smeh)

a bad shot to show in February.

loš grafikon za pokazivanje u februaru.

But it's also more responsible data visualization,

Međutim, takođe je odgovornija vizualizacija podataka,

because if you were to show the exact probabilities,

jer ako biste pokazali tačne verovatnoće,

možda bi to podstaklo ljude da dobiju vakcine protiv gripa

maybe that would encourage people to get their flu jabs

u pogrešno vreme.

at the wrong time.

The point of these shaky lines

Svrha ovih nesigurnih linija

is so that people remember these imprecisions,

je da ljudi upamte te nepreciznosti,

but also so they don't necessarily walk away with a specific number,

ali i da ne ponesu nužno sa sobom određeni broj,

but they can remember important facts.

već da mogu zapamtiti važne činjenice.

Facts like injustice and inequality leave a huge mark on our lives.

Činjenice poput toga da nepravde i nejednakosti

ostavljaju veliki trag u našem životu.

Facts like Black Americans and Native Americans have shorter life expectancies

Činjenice poput toga da američki crnci i Indijanci imaju kraći životni vek

od pripadnika drugih rasa,

than those of other races,

and that isn't changing anytime soon.

a to se neće u skorije vreme promeniti.

Facts like prisoners in the US can be kept in solitary confinement cells

Činjenice poput toga da se zatvorenici u SAD mogu držati u samicama

that are smaller than the size of an average parking space.

koje su manje od veličine prosečnog mesta za parkiranje.

The point of these visualizations is also to remind people

Svrha ovih vizualizacija takođe je da se ljudi podsete

of some really important statistical concepts,

nekih veoma važnih statističkih koncepata,

concepts like averages.

kao što su prosečne vrednosti.

So let's say you hear a claim like,

Recimo da čujete tvrdnju kao što je:

"The average swimming pool in the US contains 6.23 fecal accidents."

„Prosečan bazen u SAD sadrži 6,23 fekalnih nezgoda.“

That doesn't mean every single swimming pool in the country

To ne znači da svaki bazen u zemlji

sadrži tačno 6,23 komada izmeta.

contains exactly 6.23 turds.

So in order to show that,

Kako bih to pokazala,

okrenula sam se prvobitnim podacima iz Centra za kontrolu i prevenciju bolesti

I went back to the original data, which comes from the CDC,

who surveyed 47 swimming facilities.

koji je izvršio procenu 47 objekata za plivanje.

And I just spent one evening redistributing poop.

A ja sam samo provela jedno veče u preraspodeli kake.

So you can kind of see how misleading averages can be.

Tako da možete videti kako prosek može da obmane.

(Laughter)

(Smeh)

OK, so the second question that you guys should be asking yourselves

U redu, drugo pitanje koje treba da postavite sebi

to spot bad numbers is:

da biste uočili loše brojeve

je da li vidite sebe u podacima.

Can I see myself in the data?

This question is also about averages in a way,

Ovo pitanje se na neki način odnosi i na prosečne vrednosti,

because part of the reason why people are so frustrated

jer je deo razloga zašto ljude toliko frustriraju

with these national statistics,

ovi nacionalni statistički podaci

to što ne iznose priču o tome ko pobeđuje a ko je na gubitku

is they don't really tell the story of who's winning and who's losing

usled državne politike.

from national policy.

Lako je razumeti zašto ljude frustriraju globalne prosečne vrednosti

It's easy to understand why people are frustrated with global averages

kada se ne poklapaju sa njihovim ličnim iskustvom.

when they don't match up with their personal experiences.

Htela sam da pokažem ljudima

I wanted to show people the way data relates to their everyday lives.

kako podaci imaju veze sa njihovim svakodnevnim životom.

Pokrenula sam rubriku za savete pod nazivom „Draga Mona“,

I started this advice column called "Dear Mona,"

where people would write to me with questions and concerns

gde bi mi ljudi pisali i iznosili svoja pitanja i probleme,

a ja bih pokušala da im odgovorim pomoću podataka.

and I'd try to answer them with data.

People asked me anything.

Ljudi su me svašta pitali,

questions like, "Is it normal to sleep in a separate bed to my wife?"

na primer: „Da li je normalno da spavam u odvojenom krevetu od svoje žene?“

"Do people regret their tattoos?"

„Da li se ljudi kaju zbog svojih tetovaža?“

"What does it mean to die of natural causes?"

„Šta znači umreti prirodnom smrću?“

All of these questions are great, because they make you think

Sva ta pitanja su sjajna, jer vas teraju da razmislite

about ways to find and communicate these numbers.

o tome kako da saznate i saopštite ove brojeve.

If someone asks you, "How much pee is a lot of pee?"

Ako vas neko pita: „Koliko piškenja je mnogo?“,

što je pitanje koje sam ja dobila,

which is a question that I got asked,

you really want to make sure that the visualization makes sense

želite da se postarate da vizualizacija ima smisla

za što je više ljudi moguće.

to as many people as possible.

These numbers aren't unavailable.

Ovi brojevi nisu nedostupni.

Sometimes they're just buried in the appendix of an academic study.

Ponekad su samo zakopani u prilogu akademske studije.

And they're certainly not inscrutable;

A svakako nisu nedokučivi;

ako zaista želite da proverite ove brojeve o količini mokrenja,

if you really wanted to test these numbers on urination volume,

you could grab a bottle and try it for yourself.

možete uzeti bočicu i pokušati sami.

(Laughter)

(Smeh)

The point of this isn't necessarily

Suština ovoga nije nužno

da svaki skup podataka mora da se izričito odnosi na vas.

that every single data set has to relate specifically to you.

Mene zanima koliko žena je dobilo novčanu kaznu u Francuskoj

I'm interested in how many women were issued fines in France

za nošenje vela na licu, ili nikaba,

for wearing the face veil, or the niqab,

čak iako ne živim u Francuskoj niti nosim veo preko lica.

even if I don't live in France or wear the face veil.

The point of asking where you fit in is to get as much context as possible.

Poenta postavljanja pitanja gde se vi uklapate

je da dobijete što je više konteksta moguće.

So it's about zooming out from one data point,

Dakle, radi se o tome da umanjite sliku sa jednog podataka,

like the unemployment rate is five percent,

na primer, stopa nezaposlenosti je 5%

and seeing how it changes over time,

i vidite kako se menja tokom vremena,

or seeing how it changes by educational status --

ili da vidite kako se menja s obzirom na status obrazovanja -

this is why your parents always wanted you to go to college --

zato su roditelji uvek želeli da idete na fakultet -

or seeing how it varies by gender.

ili da vidite kako varira s obzirom na pol.

Nowadays, male unemployment rate is higher

Danas je stopa nezaposlenosti muškaraca viša

than the female unemployment rate.

nego stopa nezaposlenosti žena.

Up until the early '80s, it was the other way around.

Do ranih '80-ih godina, bilo je obrnuto.

This is a story of one of the biggest changes

Ovo je priča o jednoj od najvećih promena

that's happened in American society,

koja se dogodila u američkom društvu.

i sve je na tom grafikonu, kada sagledate stvari izvan proseka.

and it's all there in that chart, once you look beyond the averages.

The axes are everything;

Sve je u osama;

once you change the scale, you can change the story.

kada promenite nivo sagledavanja, možete promeniti priču.

OK, so the third and final question that I want you guys to think about

U redu, treće i poslednje pitanje o kojem želim da razmišljate

when you're looking at statistics is:

kada posmatrate statističke podatke

je kako su podaci prikupljeni.

How was the data collected?

So far, I've only talked about the way data is communicated,

Do sada sam govorila samo o načinu na koji se podaci saopštavaju,

but the way it's collected matters just as much.

ali način njihovog prikupljanja podjednako je bitan.

I know this is tough,

Znam da je ovo teško,

because methodologies can be opaque and actually kind of boring,

jer metodologija može biti nejasna i nekako dosadna,

ali postoje jednostavni koraci pomoću kojih možete ovo proveriti.

but there are some simple steps you can take to check this.

I'll use one last example here.

Ovde ću upotrebiti jedan poslednji primer.

One poll found that 41 percent of Muslims in this country support jihad,

Jedna anketa je otkrila da 41 odsto muslimana u ovoj zemlji podržava džihad,

što je očigledno prilično zastrašujuće

which is obviously pretty scary,

and it was reported everywhere in 2015.

i o tome se izveštavalo svuda 2015. godine.

When I want to check a number like that,

Kada hoću da proverim takvu brojku,

I'll start off by finding the original questionnaire.

počeću pronalaženjem originalnog upitnika.

It turns out that journalists who reported on that statistic

Ispostavilo se da su novinari koji su izveštavali o tom podatku

ignored a question lower down on the survey

zanemarili pitanje nešto niže na anketi

that asked respondents how they defined "jihad."

koje je pitalo ispitanike kako definišu „džihad“,

And most of them defined it as,

a većina njih ga je definisala

"Muslims' personal, peaceful struggle to be more religious."

kao „ličnu, mirnu borbu muslimana da budu religiozniji“.

Only 16 percent defined it as, "violent holy war against unbelievers."

Samo 16 procenata ga je definisalo kao „nasilan sveti rat protiv nevernika“.

This is the really important point:

To je zaista bitan deo;

based on those numbers, it's totally possible

na osnovu tih brojeva, sasvim je moguće

that no one in the survey who defined it as violent holy war

da niko ko ga je u istraživanju definisao kao nasilni sveti rat

also said they support it.

nije rekao i da ga podržava.

Those two groups might not overlap at all.

Te dve grupe se možda uopšte ne preklapaju.

It's also worth asking how the survey was carried out.

Takođe, vredi pitati kako je istraživanje sprovedeno.

This was something called an opt-in poll,

Ovo je bilo nešto što se zove opciona anketa,

which means anyone could have found it on the internet and completed it.

što znači da je bilo ko mogao da je nađe na internetu i popuni je.

Nema načina da se sazna da li se ti ljudi uopšte identifikuju kao muslimani.

There's no way of knowing if those people even identified as Muslim.

And finally, there were 600 respondents in that poll.

Naposletku, u toj anketi je bilo 600 ispitanika.

U ovoj zemlji ima približno tri miliona muslimana,

There are roughly three million Muslims in this country,

according to Pew Research Center.

prema Centru za istraživanje Pju.

To znači da se anketa obraćala otprilike jednom od svakih 5 000 muslimana

That means the poll spoke to roughly one in every 5,000 Muslims

in this country.

u ovoj zemlji.

This is one of the reasons

To je jedan od razloga

why government statistics are often better than private statistics.

zašto su vladini statistički podaci često bolji od privatnih.

A poll might speak to a couple hundred people, maybe a thousand,

Anketa se može obratiti par stotina ljudi, možda hiljadu,

or if you're L'Oreal, trying to sell skin care products in 2005,

ili ako ste Loreal i pokušavate da prodate proizvode za negu kože 2005. godine,

then you spoke to 48 women to claim that they work.

onda ste razgovarali sa 48 žena da biste tvrdili da deluju.

(Laughter)

(Smeh)

Private companies don't have a huge interest in getting the numbers right,

Privatne kompanije nemaju veliki interes da dobiju ispravne brojeve,

they just need the right numbers.

već su im samo potrebni odgovarajući brojevi.

Government statisticians aren't like that.

Vladini statističari nisu takvi.

In theory, at least, they're totally impartial,

Makar u teoriji, sasvim su nepristrasni,

not least because most of them do their jobs regardless of who's in power.

ne samo zato što većina njih obavlja svoj posao

bez obzira na to ko je na vlasti.

They're civil servants.

Oni su državni službenici.

And to do their jobs properly,

A da bi valjano radili svoj posao,

they don't just speak to a couple hundred people.

ne govore samo sa par stotina ljudi.

Oni brojevi vezani za nezaposlenost na koje se uporno pozivam

Those unemployment numbers I keep on referencing

come from the Bureau of Labor Statistics,

su iz odeljenja za statistiku Ministarstva za rad,

and to make their estimates,

a da bi izvršili svoje procene,

they speak to over 140,000 businesses in this country.

oni se obraćaju preko 140 000 firmi u ovoj zemlji.

I get it, it's frustrating.

Kapiram, to frustrira.

Ako želite da proverite podatke koji dolaze iz privatne kompanije,

If you want to test a statistic that comes from a private company,

you can buy the face cream for you and a bunch of friends, test it out,

možete da kupite kremu za lice za sebe i gomilu prijatelja, isprobate,

if it doesn't work, you can say the numbers were wrong.

a ako ne deluje, možete reći da su brojevi bili pogrešni.

But how do you question government statistics?

Međutim, kako da preispitate vladine podatke?

You just keep checking everything.

Samo uporno sve proveravajte.

Saznajte kako su prikupili brojeve.

Find out how they collected the numbers.

Find out if you're seeing everything on the chart you need to see.

Otkrijte da li na grafikonu vidite sve što treba da vidite.

Ali, ne odustajte sasvim od brojeva, jer ako odustanete,

But don't give up on the numbers altogether, because if you do,

donosićemo odluke o javnoj politici u neznanju,

we'll be making public policy decisions in the dark,

isključivo koristeći lične interese kao smernice.

using nothing but private interests to guide us.

Thank you.

Hvala.

(Aplauz)

(Applause)