Not all scientific studies are created equal - David H. Schwartz

Studies have shown that

taking vitamins is good for your health

and bad for your health.

That newly discovered herb can improve your memory

or destroy your liver.

Headlines proclaim a promising new cancer treatment

and never mention it again.

On a daily basis,

we are bombarded with attention-grabbing news,

backed up by scientific studies,

but what are these studies?

How are they performed?

And how do we know whether they're reliable?

When it comes to dietary or medical information,

the first thing to remember

is that while studies on animals or individual cells

can point the way towards further research,

the only way to know how something will affect humans

is through a study involving human subjects.

And when it comes to human studies,

the scientific gold standard is

the randomized clinical trial, or RCT.

The key to RCTs is that the subjects are randomly assigned

to their study groups.

They are often blinded to make them more rigorous.

This process attempts to ensure

that the only difference between the groups

is the one the researchers are attempting to study.

For example,

when testing a new headache medication,

a large pool of people with headaches

would be randomly divided into two groups,

one receiving the medication

and another receiving a placebo.

With proper randomization,

the only significant overall difference

between the two groups

will be whether or not they received the medication,

rather than other differences that could affect results.

Randomized clinical trials are incredible tools,

and, in fact, the US Food and Drug Administration

often requires at least two to be conducted

before a new drug can be marketed.

But the problem is that an RCT is not possible

in many cases,

either because it's not practical

or would require too many volunteers.

In such cases,

scientists use an epidemiological study,

which simply observes people going about their usual behavior,

rather than randomly assigning active participants

to control invariable groups.

Let's say we wanted to study

whether an herbal ingredient on the market

causes nausea.

Rather than deliberately giving people something

that might make them nauseated,

we would find those who already take the ingredient

in their everyday lives.

This group is called the cohort.

We would also need a comparison group

of people who do not have exposure to the ingredient.

And we would then compare statistics.

If the rate of nausea is higher in the herbal cohort,

it suggests an association

between the herbal supplement and nausea.

Epidemiological studies are great tools

to study the health effects of almost anything,

without directly interfering in people's lives

or assigning them to potentially dangerous exposures.

So, why can't we rely on these studies

to establish causal relationships

between substances and their effects on health?

The problem is

that even the best conducted epidemiological studies

have inherent flaws.

Precisely because the test subjects

are not randomly assigned to their groups.

For example, if the cohort in our herbal study

consisted of people who took the supplement

for health reasons,

they may have already had higher rates of nausea

than the other people in the sample.

Or the cohort group could've been composed of

people who shop at health food stores

and have different diets

or better access to healthcare.

These factors that can affect results,

in addition to the factor being studied,

are known as confounding variables.

These two major pitfalls,

combined with more general dangers,

such as conflicts of interest or selective use of data,

can make the findings of any particular epidemiological study suspect,

and a good study must go out of its way

to prove that its authors have taken steps

to eliminate these types of errors.

But even when this has been done,

the very nature of epidemiological studies,

which examine differences between preexisting groups,

rather than deliberately inducing changes within the same individuals,

means that a single study

can only demonstrate a correlation

between a substance and a health outcome,

rather than a true cause and effect relationship.

At the end of the day,

epidemiological studies have served as excellent guides to public health,

alerting us to critical health hazards,

such as smoking, asbestos, lead, and many more.

But these were demonstrated through

multiple, well-conducted epidemiological studies,

all pointing in the same direction.

So, the next time you see a headline

about a new miracle cure

or the terrible danger posed by an everyday substance,

try to learn more about the original study

and the limitations inherent in any epidemiological study or clinical trial

before jumping to conclusions.