Is there a reproducibility crisis in science? - Matt Anticole

譯者: Helen Chang 審譯者: 庭芝 梁

In 2011, a team of physicists reported a startling discovery:

2011 年一組物理學家團隊 宣布了一項驚人的發現

他們從日內瓦發射微中子

neutrinos traveled faster than the speed of light

by 60 billionths of a second

經過 730 公里後 抵達義大利的偵測器

in their 730 kilometer trip from Geneva to a detector in Italy.

它的速度居然比光速快了六十奈秒 ( 0.00000006 秒 )

Despite six months of double checking, the bizarre discovery refused to yield.

經過六個月之後再次進行實驗 仍然出現這個怪異的結果

But rather than celebrating a physics revolution,

研究人員並未慶祝這項物理界的革命

the researchers published a cautious paper

反而發表了一篇謹慎的論文： 《微中子快過光速？不會吧！》

arguing for continued research in an effort to explain the observed anomaly.

他們主張應該持續研究 以解釋觀察到的異常現象

In time, the error was tracked to a single incorrectly connected fiber optic cable.

過了不久，他們發現錯誤的原因 有一條光纖被接錯了

這個例子提醒我們 真正的科學並不是只存在教科書中

This example reminds us that real science is more than static textbooks.

Instead, researchers around the world are continuously publishing

事實上，全球各地的研究人員

不斷發布著最新的研究發現

their latest discoveries

with each paper adding to the scientific conversation.

並且透過論文，參與科學交流

Published studies can motivate future research,

學術發表可以激勵未來的研究

inspire new products,

啟發新產品

and inform government policy.

或是告知政府決策

So it's important that we have confidence in the published results.

因此，相信已公開發表的研究成果 是件非常重要的事

如果研究結論出錯

If their conclusions are wrong,

we risk time,

我們就必須承擔時間與資源的風險

resources,

and even our health in the pursuit of false leads.

甚至會由於錯誤的引導而危害健康

When findings are significant,

當研究成果特別重要時

they are frequently double-checked by other researchers,

通常會被其他的研究者進行再次檢驗

either by reanalyzing the data

包括重新分析數據

or by redoing the entire experiment.

或是重複進行整個實驗

For example, it took repeated investigation of the CERN data

例如前面提到的微中子事件 在查出錯誤之前

before the timing error was tracked down.

歐洲核子研究組織的研究數據 就被反覆地進行檢驗

Unfortunately, there are currently neither the resources nor professional incentives

很遺憾，目前沒有足夠的資源 或是專業上的誘因

to double check the more than 1 million scientific papers published annually.

來複查每年發表的 一百多萬篇科學論文

Even when papers are challenged, the results are not reassuring.

即使是能夠通過審核的論文， 結果也無法讓人放心

Recent studies that examined dozens of published pharmaceutical papers

最近的研究顯示 數十篇已發表的藥物學論文中

managed to replicate the results of less than 25% of them.

只有不到 25% 的實驗結果夠被重現

And similar results have been found in other scientific disciplines.

其他科學領域的結果也有類似現象

There are a variety of sources for irreproducible results.

研究結果無法重現，有很多種原因

Errors could hide in their original design, execution, or analysis of the data.

錯誤可能隱藏在原始的實驗設計

執行過程，或是數據分析時

Unknown factors,

未知的研究變因

such as patients' undisclosed condition in a medical study,

例如醫學研究中 病人未揭露的病史

can produce results that are not repeatable in new test subjects.

都可能讓其他患者 無法出現同樣的結果

And sometimes, the second research group can't reproduce the original results

有時第二組研究人員 不能重現實驗結果

simply because they don't know exactly what the original group did.

只是因為他們無法得知 先前的研究者如何進行實驗

However, some problems might stem from systematic decisions

然而，有些問題來自於

從事科學研究工作的系統性決策

in how we do science.

研究者

Researchers,

the institutions that employ them,

僱用研究人員的機構

and the scientific journals that publish findings

和發佈研究結果的科學期刊

are expected to produce big results frequently.

被期望能夠經常發表重要的研究成果

Important papers can advance careers,

重量級的研究論文

能使研究者升等

generate media interest,

吸引媒體的興趣

and secure essential funding,

以及確保得到重要的研究經費

so there's slim motivation for researchers to challenge their own exciting results.

所以研究者很少有意願 去挑戰他們已有的重大發現

In addition, little incentive exists

此外，幾乎沒有任何動機

to publish results unsupportive of the expected hypothesis.

會讓研究者願意公開 「不支持原本假設」的研究結果

That results in a deluge of agreement between what was expected

所以在發表的論文中 研究結果大多能符合預期的假設

and what was found.

In rare occasions, this can even lead to deliberate fabrication,

在極少數情況下甚至導致刻意造假

such as in 2013, when a researcher spiked rabbit blood with human blood

例如在 2013 年，某位研究者 在兔血當中加入人血

to give false evidence that his HIV vaccine was working.

製造假證據來證明 他的愛滋病疫苗有效

The publish or perish mindset

對於學術成果 「不發表就死亡」的心態

can also compromise academic journals' traditional peer-review processes

也可能使學術期刊的同儕審查制度 變成一種折衷、妥協的過程

which are safety checks

原本這項制度的初衷 是要執行安全檢查

where experts examine submitted papers for potential shortcomings.

也就是讓專家檢視論文的潛在缺點

The current system,

現今的論文審查系統

which might involve only one or two reviewers,

可能只有一至兩位審查委員參與

can be woefully ineffective.

結果導致整個機制失靈

That was demonstrated in a 1998 study

正如 1998 年一項研究所顯示的

where eight weaknesses were deliberately inserted into papers,

在一篇論文中刻意插入八個弱點

but only around 25% were caught upon review.

後來只有 25% 在審核過程中被發現

Many scientists are working toward improving reproducibility in their fields.

許多科學家正在努力提升 其研究結果的可複製性

There's a push to make researchers raw data,

他們正在推廣 要讓研究者的原始數據

experimental procedures,

實驗流程

and analytical techniques more openly available

以及分析技術都能更加開放

in order to ease replication efforts.

使得研究結果更容易被重現

The peer review process can also be strengthened

同儕審查制度也應該更加強化

to more efficiently weed out weak papers prior to publication.

在論文出版之前 更有效地進行汰弱留強

And we could temper the pressure to find big results

另外也應該減緩 發表重大研究結果的壓力

by publishing more papers that fail to confirm the original hypothesis,

轉而鼓勵更多學者發表 不支持原始假設的論文

an event that happens far more than current scientific literature suggests.

與目前的科學文獻相比 過去這種情形其實經常發生

Science always has, and always will, encounter some false starts

無論過去或是未來 科學研究一定都會出現造假的情形

as part of the collective acquisition of new knowledge.

這是新知識累積過程的一部分

Finding ways to improve the reproducibility of our results

找到有效的方法 增進研究結果的可重現性

can help us weed out those false starts more effectively,

能幫助我們更有效地 去除造假的研究成果

keeping us moving steadily toward exciting new discoveries.

讓我們穩步邁向 更多令人興奮的新發現