What doctors don't know about the drugs they prescribe | Ben Goldacre

707,633 views ・ 2012-09-27

TED


Please double-click on the English subtitles below to play the video.

00:00
Translator: Joseph Geni Reviewer: Morton Bast
0
0
7000
00:15
Hi. So, this chap here,
1
15931
2785
00:18
he thinks he can tell you the future.
2
18716
2262
00:20
His name is Nostradamus, although here the Sun have
3
20978
1979
00:22
made him look a little bit like Sean Connery. (Laughter)
4
22957
3670
00:26
And like most of you, I suspect, I don't really believe
5
26627
2903
00:29
that people can see into the future.
6
29530
1108
00:30
I don't believe in precognition, and every now and then,
7
30638
2704
00:33
you hear that somebody has been able to predict something that happened in the future,
8
33342
3182
00:36
and that's probably because it was a fluke, and we only
9
36524
3059
00:39
hear about the flukes and about the freaks.
10
39583
2615
00:42
We don't hear about all the times that people got stuff wrong.
11
42198
4079
00:46
Now we expect that to happen with silly stories
12
46277
2160
00:48
about precognition, but the problem is,
13
48437
3139
00:51
we have exactly the same problem in academia
14
51576
3344
00:54
and in medicine, and in this environment, it costs lives.
15
54920
4501
00:59
So firstly, thinking just about precognition, as it turns out,
16
59421
3365
01:02
just last year a researcher called Daryl Bem conducted
17
62786
2529
01:05
a piece of research where he found evidence
18
65315
1836
01:07
of precognitive powers in undergraduate students,
19
67151
3658
01:10
and this was published in a peer-reviewed academic journal
20
70809
2574
01:13
and most of the people who read this just said, "Okay, well,
21
73383
2281
01:15
fair enough, but I think that's a fluke, that's a freak, because I know
22
75664
2181
01:17
that if I did a study where I found no evidence
23
77845
2834
01:20
that undergraduate students had precognitive powers,
24
80679
2512
01:23
it probably wouldn't get published in a journal.
25
83191
3552
01:26
And in fact, we know that that's true, because
26
86743
2855
01:29
several different groups of research scientists tried
27
89598
2529
01:32
to replicate the findings of this precognition study,
28
92127
3520
01:35
and when they submitted it to the exact same journal,
29
95647
2635
01:38
the journal said, "No, we're not interested in publishing
30
98282
3152
01:41
replication. We're not interested in your negative data."
31
101434
4525
01:45
So this is already evidence of how, in the academic
32
105959
2754
01:48
literature, we will see a biased sample of the true picture
33
108713
4870
01:53
of all of the scientific studies that have been conducted.
34
113583
3467
01:57
But it doesn't just happen in the dry academic field of psychology.
35
117050
4429
02:01
It also happens in, for example, cancer research.
36
121479
4367
02:05
So in March, 2012, just one month ago, some researchers
37
125846
4077
02:09
reported in the journal Nature how they had tried
38
129923
2896
02:12
to replicate 53 different basic science studies looking at
39
132819
3846
02:16
potential treatment targets in cancer,
40
136665
3555
02:20
and out of those 53 studies, they were only able
41
140220
2638
02:22
to successfully replicate six.
42
142858
3076
02:25
Forty-seven out of those 53 were unreplicable.
43
145934
4333
02:30
And they say in their discussion that this is very likely
44
150267
3913
02:34
because freaks get published.
45
154180
2639
02:36
People will do lots and lots and lots of different studies,
46
156819
2096
02:38
and the occasions when it works they will publish,
47
158915
2120
02:41
and the ones where it doesn't work they won't.
48
161035
1679
02:42
And their first recommendation of how to fix this problem,
49
162714
3941
02:46
because it is a problem, because it sends us all down blind alleys,
50
166655
3289
02:49
their first recommendation of how to fix this problem
51
169944
1706
02:51
is to make it easier to publish negative results in science,
52
171650
3393
02:55
and to change the incentives so that scientists are
53
175043
2907
02:57
encouraged to post more of their negative results in public.
54
177950
4352
03:02
But it doesn't just happen in the very dry world
55
182302
3852
03:06
of preclinical basic science cancer research.
56
186154
3851
03:10
It also happens in the very real, flesh and blood
57
190005
3657
03:13
of academic medicine. So in 1980,
58
193662
3591
03:17
some researchers did a study on a drug called lorcainide,
59
197253
3008
03:20
and this was an anti-arrhythmic drug,
60
200261
2331
03:22
a drug that suppresses abnormal heart rhythms,
61
202592
2251
03:24
and the idea was, after people have had a heart attack,
62
204858
2228
03:27
they're quite likely to have abnormal heart rhythms,
63
207086
1537
03:28
so if we give them a drug that suppresses abnormal heart
64
208623
2377
03:31
rhythms, this will increase the chances of them surviving.
65
211000
3713
03:34
Early on its development, they did a very small trial,
66
214713
3008
03:37
just under a hundred patients.
67
217721
1644
03:39
Fifty patients got lorcainide, and of those patients, 10 died.
68
219365
3652
03:43
Another 50 patients got a dummy placebo sugar pill
69
223017
3043
03:46
with no active ingredient, and only one of them died.
70
226060
2958
03:49
So they rightly regarded this drug as a failure,
71
229018
2649
03:51
and its commercial development was stopped, and because
72
231667
2869
03:54
its commercial development was stopped, this trial was never published.
73
234536
4348
03:58
Unfortunately, over the course of the next five, 10 years,
74
238884
5397
04:04
other companies had the same idea about drugs that would
75
244281
3825
04:08
prevent arrhythmias in people who have had heart attacks.
76
248106
2592
04:10
These drugs were brought to market. They were prescribed
77
250698
1720
04:12
very widely because heart attacks are a very common thing,
78
252418
3412
04:15
and it took so long for us to find out that these drugs
79
255830
3843
04:19
also caused an increased rate of death
80
259673
2911
04:22
that before we detected that safety signal,
81
262584
2747
04:25
over 100,000 people died unnecessarily in America
82
265331
6051
04:31
from the prescription of anti-arrhythmic drugs.
83
271382
3451
04:34
Now actually, in 1993,
84
274833
3598
04:38
the researchers who did that 1980 study, that early study,
85
278431
3560
04:41
published a mea culpa, an apology to the scientific community,
86
281991
3841
04:45
in which they said, "When we carried out our study in 1980,
87
285832
3125
04:48
we thought that the increased death rate that occurred
88
288957
1936
04:50
in the lorcainide group was an effect of chance."
89
290893
3358
04:54
The development of lorcainide was abandoned for commercial reasons,
90
294251
2032
04:56
and this study was never published;
91
296283
1638
04:57
it's now a good example of publication bias.
92
297921
2386
05:00
That's the technical term for the phenomenon where
93
300307
1891
05:02
unflattering data gets lost, gets unpublished, is left
94
302198
4238
05:06
missing in action, and they say the results described here
95
306436
3371
05:09
"might have provided an early warning of trouble ahead."
96
309807
4808
05:14
Now these are stories from basic science.
97
314615
3213
05:17
These are stories from 20, 30 years ago.
98
317828
4787
05:22
The academic publishing environment is very different now.
99
322615
3147
05:25
There are academic journals like "Trials," the open access journal,
100
325762
3995
05:29
which will publish any trial conducted in humans
101
329757
2655
05:32
regardless of whether it has a positive or a negative result.
102
332412
3303
05:35
But this problem of negative results that go missing in action
103
335715
3969
05:39
is still very prevalent. In fact it's so prevalent
104
339684
3559
05:43
that it cuts to the core of evidence-based medicine.
105
343243
5851
05:49
So this is a drug called reboxetine, and this is a drug
106
349094
3015
05:52
that I myself have prescribed. It's an antidepressant.
107
352109
2535
05:54
And I'm a very nerdy doctor, so I read all of the studies
108
354644
2536
05:57
that I could on this drug. I read the one study that was published
109
357180
3052
06:00
that showed that reboxetine was better than placebo,
110
360232
2947
06:03
and I read the other three studies that were published
111
363179
1864
06:05
that showed that reboxetine was just as good as any other antidepressant,
112
365043
3571
06:08
and because this patient hadn't done well on those other antidepressants,
113
368614
2187
06:10
I thought, well, reboxetine is just as good. It's one to try.
114
370801
2466
06:13
But it turned out that I was misled. In fact,
115
373267
3392
06:16
seven trials were conducted comparing reboxetine
116
376659
2449
06:19
against a dummy placebo sugar pill. One of them
117
379108
2712
06:21
was positive and that was published, but six of them
118
381820
2312
06:24
were negative and they were left unpublished.
119
384132
4048
06:28
Three trials were published comparing reboxetine
120
388180
1739
06:29
against other antidepressants in which reboxetine
121
389919
2226
06:32
was just as good, and they were published,
122
392145
1793
06:33
but three times as many patients' worth of data was collected
123
393938
4389
06:38
which showed that reboxetine was worse than
124
398327
1871
06:40
those other treatments, and those trials were not published.
125
400198
4701
06:44
I felt misled.
126
404899
3759
06:48
Now you might say, well, that's an extremely unusual example,
127
408658
2130
06:50
and I wouldn't want to be guilty of the same kind of
128
410788
2008
06:52
cherry-picking and selective referencing
129
412796
2981
06:55
that I'm accusing other people of.
130
415777
1791
06:57
But it turns out that this phenomenon of publication bias
131
417568
1884
06:59
has actually been very, very well studied.
132
419452
2127
07:01
So here is one example of how you approach it.
133
421579
2218
07:03
The classic model is, you get a bunch of studies where
134
423797
2440
07:06
you know that they've been conducted and completed,
135
426237
2185
07:08
and then you go and see if they've been published anywhere
136
428422
2321
07:10
in the academic literature. So this took all of the trials
137
430743
2863
07:13
that had ever been conducted on antidepressants
138
433606
2154
07:15
that were approved over a 15-year period by the FDA.
139
435760
3642
07:19
They took all of the trials which were submitted to the FDA as part of the approval package.
140
439402
3756
07:23
So that's not all of the trials that were ever conducted on these drugs,
141
443158
3200
07:26
because we can never know if we have those,
142
446358
2098
07:28
but it is the ones that were conducted in order to get the marketing authorization.
143
448456
3494
07:31
And then they went to see if these trials had been published
144
451950
2349
07:34
in the peer-reviewed academic literature. And this is what they found.
145
454299
2572
07:36
It was pretty much a 50-50 split. Half of these trials
146
456871
3169
07:40
were positive, half of them were negative, in reality.
147
460040
3597
07:43
But when they went to look for these trials in the peer-reviewed academic literature,
148
463637
4741
07:48
what they found was a very different picture.
149
468378
2234
07:50
Only three of the negative trials were published,
150
470612
4372
07:54
but all but one of the positive trials were published.
151
474984
4642
07:59
Now if we just flick back and forth between those two,
152
479626
3761
08:03
you can see what a staggering difference there was
153
483387
2594
08:05
between reality and what doctors, patients,
154
485981
3450
08:09
commissioners of health services, and academics
155
489431
2622
08:12
were able to see in the peer-reviewed academic literature.
156
492053
3281
08:15
We were misled, and this is a systematic flaw
157
495334
4454
08:19
in the core of medicine.
158
499788
3330
08:23
In fact, there have been so many studies conducted on
159
503118
2663
08:25
publication bias now, over a hundred, that they've been
160
505781
3384
08:29
collected in a systematic review, published in 2010,
161
509165
3194
08:32
that took every single study on publication bias
162
512359
2766
08:35
that they could find.
163
515125
1299
08:36
Publication bias affects every field of medicine.
164
516424
2852
08:39
About half of all trials, on average, go missing in action,
165
519276
4313
08:43
and we know that positive findings are around twice as likely
166
523589
3058
08:46
to be published as negative findings.
167
526647
3054
08:49
This is a cancer at the core of evidence-based medicine.
168
529701
4061
08:53
If I flipped a coin 100 times but then
169
533762
3871
08:57
withheld the results from you from half of those tosses,
170
537633
3259
09:00
I could make it look as if I had a coin that always came up heads.
171
540892
3400
09:04
But that wouldn't mean that I had a two-headed coin.
172
544292
1806
09:06
That would mean that I was a chancer
173
546113
1712
09:07
and you were an idiot for letting me get away with it. (Laughter)
174
547825
3114
09:10
But this is exactly what we blindly tolerate
175
550939
3637
09:14
in the whole of evidence-based medicine.
176
554576
3789
09:18
And to me, this is research misconduct.
177
558365
4432
09:22
If I conducted one study and I withheld
178
562797
2743
09:25
half of the data points from that one study,
179
565540
3000
09:28
you would rightly accuse me, essentially, of research fraud.
180
568540
4707
09:33
And yet, for some reason, if somebody conducts
181
573247
2783
09:36
10 studies but only publishes the five that give the result that they want,
182
576030
4558
09:40
we don't consider that to be research misconduct.
183
580588
2788
09:43
And when that responsibility is diffused between
184
583376
2567
09:45
a whole network of researchers, academics,
185
585943
3161
09:49
industry sponsors, journal editors, for some reason
186
589104
3528
09:52
we find it more acceptable,
187
592632
1453
09:54
but the effect on patients is damning.
188
594085
3675
09:57
And this is happening right now, today.
189
597760
5018
10:02
This is a drug called Tamiflu. Tamiflu is a drug
190
602778
2711
10:05
which governments around the world have spent billions
191
605489
2596
10:08
and billions of dollars on stockpiling,
192
608085
2572
10:10
and we've stockpiled Tamiflu in panic,
193
610657
3148
10:13
in the belief that it will reduce the rate of complications of influenza.
194
613805
3949
10:17
Complications is a medical euphemism for pneumonia
195
617754
2684
10:20
and death. (Laughter)
196
620438
4814
10:25
Now when the Cochrane systematic reviewers
197
625252
3208
10:28
were trying to collect together all of the data from all
198
628460
2525
10:30
of the trials that had ever been conducted on whether Tamiflu actually did this or not,
199
630985
3648
10:34
they found that several of those trials were unpublished.
200
634633
2951
10:37
The results were unavailable to them.
201
637584
1842
10:39
And when they started obtaining the writeups of those trials through various different means,
202
639426
3964
10:43
through Freedom of Information Act requests, through
203
643390
1682
10:45
harassing various different organizations, what they found was inconsistent.
204
645072
4809
10:49
And when they tried to get a hold of the clinical study reports,
205
649881
2466
10:52
the 10,000-page long documents that have
206
652347
3046
10:55
the best possible rendition of the information,
207
655393
3600
10:58
they were told they weren't allowed to have them.
208
658993
2888
11:01
And if you want to read the full correspondence
209
661881
2683
11:04
and the excuses and the explanations given by the drug company,
210
664564
3290
11:07
you can see that written up in this week's edition
211
667854
2717
11:10
of PLOS Medicine.
212
670571
4367
11:14
And the most staggering thing of all of this, to me,
213
674938
3859
11:18
is that not only is this a problem, not only do we recognize
214
678797
3299
11:22
that this is a problem, but we've had to suffer fake fixes.
215
682096
4195
11:26
We've had people pretend that this is a problem that's been fixed.
216
686291
3058
11:29
First of all, we had trials registers, and everybody said,
217
689349
2188
11:31
oh, it's okay. We'll get everyone to register their trials, they'll post the protocol,
218
691537
3603
11:35
they'll say what they're going to do before they do it,
219
695140
2024
11:37
and then afterwards we'll be able to check and see if all the trials which
220
697164
2121
11:39
have been conducted and completed have been published.
221
699285
2468
11:41
But people didn't bother to use those registers.
222
701753
2196
11:43
And so then the International Committee of Medical Journal Editors came along,
223
703949
2619
11:46
and they said, oh, well, we will hold the line.
224
706568
1543
11:48
We won't publish any journals, we won't publish any trials,
225
708111
2633
11:50
unless they've been registered before they began.
226
710744
2682
11:53
But they didn't hold the line. In 2008, a study was conducted
227
713426
3531
11:56
which showed that half of all of trials published by journals
228
716957
3015
11:59
edited by members of the ICMJE
229
719972
2667
12:02
weren't properly registered, and a quarter of them weren't registered at all.
230
722639
4813
12:07
And then finally, the FDA Amendment Act was passed
231
727452
2801
12:10
a couple of years ago saying that everybody who conducts
232
730253
2349
12:12
a trial must post the results of that trial within one year.
233
732602
3443
12:16
And in the BMJ, in the first edition of January, 2012,
234
736045
4096
12:20
you can see a study which looks to see if people kept
235
740141
2704
12:22
to that ruling, and it turns out that only one in five
236
742845
3719
12:26
have done so.
237
746564
2864
12:29
This is a disaster.
238
749428
3283
12:32
We cannot know the true effects of the medicines
239
752711
3564
12:36
that we prescribe if we do not have access
240
756275
3216
12:39
to all of the information.
241
759491
3180
12:42
And this is not a difficult problem to fix.
242
762671
3959
12:46
We need to force people to publish all trials
243
766630
5128
12:51
conducted in humans, including the older trials,
244
771758
2971
12:54
because the FDA Amendment Act only asks that you publish the trials conducted after 2008,
245
774729
3945
12:58
and I don't know what world it is in which we're only
246
778674
2613
13:01
practicing medicine on the basis of trials that completed in the past two years.
247
781287
4456
13:05
We need to publish all trials in humans,
248
785743
2105
13:07
including the older trials, for all drugs in current use,
249
787848
3074
13:10
and you need to tell everyone you know
250
790922
2916
13:13
that this is a problem and that it has not been fixed.
251
793838
3442
13:17
Thank you very much. (Applause)
252
797280
2951
13:20
(Applause)
253
800231
3273
About this website

This site will introduce you to YouTube videos that are useful for learning English. You will see English lessons taught by top-notch teachers from around the world. Double-click on the English subtitles displayed on each video page to play the video from there. The subtitles scroll in sync with the video playback. If you have any comments or requests, please contact us using this contact form.

https://forms.gle/WvT1wiN1qDtmnspy7