The notion that the pharmaceutical industry fiddles with drug studies is one that I harp on constantly. Sometimes they fiddle with the data interpretation process, but more commonly they commit what amounts to lies of omission.
Here’s what most do. If a drug company spends millions of dollars to sponsor a study designed to show that Drug X reduces inflammation and finds when all the data is assessed at the end that Drug X does indeed reduce inflammation to some extent, then you can bet that the company will rush the study in to print. But if this study instead of showing improvement shows that Drug X doesn’t help reduce inflammation or, horror of horrors, actually makes it worse, odds are that the company won’t publish the study.
We end up with a situation in which mainly positive studies get published. These positive studies then lend a aura of efficacy that really doesn’t exist to most of the drugs out there. No one can really point to studies nay saying the drugs in question since those studies were never published.
Yesterdays New England Journal of Medicine (NEJM) outed this execrable practice, at least as it applies to studies of antidepressant drugs. As reported in the New York Times:
In published trials, about 60 percent of people taking the drugs report significant relief from depression, compared with roughly 40 percent of those on placebo pills. But when the less positive, unpublished trials are included, the advantage shrinks: the drugs outperform placebos, but by a modest margin, concludes the new report, …
Previous research had found a similar bias toward reporting positive results for a variety of medications; and many researchers have questioned the reported effectiveness of antidepressants. But the new analysis, reviewing data from 74 trials involving 12 drugs, is the most thorough to date. And it documents a large difference: while 94 percent of the positive studies found their way into print, just 14 percent of those with disappointing or uncertain results did.
The authors of the NEJM study had this to say:
We found a bias toward the publication of positive results. Not only were positive results more likely to be published, but studies that were not positive, in our opinion, were often published in a way that conveyed a positive outcome. We analyzed these data in terms of the proportion of positive studies and in terms of the effect size associated with drug treatment. Using both approaches, we found that the efficacy of this drug class is less than would be gleaned from an examination of the published literature alone. According to the published literature, the results of nearly all of the trials of antidepressants were positive. In contrast, FDA analysis of the trial data showed that roughly half of the trials had positive results. The statistical significance of a study’s results was strongly associated with whether and how they were reported, and the association was independent of sample size. The study outcome also affected the chances that the data from a participant would be published. As a result of selective reporting, the published literature conveyed an effect size nearly one third larger than the effect size derived from the FDA data.
So, we’ve got sins both of omission and commission by Big Pharma. These guys don’t publish the negative studies (the sin of omission) or they publish them in a way that makes them look positive (the sin of commission).
The other way that Big Pharma sins is by sending its henchmen (doctors and researchers on the Big Pharma teat) out to exaggerate the significance of a single study. This sort of chicanery goes on all the time. I’ve said it before and I’ll say it again: there are no definitive medical studies. In other words, there are no single studies that are the last word on anything. All studies are just pieces of the puzzle. Only when enough studies have been reproduced in other labs showing consistent results we can start to feel some confidence in these results.
But what happens is that researchers are so sure of their hypothesis that they tend to disregard all studies showing that their hypothesis doesn’t hold up under investigation. Then when a study comes along that confirms their hypothesis, they declare the study the smoking gun.
This has happened with statins. Study after study has shown no decrease in all-cause mortality, but Big Pharma keeps funding more studies in hopes that one will be positive. Then when one positive one comes along, they shout it to the rooftops.
But what they don’t shout about are the failures that have been published and the many failures that haven’t. When you take it all in the aggregate the picture looks a lot different than the drug companies and the statinators who pimp for them would have you believe.
For these reasons, I’m extremely hesitant about prescribing any medications that I don’t feel have passed the hurdle of fairly long term use without serious problems and that haven’t been shown to be efficacious to my satisfaction.