Tamiflu & Relenza: how effective are they?

Today we found out that Tamiflu doesn't work so well after all. Roche, the drug company behind it, withheld vital information on its clinical trials for half a decade, but the Cochrane Collaboration, a global not-for-profit organisation of 14,000 academics, finally obtained all the information. Putting the evidence together, it has found that Tamiflu has little or no impact on complications of flu infection, such as pneumonia. 

That is a scandal because the UK government spent £0.5bn stockpiling this drug in the hope that it would help prevent serious side-effects from flu infection. But the bigger scandal is that Roche broke no law by withholding vital information on how well its drug works. In fact, the methods and results of clinical trials on the drugs we use today are still routinely and legally being withheld from doctors, researchers and patients. It is simple bad luck for Roche that Tamiflu became, arbitrarily, the poster child for the missing-data story.

 

And it is a great poster child. The battle over Tamiflu perfectly illustrates the need for full transparency around clinical trials, the importance of access to obscure documentation, and the failure of the regulatory system. Crucially, it is also an illustration of how science, at its best, is built on transparency and openness to criticism, because the saga of the Cochrane Tamiflu review began with a simple online comment.

 

In 2009, there was widespread concern about a new flu pandemic, and billions were being spent stockpiling Tamiflu around the world. Because of this, the UK and Australian governments specifically asked the Cochrane Collaboration to update its earlier reviews on the drug. Cochrane reviews are the gold-standard in medicine: they summarise all the data on a given treatment, and they are in a constant review cycle, because evidence changes over time as new trials are published. This should have been a pretty everyday piece of work: the previous review, in 2008, had found some evidence that Tamiflu does, indeed, reduce the rate of complications such as pneumonia. But then a Japanese paediatrician called Keiji Hayashi left a comment that would trigger a revolution in our understanding of how evidence-based medicine should work. This wasn't in a publication, or even a letter: it was a simple online comment, posted informally underneath the Tamiflu review on the Cochrane website, almost like a blog comment.

 

Cochrane had summarised the data from all the trials, explained Hayashi, but its positive conclusion was driven by data from just one of the papers it cited: an industry-funded summary of 10 previous trials, led by an author called Kaiser. From these 10 trials, only two had ever been published in the scientific literature. For the remaining eight, the only available information on the methods used came from the brief summary in this secondary source, created by industry. That's not reliable enough.

 

This is science at its best. The Cochrane review is readily accessible online; it explains transparently the methods by which it looked for trials, and then analysed them, so any informed reader can pull the review apart, and understand where the conclusions came from. Cochrane provides an easy way for readers to raise criticisms. And, crucially, these criticisms did not fall on deaf ears. Dr Tom Jefferson is the head of the Cochrane respiratory group, and the lead author on the 2008 review. He realised immediately that he had made a mistake in blindly trusting the Kaiser data. He said so, without defensiveness, and then set about getting the information needed.

 

First, the Cochrane researchers wrote to the authors of the Kaiser paper. By reply, they were told that this team no longer had the files: they should contact Roche. Here the problems began. Roche said it would hand over some information, but the Cochrane reviewers would need to sign a confidentiality agreement. This was tricky: Cochrane reviews are built around showing their working, but Roche's proposed contract would require them to keep the information behind their reasoning secret from readers. More than this, the contract said they were not allowed to discuss the terms of their secrecy agreement, or publicly acknowledge that it even existed. Roche was demanding a secret contract, with secret terms, requiring secrecy about the methods and results of trials, in a discussion about the safety and efficacy of a drug that has been taken by hundreds of thousands of people around the world, and on which governments had spent billions. Roche's demand, worryingly, is not unusual. At this point, many in medicine would either acquiesce, or give up. Jefferson asked Roche for clarification about why the contract was necessary. He never received a reply.

 

Then, in October 2009, the company changed tack. It would like to hand over the data, it explained, but another academic review on Tamiflu was being conducted elsewhere. Roche had given this other group the study reports, so Cochrane couldn't have them. This was a non-sequitur: there is no reason why many groups should not all work on the same question. In fact, since replication is the cornerstone of good science, this would be actively desirable. Then, one week later, unannounced, Roche sent seven documents, each around a dozen pages long. These contained excerpts of internal company documents on each of the clinical trials in the Kaiser meta-analysis. It was a start, but nothing like the information Cochrane needed to assess the benefits, or the rate of adverse events, or fully to understand the design of the trials.

 

At the same time, it was rapidly becoming clear that there were odd inconsistencies in the information on this drug. Crucially, different organisations around the world had drawn vastly different conclusions about its effectiveness. The US Food and Drug Administration (FDA) said it gave no benefits on complications such as pneumonia, while the US Centers for Disease Control and Prevention said it did. The Japanese regulator made no claim for complications, but the European Medicines Agency (EMA) said there was a benefit. There are only two explanations for this, and both can only be resolved by full transparency. Either these organisations saw different data, in which case we need to build a collective list, add up all the trials, and work out the effects of the drug overall. Or this is a close call, and there is reasonable disagreement on how to interpret the trials, in which case we need full access to their methods and results, for an informed public debate in the medical academic community.
 

This is particularly important, since there can often be shortcomings in the design of a clinical trial, which mean it is no longer a fair test of which treatment is best. We now know this was the case in many of the Tamiflu trials, where, for example, participants were sometimes very unrepresentative of real-world patients. Similarly, in trials described as "double blinded" – where neither doctor nor patient should be able to tell whether they're getting a placebo or the real drug – the active and placebo pills were different colours. Even more oddly, in almost all Tamiflu trials, it seems a diagnosis of pneumonia was measured by patients' self-reporting: many researchers would have expected a clear diagnostic algorithm, perhaps a chest x-ray, at least.
 

Since the Cochrane team were still being denied the information needed to spot these flaws, they decided to exclude all this data from their analysis, leaving the review in limbo. It was published in December 2009, with a note explaining their reasoning, and a small flurry of activity followed. Roche posted their brief excerpts online, and committed to make full study reports available. For four years, they then failed to do so. During this period, the global medical academic community began to realise that the brief, published academic papers on trials – which we have relied on for many years – can be incomplete, and even misleading. Much more detail is available in a clinical study report (CSR), the intermediate document that stands between the raw data and a journal article: the precise plan for analysing the data statistically, detailed descriptions of adverse events, and so on.

 

By 2009, Roche had shared just small portions of the CSRs, but even this was enough to see there were problems. For example, looking at the two papers out of 10 in the Kaiser review that were published, one said: "There were no drug-related serious adverse events", and the other doesn't mention adverse events. But in the CSR documents shared on these same two studies, 10 serious adverse events were listed, of which three are classified as being possibly related to Tamiflu.

 

By setting out all the known trials side by side, the researchers were able to identify peculiar discrepancies: for example, the largest "phase three" trial – one of the large trials that are done to get a drug on to the market – was never published, and is rarely mentioned in regulatory documents. The chase continued, and it exemplifies the attitude of industry towards transparency. In June 2010, Roche told Cochrane it was sorry, but it had thought they already had what they wanted. In July, it announced that it was worried about patient confidentiality. By now, Roche had been refusing to publish the study reports for a year. Suddenly, it began to raise odd personal concerns. It claimed that some Cochrane researchers had made untrue statements about the drug, and about the company, but refused to say who, or what, or where. "Certain members of Cochrane Group," it said, "are unlikely to approach the review with the independence that is both necessary and justified." This is hard to credit, but even if true, it should be irrelevant: bad science is often published, and is shot down in public, in academic journals, by people with good arguments. This is how science works. No company or researcher should be allowed to choose who has access to trial data. Still Roche refused to hand over the study reports.
 

Then Roche complained that the Cochrane reviewers had begun to copy in journalists, on their emails when responding to Roche staff. At the same time, the company was raising the broken arguments that are eerily familiar to anyone who has followed the campaign for greater trials transparency. Key among these was one that cuts to the core of the culture war between evidence-based medicine, and the older "eminence-based medicine" that we are supposed to have left behind. It is simply not the job of academics to make these decisions about benefit and risk, said Roche, it is the job of regulators.

This argument fails on two fronts. First, as with many other drugs, it now seems that not even the regulators had seen all the information on all the trials. But more than that, regulators miss things. Many of the most notable problems with medicines over the past few years – with the arthritis drug Vioxx; with the diabetes drug rosiglitazone, marketed as Avandia; and with the evidence base for Tamiflu – weren't spotted primarily by regulators, but rather by independent doctors and academics. Regulators don't miss things because they are corrupt, or incompetent. They miss things because detecting signals of risk and benefit in reviews of clinical trials is a difficult business and so, like all difficult questions in science, it benefits from having many eyes on the problem.

 

Tamiflu (the antiviral drug oseltamivir) shortens symptoms of influenza by half a day, but there is no good evidence to support claims that it reduces admissions to hospital or complications of influenza. This is according to the updated Cochrane evidence review, published today by The Cochrane Collaboration, the independent, global healthcare research network, and the BMJ. Evidence from treatment trials confirms increased risk of suffering from nausea and vomiting. And when Tamiflu was used in prevention trials, there was an increased risk of headaches, psychiatric disturbances, and renal events. Although when used as a preventative treatment, the drug can reduce the risk of people suffering symptomatic influenza, it is unproven that it can stop people carrying the influenza virus and spreading it to others. 

The latest updated Cochrane Review: Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children, is based on full internal reports of 20 Tamiflu (oseltamivir) and 26 Relenza (zanamivir) trials. These trials involved more than 24,000 people and the findings challenge the historical assumption that neuraminidase inhibitors are effective in combating influenza. The evidence also suggests there are insufficient grounds to support the use of Tamiflu in preventing person-to-person spread of influenza. This raises further questions about the most effective way to support drug regulation and public health policy decision-making. Claims about the effectiveness of Tamiflu against complications were a key factor in decisions made by governments around the world to stockpile these drugs in case of a pandemic. The US has spent more than $1.3 billion buying a strategic reserve of antivirals, while in the UK the government has spent almost £424 million  for a stockpile of about 40 million doses.

In 2009, a lack of access to available trial data hampered the efforts of the Cochrane researchers to verify the safety and effectiveness of Tamiflu – and led to questions over decisions to stockpile the drug while the risks and benefits remained uncertain. Today, the BMJ and Cochrane issue a joint call to government and health policy decision makers the world over asking, in light of the latest findings from the Cochrane Review, would you make the same recommendations today, choosing to stockpile Tamiflu?

Compared with a placebo, taking Tamiflu led to a quicker alleviation of influenza-like symptoms of just half a day (from 7 days to 6.3 days) in adults, but the effect in children was more uncertain. There was no evidence of a reduction in hospitalisations or serious influenza complications; confirmed pneumonia, bronchitis, sinusitis or ear infection in either adults or children. Tamiflu also increased the risk of nausea and vomiting in adults by around 4 percent and in children by 5 percent. There was a reported increased risk of psychiatric events of around 1 percent when Tamiflu was used to prevent influenza. Evidence also suggests that Tamiflu prevented some people from producing sufficient numbers of their own antibodies to fight infection.

The worldwide use of Tamiflu has increased dramatically since the outbreak of H1N1 virus (Swine Flu) in April 2009. It was initially believed that it would reduce hospital admissions and complications of influenza, such as pneumonia, during influenza pandemics. However, the original evidence presented to government agencies around the world was incomplete. The Cochrane Review has benefited from access to more complete reports of the original research, now made available by the manufacturers, Roche and GlaxoSmithKline. Although the review has confirmed small benefits on symptom relief, there is little to justify any belief that it reduces hospital admission or the risk of developing confirmed pneumonia. Along with the evidence of harms from the medication, it raises the question of whether global stockpiling of the drugs is still justifiable given the lack of reliable evidence to support the original claims of its benefits.

Source:  http://www.theguardian.com/business/2014/apr/10/tamiflu-saga-drug-trials-big-pharma
              http://www.cochrane.org/features/tamiflu-relenza-how-effective-are-they