Read this quote, from the FDA Website, carefully:
There is insufficient information in the application to conclude that ocrelizumab [Ocrevus] is a safe or effective treatment for PPMS. There is only one PPMS trial.
As it is, the trial results count events that may not have occurred, show inconsistencies among important subgroups, and lack independent confirmation.
In women, no beneficial effect balances the potential risk of breast cancer. In addition, there are problems with trial conduct and reasons to suspect the quality of the data.
John Marler, MD, Leader of the FDA Ocrevus PPMS review team.
Whether or not to begin an Ocrevus regimen is a decision many PPMS patients struggle with. Risks, costs, and benefits have to be weighed. While it’s natural to seek medication for such a debilitating disease, Ocrevus may be a bad choice for many. Below are six reasons you may want to say No to this powerful drug.
In the text below, “the Study” is the Phase 3 “ORATORIO” PPMS trial. “CDP” refers to Confirmed Disease Progression. The primary endpoint of the Study was evidence of disease progression, confirmed for at least 12 weeks, or “12 week CDP”.
1. You’re a woman
The primary endpoint of the Study found 32.9% of people in the Ocrevus group had symptom progression, versus 39.3% of those on the placebo:
Unfortunately, as the FDA quote at the top calls out, it had virtually no benefit for women. We while you’ll never see it in their marketing materials, only 1% more women in the Ocrevus group than the placebo group failed to progress:
You might think an important fact like this would be disclosed to patients, but that’s not the way for-profit drug companies work. However, this data was disclosed to the FDA, though you’ll have do dig through their 477 page Medical Reviews doc to find it (I’ll save you the trouble, page 168).
There are ways to spin Ocrevus’ benefit for women in a (slightly) more positive light. For example, Genentech published a hazard ratio (HR), is the relative risk of an event occurring at a point in time, given two sets of results. HRs have their own set of problems (they’re complex statistical regressions that can be manipulated, they suffer from survivorship bias, etc.). However, if you take the female HR Genentech reported for Ocrevus, .94, to be the actual odds that Ocrevus will halt your progression, you have a 6% chance of being helped by the drug. But there’s much data, including Genentech’s own long term studies, to make this blogger skeptical of that number. In any case, that .94 HR is only after Genentech, arguably, chose patients more likely to benefit from Ocrevus and used some creative accounting to make the results look better than they should have (see below). An HR of .76, for all patients was barely enough for FDA approval for Ocrevus. An HR of .94 would never be deemed effective, for any drug.
What caused this lopsided gender result? Apparently nothing in the design of the Study, which had a large population broken down evenly by gender. In 2017, the European Medicines Agency, the drug regulatory authority for the EU, demanded Genentech explain the discrepancy. The best it could come up with was to note slight differences between genders with active lesions (25.7% male vs 29.4% female of those receiving Ocrevus). The EMA was not convinced, stating:
In terms of biological plausibility, the figures on the potential influence of Gd-enhancing lesion distribution on the effect of OCR [Ocrevus] by sex reported in descriptive, univariate and multivariate analyses would not substantially explain the lower treatment effect observed in female PPMS patients. The same reduction of OCR efficacy in females is not seen in RMS.
In other words, nobody knows why the fairer sex didn’t respond to Ocrevus. But it’s not surprising. Many drugs affect males and females differently. Women considering this drug should ask their medical team hard questions about the benefits they can expect. And, incidentally, one of those should be about breast cancer – at least seven female patients in the Ocrevus PPMS and RMS studies reported a diagnosis of it, versus none in the placebo groups.
2. The Study didn’t select typical PPMS patients
As the quote from the FDA alludes to, Genentech made a number of suspicious decisions in designing and carrying out the Study – too many for a single blog post, but we’ll highlight a couple here.
Update: you can read more about Genentech’s creative patient accounting here: https://ppms.blog/2019/09/22/ocrevus-ppms-and-the-102-mystery-patients.
It’s important to recall that Genentech didn’t create the Study naively, as it would have a new class of drug. It had previously tested an almost identical molecule, Rituximab, for PPMS, and determined the subgroups for which is was most effective. So in this second study it could choose patients with characteristics similar to those who responded well in the first one. Unfortunately, regulatory agencies don’t dictate drug trial parameters – they’re at the discretion of the drug company.
For the Rituximab study, the patient age range was 18-65. Other PPMS studies recruited a even older age range. The range for the 2003 Interferon-beta Phase 3 PPMS trial, for example, was 30-65. The age range for the ORATORIO study, however, was 18-56.
This was the same youthful population that responded well to Rituximab, but it is not typical of the average PPMS sufferer. The Cleveland Clinic, which has one of the leading MS practices in the world, notes, for example, that:
Most people start to have symptoms [of PPMS] at about age 50 or older, or about 10 years later than is typical of relapsing forms of MS.
The median age of the Study patient was 46, which means the majority of patients were younger than when the typical person even begins to experience symptoms, per the statement above.
This would be less of a problem if a drug were only allowed to be sold to the types of patients on which it was more effective (countries outside the US do a better job limiting drugs to certain groups). But Genentech markets Ocrevus to everyone. Study details – like effectiveness by age or gender – are buried in footnotes that many long suffering, neurologically compromised patients are unlikely to dig into.
As you might imagine, these younger folks hadn’t suffered from PPMS nearly as long as the typical patient. In fact, the patients in the Study were PPMS newbies, having been diagnosed only 1.5 years prior (median). Given that the median life expectancy for PPMS patients is about 70, the typical PPMS patient alive today has had the disease for more like 10 years. While the median patient in the Study had experienced symptoms for 5 or 6 years, this still doesn’t reflect the actual patient population.
3. Genentech Fudged the Study Results
Reducing the results of a drug study to numbers – p-values, hazard ratios – ignores the messy reality of drug trials. Despite the best efforts of all involved parties, the results are usually less scientifically precise than they appear. For example, trials often span multiple countries, causing variable results. The protocols and equipment used for an MRI in Canada may be very different from those in Croatia, for example.
A key problem in the Study was progression confirmation. Recall that the primary endpoint was confirmed progression. For the progression to be confirmed, each patient had to return to the doctor 12 weeks later. A few people – 9 in the Ocrevus group, 12 in the placebo group – never came back for that follow up appointment.
The conservative, arguably prudent, thing to do here would be to throw out these unconfirmed records, and consider only those who completed the trial. In a clearly effective drug, the removal of just 3% of the data wouldn’t alter the results. But in a marginally effective one, like Ocrevus, every data point counts. In fact, removing the non-confirmations in the Study would have resulted in a p-value of .14 for the primary endpoint, rendering the drug ineffective by FDA standards. So Genentech found another way around the problem. It assumed these patients did in fact return for a second visit, and that at this imaginary appointment, all disabilities were confirmed.
This is not standard practice. The FDA described it as “unusual” it its review of the study. But you can see why Genentech did it – 12 of the 21 non-confirmed patients were in the placebo group, despite there being only half as many total placebo patients. It’s not hard to imagine the many other creative ways the company might have solved this problem if those 12 patients had been on Ocrevus, not the placebo.
4. John Marler, Billy Dunn and the FDA
John Marler, MD, has worked for over a decade as a Clinical Team Leader for the Division of Neurology Products at the FDA. He was one of five physicians deciding whether Ocrevus should be approved for PPMS. After a careful review of the Study, Dr. Marler voted No.
Marler noted that the effectiveness of Ocrevus was marginal at best, and that it only passed the FDA p-value threshold due to Genentech’s creative accounting of the patients without confirmed diagnoses, as noted above.
But Billy Dunn, MD, Director of the Division, disagreed with Dr. Marler, and it was he who made the final approval decision on Ocrevus.
If Marler’s concerns were valid, why did Dr. Dunn overrule them? Dunn made two points repeatedly in the FDA review, which was written in the first person by him. First, it appears he decided to approve the drug simply because there was no other treatment available. Here’s one of many quotes from Dunn’s review:
Regarding product quality, I must reiterate that identified deficiencies would ordinarily preclude approval, but resolution of remaining product quality issues via initial adjustments to the control strategy, appropriate PMCs, continued process verification by the applicant, and continued implementation of corrective and preventive actions, is an appropriate strategy to support the approval of OCR [Ocrevus] given the unmet medical need that it will address.
If this seems like a questionable reason to approve a drug, consider the enormous pressure decision makers like Dr. Dunn face. In 2016, for example, Dunn refused to approve the Muscular Dystrophy drug eteplirsen because the study population in the trial was too small to show effectiveness. How small? Well, the ORATORIO study had 737 participants, and even then, had issues with statistical power. The eteplirsen study had 12.
The decision likely cost its maker, Sarepta Therapeutcs, millions of dollars in profits, and the pro-business Wall Street Journal pounced on the decision. In an editorial titled “Mental Dystrophy at the FDA” that ranted against “government bureaucracy”, it blasted the decision, claiming the FDA should have approved the drug anyway. It even attacked Dunn personally, calling him “arrogant” for suggesting that anecdotal evidence was not enough for FDA approval.
Anti-government rhetoric sells newspapers, boring scientific details don’t, and the Wall Street Journal knows it. And as the long history of snake oil shows, patients want medicine and hope, often whether or not a drug is proven to work.
I’m sure Dr. Dunn made more enemies that day than your or I will in a lifetime. Whatever the stresses you face in your career, having your decisions attacked in the global media probably isn’t one of them. While I have absolutely no proof that Dr. Dunn feels any such pressures when making decisions on marginally effective drugs, I do know how human nature works.
Compare this to the experience one has when approving a drug. From the FDA Ocrevus press release:
“Multiple sclerosis can have a profound impact on a person’s life,” said Billy Dunn, M.D., director of the Division of Neurology Products in the FDA’s Center for Drug Evaluation and Research. “This therapy not only provides another treatment option for those with relapsing MS, but for the first time provides an approved therapy for those with primary progressive MS.”
Billy Dunn. Hero.
The second reason Dunn overruled Marler is that the two disagreed on a key point – that Ocrevus’ effectiveness in treating RRMS should somehow spill over to PPMS. And Ocrevus clearly is effective for RRMS. In two Phase 3 trials, it reduced relapses in RRMS patients by almost 50% (p value < .0001) versus a control group taking another RRMS drug, Rebif, which itself is already an effective RRMS drug.
The problem is that PPMS and RRMS are distinct diseases in many ways. For example, while there were 10 RRMS FDA approved drugs at the time of the Ocrevus review, not a one had been shown to be effective for PPMS. In fact, over 50 trials using various drugs for PPMS have been attempted, all of which have failed.
RRMS, as the name implies, is a disease of relapses – reduce the relapses, and you reduce the progression. It’s also a disease of lesions, readily seen on a brain MRI. The more mysterious PPMS, on the other hand, is a steadily worsening condition resulting in damage mainly to diffuse brain matter. Its pathological mechanisms happen mostly behind, not across, the blood brain barrier, making it exceptionally difficult to treat. It shows up later in life than RRMS, and affects mostly men. RRMS affects mostly women.
The differences between the two MS types are apparent in the Study results as well. As noted above, Ocrevus showed almost no benefit to women with PPMS, but the positive effects on RRMS were just as strong for women as men. Given these differences, assigning a benefit to a PPMS drug based on RRMS success, without evidence, feels like wishful thinking. A flu vaccine won’t prevent the common cold, though the two diseases may have similar symptoms and causes.
5. The Side Effects
Genentech could be accused of misleading advertising in its claims about the benefits of Ocrevus. However, it doesn’t mince words when it comes to the side effects of the drug:
“OCREVUS can cause infusion reactions that can be serious and require you to be hospitalized.”
“OCREVUS increases your risk of getting upper respiratory tract infections, lower respiratory tract infections, skin infections, and herpes infections. ”
“If you have ever had hepatitis B virus infection, the hepatitis B virus may become active again during or after treatment with OCREVUS”
“OCREVUS taken before or after other medicines that weaken the immune system could increase your risk of getting infections.”
It’s not hard to see why these warnings should be taken seriously. Ocrevus has only one effect – to destroy “b-cells”, one of the two main types of white blood cells that protect your body from disease. And that’s certain to weaken your immune system. In fact, Genentech ended its Phase 3 study of Ocrevus for rheumatoid arthritis, despite the fact that the drug was effective at the 500 ML dose, because too many patients reported serious infections. During the ORATORIO study, four Ocrevus patients suffered non-accidental deaths, versus none in the placebo group.
Infusion reactions are another side effect suffered by 40% of Ocrevus patients in the Study (vs 25% in the placebo group). To counteract this, patients are administered powerful steroids during each infusion, and often antihistamines.
Visit any MS internet forum where Ocrevus is discussed and you’ll find a large number of patients who had to stop treatment due to repeated infections, despite the fact the drug has only been on the market since 2017. You’ll also meet plenty of people who tolerated the drug’s effects well. But make no mistake, Ocrevus is a serious antibody you don’t want attacking your immune system without a clear offsetting benefit.
6. You’re Paying
This is more of a philosophical point, since, if you’re an Ocrevus patient, your insurance is almost certainly covering your treatment. But bear with me.
The wholesale price of a year of the drug alone is $65,000, and the total infusion cost runs around $200,000 per year for many. If I offered you, a PPMS patient, that amount of money, or two Ocrevus infusions, which would you take?
That’s the true test of the usefulness of any drug – whether the benefits are worth the sum of the costs and risks. In the case of very expensive, marginally effective products, it’s a question worth asking. And it’s a question countries with more cost effective health care systems do ask.
Take England. Genentech offered to sell Ocrevus to their National Health Service, the NHS, for about $22,000 per patient per year, versus the $65,000 cost in the US. At this price, the NHS determined the benefits were not worth the cost. After further review, and negotiations to lower the price of the drug even further, the NHS did approve the drug, but only, as noted below, in limited cases where it was proved most effective.
It’s not that the NHS is especially conservative when it comes to MS treatments – in fact, it offers stem cell therapy to some MS patients, which no insurance plan in the US will cover. But, looking at the points above, I hope it’s not hard to see why they found the drug only worth purchasing at modest price points.
Two Reasons to Consider Ocrevus for PPMS
Here are a couple of reasons, despite the arguments above, that you might still consder Ocrevus for PPMS.
1. You’re in a demographic for which Ocrevus is more effective
We know that men respond better to Ocrevus that women – following our hazard ratio logic above, about 11% of male patients will be helped by Ocrevus, and as possibly much as 39%. And we can try to segment that further. For example, the hazard ratio for people 45 and younger was .64 (for both sexes) versus .88 for those over 45. So Ocrevus was exactly three times as effective in younger patients.
Those with active T1 lesions are also better than average candidates for Ocrevus. “Active” means you were injected with a chemical called gadolinium, then had an MRI, and the gadolinium caused the lesions to appear as bright spots on the images. This proves two things: one, that the gadolinium penetrated the “blood/brain” barrier, and two, that blood is flowing through some of your lesions, which labels them “active”. A “T1” MRI is calibrated to highlight active lesions, while “T2” images show both active and inactive ones.
Anyway, the hazard ratio, regardless of age, was .65 for this group, versus .84 for those without them. This makes sense, as we know b-cells play key roles in MS lesion activity. It’s no wonder that England will approve the drug only for patients with active lesions in the early stages of the disease.
If you’re on the fence with Ocrevus injections, demographics – and lesion activity – may be your best guides. You’ll also want to consider your overall health, beyond PPMS. The weaker your immune system due to other health problems, the more dangerous Ocrevus is probably going to be.
2. You want to roll the dice, and hope you get lucky
Ocrevus isn’t generally effective for women, but it did have a positive effect for a very small number of them. The same is true for patients over 50. You might get lucky and be in the minorities for which it works. You might get lucky a second time and escape serious side effects of the drug.
All this makes a critical assumption, of course: that the patients in the Study had the same response to Ocrevus as the general population would. We can say this confidently about the trial as a whole, but less confidently about the subgroups (age, sex, lesion activity, etc.). This is because the smaller number of patients in each subgroup reduces the predictive power of the results. So you’re going have to get lucky once more.
PPMS Blog 