Stop Ignoring the Evidence on Covid-19 Treatments

Thanks to Twitter, you can now watch a doctor’s heart break in real time. Like everyone else, we’ve often made our feelings plain during the pandemic—our despondence over all the deaths, our anger over their preventability—but there’s another sort of public display that’s more special to our discipline. I like to call it publication humiliation. It comes out when you realize that the published data on a favored treatment just aren’t on your side.

There was plenty of publication humiliation to go around a couple years ago, when studies started coming out against the magical healing powers of Vitamin D. Researchers had noticed that people with low Vitamin D levels seemed to have a greater chance of developing a range of medical problems, and many serious physicians bought right in. Recent data from well-designed clinical trials suggested otherwise. Could taking Vitamin D prevent cancer or heart disease? Well, no. What about diabetes and depression? No, and no again. But heartbreak, as it often does, played out as denial. It wasn’t the treatment that was wrong; it was the science used to study it. If randomized controlled trials came out against the use of Vitamin D, that’s because they weren’t done correctly. Maybe the doses were too low to have an effect; or else, if the doses were high enough, then the timing wasn’t right. “If you are already too sick or have a disease, it is too late for Vitamin D,” one doctor tweeted when a major trial found the treatment wasn’t saving any lives. (Never mind the fact that prevention trials also come up short.)

If some doctors like to close their eyes in grief, others dig for deeper answers in the data. A “subgroup analysis”—for which you may end up picking out only the parts of a data set that happen to support your theory—is a useful tool in this regard. Doctors who were undeterred by the 26,000-person study on Vitamin D supplements and cancer quickly got to work on a second publication drawing from the same results. This one suggested that the vitamin could, at least, prevent more serious cancers … so long as you were only looking at the skinny patients. (If that hadn’t panned out, they might have tried dividing up the patients by eye color or favorite Seinfeld episode.)

The latest source of publication humiliation is convalescent plasma, the alleged Covid-19 wonder drug drawn through a needle from people who have recovered from infection. Last summer, former head of the Food and Drug Administration Stephen Hahn promised that recipients would see a “35 percent improvement in survival.” Experts quickly pointed out that he was looking at only the tiny subset of the data which was most favorable to plasma. Subgroup analysis strikes again! But still, doctors jumped at any chance to help their patients. In December alone, more than 100,000 units of convalescent plasma were given out in the US.

Our enthusiasm was not entirely unfounded. It’s reasonable to think that giving sick patients someone else’s naturally occurring antibodies might help their recovery along, even save their life, and doctors have tried convalescent plasma to treat viral illnesses at least as far back as the 1918 Spanish flu. Here’s the problem, though: The evidence for its benefit has never been very good. I can forgive those old-timey 1918 doctors, but a systematic review of published work as of 2013 drew from “predominately low-quality, uncontrolled studies.” Then this month, the most important medical study of the pandemic—the UK’s Recovery trial—put out its preliminary results on plasma, and they aren’t promising at all. Eighteen percent of hospitalized Covid patients who received the treatment died within 28 days, versus 18 percent of patients who didn’t receive plasma. You don’t need to be a scientist to understand the implication: Plasma didn’t help.

You might think this would be enough to change some minds. After all, Recovery’s simple, randomized design has been definitive for other drugs. Enrolling tens of thousands of participants, it has been able to answer our most important question about a number of potential Covid treatments: Does it actually prevent you from dying? For hydroxychloroquine or the antibiotic azithromycin, the answer was no. For the steroid dexamethasone, it was yes. Now, for convalescent plasma, it appears we have another no.