This “Waning Immunity” Argument Against the FDA’s Covid Vaccine Approval Is a Scientific Quagmire

I sure don’t envy the people under all that pressure at the U.S. Food and Drug Administration. They’ve been at the brunt of an extended media barrage to approve Covid vaccines more quickly. I didn’t agree they were moving too slowly – I wrote about why over at The Atlantic – but at least I could see a basis for taking that position. In the U.S., a lot of mandates were waiting for that green light, and the country is suffering a heavy toll, given extensive resistance to both vaccination and containment measures.

There’s been a much smaller push in the opposite direction, though, urging them to slow way down. Peter Doshi, from the BMJ, continued arguing that case last week. If there is a strong science-based case to make against the FDA approving the vaccines now, though, this wasn’t it.

Doshi added a “waning immunity” argument, and looped in others he’s made previously – in January, contesting the efficacy of the mRNA vaccines, and then in June, when he and a group of others petitioned the FDA against approving the vaccines until deep into 2022 or 2023 when follow-up for the phase 3 trials is completed. That’s a lot to untangle. I wrote a detailed analysis of the January blog post here. Doshi had relied on some very weird arguments. For example, he estimated vaccine efficacy based not on the people whose cough- and cold-like symptoms were diagnosed as Covid-19, but on everyone who got tested for those symptoms. It didn’t matter, he claimed, whether someone’s respiratory illness was not caused by SARS-CoV-2: as if a Covid vaccine can only be considered effective if it also prevents the common cold.

What about the petition to slow down FDA approval? That essentially boils down to this: since the vaccines can already be used with their emergency authorization, the bar for full approval should be raised far higher than usual. They don’t provide a solid justification for this, especially in a pandemic. The FDA put it this way when they denied the petition: it “does not contain facts demonstrating any reasonable grounds for the requested action”.

Doshi’s latest argument was mostly about waning immunity against non-severe Covid-19. According to the FDA report of the data underpinning their approval of the BNT-Pfizer vaccine, about 60% of the people in the big phase 3 trial were 4 to 6 months past their vaccination. Vaccine efficacy across the whole time was 91% (CI 89-93) against symptomatic disease, and 95% (CI 71-99.9) against severe disease (only 1 vaxed person severely ill versus 21 in the placebo group). This was before the Delta variant arrived in force.

In a preprint from BNT-Pfizer, the authors reported efficacy of 84% (CI 75-90) against symptomatic disease in the time period from 4 months. It’s likely that at some point, the approval will be amended to cover boosters or for a 3-dose regimen, at least for some people. However, the possibility that there could be a modification to how a vaccine is used in the future is not a reason to wait another year for approval. And not needing further doses down the line was never required for a Covid vaccine.

There are serious problems in the picture Doshi paints. He argues that because the people in the placebo group were pretty much all vaccinated by 6 months, we won’t have good long-term data for the BNT-Pfizer vaccine. That focus on the specifics of a single trial is fundamentally misleading, though. The original phase 3 trial is not the only way to track declining vaccine effectiveness. For example, they have a trial underway for 10,000 people randomized to a booster dose or placebo 6 months after being vaccinated. That’s just from the manufacturer. There’ll be a vast amount of data and analysis on this around the world now the vaccine is in wide use. Helen Branswell reports that the CDC’s vaccination committee was just told that in addition to the over 40,000 people in the phase 3 trial, they are looking at studies with a total of 680,000 people in them. On top of all that, there’s a vast amount of safety monitoring globally.

Doshi writes he’s concerned about declining efficacy, “And so the recent reports from Israel’s Ministry of Health caught my eye. In early July, they reported that efficacy against infection and symptomatic disease ‘fell to 64%.’ By late July it had fallen to 39% where Delta is the dominant strain. This is very low. For context, the FDA’s expectation is of ‘at least 50%’ efficacy for any approvable vaccine”.

“And so…[this]…caught my eye”… It’s not often you see someone spell out their own cherry-picking and confirmation bias so explicitly!

The 2 “reports” Doshi links to aren’t reports from the Ministry of Health – they’re very thin media stories. So let’s put this data into some more accurate perspective before we get to the bunch of other studies on this question that Doshi doesn’t mention.

Israel vaccinated very fast before they hit something of a ceiling in people old enough for vaccines – and they have a large proportion of the population too young for them. They started on December 20, using the BNT-Pfizer vaccine. According to Ran Balicer, Chair of the Covid-19 National Experts Advisory Team, more than 80% of vaccinated Israelis had their first dose by March 1. The people most vulnerable to getting infected or suffering severe disease went first. The vaccinated and unvaccinated are fundamentally different groups at this point: statistician Jeffrey Morris estimated that more than 90% of people aged 50 or older are vaccinated, while more than 85% of the unvaccinated are under 50.

People who aren’t vaccinated are reportedly more geographically concentrated in parts of the country that the Delta outbreak didn’t reach – broad brush national data could be distorting the perspective. Likelihood of getting tested is another complication. And people living in the areas with major outbreaks (and more likely to be vaccinated) would be likely to be getting tested more. In other words, there are lots of confounders complicating the national picture.

Let’s look at the very low rate of effectiveness Doshi pointed to. The data is for a month from late June in a Delta outbreak. For symptomatic Covid-19, the vaccine effectiveness rate they estimated was 40.5%, with extraordinarily wide confidence intervals (CI) – from 9 to 61. For severe Covid-19, it was 91% (CI 83-96), but Doshi doesn’t mention that this was still high. The effectiveness for symptomatic disease was much lower for people who had their second vaccine dose in January and February versus March and April. For severe Covid-19, the January-vaxed rate was 86%; February-vaxed was 91%; March-vaxed was 94%; April-vaxed was 84% (with very large uncertainty). (If you want to dig into the methodology Israel is using for these national estimates, it was described with an earlier batch of the data here.)

There’s now a preprint of a different study on the Israeli data for a shorter period in July that has similar results. Plus very early data on boosters given to people aged 60 and over suggest it might have brought effectiveness back to the high levels of first being fully immunized. Small numbers and the short time period mean there’s too much uncertainty about the difference it might have made for severe disease. In addition, there were differences in who got the boosters – the proportion of people who weren’t Arabic, were originally vaccinated in January rather than February, and/or were over 70 was higher among the booster-vaxed, for example. Of course, there are groups of people who have weaker responses in the first place, and they were also more likely to be in the vulnerable groups vaccinated first. For at least some, a 3-dose course may make an important difference. It’s still early days for publicly available data on third doses for others, though. And we’re nowhere near the stage of discussing boosters, in the sense of regular “top-ups”.

Severe disease and the life-threatening outcomes that come with it are not the only outcomes that matter, but they are critical. As Branswell writes, before we saw the early results of the BNT-Pfizer and Moderna phase 3 trials, “the world would have been ecstatic at the prospect of vaccines that prevented severe Covid infections in most vaccinated people. But vaccine efficacy estimates in the 90% range against even mild infections reset what we thought these vaccines would be able to do for us”. She describes us being in a process of having our expectations “sinking back to earth”.

Doshi describes severe disease as crucial, too, and points out that the BNT-Pfizer phase 3 trial wasn’t designed to definitively answer this question. That’s true. It’s important to know, he writes, whether there’s a decline in efficacy against severe disease, and he criticizes BNT-Pfizer for not reporting “the results in a way that allows for evaluating this question” in the preprint. But that’s absurd. The supplementary file to the preprint makes it clear that in the entire period from dose 1, there was just a single case of severe disease in the vaxed group. And we already know all we need to about that single vaxed person counted as severe, because that happened early in the trial, not after several months.

Here’s what the FDA reported about that person back at the emergency use authorization stage: “The vaccine recipient who had severe COVID-19 disease met the severe case definition because oxygen saturation at the COVID-19 illness visit was 93% on room air. The subject was not hospitalized, did not seek further medical care, and did not have risk factors for severe disease”. Most of us would consider not needing oxygen or medical care only technically “severely ill”. And according to the FDA, this person didn’t meet the CDC definition for severe, either. But even if that person had gotten sick in the later months, the numbers would still be too small for meaningful evaluation.

While the antibodies that protect against getting infected decline, it’s expected to do that to some extent. It doesn’t necessarily mean people are no longer protected. Cellular immunity (non-antibody immunity, such as from T-cells) can help clear an infection without the person getting severely ill. (There’s a good article by Céline Gounder explaining this here.)

I think there’s a convincing body of data that the BNT-Pfizer and Moderna vaccines’ protection against severe disease and life-threatening outcomes holds up well, both against Delta and against 6 months of time. (The J&J vaccine hasn’t been used as long, and other vaccines are not FDA-approved, so aren’t relevant to the FDA argument.) Israel is not the only place that has substantial numbers of people vaccinated 6 months ago, and differences in other studies aren’t typically as dramatic as the reports coming from there. If you want to dig into the detail of how I reached this conclusion, I’ve summarized another 14 studies I’ve seen so far on durability of the BNT-Pfizer and Moderna vaccines below this post – 4 of them measuring immune responses, and 10 analyzing effectiveness/health outcomes. Some of them were released after Doshi’s BMJ blog post was published, but many were already available.

There’s sure to be so much data coming now, that it’s going to be a difficult phase to navigate. Be prepared for a lot of fear-mongering about vaccine effectiveness, and some awfully confusing conflicting messages. However, one thing is still unambiguously clear. We need to vaccinate the unvaccinated and under-vaccinated – everywhere. That remains urgent, both scientifically and morally. Casting doubts on the vaccines with smoke and mirrors, and scaring well-protected people into boosters before they’re needed, are both natural enemies of that absolute goal. Superficial and scientifically specious hot takes on powerful medical platforms is the last thing we need.

Please don't look away!

My new post @NatureMicrobiol offers

10 Images That Illustrate The Shameful Global Vaccine Inequityhttps://t.co/5iwINN1IeN #COVID19 #vaccinequity

— Madhu Pai, MD, PhD (@paimadhu) August 28, 2021

~~~~

All my Absolutely Maybe Covid-19 vaccine posts

All previous Covid-19 posts at Absolutely Maybe

My posts at The Atlantic, at WIRED, and debunking posts at my personal website.

Disclosures: My interest in Covid-19 vaccine trials is as a person worried about the virus, as one of my sons is immunocompromised: I have no financial or professional interest in the vaccines. I have worked for an institute of the NIH in the past, but it was NCBI at the National Library of Medicine, not the one working on vaccines (NIAID). More about me. I am an occasional (unpaid) contributor to BMJ Opinions, but haven’t done so since the pandemic started (though that may be about to change). I’m a (paid) adviser to a BMJ publication, The Drugs and Therapeutics Bulletin. Doshi and I have agreed and disagreed on important issues over the years, but I don’t think I’ve ever criticized his work or views publicly before rebutting his claims about the BNT-Pfizer vaccine in January this year. However, we were on opposing sides of a long-running debate about HPV vaccines, and I actively disputed claims from some of his colleagues (Peter Gøtzsche, Carl Heneghan, and Tom Jefferson): that may at some point have included criticism of commentary he co-authored on that vaccine. 

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny.)

List of additional* studies on effectiveness of BNT-Pfizer and Moderna vaccines over time, as at August 31

* The national studies from Israel discussed in the post above are not included here. The studies below include many preprints.

Immunogenicity studies (studies of immune response, not health outcomes)

• Researchers from Pennsylvania analyzed blood samples from people vaccinated with BNT-Pfizer or Moderna vaccines at 6 times, up to 6 months. While antibodies declined substantially, cellular immunity didn’t – and memory B cells continued to increase between 3 and 6 months.
• A study of antibodies in 120 healthcare workers and 92 nursing home residents in New Hampshire 6 months after BNT-Pfizer vaccination found a decline, which was steep in older people. Cellular immunity wasn’t measured.
• In Germany, a study of both types of immune responses in 107 healthcare workers and 82 elderly people (median age of 83) found that both types declined substantially in the elderly 6 months after BNT-Pfizer vaccination; much less so in the healthcare workers (median age of 35).
• A study from a database of test results of members of a health insurer in Israel, analyzing antibody decline only in 2,653 people vaccinated with BNT-Pfizer, and 4,361 people who had tested positive without being vaccinated, between January and July 2021. Rates for the vaccinated people started out higher, and declined more rapidly than those for the unvaccinated people who had Covid-19. It wasn’t a matched control study. The group who had Covid-19 were very different: for example, they were much younger, and more likely to be Arabic and of lower socioeconomic status.

Effectiveness and health outcome studies

• A study in New York found some decline in vaccine effectiveness against infection between May and July – from 92% to 80% – with a lot of decline in the young but not in those aged 65+; protection against hospitalization remained over 90% across the board. The vaccines were mostly BNT-Pfizer and Moderna, but this study included some people vaccinated with J&J. Additional potential issues the authors raised that may contribute to decline was changes in protective behavior (especially since the decline was in younger people), high levels of vaccinated people providing some protection to the unvaccinated, and increasing immunity in unvaccinated people from infection. Note: No data on effectiveness against symptomatic disease. The US also began vaccinating in December and started off very quickly.
• A case-control study of durability of vaccine protection against hospitalization with data from 21 hospitals in 18 U.S. states from March to July estimated vaccine effectiveness of 86% (CI 82-90) between 2 and 12 weeks after vaccination, and 84% (CI 77-90) between 13 and 24 weeks (around 3 to 6 months). (There were 1,195 case-patients vaccinated with BNT-Pfizer and Moderna, and 1,895 control-patients.)
• An analysis of Kaiser Permanente records in Southern California adjusted for some demographic and comorbidity variables found that BNT-Pfizer vaccine effectiveness against infection known to be caused by Delta was 93% (CI 85-97) in the first month, declining to 53% (CI 39-65) for longer than 4 months, but it stayed over 90% for hospitalization. However, they didn’t know the variants for all infections, and they didn’t have data on protection against symptomatic disease.
• An analysis from the Mayo Clinic in Minnesota compared people vaccinated with BNT-Pfizer or Moderna vaccines with a group matched for age, sex, race, history of prior SARS-CoV-2 PCR testing, and date of full vaccination. In July when Delta was prevalent, vaccine effectiveness against hospitalization was 75% for BNT-Pfizer vax (CI 24-94) and 81% for Moderna (CI 33-96). That was similar to the rates from January, but the estimated rate of effectiveness against infection dropped substantially: from 76% down to 42% (CI 13-62) for BNT-Pfizer and from 86% to 76% (58-87) for Moderna. (No estimates for symptomatic disease.) Comparing data from several states, they found a substantial difference between BNT-Pfizer and Moderna vaccines.
• The HEROES-RECOVER study, in which American healthcare workers and other essential workers were tested weekly, released data up to August. Among the 3,483 vaccinated (most of the participants), almost all were BNT-Pfizer or Moderna, mostly BNT-Pfizer.
• A study from the Utah Department of Health involved limited data. They estimated that vaccine (mostly BNT-Pfizer and Moderna) effectiveness against testing positive was 90% in mid-May and 83% by the end of June, with the drop coinciding with the rise of Delta.
• Randomly selected households across the UK were studied, adjusting for many key variables, and analyzing the effectiveness of BNT-Pfizer and AstraZeneca vaccines. The BNT-Pfizer vaccine had higher initial effectiveness against infection and symptomatic disease than AstraZeneca, but this waned more quickly. However, they did not report the estimates for vaccine effectiveness, and did not have data on severe disease and hospitalization. Vaccine effectiveness against infections for 18-64 year-olds in the Delta-dominant phase was 82% for BNT-Pfizer (CI 79-85) and 67% for AstraZeneca (CI 62-71). (The authors of this study concluded longer dosing intervals didn’t improve effectiveness in this group.)
• A study in people 65 and older in Portugal found high protection against hospitalization and death from BNT-Pfizer and Moderna vaccines (over 94% for 65-79 year-olds and over 81% for people 80 and older), with no waning up to about 3 months after the second dose. Vaccine effectiveness against infection was estimated as 85% for that vaxed up to about 4 months earlier (CI 68-93), 81% for those vaxed up to about 5 months previously (CI 34–95), and 73% for those vaxed more than 5 months previously (CI 49–86). For Delta overall, effectiveness was estimated at 66% (CI 26–84) – for a quarter of the vaccinated people infected with Delta, there were no symptoms. This study didn’t report vaccine effectiveness against symptomatic and severe disease.
• A study from a health insurer in Israel found that the risk of testing positive was higher for those who were vaccinated earlier, but the authors don’t report effectiveness rates, or the risk of symptomatic disease, hospitalization, or severe disease.
• Then there’s a real outlier, a test-negative matched case-control study from Qatar, and it leaves me with a lot of questions. For example, if I’m understanding their description of their methods correctly, all the people who got Covid-19 in 2020 – and their outbreak then was far larger than their outbreaks in 2021 – and didn’t get vaccinated, could be included as controls if they got a Covid test, even though it would mean they could have some non-vaccine-induced immunity. (There have already been results that need explaining from previous uses of this study’s methodology with the Qatar data. Statistician Peter Dalgaard has pointed to several issues.) The authors estimated that BNT-Pfizer vaccine effectiveness against symptomatic disease only ever got as high as 80% (CI 78-81), then declined gradually until 20 weeks (about 5 months), and suddenly fell off a cliff. There was almost no change in effectiveness against hospitalization – about 95% – up to 6 months, and protection against severe, critical or fatal disease was about 90% across time. Longer than 6 months, effectiveness against severe, critical, or death dropped to 72%, but the uncertainty was huge because there was too little data: CI was from 9 to 93.