A Unique Pandemic Control Trial

Image credit

Cartoon by Hilda Bastian, CC BY-NC-ND 4.0

When I was young, while I understood pandemics that kill millions could happen in theory, I really thought they were a thing of the past. But now two of the five biggest in recorded history have emerged in my adulthood (HIV and Covid-19). Since SARS crossed over from animals to humans in 2002, other life-threatening corona- or influenza viruses have spread from animals to us every few years. [*] Scientists predict these outbreaks of novel diseases will be more frequent as climate change pushes previously isolated wildlife into close-enough contact to spread viruses to humans.

That makes knowledge about what definitely works to protect communities in pandemics critical. However it’s extremely difficult to run large-scale research projects quickly in such complicated situations—and fear of leaving people exposed in control groups can prevent research getting off the ground. At the same time, opponents of interventions use that lack of data to whip up fear and opposition to acting. As a result, each time a pandemic hits we’re exposed to both infection, and campaigns against the interventions that could protect us.

So it’s impressive when anyone even tries to get an ambitious trial in the field, let alone when a community pulls it off. We need them for vaccines, too. As amazing as the large Covid vaccine drug approval trials were, they could only provide data about some individual-level effects. That makes it easy for critics to suppress uptake and discourage policies to enable mass vaccination: They can focus on known limitations for individuals while fear-mongering about harms.

A powerful way to get that population-level evidence would be controlled trials of vaccine rollout in a pandemic. In addition, as Hemkens and Goodman (2021) pointed out, that has the potential to improve our knowledge of individual-level effects. The vaccine approval studies weren’t designed to evaluate many outcomes, such as rates of mortality, hospitalization, and uncommon adverse effects. They argued if a few million of people in a state or country were randomized to be vaccinated first in a rollout, pandemic surveillance data would be akin to having a randomized trial with 200,000 participants: Many people have to wait longer than others anyway. Hemkens and Goodman point to examples of places where people on waiting lists are randomized to medical appointments to ensure fair access—and it was done with the Medicaid expansion in Oregon as well.

I only saw two randomized trials of Covid vaccine rollout in communities get as far as being registered in 2021. One of them didn’t get off the ground in the end. It was a trial set up by a team at McMaster University in Canada with a Hutterite community. About 4,000 people would have been randomized to either an experimental group receiving early vaccination with the Moderna vaccine, or to waiting till the national rollout reached them. That community was relatively isolated from other communities, but with a lot of communal interaction within it—a particularly suitable setting to test for community (“herd”) immunity. The trial didn’t happen, though, because the researchers’ plans were overtaken by the speed of Canada’s national vaccine rollout.

The other community did pull their trial off, and it was far bigger. The town was Serrana, near São Paulo in Brazil. The trial was proposed and run by a team from the Butantan Institute, a public research agency in São Paulo. The Institute was manufacturing doses of CoronaVac, the inactivated Covid vaccine developed by Sinovac in China. It was the most widely used Covid vaccine in the world, but as with most very large countries, Brazil was not going to have enough vaccine for its whole population in 2021.

Serrana has a population of about 45,000 people. Many people who live there work in other towns, and Serrana was particularly hard hit by Covid. The mayor told reporters, “Our small health system collapsed. It was like a very dark cloud was above the town.” The community welcomed the chance for priority vaccination that the trial offered.

This unique trial’s results were published a few weeks ago (Carvalho Borges 2026). Researchers had divided the town into 25 subareas, and then grouped them into four clusters which were randomized to a spot in the vaccine queue. Residents in the first cluster could get a vaccine dose in week one. One week later, the next cluster would be eligible for their first dose, and so on, until all the trial participants had a chance at vaccine. At the end of a month, when all clusters had been offered the first dose, people in the first cluster were due for their second vaccine dose, and the process was repeated.

This is called a stepped wedge design, because a table showing the step-by-step progress towards treatment for everybody looks like a set of stairs:

Image credit

Image by Hilda Bastian, CC BY-NC-ND 4.0

That shape also highlights one of the weaknesses of stepped-wedge trials: The size of the control group shrinks relative to the treatment/intervention arm as time goes on, and changes over time in the community unrelated to the intervention aren’t controlled for as they are when treatment and control groups happen in parallel (Hemming 2015). In the Serrana trial, there was only a week between each cluster, though.

It was exciting watching reports from Brazil as this trial unfolded and the people of Serrano stepped up. Trial enrolment began soon after the study was announced, and it was organized through schools like a local election. Adults were eligible for the trial (with some exceptions), and 83% of all adults in the town ended up getting at least the first dose: 27,390 participants. That was 62% of the whole population—adults and children—in Serrano. (You can read more about the trial in this media report from June 2021.)

Vaccination started in February 2021, and data was collected for a year. The Gamma variant was dominating the pandemic there at the time, with Delta taking over in late August. By the middle of October, Omicron was prevalent. Boosters (third doses) hadn’t been part of the trial design, and most of the trial participants ended up getting a booster—not necessarily CoronaVac—under the national program from August.

CoronaVac was one of the Covid vaccines with the lowest effectiveness (and very low rate of adverse reactions), and there was less data from vaccine approval studies. A systematic review concluded that the trial evidence only provided low-certainty evidence even for the primary outcome of confirmed symptomatic Covid (Graña 2022). The WHO assessment of the vaccine cited a vaccine efficacy rate of 51% with a large range of uncertainty [CI 36–62%] for symptomatic Covid, with little confidence in the rate of more serious outcomes, for example hospitalization [CI 56–100%]. Those early trials were in relatively lower risk people, and they weren’t designed to provide strong certainty about very uncommon outcomes.

The Serrana trial provides more data on serious outcomes. Before the more severe variants arrived and vaccine efficacy waned, the efficacy rate against hospitalization and death was 89.2% [CI 68.1-96.3] from February to May, and 86.8% [CI 72.2-93.7] from May to August.

For the first seven weeks after vaccination started, the rate of hospitalization and death in Serrana was as high as other cities in the region. Then the rate in Serrana dropped while it stayed high in the other cities.

Based on the city’s surveillance data for Covid, mass vaccination had a major impact on circulation of the disease. Carvalho Borges and colleagues report “When approximately 50% of the adult population was fully vaccinated, a reduction in symptomatic COVID-19 was also observed among participants who were not yet fully vaccinated.”

We need multiple studies to build up this knowledge base, especially given the inevitable messiness of a large community trial like the Serrana trial. Still, this study provides an indication about the impact vaccination can have in a pandemic, even if the vaccine isn’t the most powerful one at an individual level—as long as enough people get vaccinated. Perhaps even more importantly, it shows that ambitious pandemic trials are possible. Here’s hoping the Brazilian precedent helps other researchers and communities to aim higher next time.

You can keep up with my work at my newsletter, Living With Evidence. And I’m active on Mastodon: @hildabast@mastodon.online and less so on BlueSky (hildabast.bsky.social).

~~~~

The cartoon and stepped wedge trial graph are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny.)

Disclosures: My interest in Covid-19 vaccine trials began as a person worried about the virus, as my son was immunocompromised: I have no financial or professional interest in the vaccines. I have worked for an institute of the NIH in the past, but not one working on vaccines. I maintain a list of financial disclosures here.

* List of coronavirus or influenza epidemics caused by zoonoses (diseases crossing over from animals) in the 2000s (via Wikipedia):

2002 SARS (coronavirus)

2003 H5N1 (avian flu)

2009 H1N1 (swine flu) (pandemic)

2012 MERS (coronavirus)

2013 H7N9 (avian flu)

2015 H1N1 (swine flu)

2019 Covid-19 (coronavirus) (pandemic)

Return to para