Blog

In the past few weeks, we’ve heard from multiple pharmaceutical companies that interim results of their Covid-19 vaccines are showing surprisingly good results. Pfizer and BioNTech announced their vaccine candidate was found to be more than 90% effective and less than a week later Moderna announced that their vaccine was protecting 94.5% of people.

But what does this actually mean? How do we know what percentage of a population a vaccine is able to protect when we don’t know how many people may have been exposed to the virus? In standard clinical drug trials the treatments are given to people known to be suffering from the disease in question. This is not the case with infectious diseases where not everyone will be exposed to the infection, and even if they are exposed, they may not develop a case of their own. Researchers don’t know for certain who will contract COVID-19 which makes it difficult to decipher how well the vaccine has protected people.

How do we know who a vaccine is able to protect when we don’t know how many people may have been exposed?

All clinical trials go through several phases and it’s not until phase III where they look at how well the treatment or vaccine works on a large number of participants. Usually half of the participants who are enrolled into a trial will receive the vaccine and the other half (known as the control group) will receive a placebo — likely an identical looking vaccine which contains no therapeutic effect. Using a placebo means that we can calculate the prevalence of the virus (the proportion of people suffering from the disease) in the control group, and then compare this to the group who did receive the active vaccine.

Trials are usually run blind, meaning that the participants don’t know whether they receive the treatment or the placebo. In double-blind trials the scientists as well as the participants do not know which participants receive the treatment or placebo. So scientists running the trial will count the number of participants who are confirmed to have COVID-19, without knowing whether those people had been given the candidate vaccine or the placebo. Once the number of cases reaches a previously agreed upon number (determined by sample size calculation), the results are “unblinded” and it is revealed how many of the cases were contracted by participants given the vaccine compared to those given the placebo.

To calculate how effective the vaccine is, we compare the proportion of positive cases in the vaccinated group to the proportion of positive cases in the placebo group:

But, since we have the same numbers of participants in each group, we can simply calculate

we get the percentage effectiveness of the vaccine.

Now, looking at the results that have been in the news we can use this to interpret what they actually mean. Pfizer’s interim analysis reported that the number of cases in their trial participants had reached 94. At this point they would have unblinded the trial so that the scientists could see the number of cases among individuals who had been given the candidate vaccine. To reach 90% effectiveness, they would have found 8 or fewer cases among the vaccinated group out of the 94 total.

More recently, Pfizer and BioNTech have announced that they’ve reached the end of their phase III study. They detail that there were 170 confirmed cases of COVID-19 within their study participants, with 162 in the placebo group and still only 8 in the vaccinated group, increasing their effectiveness to 95%.

Moderna also recently published their interim analysis results. Out of 95 cases, only 5 were in the vaccine group which gives an effectiveness of 94.5%.

More and more companies are announcing similarly positive results, including encouraging results from the Oxford vaccine. Thanks to the use of placebos, researchers are able to calculate how effective the vaccine is which is imperative in the process of a vaccine gaining approval.