Blog

As we reported in our previous blog, the preliminary results from phase III trials of COVID-19 vaccines have been reported in the media. The vaccine developed and trialled by Pfizer and BioNTech is reported to be 95% effective and Moderna’s is 94.5% effective.

In contrast to these precise figures, reports about the vaccine being developed by Oxford University and AstraZeneca say it “may be up to 90 per cent effective“. The results from the trial show that the vaccine is 70% effective, but “with potential for that to rise to 90 per cent”.

These results are less conclusive than the other studies because of a dosing error. Scientists noticed that some trial participants had weaker adverse reactions to the vaccine. They quickly realised that the first dose, out of a 2 dose regimen, was half the strength. This was not part of the trial’s design as it did not follow what should have happened according to the trial’s protocol. After reporting this to the regulators the trial was allowed to continue with the doses as originally planned.

Reporting the results for the trial overall (following the protocol, as required) the vaccine is 70% effective, however, for the group of participants who received an initial half dose, the results show the vaccine to be 90% effective. This is good news – a serendipitous consequence of an accident. There have been many important scientific discoveries that were the result of an accident; Penicillin being one well-known example.

You’ve taken two studies for which different doses were used and come up with a composite that doesn’t represent either of the doses.

While these are genuine results from the trial, the scientists are being cautious in their interpretation. This is for a number of reasons.

While the data have been analysed according to the protocol, finding the vaccine to be 70% effective overall, this is actually an average across two subgroups of participants, which may be problematic. As one expert explained “You’ve taken two studies for which different doses were used and come up with a composite that doesn’t represent either of the doses.”

Reporting results for the two subgroups separately also has problems. The original intention of the trial was for everyone to receive two equal doses and the sample size was calculated accordingly. Analysing the data as two subgroups is likely to mean that each analysis is underpowered. This means that the number of participants in the sample (the number in the subgroup) is too small to be confident that the results are not due to chance, and that the same results would be obtained if the study were to be repeated with another sample. This would particularly apply to the smaller group (where the vaccine was found to be 90% effective).

An additional reason to be careful interpreting the results is that since they were obtained from an unplanned subgroup of participants, the characteristics of the subgroup may not reflect the characteristics of the general population. The full sample of the trial was designed to reflect the general population, and since those who received the initial half dose were the first participants to take part, there may be features of the way the trial was rolled out which mean that this group is not representative of the population. Indeed, the participants who received the initial half dose were all under 55, so we do not know the effectiveness of the initial half dose in older participants who are more severely affected by COVID-19.

So while the preliminary results of the Oxford University and AstraZeneca trial seem encouraging, and full results will be published in a peer-reviewed journal which will allow further scrutiny of the data, there is only one way to resolve these issues. Oxford University and AstraZeneca are now conducting a new trial to properly test the half dose regimen.