Efficacy and Effectiveness

From the headlines, you might well assume that Big Pharma’s Covid vaccines are going to protect 95 out of 100 people who get them. 

Truth be told, that’s not actually what the trials have shown. Exactly how the vaccines perform out in the real world will depend on a lot of factors we just don’t have answers to yet – e.g. whether vaccinated people can get asymptomatic infections. 

The fundamental logic behind today’s vaccine trials was worked out by statisticians over a century ago. 

Researchers vaccinate some people and give a placebo to others. They then wait for participants to get sick and look at how many of the illnesses came from each group.

In the case of Pfizer, the company recruited 43,661 volunteers and waited for 170 people to come down with symptoms of Covid-19 and then get a positive test. Out of these 170, 162 had received a placebo shot, and just eight had received the real vaccine. 

From these numbers, Pfizer’s researchers calculated the fraction of volunteers in each group who got sick. Both fractions were small, but the fraction of unvaccinated volunteers who got sick was much bigger than the fraction of vaccinated ones. The scientists then determined the relative difference between those two fractions. Scientists express that difference with a value they call efficacy. 

If there’s no difference between the vaccine and placebo groups, the efficacy is zero. If none of the sick people had been vaccinated, the efficacy is 100 percent. 

A 95 percent efficacy is certainly compelling evidence that a vaccine works well. But that number doesn’t tell you what your chances are of becoming sick if you get vaccinated. 

And on its own, it also doesn’t say how well the vaccine will bring down Covid-19 all across the globe. 

Efficacy and effectiveness are related to each other, but they’re not the same thing. And vaccine experts say it’s crucial not to mix them up. 

Efficacy is just a measurement made during a clinical trial. “Effectiveness is how well the vaccine works out in the real world,” said Naor Bar-Zeev, an epidemiologist at the Johns Hopkins Bloomberg School of Public Health. It’s, of course, possible that the effectiveness of coronavirus vaccines will match their impressive efficacy in clinical trials. 

But if previous vaccines are any useful guide, effectiveness may prove somewhat lower. The mismatch comes about because the people who join clinical trials are not a perfect reflection of the population at large. Out in the real world, people may have a host of chronic health problems that could interfere with a vaccine’s protection, for example. 

The clinical trials run by Pfizer and other companies were specifically designed to see whether vaccines protect people from getting sick from Covid-19. If volunteers developed symptoms like a fever or cough, they were then tested for the coronavirus. 

But there’s abundant evidence that people can get infected with the coronavirus without ever showing symptoms. And so it’s possible that a number of people who got vaccinated in the clinical trials got infected, too, without ever realizing it. If those cases indeed exist, none of them are reflected in the 95 percent efficacy rate. 

Anyway, good luck to those who are getting vaccinated!