What Does Vaccine Effectiveness Mean?

 When technical specialists adopt an everyday word, they often give it a meaning that differs from its everyday use.  This can be misleading for nonspecialists, especially when little effort is made to explain the difference.  A well known example is “significance”, which means one thing when used in statistical work and another when it just denotes whether something is important.

Let’s look at another example, vaccine “effectiveness”.  What do people most want to know about the coronavirus vaccines?  How much protection they give you against the risk of getting infected with the virus, right?  And how much protection they give against more severe symptoms, such as those requiring hospitalization or resulting in long Covid.  When public health authorities throw out numbers about vaccine effectiveness, that’s probably how most people interpret them.

But that’s not what effectiveness means in medical research.  When pharmaceutical companies or public health outfits conduct effectiveness tests, they assemble and compare two groups, a treatment and a control (or multiple treatment groups with different protocols).  The treatment group gets the vaccine, the control group doesn’t.  Who gets assigned to which group is determined randomly, and participants don’t know which one they’re in.  (The controls get injected with a placebo.)  Then they go about their life, monitored to see if they get infected or not.  Vaccine effectiveness is a ratio, the fraction of the control group that gets infected divided by the corresponding fraction of the treatment group; it’s a ratio of two ratios.  You can also calculate effectiveness within subgroups, like treatments-over-65 and controls-over-65.  If the trial is conducted properly, the samples are representative and large and the public health context, including the virus variant, is stable, you can generalize effectiveness in the samples to the population as a whole.

Now notice a subtle difference in language.  The everyday use of “effectiveness” is effectiveness against the virus.  The research use is effectiveness relative to the control group.  This is immense, but widely misunderstood and seldom explained.

Here’s a numerical example.  Suppose a typical unvaccinated person going about life in a typical way faces a 1% risk of getting infected with Covid over the course of a month.  Suppose also that a vaccine is introduced with 95% effectiveness, as that term is used in medical research.  This means that a vaccinated person exposed to the same risk factors would have only a .05% chance of getting infected during the same time period.

Next, imagine that a new virus variant appears, combined with more relaxed public behavior—more indoor gathering, less masking.  Let’s say that an unvaccinated person now has a 5% monthly risk of infection.  If the vaccine is equally effective against the new variant, our typical vaccinated person now has a .25% risk of infection.  The numerator and denominator have both risen fivefold, but the effectiveness ratio of treatment vs control is unchanged.  Suppose further that the vaccine loses effectiveness against the new variant; it is now just 40% rather than 95%.  40% of 5% is 2%, the new monthly infection risk of those who have been vaccinated.

Effectiveness in the research sense has fallen from 95% to 40% from the first scenario to the second, smaller but still noticeably positive, but the risk of infection faced by someone who has been vaccinated in the second scenario is greater than the risk faced by someone unvaccinated in the first.

These numbers were made up, and of course the notion of a typical individual is a gross oversimplification, but the point applies to the current situation.  We now have vaccines against the coronavirus, and we also have a new, much more transmissible variant.  Effectiveness as researchers understand it has fallen, perhaps to about 40% for those vaccinated more than four months ago.  It’s still very important to get vaccinated, since it reduces both the risk of infection and the risk of infecting others relative to not being vaccinated, but even so you may well be at greater risk of infection now than you were a year ago before the vaccines were introduced.

The bottom line: vaccine effectiveness measures the risk faced by vaccinated individuals compared to those who aren’t vaccinated.  If the risk rises for the second group it rises for the first, even more if effectiveness is also falling.