Wednesday, August 4, 2021

What Does Vaccine Effectiveness Mean?

 When technical specialists adopt an everyday word, they often give it a meaning that differs from its everyday use.  This can be misleading for nonspecialists, especially when little effort is made to explain the difference.  A well known example is “significance”, which means one thing when used in statistical work and another when it just denotes whether something is important.

Let’s look at another example, vaccine “effectiveness”.  What do people most want to know about the coronavirus vaccines?  How much protection they give you against the risk of getting infected with the virus, right?  And how much protection they give against more severe symptoms, such as those requiring hospitalization or resulting in long Covid.  When public health authorities throw out numbers about vaccine effectiveness, that’s probably how most people interpret them.

But that’s not what effectiveness means in medical research.  When pharmaceutical companies or public health outfits conduct effectiveness tests, they assemble and compare two groups, a treatment and a control (or multiple treatment groups with different protocols).  The treatment group gets the vaccine, the control group doesn’t.  Who gets assigned to which group is determined randomly, and participants don’t know which one they’re in.  (The controls get injected with a placebo.)  Then they go about their life, monitored to see if they get infected or not.  Vaccine effectiveness is a ratio, the fraction of the control group that gets infected divided by the corresponding fraction of the treatment group; it’s a ratio of two ratios.  You can also calculate effectiveness within subgroups, like treatments-over-65 and controls-over-65.  If the trial is conducted properly, the samples are representative and large and the public health context, including the virus variant, is stable, you can generalize effectiveness in the samples to the population as a whole.

Now notice a subtle difference in language.  The everyday use of “effectiveness” is effectiveness against the virus.  The research use is effectiveness relative to the control group.  This is immense, but widely misunderstood and seldom explained.

Here’s a numerical example.  Suppose a typical unvaccinated person going about life in a typical way faces a 1% risk of getting infected with Covid over the course of a month.  Suppose also that a vaccine is introduced with 95% effectiveness, as that term is used in medical research.  This means that a vaccinated person exposed to the same risk factors would have only a .05% chance of getting infected during the same time period.

Next, imagine that a new virus variant appears, combined with more relaxed public behavior—more indoor gathering, less masking.  Let’s say that an unvaccinated person now has a 5% monthly risk of infection.  If the vaccine is equally effective against the new variant, our typical vaccinated person now has a .25% risk of infection.  The numerator and denominator have both risen fivefold, but the effectiveness ratio of treatment vs control is unchanged.  Suppose further that the vaccine loses effectiveness against the new variant; it is now just 40% rather than 95%.  40% of 5% is 2%, the new monthly infection risk of those who have been vaccinated.

Effectiveness in the research sense has fallen from 95% to 40% from the first scenario to the second, smaller but still noticeably positive, but the risk of infection faced by someone who has been vaccinated in the second scenario is greater than the risk faced by someone unvaccinated in the first.

These numbers were made up, and of course the notion of a typical individual is a gross oversimplification, but the point applies to the current situation.  We now have vaccines against the coronavirus, and we also have a new, much more transmissible variant.  Effectiveness as researchers understand it has fallen, perhaps to about 40% for those vaccinated more than four months ago.  It’s still very important to get vaccinated, since it reduces both the risk of infection and the risk of infecting others relative to not being vaccinated, but even so you may well be at greater risk of infection now than you were a year ago before the vaccines were introduced.

The bottom line: vaccine effectiveness measures the risk faced by vaccinated individuals compared to those who aren’t vaccinated.  If the risk rises for the second group it rises for the first, even more if effectiveness is also falling.


Sandwichman said...

What people seem not to grasp is that the 95% effectiveness finding assumes that all else is equal. If vaccinated people behave as if they are now invulnerable, their infection rate will increase regardless of the vaccine's rated effectiveness. A lot of people are letting down their guard and being surprised when it has consequences.

paddy kivlin said...

there are more categories of adjectives about a decision to buy than effective.

commonly another group is efficient which includes among a long list of testables: safety

i see no assurance that any vax is safe; none at an acceptable consumer risk.

the original B737 max 8 tests were more safety conscious than vax tests.

Dan Crawford said...


Even in MA where counties have high vaccination rates the feeling of its basically over prevails, less so with older folk. Peter has started a conversation needed to clarify the confused conversation currently happening, down to reviewing what is available given the limited choices from Moderna and Pfizer.

Dan Crawford said...

Thank you for starting this conversation. While the politics of the pandemic have replaced the medical, it is still worth figuring out the realities of ‘the science’ as well as how the money trail of doing business has severely limited our discussion of what is effective and what is safe. While phase three results for other vaccines are published, the US has had little public discussion about alternative vaccines effectiveness other than Moderna and Pfizer. Now that boosters are in the mix and federal monies announced and mandates being explored, media and ‘the science’ appear completely co-opted by current limited options.