This study evaluates the effectiveness of the novel BNT162b2 mRNA vaccine1 against Covid-19 in a nationwide mass vaccination setting. Estimated vaccine effectiveness during the follow-up period starting 7 days after the second dose was 92% for documented infection, 94% for symptomatic Covid-19, 87% for hospitalization, and 92% for severe Covid-19. Estimated effectiveness during days 14 through 20 (after one dose) and days 21 through 27 (gradual shifting between the first and second vaccine doses) was 46% and 60% for documented infection, 57% and 66% for symptomatic Covid-19, 74% and 78% for hospitalization, 62% and 80% for severe Covid-19, and 72% and 84% for Covid-19–related death, respectively.
The first primary end point evaluated in the randomized trial of the BNT162b2 vaccine was symptomatic Covid-19. In both the randomized trial and our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and unvaccinated groups began to diverge around day 12 after the first dose.1 The estimated vaccine efficacy for symptomatic Covid-19 starting at day 7 after the second dose was 95% in the randomized trial, as compared with 94% in our study. The estimated efficacy between the first dose and the second dose was 52% in the trial, as compared with 29% in our study. This difference may reflect the high level of transmission in Israel during the study period,14 which affected both the vaccinated persons and the controls equally during the first 12 days after administration of the first dose. To eliminate this distortion, we estimated first-dose effectiveness of the vaccine against Covid-19 for the period from days 14 through 20; the estimated effectiveness was 57%.
The estimated effectiveness for documented infection during days 14 through 20 was 46% in our study. A relatively similar effectiveness of 51% was reported by Chodick et al.,15 who evaluated a cohort from another health care organization in Israel and used a different study design that compared infection among vaccinated persons at days 13 through 24 after the first dose against infection during days 0 through 12.
In the randomized trial, the estimated vaccine efficacy for severe Covid-19 (89% over the entire study period) was based on only 10 cases. Our study recorded 229 cases of severe Covid-19 — 55 in the vaccinated group and 174 in the unvaccinated group — resulting in an estimated effectiveness of 62% for days 14 through 20 after the first dose, 80% for days 21 through 27, and 92% for 7 or more days after the second dose.
The large sample size in our study also allowed us to estimate vaccine effectiveness for specific subpopulations that the randomized trial was not sufficiently powered to evaluate. In the trial, the estimated efficacy for Covid-19 among persons up to 55 years of age, older than 55 years, and 65 years or older 7 days after the second dose was 94 to 96%. We were able to study more granular age groups, and we estimated that the vaccine effectiveness was similar for adults 70 years of age or older and for younger age groups for the same time period.
The randomized trial estimated vaccine efficacy for patients with one or more coexisting conditions according to the Charlson comorbidity index16 and specifically for patients with obesity or hypertension. These measures do not provide clarity regarding effectiveness in patients with multiple coexisting conditions. We estimated vaccine effectiveness in relation to various numbers of coexisting conditions and found indications that effectiveness may be slightly lower among persons with higher numbers of coexisting conditions.
Two factors make the present study uniquely suited to evaluating the effectiveness of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical background data, Covid-19 PCR test results (for the documented infection outcome), and patient follow-up data in both community (for the symptomatic Covid-19 outcome) and inpatient (for all other outcomes) settings — CHS has maintained such an integrated data repository for over half the Israeli population, and has updated it daily, for more than two decades; and second, the rapid pace and high uptake of Covid-19 vaccine in Israel and the high disease rates during the vaccination campaign. On the other hand, the rapid pace of the vaccination campaign contributed to the frequent censoring of data for matched unvaccinated controls, especially among those over the age of 60 years (often only a few days after matching) and the corresponding reduction in the average follow-up period of the study.
Concerns have emerged regarding the possible resistance of SARS-CoV-2 variants to Covid-19 vaccines17,18 and neutralizing antibodies.19,20 During the study period, an increasing share of SARS-CoV-2 isolates in Israel — up to 80% in the days before data extraction — were of the B.1.1.7 variant.21 Thus, this study estimates an average effectiveness of the vaccine over multiple strains. Although we cannot provide a specific effectiveness estimate for the B.1.1.7 variant, the plateau observed during the later periods in the cumulative incidence curve for vaccinated persons suggests that the BNT162b2 vaccine is also effective for this variant, an observation consistent with previous reports that showed preserved neutralizing antibody titers.22 The B.1.351 variant was estimated to be rare in Israel at the time of data extraction.23
As with any observational study, our study may have been affected by residual confounding due to differences between vaccinated persons and unvaccinated controls, especially in terms of health-seeking behavior. We therefore performed rigorous matching on a wide range of factors that may be expected to confound the causal effect of the vaccine on the various outcomes. After the matching process, we found a consistent pattern of similarity between the groups in the days just before day 12 after the first dose (the anticipated onset of the vaccine effect), which thus serve as a “negative control”24 period (Figure 2, Fig. S6, and Table S7). This similarity occurred despite a temporary increase in events among unvaccinated controls during the very first days after the first vaccine dose, most likely stemming from the fact that persons who choose to be vaccinated on a specific day are feeling well at the time of vaccination. The similarity of the study groups in coexisting conditions and known risk factors for severe Covid-19 (Table 1 and Fig. S2) provides further evidence of exchangeability (i.e., absence of confounding). However, this rigorous matching process came at the cost of not including in the final cohort approximately 34% of the vaccinated persons who otherwise met the study’s eligibility criteria. Limited matching on age and sex only would have been insufficient to eliminate the early confounding (Fig. S6).
We also excluded population groups with high internal variability in the probability of vaccination or outcome, such as health care workers, persons confined to the home for medical reasons, and nursing home residents, to avoid residual confounding. Although the randomized trial was also less likely to include persons who were not healthy enough to comply with the scheduled visits and vaccination plan, it did not exclude health care workers.
To assess a possible selection bias that could stem from informative censoring, whereby controls who are vaccinated feel well around the time of vaccination, we performed a sensitivity analysis in which they were kept in the unvaccinated group for a period of time that was set differently for each outcome (Fig. S7 and Table S5). This analysis showed results similar to those of the main analysis, which suggests that any such bias was small in our analysis.
Finally, the date of onset of symptoms was not available for the analysis. Instead, for infection outcomes, the date was set to the date of swab collection for the first positive PCR test. Given that there was likely to have been a time gap between the onset of symptoms and swab collection, the observed divergence of the cumulative incidence plots for the infection outcomes between the vaccinated persons and unvaccinated controls may be slightly delayed. In parallel, there might be an underestimation of the vaccine effectiveness at each time window, since the estimate actually reflects a narrower window for the vaccine to be active. Because SARS-CoV-2 PCR testing is highly accessible in Israel and can be done without referral in a matter of hours, we estimate this potential time gap and thus the vaccine effectiveness underestimation to be small. In interpreting the effectiveness estimates for more severe outcomes, longer median gaps should be kept in mind (Fig. S3): 1 day for hospitalization, 5 days for severe Covid-19, and 11 days for Covid-19 death.
This study estimates a high effectiveness of the BNT162b2 vaccine for preventing symptomatic Covid-19 in a noncontrolled setting, similar to the vaccine efficacy reported in the randomized trial. Our study also suggests that effectiveness is high for the more serious outcomes: hospitalization, severe illness, and death. Furthermore, the estimated benefit increases in magnitude as time passes. These results strengthen the expectation that newly approved vaccines can help to mitigate the profound global effects of the Covid-19 pandemic.