Introduction

As of May 2023, COVID-19 has resulted in >6.1 million hospitalizations and >1.1 million deaths in the United States1. Monovalent vaccines designed to protect against the original strain of SARS-CoV-2 were highly effective in reducing COVID-19 morbidity and mortality2,3. However, effectiveness of vaccines against SARS-CoV-2 infection has declined over time due to waning immunity as well as the emergence of SARS-CoV-2 viral variants leading to immune evasion from naturally acquired and/or vaccine-elicited immunity4,5,6. To address this concern, mRNA BA.4/BA.5 bivalent booster vaccines, containing equal amounts of spike protein sequences for Omicron subvariants BA.4/BA.5 and original SARS-CoV-2 spike proteins, were developed.

Pre-clinical data suggests that BA.4/BA.5-containing mRNA vaccine provides greater neutralizing activity against BA.4/BA.5 and other emerging SARS-CoV-2 Omicron sublineages compared to original mRNA vaccines7,8. Based on these factors and in anticipation of an increase in cases during the winter respiratory season9, the U.S. Food and Drug Administration (FDA) authorized the Moderna and Pfizer-BioNTech bivalent mRNA vaccines on 31 August 2022 for individuals ≥18 years of age who had received their last of at least two monovalent doses at least 2 months earlier10,11. Subsequently, as of 18 April 2023, the Moderna and Pfizer-BioNTech bivalent mRNA vaccines were authorized for all doses administered to individuals aged 6 months of age and older12.

Recent real-world studies in the United States have demonstrated improved effectiveness of BA.4/BA.5-containing mRNA bivalent boosters against COVID-19 outcomes such as hospitalization, symptomatic infection, and death relative to monovalent vaccination and no vaccination10,13,14,15. However, these early studies evaluated VE against non-medically attended symptomatic infection10,15, medically attended infection13, or hospitalization during September to early December 2022 when BA.5 and BQ.1/BQ.1.1 were predominant5,13,16.

We conducted a matched prospective cohort study at Kaiser Permanente Southern California (KPSC) with follow-up through 01/31/2023 to assess the effectiveness of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine in preventing hospitalization for COVID-19 (primary outcome), medically attended SARS-CoV-2 infection, and COVID-19 hospital death in a real-world setting during a period of BA.5 and BQ.1 predominance and subsequent emergence of XBB sublineages.

Results

The study included 290,292 recipients of the bivalent mRNA-1273 vaccine (follow-up time for primary outcome analysis: median 2.46 months, maximum 4.57 months) and 580,584 age-, sex-, and race/ethnicity-matched individuals who received ≥2 doses of monovalent mRNA COVID-19 vaccine only (follow-up time for primary outcome analysis: median 1.45 months, maximum 4.57 months) (Fig. 1). Detailed distribution of sociodemographic variables and baseline characteristics of the two cohorts is presented in Table 1. The bivalent vaccine group and the ≥2 monovalent mRNA vaccine group had different distributions (ASD > 0.1) for number of outpatient and virtual visits, preventive care, number of monovalent vaccines prior to index date, time between latest monovalent vaccine and index date, and medical center area. The study also included 204,655 individuals who never received any COVID-19 vaccine (follow-up time for primary outcome analysis: median 2.33 months, maximum 4.57 months) that were matched up to 1:1 to the bivalent cohort on age-, sex-, and race/ethnicity (Fig. 1). The detailed distribution of sociodemographic variables and baseline characteristics of the two cohorts is presented in Supplementary Table 1. They had different distributions (ASD > 0.1) of age and race/ethnicity due to inability to completely match bivalent boosted with unvaccinated persons, as described in the Methods section. As expected, the bivalent vaccine group and the COVID-19 unvaccinated group differed across several variables, including body mass index, smoking, Charlson comorbidity score, frailty index, kidney disease, lung disease, diabetes, immunocompromised (IC) status, history of SARS-CoV-2 infection, history of SARS-CoV-2 molecular test, number of outpatient and virtual visits, preventive care, Medicaid status, neighborhood median household income, and medical center area.

Fig. 1: Cohort selection for evaluating effectiveness of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine in Kaiser Permanente Southern California, 2022–2023 (N = 1,946,407).
figure 1

Steps for selection of the bivalent mRNA-1273 cohort (n = 290,292), the ≥2 monovalent mRNA vaccine cohort (n = 580,584), and the COVID-19 unvaccinated cohort (n = 204,655) by inclusion and exclusion criteria in Kaiser Permanente Southern California (KPSC). aIndex date for bivalent-vaccinated recipients is the date of their bivalent mRNA-1273 dose. For non-bivalent comparators, index date is assigned based on the bivalent dose date of their matched counterpart. bDuring the matching process, the following inclusion criteria were applied to non-bivalent comparators based on their assigned index date: no receipt of any bivalent COVID-19 vaccine on or prior to index date, no receipt of any COVID-19 vaccine <14 days after index date, no receipt of any non-U.S. Food and Drug Administration (FDA) authorized COVID-19 vaccines prior to or during study period, KPSC member for ≥365 days prior to index date and 14 days after index date, no death <14 days after index date, no COVID-19 diagnosis code or SARS-CoV-2 positive molecular or antigen test <14 days after index date, and any health care utilization or vaccination within two years prior to index date. cNot all bivalent-vaccinated individuals were able to match to a COVID-19 unvaccinated comparator. Unmatched bivalent-vaccinated individuals were kept in the analysis.

Full size image
Table 1 Comparison of baseline characteristics between individuals in the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine cohort and the ≥2 monovalent mRNA vaccine cohort
Full size table

Figure 2a shows the adjusted relative vaccine effectiveness (rVE) of the bivalent vaccine, compared to the ≥2 monovalent mRNA vaccine group, against hospitalization for COVID-19 disease (defined in the outcomes of interest section under Methods); case numbers and incidence rates are presented in Supplementary Table 2. Overall, the rVE against hospitalization for COVID-19 was 70.3% (95% confidence interval [CI]: 64.0%–75.4%). The rVE was slightly lower in the 45–64-year-old group (56.2%, 95% CI: 22.5%–75.2%), but in general was consistent across age, sex, and race/ethnicity group. The rVE was 64.7% (95% CI: 44.0%–77.7%) in IC individuals, compared to 71.3% (95% CI: 64.5%–76.7%) in immunocompetent individuals. The rVE was 60.7% (95% CI: 33.8%–76.7%) in individuals with a history of SARS-CoV-2 infection and 71.1% (95% CI: 64.6%–76.5%) in those without known history. There was no apparent waning in protection against hospitalization for COVID-19, with rVE remaining at 79.6% (95% CI: 43.2%–92.7%) at ≥3 months since bivalent vaccination, although the confidence interval was relatively wide. The number of prior monovalent vaccine doses received and the time since the last dose of monovalent vaccine did not appear to substantially modify rVE.

Fig. 2: Relative vaccine effectiveness (rVE) and vaccine effectiveness (VE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine in preventing hospitalization for COVID-19, overall and by subgroups.
figure 2

a Relative vaccine effectiveness (rVE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine, compared to the ≥2 monovalent mRNA vaccine group, and (b) Absolute vaccine effectiveness (VE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine, compared to the COVID-19 unvaccinated group and their 95% confidence intervals (CI) in preventing hospitalization for COVID-19, overall and by subgroups. Data are presented as rVE and VE and their 95% confidence intervals. Tabulated data and unadjusted estimates are available in Supplementary Tables 2 and 3. When the hazard ratio or its 95% CI was >1, the rVE or its 95% CI was transformed as ([1/hazard ratio]–1) × 100. arVE models adjusted for covariates age group, sex, race/ethnicity, index date (in months), history of SARS-CoV-2 infection, number of outpatient and virtual visits, preventive care, number of monovalent vaccines prior to index date, time between latest monovalent vaccine and index date, and antiviral therapy. Medical center area removed from adjustment set due to lack of model convergence. bVE models adjusted for covariates age group, sex, race/ethnicity, index date (in months), body mass index, smoking, Charlson comorbidity score, frailty index, kidney disease, lung disease, diabetes, immunocompromised status, history of SARS-CoV-2 infection, history of SARS-CoV-2 molecular test, number of outpatient and virtual visits, preventive care, Medicaid, and antiviral therapy. Neighborhood median household income and medical center area removed from adjustment set due to lack of model convergence. cAdjusted for continuous age (in years) in addition to covariates above. dSmoking removed from adjustment set in indicated VE models due to lack of model convergence. eKidney disease, lung disease, and immunocompromised status removed from adjustment set in indicated VE models due to lack of model convergence. fAdjusted for immunocompromising sub-conditions in addition to covariates above. gAdjusted for time since prior SARS-CoV-2 infection in addition to covariates above. hTime between latest monovalent vaccine and index date removed from adjustment set due to lack of model convergence. iNumber of monovalent vaccines prior to index date removed from adjustment set due to lack of model convergence.

Full size image

Figure 2b shows the adjusted absolute vaccine effectiveness (VE) of the bivalent vaccine, compared to the COVID-19 unvaccinated group, against hospitalization for COVID-19 disease; case numbers and incidence rates are presented in Supplementary Table 3. Overall, the VE against hospitalization for COVID-19 was 82.8% (95% CI: 78.8%–86.0%). It was generally consistent across age, sex, and race/ethnicity group. The VE was 71.8% (95% CI: 48.8%–84.5%) in IC individuals, compared to 84.1% (95% CI: 80.1%–87.4%) in immunocompetent individuals. The VE was 68.3% (95% CI: 45.4%–81.6%) in individuals with history of SARS-CoV-2 infection and 84.3% (95% CI: 80.3%–87.5%) in those without known history. There was no apparent waning in protection against hospitalization for COVID-19 with VE remaining at 75.5% (95% CI: 43.8%–89.3%) at ≥3 months since bivalent vaccination.

Cumulative incidence of hospitalization for COVID-19 was significantly higher in the two comparator groups compared to the bivalent vaccine group (log-rank test p < 0.0001; Figs. 3a and 3b). At the end of follow-up, the cumulative incidence was 0.09%, 0.22%, and 0.27% in the bivalent vaccine, ≥2 monovalent mRNA vaccine, and COVID-19 unvaccinated groups, respectively.

Fig. 3: Cumulative incidence of hospitalization for COVID-19 estimated by Kaplan Meier methods, comparing the bivalent mRNA-1273 cohort and the ≥2 monovalent mRNA vaccine cohort, and between the bivalent mRNA-1273 cohort and the COVID-19 unvaccinated cohort.
figure 3

a Cumulative incidence of hospitalization for COVID-19 between individuals in the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine cohort and the ≥2 monovalent mRNA vaccine cohort, and (b) cumulative incidence of hospitalization for COVID-19 between individuals in the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine cohort and the COVID-19 unvaccinated cohort. The red line indicates the bivalent vaccine group, and the blue line indicates the comparison group. The difference in each comparison was tested by a log-rank test.

Full size image

Figure 4a shows the adjusted rVE of the bivalent vaccine, compared to the ≥2 monovalent mRNA vaccine group, in preventing medically attended SARS-CoV-2 infection and COVID-19 hospital death as the secondary outcomes; case numbers and incidence rates are presented in Supplementary Table 4. The rVE against medically attended SARS-CoV-2 infection in all care settings and in emergency department/urgent care (ED/UC) settings was 35.9% (95% CI: 32.7%–39.0%) and 55.0% (95% CI: 50.8%–58.8%), respectively. The rVE against COVID-19 hospital death was 82.7% (95% CI: 63.7%–91.7%).

Fig. 4: Relative vaccine effectiveness (rVE) and vaccine effectiveness (VE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine in preventing medically attended SARS-CoV-2 infection in different settings and COVID-19 hospital death.
figure 4

a Relative vaccine effectiveness (rVE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine, compared to the ≥2 monovalent mRNA vaccine group, and (b) Absolute vaccine effectiveness (VE) of the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine, compared to the COVID-19 unvaccinated group and their 95% confidence intervals (CI) in preventing medically attended SARS-CoV-2 infection and COVID-19 hospital death. Data are presented as rVE and VE and their 95% confidence intervals. Tabulated data and unadjusted estimates are available in Supplementary Tables 4 and 5. arVE models adjusted for covariates age group, sex, race/ethnicity, index date (in months), history of SARS-CoV-2 infection, number of outpatient and virtual visits, preventive care, number of monovalent vaccines prior to index date, time between latest monovalent vaccine and index date, and medical center area. bVE models adjusted for covariates age group, sex, race/ethnicity, index date (in months), BMI, smoking, Charlson comorbidity score, frailty index, kidney disease, lung disease, diabetes, immunocompromised status, history of SARS-CoV-2 infection, history of SARS-CoV-2 molecular test, number of outpatient and virtual visits, preventive care, Medicaid, neighborhood median household income, and medical center area. cAdjusted for antiviral therapy in addition to covariates above. Neighborhood median household income (VE models only) and medical center area (rVE and VE models) removed from adjustment set due to lack of model convergence.

Full size image

Figure 4b shows the adjusted absolute VE of the bivalent vaccine, compared to the COVID-19 unvaccinated group, in preventing medically attended SARS-CoV-2 infection and COVID-19 hospital death; case numbers and incidence rates are presented in Supplementary Table 5. The VE against medically attended SARS-CoV-2 infection in all care settings and in ED/UC settings was 10.7% (95% CI: 4.4%–16.6%) and 55.4% (95% CI: 50.3%–60.1%), respectively. The VE against COVID-19 hospital death was 89.7% (95% CI: 77.7%–95.2%).

Discussion

In this cohort study conducted among a socio-demographically diverse population in a large U.S. healthcare system, the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine improved protection against a wide range of COVID-19 outcomes. mRNA-1273 BA.4/BA.5 bivalent vaccine effectiveness was high against hospitalization for COVID-19 disease and COVID-19 hospital death, even when compared to individuals who received ≥2 doses of monovalent mRNA vaccine; however, vaccine effectiveness was moderate against medically attended SARS-CoV-2 infection. The observed vaccine protection against hospitalization for COVID-19 was consistent across age (not estimated in the 6-17 years group), sex, race/ethnicity, IC status, history of SARS-CoV-2 infection, number of prior monovalent doses, and time since last monovalent dose. Additionally, the protection against hospitalization for COVID-19 was durable for at least 3 months of follow-up. These data from a real-world setting underscore the importance of remaining up-to-date with recommended COVID-19 vaccines, including receipt of a COVID-19 bivalent booster dose, to optimize protection, particularly against COVID-19 hospitalizations and death.

Initial estimates of vaccine effectiveness of bivalent mRNA vaccines have been generated using national outpatient and hospital-based surveillance networks in the United States. Early estimates from the VISION network of the VE and rVE (56% and 31–50%, respectively) against COVID-19 associated ED/UC visits during September–November 2022, when the BA.5 and other Omicron sublineages were the predominant SARS-CoV-2 variants in the United States, were similar to those in our study13. On the other hand, VISION’s estimates of VE and rVE against hospitalization (57% and 38%–45%, respectively13) were lower than our estimates (82.8% and 70.3%, respectively). It is possible that limiting our analysis to hospitalizations confirmed to have been for COVID-19 reduced the bias of VE estimates resulting from including hospitalizations for other reasons with incidental SARS-CoV-2 infection in the analysis17. The high VE and rVE of the bivalent booster against COVID-19 death in our study (89.7% and 82.7%, respectively) are higher than previous reports of bivalent VE and rVE against severe infection resulting in hospitalization or death16,18.

Reduced VE of the mRNA monovalent vaccine series among IC adults has been found to be much more pronounced against infection with Omicron than with earlier variants4, although VE against more severe COVID-19 outcomes was generally preserved. However, Britton and colleagues found that the VE of monovalent boosters against hospitalization with Omicron BA.2/BA.2.12.1/BA.4/BA.5 sublineages was markedly reduced among IC adults (32% ≥90 days after dose 3, and 43% ≥7 days after dose 4)19. In our study, we found that absolute VE of the bivalent booster against hospitalization for COVID-19 among a predominantly adult, IC population during periods of Omicron BA.5 and BQ.1 predominance was higher at 71.8%. The evidence supports the recommendation to boost the IC population with bivalent COVID-19 vaccine to lower the risk of hospitalization for COVID-19.

This study reports results that support early estimates of the effectiveness of bivalent COVID-19 boosters against an array of COVID-19 outcomes with data on durability and various subgroups. It has several strengths and limitations, with these limitations partly addressed through the strengths of the data source and analytic methods used in the study. First, there are various risk factors for infections and severe outcomes associated with testing and with vaccination that were unavailable in electronic health record (EHR) data. This could introduce bias, including mask-wearing, social distancing, and healthcare-seeking behavior. However, we attempted to mitigate potential bias by matching individuals and by adjusting for sociodemographic characteristics, prior healthcare utilization prior SARS-CoV-2 testing, and comorbidities in the models. In addition, we compared vaccinated and unvaccinated individuals at the same point in time, which balances community infection rates, exposure risk, and other secular impacts that might affect testing behaviors and risk of infection between vaccinated and unvaccinated participants. Furthermore, we estimated rVE, a comparison that should be less confounded by vaccination behavior, health care use, and social behavior differences than VE estimates with unvaccinated individuals as comparators. Although potential residual confounding could remain, it is unlikely to impact the conclusions of the study. Second, to reflect real-world conditions, we included in the analysis self-reported positive tests that were included in the EHR, but also required a COVID-19 diagnosis code in an encounter from 3 days before to 7 days after the test to ensure that the infection was medically attended. This likely reflected less severe disease. Third, although we attempted to account for a prior history of SARS-CoV-2 infection, it could be under-ascertained. Differential under-ascertainment could potentially generate bias. For example, some unvaccinated individuals could obtain some protection from prior infections that were not documented in the EHR, resulting in underestimated VE. On the other hand, prior infections of vaccinated individuals might provide hybrid immunity from both infections and vaccinations, and if some of these prior infections were not accounted for in analyses, VE could be overestimated20,21. Fourth, misclassification of vaccination status is possible but unlikely to impact the results substantially. KPSC vaccination records capture all immunizations given within KPSC and are updated daily with California Immunization Registry data to which all providers are required by law to report COVID-19 vaccinations within 24 hours of administration. Fifth, we did not assess VE against asymptomatic infection. However, VE against a range of outcomes of varying severity (including medically attended infection, ED/UC visits, hospitalizations, and COVID-19 hospital deaths) provides estimates of protection from receipt of a bivalent booster that helps inform policy decisions. Sixth, the VE of COVID-19 vaccine against severe outcomes could potentially be more sustained given that cell-mediated immunity mechanisms appear to play a more significant role than humoral immunity in the prevention of severe outcomes22,23. However, with the wide confidence intervals for VE estimates and the replacement of BA.5 and BQ.1 by XBB sublineages, the durability of the bivalent vaccine (targeting Omicron BA.4/BA.5) against hospitalization due to BA.5 beyond 3 months is not as clear. Seventh, there were no cases of hospitalization for COVID-19 among exposed or unexposed individuals 6–17 years of age, preventing estimation of rVE and VE in this age group. Finally, while our study was conducted during a period when the proportion of infections with highly immune-evasive XBB sublineages was growing, our data were insufficient to assess VE against XBB subvariants alone. Recent data suggest that short-term bivalent VE against COVID-19 disease with BA.5 and XBB sublineages may be similar10.

This study found that vaccination with the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine was associated with a lower risk of COVID-19 disease among recipients compared to those who received two or more doses of monovalent mRNA vaccines only or to those who were never vaccinated with a COVID-19 vaccine. These data suggest the potential benefit of receiving a COVID-19 booster dose directed against circulating SARS-CoV-2 variants, particularly for preventing severe outcomes associated with COVID-19. Additional studies are needed to assess the durability of protection afforded by BA.4/BA.5 bivalent vaccination, and other newly formulated vaccines against severe COVID-19 and their effectiveness against emerging SARS-CoV-2 variants, including XBB sublineages.

Methods

Study setting

KPSC is an integrated health system that provides health care services and insurance coverage to >4.8 million members with sociodemographic characteristics representative of the diverse Southern California population. Comprehensive EHRs capture details of patient care, including vaccinations, diagnoses, laboratory tests, procedures, and pharmacy records, from inpatient, ED, outpatient, and virtual care settings, with care received outside of the KPSC system captured through claims. In addition, vaccinations received outside of KPSC are imported daily from external sources, including the California Immunizations Registry (CAIR), Care Everywhere (system on the Epic EHR platform that allows different health care systems to exchange patients’ medical information), claims (for example, retail pharmacies), and self-report by members (with valid documentation). The study was approved by the KPSC Institutional Review Board (#12758), which waived requirements for written informed consent and written Health Insurance Portability and Accountability Act authorization, as the use of EHRs for this observational study involved minimal risk. The study protocol was submitted to regulatory agencies prior to the conduct of the study6.

Study population

KPSC members were eligible for inclusion in the study if they were ≥6 years of age at the index date, had membership ≥12 months prior to the index date through 14 days after the index date, and had none of the exclusion criteria listed below. The exposure of interest was the bivalent (original and Omicron BA.4/BA.5) mRNA-1273 COVID-19 vaccine, as this study was conducted as part of a regulatory commitment from the manufacturer to multiple health authorities. The index date was the calendar date of receiving the first eligible bivalent mRNA-1273 COVID-19 vaccine for individuals in the exposed group. For unexposed individuals, the index date was assigned based on the index date of their exposed matched counterpart; these unexposed individuals had no receipt of bivalent vaccine prior to the index date. The exclusion criteria for both groups included receiving any bivalent COVID-19 vaccine other than bivalent mRNA-1273 vaccine on or prior to the index date, receipt of any non-FDA authorized COVID-19 vaccines prior to or during the follow-up, no health care utilization, and no vaccination within 2 years prior to the index date, receipt of any COVID-19 vaccine <14 days after the index date, death <14 days after the index date, or occurrence of a COVID-19 diagnosis code or a SARS-CoV-2 positive molecular or antigen test <14 days after the index date.

Exposed cohort

Individuals were included in the exposed cohort if they received a dose of the bivalent mRNA-1273 COVID-19 vaccine (mRNA-1273.222 [original and Omicron BA.4/BA.5]) during the study accrual period (for ages ≥18 years: 8/31/2022-12/31/2022, and for ages 6–17 years: after FDA authorization on 10/12/2022-12/31/2022), received at least 2 monovalent mRNA COVID-19 vaccines prior to the bivalent dose, and did not receive any COVID-19 vaccine <52 days prior to the bivalent COVID-19 vaccine (recommended interval of ≥8 weeks, allowing a 4-day grace period).

Unexposed cohorts

Two unexposed cohorts were used as comparators in the study.

1) ≥2 doses of monovalent mRNA COVID-19 vaccinated cohort (≥2 monovalent mRNA vaccine group): Individuals who had not received any bivalent dose but had received at least two doses of monovalent mRNA COVID-19 vaccine by the index date, were randomly selected and individually matched 2:1 to the bivalent exposed cohort.

2) COVID-19 unvaccinated group: Individuals who never received any COVID-19 vaccine dose by the index date were randomly selected and individually matched 1:1 to the bivalent exposed cohort.

The matching factors for both cohorts included age group (6–17 years, 18–44 years, 45–64 years, 65–74 years, and ≥75 years), sex (male, female), and race/ethnicity (Non-Hispanic White, Non-Hispanic Black, Hispanic, Non-Hispanic Asian, and Other/Unknown). Not all bivalent exposed individuals were matched with a COVID-19 unvaccinated individual due to a limited number of COVID-19 unvaccinated individuals, but all bivalent exposed individuals were retained in the analysis.

Follow-up

The follow-up period ended on 01/31/2023. Each individual in the study was followed from 14 days after their index date till the end of the study follow-up period, membership end date, receipt of any COVID-19 vaccine, death, or outcome of interest, whichever came first.

Outcome of interest

The primary outcome of interest was hospitalization for COVID-19, which included patients having a COVID-19 diagnosis code in the inpatient setting or a SARS-CoV-2 positive molecular or antigen test ≤7 days prior to or during the hospitalization stay. We ascertained the first occurrence of COVID-19 hospitalization ≥14 days after the index date. To ensure that hospitalization was for severe COVID-19 rather than coincident with SARS-CoV-2 infection, hospitalization for COVID-19 was further confirmed by (1) ≥1 documented oxygen saturation (SpO2) of <90% during hospital stay for all patients or during a labor/delivery stay >2 days for pregnant patients or (2) manual chart review, as needed, performed by a physician investigator (B.K.A.) and trained chart abstractors to verify the presence of severe COVID-19 symptoms.

The secondary outcomes of interest included medically attended incident SARS-CoV-2 infection and COVID-19 hospital death. Medically attended SARS-CoV-2 infection was defined as infection resulting in seeking medical care in (1) all care settings: a SARS-CoV-2 positive molecular or antigen test, with a COVID-19 diagnosis code in the inpatient, emergency, outpatient, or virtual visit setting from 3 days before to 7 days after the test, or (2) ED/UC settings only: a SARS-CoV-2 positive molecular or antigen test, with a COVID-19 diagnosis code in the ED/UC settings from 3 days before to 7 days after the test. In both settings, the SARS-CoV-2 antigen test results could be self-reported. We ascertained the first occurrence of medically attended incident SARS-CoV-2 infection ≥14 days after the index date. COVID-19 hospital death occurred during a hospitalization for COVID-19.

Covariates

Potential confounders were identified a priori based on the literature. Variables collected from EHRs at the index date included age, sex, race/ethnicity, socioeconomic status (Medicaid and neighborhood median household income), medical center area, and pregnancy status. Variables assessed prior to the index date included body mass index, smoking, Charlson comorbidity score, frailty index, chronic diseases, immunocompromised status, autoimmune conditions, health care visits (outpatient, virtual, ED, and inpatient), and preventive care (other vaccinations, screenings, and wellness visits). Race/ethnicity data (non-Hispanic White, non-Hispanic Black, Hispanic, non-Hispanic Asian, and other/unknown) were collected in the EHRs through self-report. Additional variables included history of SARS-CoV-2 infection and history of SARS-CoV-2 molecular test from 03/01/2020 to index date, and receipt of concomitant vaccine with the bivalent dose. For the analysis of the exposed cohort group versus the ≥2 monovalent mRNA vaccine group only, the number of monovalent COVID-19 vaccine doses prior to index date and time between the latest monovalent COVID-19 vaccine dose and index date were included. For analyses of hospitalization for COVID-19 and COVID-19 hospital death outcomes only, additional variables included were any antiviral therapy (nirmatrelvir/ritonavir, molnupiravir, or remdesivir) during follow-up (yes/no).

Statistical analysis

We described attributes of the exposed and two unexposed cohorts. Categorical variables were compared using the χ2 test or Fisher’s exact test, as appropriate, and continuous variables were compared using the two-sample t test or Wilcoxon rank-sum test, as appropriate. Absolute standardized differences (ASD) were calculated to assess the balance of covariates; potential confounders were determined by ASD > 0.1 and were included in the adjusted models, along with matching variables (age, sex, and race/ethnicity), month of index date, and other select covariates based on scientific relevance.

We calculated overall incidence rates of hospitalization for COVID-19, medically attended SARS-CoV-2 infections, and COVID-19 hospital death for the exposed and the unexposed cohorts (number of incident events divided by person-years). The cumulative incidence of hospitalization for COVID-19 was estimated by the Kaplan–Meier method and compared between the exposed and unexposed cohorts by the log-rank test.

Unadjusted and adjusted hazard ratios (HR) and 95% confidence intervals (CIs) comparing hospitalization for COVID-19, medically attended SARS-CoV-2 infections, and COVID-19 hospital death between the exposed and the unexposed cohorts were estimated by Cox proportional hazards regression models without and with confounder adjustment. Relative vaccine effectiveness (rVE, %) comparing between the exposed and the ≥2 monovalent mRNA vaccine group and absolute vaccine effectiveness (VE, %) comparing between the exposed and the COVID-19 unvaccinated group, were calculated as (1−HR) × 100 when HR was ≤1, and ([1/HR]-1) × 100 when HR was >1. We also assessed rVE and VE against the primary outcome, i.e., hospitalization for COVID-19, by age, sex, race/ethnicity, immunocompromised status, history of SARS-CoV-2 infection, time since bivalent mRNA-1273 COVID-19 vaccine, number of prior monovalent doses received (rVE only), and time since the last monovalent dose (rVE only). SAS 9.4 software (SAS Institute) was used for all analyses. Results were reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist (Supplementary Table 6).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.