Introduction

There has been a global increase in COVID-19 cases associated with the Omicron variant between November 2021 and March 20221. Nevertheless, surges in severe disease, as indicated by hospitalisations, ICU admissions or deaths, have not matched those of previous waves of the pandemic2. A range of factors are likely to contribute to this divergence, including lower inherent severity of Omicron compared to previous variants, a greater proportion of the population with immunity from vaccination and/or prior infection, and sustained protection against severe disease3,4.

Early data indicated a reduced neutralising antibody response to the Omicron variant5,6,7. Real-world studies have since found reduced effectiveness of COVID-19 vaccines against infection or mild disease with the Omicron variant8,9,10. Receipt of a 3rd dose improves protection; however, this appears to wane rapidly from the second month after vaccination8. Evidence on protection against severe disease is mixed with some studies suggesting substantially reduced effectiveness against hospitalisation compared to the Delta variant even with 3rd doses11,12, whereas other studies suggest very high levels of effectiveness of over 90%9,13,14. There is currently limited data on the duration of protection against severe disease.

In this study, we assess the effectiveness of COVID-19 vaccines against hospitalisation in those testing positive by PCR for Omicron and Delta variants. In the past, we have done this using symptomatic community tested cases subsequently hospitalised through emergency care for a non-accident reason within 2 weeks of their positive test with a test-negative case–control (TNCC) design15. This has yielded estimates of the effectiveness of over 90% against Alpha and Delta variants. However, given that all individuals who are hospitalised for any reason in the UK are tested for COVID-19, and with the lower severity of Omicron and the high incidence, an increasing proportion of those hospitalised who also test positive may be hospitalised with COVID-19 as an incidental finding rather than hospitalised as a result of COVID-19. This would lead to underestimation of effectiveness against hospitalisation because the “with COVID-19” cases would be expected to have effectiveness similar to that seen against infection. To investigate this specificity of outcome issue we have obtained data on coded hospital discharges in those PCR tested, including on primary diagnosis, length of stay, oxygen use, ventilation and admission to intensive care. Whilst these data are not as timely as using emergency care admission data, they allow the identification of those more likely to have been admitted due to COVID-19.

Results

Descriptive characteristics

After linkage of testing data to hospitalised cases in ECDS or a respiratory coded SUS episode and to the NIMS vaccination database, and selection of the Delta and Omicron assigned cases and the controls, the total number of tests in the study period was 409,985 of which 115,720 were cases and 294,265 controls. A total of 51,115 (44.2%) of these cases and 34,556 (11.7%) of these controls had a Pillar 2 test as the earliest test and of these 38,150 cases and 31,552 controls were symptomatic and included in the ECDS analysis. For the ECDS analyses where all symptomatic controls were used irrespective of hospitalisation the total number of controls included was 6,759,286 whilst the analysis to assess symptomatic vaccine effectiveness using the Pillar 2 data included 27,256 cases along with these controls.

The characteristics of hospitalised cases and controls for the Omicron and Delta period analyses are shown in Supplementary Table S1 (age 18–64) and Supplementary Table S2 (age 65 years and over). Note that some controls contribute to both the Omicron and Delta analyses. Pillar 2 symptomatic ECDS admissions in cases are much lower than SUS admissions for those aged over 65, even when restricting to those with a 2-day stay and primary diagnostic field coded. This difference is less for age 18–64 and for Omicron (18–64). Of the SUS admissions the proportion with a recorded intervention (oxygen/ventilation/ICU) is significantly higher (Chi-squared P < 0.001) for Delta cases (age 18–64: 20.8%; age 65+: 21.2%) than Omicron cases (age 18–64: 2.5%; age 65+: 6.6%) and higher for cases than controls except for Omicron cases (2.5%) compared to controls (4.4%) for age 18–64. This indicates not only severity differences by variant but also that severity differences differ by age with particularly low severity in age 18–64 Omicron cases.

Post 3rd dose effectiveness by outcome

Figure 1 and Supplementary Table S3 summarise vaccine effectiveness at least 7 days post 3rd dose by age, variant and outcome. For Delta in those ages, 18–64 and 65+ VE against symptomatic infection was just over 90%. For all Delta ECDS analyses, VE was very high at over 98%, irrespective of controls used or respiratory coding or age. For the Delta SUS analysis, it is clear that those with 0 length of stay or not with a respiratory code in the primary field show lower VE, even lower than for symptomatic infection. The Delta SUS analyses with at least 2 days' stay and a primary coding all show VE of over 93%. For Omicron, results are much more variable. As previously seen, VE against symptomatic infection is much lower than for Delta with point estimates of 62% (age 18–64) and 52% (age 65+). For those age 65+, VE against hospitalisation using ECDS data is 86–91% improving to 93–95% if respiratory coding is used and with little variation according to which control group is used. VE in SUS in this age group is much lower, and more similar to symptomatic infection VE if using 0 days length of stay or a non-primary respiratory diagnosis. With the more specific and severe SUS endpoints VE increases to over 88–93% and is similar to that seen for ECDS. The picture in those aged 18–64 is more complex with ECDS data giving VE of 75–80% with an increase to 87% if respiratory coded. Using SUS data with 0 days of admission or a non-primary respiratory code gives VE similar to that for symptomatic infection, but there is a large increase in VE as the length of stay increases (to 89% VE) and with use of oxygen (to 93% VE). Only when oxygen use forms part of the definition is the VE in those aged 18–64 similar to that seen in age 65+.

Fig. 1: Vaccine effectiveness estimates with 95% confidence intervals 7+ days after a 3rd dose against symptomatic disease and different hospitalisation outcomes by age group and variant.
figure 1

ECDS Emergency Care Dataset—administrative data from emergency departments, SUS Secondary Users Service—administrative data from all secondary care which is coded on discharge, ECDS All all admissions through emergency care. ECDS Resp Coded respiratory SNOMED coded admission through emergency care, SUS All admitted with a respiratory ICD code, SUS Not Primary admitted with a respiratory ICD code not in the primary diagnosis field, SUS Primary admitted with a respiratory ICD code in the primary diagnosis field. 0 days, 1+ days, 2+ days, 3+ days: length of stay, for example, 1+ means at least one overnight stay. Vaccine effectiveness estimates are adjusted using sex, index of multiple deprivation (quintile), ethnic group, care home residence status (for age 65+), geographic region (NHS region), period (calendar week of test), health and social care worker status (for age <65), clinical risk group status (for age <65), clinically extremely vulnerable, severely immunosuppressed, and previously testing positive.

Full size image

Effectiveness by vaccine manufacture, dose and interval

Analysis using all outcomes by dose intervals post vaccination are summarised in Table 1 (for Omicron) and Supplementary Table S4 (for Delta) with full details in Supplementary Tables S5–S10. They show the same general patterns as seen when concentrating on post 3rd dose effectiveness. With Delta, with almost all of the outcomes’, limited waning is seen, in particular among 18–64-year-olds. With Omicron waning is seen with the less specific and less severe outcomes, though this is less obvious with the more specific and more severe outcomes. More waning is seen among 18–64-year-olds with all outcomes for Omicron.

Table 1 Vaccine effectiveness against different hospitalisation outcomes with Omicron by dose and interval (all vaccines combined)
Full size table

To assess effectiveness by manufacturer, only the ECDS (all controls), ECDS respiratory coded (all controls), SUS primary code > =2 days stay and SUS, primary code > =2 days stay and Oxygen/ventilation/ICU endpoints were considered, and only for Omicron since Delta VE varies less by endpoint, and results have been previously published16. These endpoints were chosen to be the same ECDS endpoint used in past analyses and to use the more specific SUS endpoints. Figure 2 shows the ECDS analysis with lower VE in those aged 18–64 and waning post 3rd dose, more so for the BNT162b2 3rd dose where there is a longer follow-up where VE declines to 38% for those primed with ChAdOx1. For those aged 65+, ECDS VE is higher at over 90% up to 14 weeks post 3rd dose irrespective of priming vaccine or the 3rd dose received and remaining over 80% from 15+ weeks after the 3rd dose. In this age group, the waned 2 dose VE is higher for BNT162b2 (76%) than ChAdOx1 (56%). ECDS results with respiratory coding show generally higher VE with similar patterns (Supplementary Tables S5–S8). Figure 3 shows the SUS results and shows VE above 80% in almost all vaccination combinations and post 3rd dose periods. For those aged over 65+, the SUS data suggest little evidence of waning, whilst in those aged 18–64 VE declines to around 66–69% 15+ weeks after a BNT162b2 boost. Within each interval, VE is similar for both ChAdOx1 and BNT162b2 primed individuals and also for BNT162b2 and mRNA-1273-boosted individuals.

Fig. 2: Vaccine effectiveness estimates with 95% confidence intervals against hospitalisations using ECDS by age group and manufacturer (all symptomatic controls, Omicron only).
figure 2

a 18–64: Two doses of ChAdOx1-S with a BNT162b2 or mRNA-1273 third dose, b 18–64: two doses of BNT162b2 with a BNT162b2 or mRNA-1273 third dose, c 65 and older: two doses of ChAdOx1-S with a BNT162b2 or mRNA-1273 third dose, d 65 and older: two doses of BNT162b2 with a BNT162b2 or mRNA-1273 third dose, Vaccine effectiveness estimates are adjusted using sex, index of multiple deprivation (quintile), ethnic group, care home residence status (for age 65+), geographic region (NHS region), period (calendar week of the test), health and social care worker status (for age <65), clinical risk group status (for age <65), clinically extremely vulnerable, severely immunosuppressed, and previously tested positive.

Full size image
Fig. 3: Vaccine effectiveness estimates with 95% confidence intervals against hospitalisations >=2 days and >=2 days and on oxygen/ventilated/on ICU using SUS by age group and manufacturer (all symptomatic controls, Omicron only).
figure 3

Vaccine effectiveness estimates are adjusted using sex, index of multiple deprivation (quintile), ethnic group, care home residence status (for age 65+), geographic region (NHS region), period (calendar week of the test), health and social care worker status (for age <65), clinical risk group status (for age <65), clinically extremely vulnerable, severely immunosuppressed, and previously tested positive. a 18–64: two doses of ChAdOx1-S with a BNT162b2 or mRNA-1273 third dose, b 18–64: two doses of BNT162b2 with a BNT162b2 or mRNA-1273 third dose, c 65 and older: two doses of ChAdOx1-S with a BNT162b2 or mRNA-1273 third dose, d 65 and older: two doses of BNT162b2 with a BNT162b2 or mRNA-1273 third dose.

Full size image

Discussion

The results of this study demonstrate that assessment and interpretation of COVID-19 vaccine effectiveness against hospitalisation has become more complicated since the less severe Omicron variant has become dominant. When the disease is less severe a higher proportion of hospitalisations are likely to have COVID as an incidental finding rather than the cause of hospitalisation. This is the case for Omicron compared to Delta and for younger adults compared to older adults. Contamination of hospitalisations with these ‘incidental’ cases appears to result in lower vaccine effectiveness estimates against hospitalisation that are likely more reflective of vaccine effectiveness against infection. Vaccine effectiveness estimates improve and waning is more limited when definitions of hospitalisation that are more specific to severe respiratory disease are used.

For the Delta variant, we found that VE was fairly robust using the emergency care admissions or SUS-coded hospital discharges as long as the SUS respiratory discharge code was in the primary field and the admission length at least one overnight stay. The results suggest that for Delta a high proportion of these admissions are likely to be truly related to COVID-19 so that the VE measure is truly against a more severe disease. Furthermore, for Delta, contamination of the hospitalised cases with cases not hospitalised due to COVID will cause less bias for VE because symptomatic VE post 3rd dose is high. For the Omicron variant VE was also high and fairly robust to the case definition in those aged 65 and over, although it did increase when using respiratory coded ECDS admissions or when restricting to SUS cases with oxygen/ventilation or ICU. This age group also had the longest available follow-up post 3rd dose and largest numbers to look at VE by the specific vaccine with these results showing similar VE by schedule post 3rd dose and with VE remaining high to 15+ weeks after the 3rd dose. In those aged 18–64 VE was lower at below 90% unless additional interventions (oxygen, ventilation or ICU) were included. VE against Omicron was particularly low and similar to symptomatic disease VE if using those without a primary respiratory code or admitted and discharged on the same day. This suggests these cases may be asymptomatically identified cases from screening of all hospitalised patients. When assessing VE against Omicron, it is therefore not sufficient to just identify hospitalisation through routine hospital datasets without using more detailed data on diagnostic codes, length of stay and interventions. In previous reports, we have given VE against hospitalisation through ECDS (all ages) and this has suggested declines by time since 3rd dose17, but this current analysis indicates how this is likely to be due, at least in part, to many of these hospitalisations not being due to COVID-19 leading to the estimates mirroring the declines seen against symptomatic infection8. Using admissions of at least 2 days with a respiratory code in the primary diagnostic field VE in both age groups started at around 91% soon after the 3rd dose, dropping to around 67% by 15+ weeks in 18–64-year-olds and 85% in 65+ year-olds. Among those on oxygen VE went from around 94% down to 80% in 18–64-year-olds and 96% down to 90% in 65+ year-olds. The lower VE and more notable waning among 18–64-year-olds suggest that even with these more specific and more severe endpoints, there are likely to be a significant number of admissions where COVID is not the primary cause of their hospitalisation. Furthermore, among the 18–64 year-olds, those who first became eligible for vaccination, and thus have the longest follow-up, are those in clinical risk groups, including immunosuppressed individuals—this is likely to contribute to the greater apparent waning in the last follow-up period.

Our findings may go some way toward explaining the differing findings among existing studies of vaccine effectiveness against severe disease with the Omicron variant. For example, Abu-Raddad et al. found dose 3 vaccine effectiveness of 76.5% (95% CI, 55.9–87.5%) against Covid-19-related hospitalisation or death, which is lower than many other estimates11. This may be related to the fact that the study was dominated by under 60-year-olds, who in general, are likely to have milder disease. Other studies where VE estimates after three doses were over 90% have included older cohorts or have used physician manual review of medical notes to confirm the presence of severe COVID-19 symptoms13,14. We only identified two studies that had stratified by period after a 3rd dose—Thompson et al. found VE of 91% in the first 2 months following a third dose and 78% >= 4 months after the third dose—this is similar to our findings in 18–64-year-olds with some of the outcomes, though generally more waning than that we observed in 65+ year-olds14. In a study by Tartof et al.18 using emergency care and hospital admissions in the USA, also using a test-negative design, the effectiveness of BNT162b2 against hospital admission due to the omicron variant was 85% (95% CI 80–89) at less than 3 months but fell to 55% (28–71), at 3 months or longer the VE against emergency care admissions was lower at 77% (72–81) at less than 3 months falling to 53% (36–66) at 3 months or longer. This is comparable to our findings using the broadest definition of hospitalisation and likely represents an underestimate of VE against severe disease due to contamination with incidental admissions.

Our study also corresponds to finding from ref. 19 which used SpO2 levels and oxygen supplementation data to assess COVID-19 disease severity to measure the impact of vaccination on trends.

Test-negative case–control design has a number of limitations and advantages8,16,20. One of the biggest limitations of this specific study is that in relies on hospital-coded data which may have coding errors or not have interventions coded when they were used (e.g., oxygen use). A study where data are collected prospectively on cases using reporting forms or detailed case note review could avoid this misclassification bias, but is much more challenging to do with sufficiently large numbers21. One potential limitation for the TNCC design when looking at severe disease in controls is test sensitivity when a large proportion of those tested are truly positive. This, however, is more likely to affect Delta than Omicron analyses (as Delta is more severe) and is one of the reasons, along with study power, that in past analyses, we have chosen to use all symptomatic Pillar 2 controls for hospitalised COVID-19 VE. The analyses in this study do show slightly higher VE when using hospitalised Pillar 2 controls which may be due to this bias, but which may also be due to residual confounding from using all controls because it is necessary to adjust for factors related to risk of hospitalisation which is unlikely to be fully captured within the available adjustment covariates. Another limitation is that we have not done a formal validation on cases using more detailed case note review to show that those with short stays and coding not in primary fields are less likely to be admitted due to COVID-19. Examining differences in VE by vaccine is particularly challenging given differences in the populations that have received either vaccine. For example, those that received ChAdOx1-S as the primary course are more likely to be in clinical risk group, particularly among younger age groups. Similarly, those in the youngest age groups that were vaccinated earliest are likely to be in clinical risk groups. While adjustments are made for age and clinical risk group, there is likely to be residual confounding.

In conclusion, we found high levels of 3rd dose VE against hospitalisation with the Omicron variant, in particular among older adults who are at greatest risk, and against more severe endpoints. Nevertheless, there is evidence of limited waning from 3 to 4 months after a 3rd dose. Care should be taken in comparison of VE against hospitalisation across different studies due to the impact of using different outcome definitions.

Methods

Ethics

This research complies with all relevant ethical regulations as surveillance of coronavirus disease 2019 (Covid-19) testing and vaccination is undertaken under Regulation 3 of the Health Service (Control of Patient Information) Regulations 2002 to collect confidential patient information (www.legislation.gov.uk/uksi/2002/1438/regulation/3/made. opens in new tab) under Sections 3(i) (a) to (c), 3(i)(d) (i) and (ii), and 3. The study protocol was subject to an internal review by the Public Health England Research Ethics and Governance Group and was found to be fully compliant with all regulatory requirements ref:CAP-2021-07-UPDATE. Given that no regulatory issues were identified, and that ethics review is not a requirement for this type of work, it was decided that a full ethics review would not be necessary.

Study design

A test-negative case–control design was used to estimate vaccine effectiveness in those aged 18 years and over against hospitalisation following a PCR test for SARS-CoV-2, as described previously22,23,24,25. Cases were those testing positive and controls those testing negative by PCR. Effectiveness was assessed using a variety of hospitalisation endpoints designed to differentiate between hospitalisations likely to be because of COVID-19 and those that may be hospitalisation with COVID-19 but potentially due to another cause. Effectiveness against Omicron and Delta was assessed using periods in which these variants were circulating and using the information on sequencing, genotyping and PCR s-gene target.

Data sources

We linked COVID-19 PCR tests from both community Pillar 2 testing (wider population testing including drive/walk in and home testing) and in-hospital Pillar 1 testing (testing for those with a clinical need, health workers and travel) with vaccination data from the national vaccination register and hospitalisation data from either the Emergency Care Dataset (ECDS) or the Secondary Users Service (SUS). The SUS dataset is a national electronic database for all NHS hospital admissions in England with ICD-10 coded discharge coding for completed hospital stays. The ECDS includes hospital admissions through NHS emergency departments but not elective admissions with the reason for attending emergency care SNOMED coded.

The data sources are described in detail in Supplementary Text 1.

Control selection

A maximum of one negative test per person within each of the following approximate 3-month periods was selected at random: 26 April to 1 August 2021, 2 August 2021 to 21 November 2021, 22 November 2021 to 23 February 2022. For analyses that involved hospitalised controls, any negative tests that led to hospitalisation within 21 days of a previous hospital negative test were excluded.

Statistical analysis

Analysis was by logistic regression with the PCR test result as the dependent variable where those testing positives were cases and those testing negative controls. Vaccination status was included as an independent variable, and effectiveness defined as 1- odds of vaccination in cases/odds of vaccination in controls. Vaccination status was defined using date of onset, or, if missing or in Pillar 1 where this was not obtained, date of sample. Status was stratified by dose and interval post vaccination at 0–27 and 28+ days post first dose, 0–13, 14–174 and 175+ days post second dose and 0–6, 7–13,14–34, 35–69, 70–104, 105+ post 3rd dose. The analysis was also stratified by the manufacturer (ChAdOx1: Astrazeneca (adenoviral vector vaccine)or BNT162b2: Pfizer/BioNTech (mRNA vaccine) 2 dose priming, and BNT162b2: Pfizer/BioNTech (mRNA vaccine) or mRNA-1273: Moderna (mRNA vaccine)boosting) and by variant (Delta and Omicron). The analyses done to assess effectiveness according to the specificity of the hospitalisation are given in Table 2. The first analysis replicates those previously done for symptomatic infection and the following analyses using different criteria to allow comparison of emergency care and SUS data sources and to assess within SUS how VE changes based on whether the respiratory code is in the primary diagnostic field, the length of stay and the presence of codes for further interventions (oxygen, ventilator, ICU admission).

Table 2 analyses to assess vaccine effectiveness against symptomatic disease and hospitalisation endpoints
Full size table

Vaccine effectiveness was adjusted in logistic regression models for age (5-year bands), sex, index of multiple deprivation (quintile), ethnic group, care home residence status (for age 65+), geographic region (NHS region), period (calendar week of test), health and social care worker status (for age <65), clinical risk group status (for age <65), clinically extremely vulnerable, severely immunosuppressed, and previously testing positive. The logistic model for the log-odds of the probability of being a positive case therefore included all of these explanatory variables as well as the variable for vaccination status. All analyses were stratified by age 18–64 and 65+. For the vaccine manufacturer stratification, only endpoints 2–5, 9 and 12 were considered and only for Omicron. Numbers were too small in those primed with mRNA-1273 to assess this schedule (Supplementary Table S1).

All analyses were conducted in STATA 17™.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.