Introduction

Lateral flow immunoassay tests (LFIAs) have been used at a population level to track antibody prevalence1 and quantify waning antibody levels post vaccination2. The clinical value of these simple rapid tests at an individual level is less clear, although they have reasonable sensitivity and specificity in detecting SARS-CoV-2 antibodies at a threshold relevant to viral neutralisation3. Clinical use of LFIAs has been suggested for tracking seroconversion post-infection4 and post-vaccination5, and monitoring waning antibody levels in the months following vaccination6, for example in those with immunosuppression7. Further, IgG antibody positivity using self-administered LFIAs has been shown to predict COVID-19 infection risk over a 200-day follow-up period in an unvaccinated population8. Nonetheless, to date there has been limited uptake of antibody LFIAs at an individual level to guide understanding of personal risk from COVID-19 and inform decision-making about behaviour and need for boosters or other protections.

The REal-time Assessment of Community Transmission-2 (REACT-2) study was a nationwide study of SARS-CoV-2 antibody prevalence in adults in England in which randomly selected adults were invited to complete an LFIA test at home and to complete a short online or telephone survey9. Here we follow up data from adults who took part in the final two rounds of REACT-2, conducted in January–May 2021, after the start of the national vaccination campaign, using their linked National Health Service (NHS) administrative health records. We use these data to assess whether LFIA tests have predictive value for adverse health outcomes in the general population, and whether they might be useful in a clinical setting. Specifically, we explore whether LFIA results are associated with future hospitalisation (all-cause, emergency and COVID-19) and mortality (all-cause and COVID-19).

Results

The study population included 361,801 participants, of whom 143,774 (39.7%) tested positive on self-administered LFIAs in rounds 5 and 6 (January–February 2021 and May 2021, respectively) of the REACT-2 study. Table 1 shows the characteristics of the study population overall and by “high” or “low” COVID-19 risk based on the UK Joint Committee on Vaccination and Immunisation (JCVI) definition of being clinically extremely vulnerable for COVID-19 (see Supplementary Methods, S.1). The mean age was 55.5 years; 31.4% were aged 50 years or under; and 56.0% were women. A single vaccination was reported by 28.9% of participants and two vaccinations by 24.3%. Over 1 in 6 (15.8%) reported prior COVID-19, although only 24.9% of these cases had been confirmed (e.g. by polymerase chain reaction (PCR) test) as most occurred in the first COVID-19 wave when community testing was limited.

Table 1 Population characteristics, by risk group status and LFIA result
Full size table

Among the 192,637 people who had received one or more vaccinations at least 14 days prior to the LFIA test, 128,282 (66.6%) were LFIA positive; 88,026 people had received two vaccinations, of whom 74,534 (84.7%) tested positive. Of the 14,271 with prior COVID-19 confirmed with a test, 12,065 (84.5%) were LFIA positive; of the 57,317 who reported confirmed or suspected prior COVID-19, 29,700 (51.8%) were LFIA positive.

We obtained follow-up data on hospitalisations to September 30, 2021, with mean follow-up time of 184 days and a total of 182,659 person years of follow-up. For mortality we obtained data to December 1, 2021, with a mean follow-up time of 245 days and a total of 243,125 person years of follow-up.

For hospitalisations, 16,802 (4.6%) people were admitted for any cause (excluding pregnancy and accident/injury related, see “Methods”), 5330 of whom were unplanned (emergency) admissions, 91 had COVID-19 as the primary diagnosis and 151 had COVID-19 listed anywhere on their hospital record. There were 389 deaths, including 19 with COVID-19 as the underlying condition and 21 with COVID-19 mentioned anywhere on the death certificate (Table 1).

In N = 25,824 high-risk individuals with one or more vaccinations, those testing LFIA negative had a risk of all-cause mortality of 3.6, compared to 1.7 per 1000 in those testing LFIA positive, an absolute difference of 1.9 per 1000 (Supplementary Table 1).

Figure 1 shows Kaplan–Meier plots of the cumulative hazard functions for hospitalisations and deaths in individuals who had received one or more vaccinations (N = 192,604). Those testing negative on the LFIA antibody test had higher cumulative hazards for all outcomes at almost all time points during follow-up. Findings were similar when stratified by risk group, by age and by vaccination status (Fig. 1 and Supplementary Fig. 1).

Fig. 1: Kaplan–Meier plots showing cumulative hazard among people reporting a positive test on the LFIA (grey line) and those reporting a negative test (red line) for (from top left to bottom right) (i) all-cause hospitalisation, (ii) emergency hospitalisation, (iii) hospitalisation with COVID-19 as the primary diagnosis, (iv) hospitalisation with COVID-19 mentioned anywhere on the hospital record, and (v) all-cause mortality, among N = 192,604 participants who had received one or more vaccines.
figure 1

95% pointwise confidence intervals are shown in shaded colour.

Full size image

In mutually adjusted Cox regression models accounting for age, sex, risk group, prior COVID-19 and vaccination count, those with a negative LFIA after one or more vaccinations had an increased mortality from all causes (HR: 1.59 [1.07, 2.37], and mortality with COVID-19 as underlying cause (HR: 20.6 [1.83, 232]), as well as increased risk of hospitalisation with COVID-19 as the primary diagnosis (2.73 [1.15, 6.48]).

After two or more vaccinations, those with a negative LFIA had increased all-cause mortality (HR: 1.87 [1.11, 3.13]), mortality with COVID-19 as underlying cause (HR: 12.46 [1.13, 138]), hospitalisation with COVID-19 as the primary diagnosis (2.79 [1.01, 7.72]) and hospitalisation with COVID-19 mentioned (2.54 [1.01, 6.41]) compared to people who tested positive (Table 2 and Supplementary Tables 2 and 3).

Table 2 Hazard ratios from multiple Cox regression models
Full size table

Stability selection

We used 100x subsampled Least Absolute Shrinkage and Selection Operator (LASSO)-penalised Cox models with demographic, biological and health variables together with LFIA result to identify which were preferentially selected as predictive of outcomes (see “Methods”). In these multivariable models, LFIA result was not selected as a predictor of subsequent hospitalisation, emergency hospitalisation, hospitalisation with COVID-19 or death in the population with one or more vaccines, nor when stratified by risk group (Fig. 2). The main positive predictors selected were risk group, age (continuous), anaemia, smoking and body mass index (BMI, continuous).

Fig. 2: Results of penalised Cox modelling with stability selection.
figure 2

Results shown for five main outcomes (l-r: hospitalisation, emergency hospitalisation, hospitalisation with COVID-19, hospitalisation with COVID-19 mentioned, death) and for vaccinated (one or more) cohort (top, N = 192,604), high-risk vaccinated cohort (middle, N = 35,812) and low-risk vaccinated cohort (bottom, N = 156,792).

Full size image

Boosted tree models

We used boosted tree models trained on different subsets of predictors, with and without LFIA, cross-validated on unseen data, to assess the predictive value of LFIA result over and above the other variables (age, sex, risk group, prior COVID-19 history and vaccination count, see “Methods”) in the population who had received one or more vaccines. Adding LFIA result to the boosted tree model did not substantially improve the prediction of hospitalisation or death (Fig. 3); however predictive accuracy was improved by including additional survey data (BMI, smoking status, deprivation, education status, comorbidities) to the model. The findings were consistent when analysis was repeated in the full population (Supplementary Fig. 3).

Fig. 3: ROC curves quantifying the additional predictive power conferred by adding LFIA test result to other survey data from REACT-2 (age, sex, BMI, smoking status, education, deprivation, vaccination status, prior COVID-19 infection, comorbidities) in the vaccinated (one or more vaccines) population (N = 192,604).
figure 3

Models built using boosted tree models (Catboost) with 5-fold cross-validation. Outcome is a binary event within follow-up y/n variable. 95% confidence intervals, derived from 1000x bootstrap resamples, are shown in shaded colour around the curves.

Full size image

Discussion

In this large, representative study of the population of England, we found that a negative LFIA antibody test result from early 2021, after receiving one or more vaccines at least 14 days previously, was associated with increased risk of hospitalisation from COVID-19 and of death from all causes and those attributed to COVID-19 over the subsequent 6–9 months. These associations persisted after adjustment for multiple factors including age. However, a combination of variable selection approaches and tree-based predictive modelling indicated that the information contained in the LFIA result could be adequately captured by the responses to a set of survey questions on basic demographics and health status.

Nonetheless, the LFIA test might provide a simple and rapid means to identify those at increased risk who might benefit from additional booster doses or rapid access to antiviral medication. The LFIA is a threshold test which correlates with SARS-CoV-2 antibody titres and live virus neutralisation10 and negativity following vaccination or infection may be a biomarker of a limited specific immune response11 and of frailty12 and therefore may be linked to hospitalisation and all-cause mortality. People with immunosuppression have lower antibody titres post-vaccination, and approximately one in five people with solid organ transplants, rare autoimmune rheumatic diseases and lymphoid malignancies are LFIA negative after three vaccine doses13.

In addition, the LFIA provides information, which can be obtained rapidly and at scale, on population antibody levels which may inform the timing and prioritisation of vaccination booster campaigns14.

Our study has limitations. LFIAs have imperfect accuracy: the tests used in this study were reported to have sensitivity of 78.7% (95% confidence interval [CI]: 71.8, 84.6)10; therefore, our LFIA test results will likely include some false negatives which may have reduced (biased) our risk estimates for hospitalisations and mortality following a negative test post-vaccination. Furthermore, the sensitivity of LFIAs may be lower in the weeks following vaccination; although we considered here a vaccine dose to be effective 14 days after injection, we cannot rule out the possibility of bias from time-since-vaccination, which is related to age and comorbidities15. In addition, self-administered LFIAs depend on user interpretation of the finding; nonetheless, validation analysis of the user-interpreted results from REACT-2 showed substantial concordance between user submitted results and those generated both by experts (Cohen’s kappa: 0.89) and by a computer vision machine learning algorithm (Cohen’s kappa: 0.80)16,17,18. There are also limitations to the routine data on outcomes including the duration of follow-up which was limited so far to between 6 and 9 months; in addition hospital and mortality data are from the monthly release of provisional data, while full quality assurance is only provided for annual extracts. It is therefore possible that some misclassification may have occurred. A further limitation is possible recruitment bias. Although the REACT-2 study included a large random population sample2, there may have been differential participation linked to risk of severe outcomes.

In conclusion, a negative LFIA result following vaccination is a marker of elevated risk of hospitalisation and death over a 6–9-month time horizon. For people designated at high risk from COVID-19, there was an additional risk of all-cause mortality of 1.9 per 1000 for those testing antibody negative compared to positive. However, the LFIA does not provide substantial predictive information over and above that which is available from detailed sociodemographic and health-related variables. Nonetheless, this simple test provides a marker which could be a valuable addition to understanding population and individual level risk.

Methods

The REACT-2 programme monitored the community prevalence of SARS-CoV-2 antibodies among adults in England, through a series of six large representative random samples of the population between July 2020 and May 2021. The study protocol9 and summary reports for each round1,2,19,20,21 have been published. Briefly: every 6–8 weeks, personalised invitations were sent to 315,000 adults aged 18 years and above on the NHS register, to achieve approximately equal numbers of responses from each of the 315 lower-tier local authority areas in England. Registration was done by telephone or online, and recruitment ceased when ~120,000 people had signed up. In the sixth and final round (May 2021), a boosted sample of adults aged 50 and above was obtained, with 749,225 invitations sent out resulting in 255,750 (34.1%) registrations and 209,482 (30.0%) completed tests.

Respondents were sent a LFIA (Fortress, United Kingdom) test kit for SARS-CoV-2 antibody self-testing and asked to report their test result and upload a photo of the completed test. Respondents also filled out a questionnaire including details of COVID-19 vaccination history, self-reported comorbidities and any history of suspected or confirmed COVID-19 (questionnaires available online at https://www.imperial.ac.uk/medicine/research-and-impact/groups/react-study/for-researchers/react-2-study-materials/).

Here, we use data on participants from rounds 5 (January–February 2021) and 6 (May 2021). We obtained Hospital Episode Statistics Admitted Patient Care data (until September 30, 2021) and Civil Registration Deaths data (until December 1, 2021) from NHS Digital for all participants with a valid LFIA result (positive or negative) who consented to this data linkage. We assessed the following outcomes from the date of completion of the LFIA to end of follow-up (December 1, 2021) or date of death: hospital admission for any reason (excluding admissions for pregnancy loss or childbirth, injuries, accidents or other external causes), unplanned/emergency admission for any reason (with the same exclusions), admission with COVID-19 as the primary diagnosis, admission where COVID-19 was listed anywhere on the record, all-cause mortality, deaths where COVID-19 was reported as the underlying condition, and deaths with COVID-19 mentioned anywhere on the death certificate (see Supplementary Methods S.1).

Participants’ survey data were used to group people into age groups (50 years and under; 51–70 years; 71 years and over); male or female; previous history of COVID-19 (no, yes; confirmed, yes; probable or suspected); and self-reported vaccination status (0, 1, 2) with people considered vaccinated 14 days after the reported date of vaccination. Participants who had received 3 or more vaccines at the time of the fieldwork were excluded from analysis. Participants were categorised as “high” or “low” risk based on the UK Joint Committee on Vaccination and Immunisation (JCVI) definition of being clinically extremely vulnerable for COVID-1922,23 using self-reported comorbidity information (see Supplementary Methods S.2). Participants were also categorised into ethnic groups (Asian, Black, Mixed, Other, White); index of multiple deprivation (IMD) quintile using the residential postcode and current cigarette smoking status (yes, no). BMI was calculated from self-reported height and weight. Educational level was self-reported and grouped into categories.

Statistical analyses

Using Kaplan–Meier plots we calculated the rate of each outcome by LFIA result and compared the cumulative hazard functions for those testing negative for the vaccinated group (N = 192,604) and for all participants, for each of the outcomes of interest. Plots were stratified by risk group, and then further stratified by age group and, for the full population, by vaccination status. We used multivariable Cox regression models to test the association between LFIA result and each outcome stratified by vaccination status in the full cohort. For each outcome, we constructed a series of nested models with increasing adjustments by age group, sex, (vaccination history), prior infection status and risk group. Hazard ratios (HRs) and 95% confidence intervals are reported.

To test whether the LFIA test result was adding substantially, as a predictor of health outcomes, to the information available in the survey (health, comorbidity and demographic), we used two multivariable approaches and included a range of potentially relevant covariates alongside the LFIA result. First, we used LASSO-penalised Cox regression models with stability analysis as implemented in the SHARP R package24 to identify a parsimonious set of stably selected predictors for each of the outcomes. Selection thresholds were calibrated using an internal stability score. We conducted this analysis among individuals who had received one or more vaccinations and in the full cohort.

Second, we used extreme gradient boosted tree models as implemented in Catboost25 to predict each of the binary outcomes in unseen data using a five-fold cross validation procedure. We evaluated various combinations of predictors (LFIA result only; covariates [age group, sex, risk group, self-reported vaccination status/count, self-reported COVID-19 history] only; LFIA + covariates; LFIA + covariates + additional health and demographic data [BMI, smoking status, comorbidities, deprivation, education]) and evaluated model performance on unseen data using the area under the Receiver Operating Characteristic (ROC) curve (which provides the predictive accuracy of a binary classifier at different discrimination thresholds). Again, we conducted this analysis among individuals who had received one or more vaccinations and in the full cohort.

All data collection for the REACT2 study was captured with Questback. All data were analysed using R v4.0.5 (2021).

Ethics

The REACT-2 study holds ethical approval from South Central—Berkshire B Research Ethics Committee (20/SC/0206; 21/SC/0163). Participants provided informed consent for their data to be used and, separately, indicated whether they were willing for their data to be linked to their NHS records.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.