Abstract
Predictive models of immune protection from COVID19 are urgently needed to identify correlates of protection to assist in the future deployment of vaccines. To address this, we analyzed the relationship between in vitro neutralization levels and the observed protection from severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection using data from seven current vaccines and from convalescent cohorts. We estimated the neutralization level for 50% protection against detectable SARSCoV2 infection to be 20.2% of the mean convalescent level (95% confidence interval (CI) = 14.4–28.4%). The estimated neutralization level required for 50% protection from severe infection was significantly lower (3% of the mean convalescent level; 95% CI = 0.7–13%, P = 0.0004). Modeling of the decay of the neutralization titer over the first 250 d after immunization predicts that a significant loss in protection from SARSCoV2 infection will occur, although protection from severe disease should be largely retained. Neutralization titers against some SARSCoV2 variants of concern are reduced compared with the vaccine strain, and our model predicts the relationship between neutralization and efficacy against viral variants. Here, we show that neutralization level is highly predictive of immune protection, and provide an evidencebased model of SARSCoV2 immune protection that will assist in developing vaccine strategies to control the future trajectory of the pandemic.
Main
Severe acute respiratory syndrome coronavirus 2 (SARSCoV2) has spread globally over the past year, infecting an immunologically naive population and causing significant morbidity and mortality. Immunity to SARSCoV2 induced either through natural infection or vaccination has been shown to afford a degree of protection against reinfection and/or reduce the risk of clinically significant outcomes. Seropositive recovered subjects have been estimated to have 89% protection from reinfection^{1}, and vaccine efficacies from 50 to 95% have been reported^{2}. However, the duration of protective immunity is presently unclear, primary immune responses are inevitably waning^{3,4,5}, and there is ongoing transmission of increasingly concerning viral variants that may escape control by both vaccineinduced and convalescent immune responses^{6}.
A critical challenge at present is to identify the immune correlate(s) of protection from SARSCoV2 infection and thereby predict how changes in immunity will be reflected in clinical outcomes. A defined correlate of protection will permit both confidence in opening up economies and facilitate rapid improvements in vaccines and immunotherapies. In influenza infection, for example, a hemagglutination inhibition (HAI) titer of 1:40 is thought to provide 50% protection from influenza infection^{7} (although estimates range from 1:17 to 1:110, refs. ^{8,9}). This level was established over many years using data from a standardized HAI assay^{10} applied to serological samples from human challenge and cohort studies. This assay is used to predict vaccine efficacy and to assist in the annual reformulation of seasonal influenza vaccines. At present, however, there are few standardized assays for assessing SARSCoV2 immunity, little data comparing immune levels in susceptible versus resistant individuals, and no human challenge model^{11}.
The data currently available for SARSCoV2 infection include immunogenicity data from phase 1 and 2 studies of vaccines, and data on protection from preliminary reports from phase 3 studies and from seropositive convalescent individuals (Supplementary Tables 1 and 2). Although antiviral T and B cell memory certainly contribute some degree of protection, strong evidence of a protective role for neutralizing serum antibodies exists. For example, passive transfer of neutralizing antibodies can prevent severe SARSCoV2 infection in multiple animal models,^{12,13} and Regeneron has recently reported similar data in humans^{14}. We therefore focus our studies on in vitro virus neutralization titers reported in studies of vaccinated and convalescent cohorts. Unfortunately, the phase 1 and 2 studies all use different assays for measuring neutralization. Normalization of responses against a convalescent serum standard has been suggested to provide greater comparability between the results from different assays^{15}. Although all studies compare immune responses after vaccination against the responses in convalescent individuals, the definition of convalescence is not standardized across studies. Similarly, among phase 3 studies, the timeframes of study and the case definitions of infection also vary (Supplementary Table 2). Recognizing these limitations, our aim was to investigate the relationship between vaccine immunogenicity and protection.
Results
Identification of neutralization titer as a correlate of immune protection
To compare neutralization titers across studies, we determined the mean and standard deviation (on a log scale) of the neutralization titer in published data from seven vaccine studies (mRNA1273, NVXCoV2373, BNT162b2, rAd26S+rAd5S, ChAdOx1 nCoV19, Ad26.COV2.S and CoronaVac) and one convalescent study^{3,16,17,18,19,20,21,22} (Supplementary Table 1). Because of the different assays used in each study, neutralization titers were normalized to the mean convalescent titer using the same assay in the same study (noting that the definition of convalescence was also not standardized across studies and a variable number of convalescent samples are analyzed in each study). We then compared this normalized neutralization level in each study against the corresponding protective efficacy reported from the seven phase 3 clinical trials^{19,23,24,25,26,27,28,29} (detailed in Supplementary Table 2). Despite the known inconsistencies between studies, comparison of normalized neutralization levels and vaccine efficacy demonstrates a remarkably strong nonlinear relationship between mean neutralization level and the reported protection across different vaccines (Spearman r = 0.905; P = 0.0046) (Fig. 1a).
Estimation of the protective neutralization level against COVID19
To further dissect the relationship between immunogenicity and protection in SARSCoV2 we considered the parallels with previous approaches to estimating a ‘50% protective titer’ in influenza infection. These historic studies in influenza involved comparison of HAI titers in infected versus uninfected subjects (in either natural infection or human challenge studies) and used logistic or receiver operating characteristic approaches to identify an HAI titer that provided protection^{7,8,9,30,31}. We adapted these approaches to analyze the existing data on reported ‘mean neutralization level’ in different studies and the observed level of protection from infection (details of statistical methods are provided in the Methods).
We first fitted a logistic model to estimate the ‘50% protective neutralization level’ (across all studies) that best predicted the protective effect observed in each study (consistent with the use of a logistic function to model protection in influenza serological studies^{30,31}). We found that this model provided a good explanation of the relationship between mean neutralization level and protection across the studies, and determined that the estimated 50% protective neutralization level was 20.2% (95% confidence interval (CI) = 14.4–28.4%) of the mean convalescent level (Fig. 1a,b). Given that different neutralization assays were used for each study^{11} (see above and Supplementary Table 1), a 50% protective neutralization level equivalent to 20% of the mean titer in the convalescent subjects equates to a measured in vitro neutralization titer of between 1:10 and 1:30 in most of the assays reported (although up to 1:200 in one assay), or we estimate approximately 54 international units (IU)/ml (95% CI 30–96 IU/ml) (Supplementary Table 4). Given that the model is dependent on the mean and distribution of neutralization levels, we also estimated these using different approaches, which led to similar estimates (see Methods and Extended Data Fig. 1).
To relax the assumption that neutralization levels are normally distributed in the above model, we also estimated the protective level using a distributionfree approach and applied this to the raw data for individual neutralization levels reported in the studies. We refer to this as the ‘protective neutralization classification model’. Although this approach may be slightly unrealistic in applying a protected or unprotected cutoff in a binary fashion (unlike the logistic approach), it has the advantage of being independent of any assumptions of the distribution of neutralization titers. Using this classification approach the estimated protective threshold was 28.6% (95% CI = 19.2–29.2%) of the mean convalescent level. As expected, the estimated protective level using the classification method was slightly higher than the 50% protective level estimated using the logistic method, given that the classification method essentially estimates a level of 100% protection instead of 50% protection.
This analysis suggests that the mean in vitro neutralization level of a vaccine measured early after vaccination is predictive of the subsequent protective efficacy measured in phase 3 trials, and estimates that the 50% neutralization level for SARSCoV2 is approximately 20% of the mean convalescent titer. To test the potential utility of this in predicting the protective efficacy of an unknown vaccine, we repeated our analysis using a leaveoneout approach. That is, we repeated our analysis by removing one of the datasets and fitting the model to the remaining seven vaccine or convalescent studies. We then used the parameters obtained from this fitting to predict the efficacy of the eighth (removed) dataset. We repeated this by removing each dataset one at a time. Figure 1c shows the results of using the logistic model of protection to predict the efficacy of each vaccine from the results of the other seven. In addition, after fitting the model to the data for eight vaccine or convalescent studies, the phase 3 efficacy results of another vaccine (BBV152) were released in a press release on 3 March 2021 (ref. ^{32}). Using the observed neutralization level (a mean of 79.2% of the convalescent titer in that study^{33} (Supplementary Table 1), the predicted efficacy of the new vaccine was 79.6% (95% predictive interval: 76.2–83.0%), which is in very close agreement with the reported efficacy of 80.6% (ref. ^{32}) and suggests good predictive value of the model (Fig. 1a).
Modeling of the duration of immune protection after vaccination
Recent studies have identified a decline in neutralization titer with time for up to 8 months after SARSCoV2 infection^{3,4,5}. A major question is whether vaccineinduced responses may be more durable than those measured following infection. Limited studies have analyzed the trajectory of neutralization titer after vaccination^{34}. To compare decay in neutralization titer we fitted a model of exponential decay to equivalent time periods in data from either convalescent^{3} or messenger RNA vaccination^{34} cohorts. Comparing neutralization titers measured between 26 and 115 d (the longest time period available for vaccination) after either mRNAbased vaccination^{34} or symptom onset for postinfection sera^{3}, we found similar halflives (65 d versus 58 d, respectively; P = 0.88, likelihood ratio test; Extended Data Fig. 2a). Although this comparison relies on limited data, it suggests that decay of vaccineinduced neutralization is similar to that observed after natural SARSCoV2 infection.
If the relationship between neutralization level and protection that we observe crosssectionally between different vaccines is maintained over time, we can use the observed relationship between neutralization and protection to predict how the observed waning of neutralization titers might affect vaccine effectiveness. Important caveats to this modeling are that (1) it assumes that neutralization is a major mechanism of protection (or that the mechanism of protection remains correlated with neutralization over time), although B cell memory and T cell responses may be more durable^{3,4,5,35} and may play a larger role later after infection or vaccination; (2) it applies the decay of neutralization observed in convalescence to the vaccine data; and (3) it assumes that the decay in titer is the same regardless of the initial starting titer (whereas others have suggested either faster^{36} or slower^{37} decay for higher initial levels). These limitations notwithstanding, we analyzed the halflife of neutralization titer using published data from a study of convalescent subjects up to 8 months after infection (using a mixedeffects model with censoring) and estimated that neutralization titer decayed with a halflife of 108 d over this period (Extended Data Fig. 2b)^{5}. We also tried alternative models of decay such as biexponential decay (consistent with rapid early decay slowing over time), but found that these did not provide a better fit to the available data. We then used this halflife of 108 d to model the decay of neutralization and protection over the first 250 d after vaccination (Fig. 2a). Our model predicts that even if the waning of neutralization titer over time is the same for different vaccines, this decay will have nonlinear effects on the level of protection from SARSCoV2 infection, depending on initial vaccine efficacy. For example, a vaccine starting with an initial efficacy of 95% would be expected to maintain 77% efficacy by 250 d. However, a response starting with an initial efficacy of 70% would be predicted to drop to 33% efficacy after 250 d. This analysis can also be used to estimate how long it would take a response of a given initial efficacy to drop to 50% (or 70%) efficacy, which may be useful in predicting the time until boosting is required to maintain a minimum level of efficacy (Fig. 2b). Clearly, data generated from standardized assays are needed to track the longterm decay of postvaccination immune responses and their relationship to clinical protection. However, this model provides a framework that can be adapted to predict outcomes as further immune and protection data become available. Indeed, if a disconnect between the decay of neutralization titer and protection is observed, this may be a direct pointer to the role of nonneutralizing responses in protection.
Modeling of the effect of viral variation on immune protection
In addition to the effect of declining neutralization titer over time, reduced neutralization titers and reduced vaccine efficacy to different viral variants have also been observed^{6,38,39,40,41}. For example, it has been reported that the neutralization titer against the B.1.351 variant in vaccinated individuals is between 7.6fold and 9fold lower compared with the early Wuhanrelated Victoria variant^{42}. Our model predicts that a lower neutralization titer against a variant of concern will have a larger effect on vaccines for which protective efficacy against the wildtype virus was lower (Fig. 2c). For example, a fivefold lower neutralization titer is predicted to reduce efficacy from 95% to 77% in a high efficacy vaccine, but from 70% to 32% for a vaccine with lower initial efficacy.
Estimation of the 50% protective level against severe infection
The analysis above investigates vaccine (and convalescent) protection against symptomatic SARSCoV2 infection (using the definitions provided in the different phase 3 and convalescent studies, Supplementary Table 2). However, it is thought that the immune response may provide greater protection from severe infection than from mild infection. To investigate this, we also analyzed data on the observed level of protection from severe infection when these were available. It is important to note that as with symptomatic infection, the definition of severe infection was not consistent across studies (the definitions for each study are detailed in Supplementary Table 3). Given that there have been under 100 severe infections reported across all the phase 3 trials combined, the 95% confidence intervals on the level of protection from severe infection are broad. The neutralization level for 50% protection from severe infection was 3.0% of the mean convalescent level (95% CI = 0.71–13%), which was significantly lower than the 20% level required for protection from any symptomatic infection (P = 0.00039, likelihood ratio test, Supplementary Table 5) (Fig. 3a). An important caveat to this analysis is the implicit assumption that neutralization titer itself confers protection from severe infection. However, it is possible that T cell responses or recall of memory B cell responses may also be important in protection from severe disease^{43,44,45,46}.
Prediction of the potential for longterm protection
The estimated neutralization level for protection from severe infection is approximately sixfold lower than the level required to protect from any symptomatic infection. Thus, a higher level of protection against severe infection is expected for any given level of vaccine efficacy against mild SARSCoV2 infection. Assuming that this relationship remains constant over time, it appears probable that immunity to severe infection may be much more durable than overall immunity to any infection. Longterm studies of antibody responses to vaccinia, measles, mumps or rubella suggest that these responses generally stabilize with halflives of >10 years^{47,48}. We therefore projected beyond the reported decay of SARSCoV2 responses (out to 8 months after infection^{5}), assuming that after 8 months following the infection the decay rate will slow down. We modeled the decay rate of the neutralization titer, assuming that it slowed linearly to a 10year halflife over 1, 1.5 or 2 years (details in Methods). This analysis predicts that even without immune boosting, a significant proportion of individuals may maintain longterm protection from severe infection by an antigenically similar strain, even though they may become susceptible to mild infection (Fig. 3b,c).
Discussion
Understanding the relationship between measured immunity and clinical protection from SARSCoV2 infection is urgently needed to plan the next steps in the COVID19 vaccine program. Placebocontrolled vaccine studies are unlikely to be possible in the development of nextgeneration vaccines, and therefore correlates of immunity will become increasingly important in planning booster doses of vaccine, prioritizing nextgeneration vaccine development, and powering efficacy studies^{49}. Our work uses available data on immune responses and protection to model both the protective titer and the longterm behavior of SARSCoV2 immunity. It suggests that neutralization titer will be an important predictor of vaccine efficacy in the future as new vaccines emerge. The model also predicts that immune protection from infection may wane with time as neutralization levels decline, and that booster immunization may be required within a year. However, protection from severe infection may be considerably more durable given that lower levels of response may be required or alternative responses (such as cellular immune responses) may play a more prominent role.
As discussed above, a major caveat of our estimate of the relative protective level of antibodies in SARSCoV2 infection is that it includes aggregation of data collected from diverse neutralization assays and clinical trial designs (Supplementary Tables 1 and 2). It is hoped that in the future a standardized neutralization assay may be developed and utilized, which will allow direct comparison of neutralization titers and further refinement of these analyses^{11}. In addition, the development of standardized trials and case definitions is necessary, given that differences in classification (particularly of severe disease) may prove to be important (Supplementary Table 3). The association of neutralization with protection across these studies does not prove that neutralizing antibodies are necessarily mechanistic in mediating protection. It is possible that neutralization is correlated with other immune responses, leading to an apparent association (as has been suggested for the use of HAI titer in influenza^{50,51,52}). Thus, it will be important to study other responses such as T cell responses or B cell memory responses as additional potential correlates of protection. Another important refinement of this approach would be to have standardized measures of other serological and cellular responses to infection, to identify if any of these provide a better predictive value than neutralization. However, despite these limitations, our work identifies neutralizing antibodies as an immune correlate of protection and provides a quantitative prediction of the link between neutralizing antibody levels and clinical protection based on evidence from clinical trials and convalescent cohort studies. An important factor that is not explored in this analysis is the role of age in immunogenicity, in the durability of neutralizing responses, and in clinical protection, given that vaccine studies tend to exclude older individuals.
Our method for estimating protective neutralization titer for COVID19 uses a very similar modeling approach to that previously used to estimate the protective titer for influenza infection^{7,8,9,30,31}. However, a major difference is the data used in the models. Our approach utilizes the wide range of immunogenicity and protective efficacy across different vaccines to estimate the 50% protective titer. By contrast, studies in influenza infection use data from the HAI titer of individual subjects and their subsequent risk of infection in either cohort or human challenge studies to estimate protection^{7,8,9,30,31}. It will be important to prospectively validate our results using a similar analysis of individual risk of COVID19 infection in future studies.
Our results are consistent with studies of both influenza and seasonal coronavirus infection, for which reinfection is possible 1 year after the initial infection, although it usually results in mild infection^{53,54}. Similarly, after influenza virus vaccination, protective efficacy is thought to decline by around 7% per month^{55}. Our modeling and predictions are based on a number of assumptions regarding the mechanisms and rate of loss of immunity. Important priorities for the field are the development of standardized assays to measure neutralization and other immune responses, as well as standardized clinical trial protocols. These data will allow further testing and validation of other potential immune correlates of protection. However, our study develops a modeling framework for integrating available, if imperfect, data from vaccination and convalescent studies to provide a tool for predicting the uncertain future of SARSCoV2 immunity.
Methods
Data extraction
When possible, data values were used as directly stated in publications. In addition, when necessary, raw data were directly extracted from the original publications using an online digitizer tool (https://automeris.io/WebPlotDigitizer/, version 4.4). All sources of data are described in Supplementary Tables 1–3.
Statistical methods
Estimation of the standard deviation of neutralization titers
Neutralization titers were extracted for each study (as above) and used to determine the standard deviation of the log_{10}transformed neutralization titers for each study. The standard deviation for each study had to consider that some measurements of neutralization titer fell below the limit of detection (LOD) of that study assay (LOD varied for each study, Supplementary Table 4). To remove the effect of LOD censoring on estimates of the standard deviation of neutralization titers we used a censored regression model to fit the distribution of the neutralization titers for each study. The likelihood function is given by
where D_{s} is a vector of all of the log_{10}neutralization titers, n_{i}, for the study s. The function f is the probability function of a normal distribution with mean μ_{cens} and standard deviation σ_{cens}, and F is the cumulative density function of the same distribution. The LOD of the assay for study s is given by L_{s}. The index variable I_{i} is 1 when n_{i} ≤ L_{s} and 0 otherwise. The negative log of this likelihood function was minimized in R using the builtin optimizer nlm to estimate the mean and standard deviation of the log_{10}transformed neutralization titers after factoring in the LOD. When no LOD was reported or when all of the values were above the LOD, L_{s} was set to negative infinity (−Inf).
Pooled standard deviation
Given the differing sample size of the neutralization data for each study, the accuracy of estimations of the standard deviation for each study varied considerably. Therefore, despite finding some limited evidence of a difference in the standard deviation between each study (P = 0.049, Fligner–Killeen test), we combined all extracted data and calculated the standard deviation of the pooled data. A test of normality showed that the pooled neutralization data were consistent with a normal distribution (P = 0.26, Shapiro–Wilk test). To estimate a pooled standard deviation, we first centered the neutralization data for each study at the reported mean of the neutralization titers for that study. The LOD given for each study was also adjusted in the same way. This provided a combined dataset of neutralization titers from all studies, which was fitted, using the likelihood model in equation (1), to the pooled data to produce an estimate of the standard deviation of the pooled data. All estimated standard deviations are reported in Supplementary Table 4.
Modeling of the relationship between neutralization and protection
In the above sections we used neutralization titer information from each vaccine and from convalescent individuals to estimate the distribution of the neutralization titers for each study. However, as discussed in the main text, owing to the diversity of assays used to assess neutralization in each study (Supplementary Table 1), from this point forward we normalized the neutralization titers in each study by the mean of the neutralization titer in the corresponding convalescent individuals from the same study. Also, in all of the analyses below we use the log_{10} transform of the normalized neutralization titers. For simplicity, in the remainder of this paper we refer to these log_{10}transformed normalized neutralization titers as the ‘neutralization levels’.
Logistic method
To model the relationship between the neutralization level of antibodies collected from individuals after vaccination (or during convalescence) and the protection from COVID19 we assumed a logistic relationship between neutralization level and protective efficacy, such that,
where E_{I} is the protective efficacy of an individual given the neutralization level n (note the definition of neutralization level above). The parameter n_{50} is the neutralization level at which an individual will have a 50% protective efficacy (that is, half the chance of being infected compared with an unvaccinated person). The steepness of this relationship between protective efficacy and neutralization level is determined by the parameter k.
We assume that a vaccine (or prior exposure) will induce a (normal) distribution of neutralization levels (n) in a population with some mean μ_{s} and standard deviation σ_{s}. The mean neutralization level for each study, μ_{s}, is the difference between the log_{10}transformed mean neutralization titers for vaccinated and for convalescent individuals in that study. The standard deviation, σ_{s}, is the standard deviation for vaccinated individuals in that study only. (Note that the distribution of neutralization level for convalescent individuals has a mean of zero by definition (that is, the log of the mean of neutralization titers for convalescent individuals normalized by itself).) Therefore the proportion of the vaccinated population for a study, s, that will be protected will be given by
where f is the probability density function of a normal distribution and E_{I} is the logistic function in equation (2). The above integral is the socalled logisticnormal integral and the mean of the logitnormal distribution, which has no analytical solution^{56}. Therefore, we use a simple numerical approximation (left Riemann sum).
Fitting of the logistic model and confidence intervals
The above model of protection was fitted to data on the protective efficacy of vaccines from phase 3 (and another large cohort study of convalescent individuals). For each vaccine and convalescence study the number of control (unvaccinated, or placebo or naive) individuals enrolled (\(N_s^c\)), the number of control individuals infected (\(I_s^c\)), the number of vaccinated (previously exposed) individuals enrolled (\(N_s^v\)), and the number of vaccinated individuals infected (\(I_s^v\)) were used in the fitting of the model. The likelihood of observing the number of infected individuals in the control and vaccinated groups for each study, given some parameters, is
where b_{s} is the probability of an unvaccinated control individual becoming infected in study s (baseline risk), \(b_s\left( {1  P(n_{50},k,\mu _s,\sigma _s)} \right)\) is the probability of infection in the vaccination group (see equation 3) and Bi(N, K, p) is the binomial probability mass function of the probability of K events from a sample size of N, for which each event has a probability p of occurring. However, we wish to fit all studies simultaneously, and so the total likelihood of observing the data in all studies, given some parameters, is
where N^{c}, I^{c}, N^{v}, I^{v}, μ, σ are vectors containing the data \(N_s^c,I_s^c,N_s^v,I_s^v,\mu _s,\sigma _s\) for each study s, and b is a vector of the baseline risk parameters b_{s} for each study. The bestfitting parameters n_{50}, k and b were found using the nlm optimizer in R by minimizing \( \log \left( {\cal{L}} \right)\). The standard error (s.e.) of these estimates was estimated using the hessian H output from this function and the formula \({\mathrm{s.e.}} = \sqrt {{\mathrm{diag}}(H^{  1})}\). The 95% CIs were taken as ±1.95 × s.e. of the estimated parameters.
The variable μ_{s} is the mean neutralization level, which can be calculated in two ways; first, by dividing the geometric mean of the neutralization titers in vaccinated individuals by the geometric mean of the neutralization titer in convalescent individuals in the same study. This is, in most cases, the ratio of two values directly reported in the immunogenicity studies. However, this approach does not account for situations in which the neutralization assay had neutralization titers below the LOD, therefore we also used a second method, in which we estimated this value by extracting the neutralization titers from the figures in each immunogenicity study (Supplementary Table 1) and computing the mean neutralization titer for vaccinated and convalescent individuals in each study using censoring regression (equation 1). Additionally, although it was in principle possible to compute the standard deviation of neutralization levels for each study (as above), these appeared somewhat confounded by the varying numbers of individuals between studies, hence we fitted the above model using (1) the standard deviation estimates for each study, (2) one standard deviation from one larger study to which we had direct access to raw data^{3} (that is, no manual data extraction required), and (3) an estimate of the standard deviation for all studies pooled together. The two different methods of estimating the mean neutralization level for each study and the three methods of estimating the standard deviation of the neutralization levels give rise to six versions of the above model. All of these versions of the model were fitted, and the estimated protective levels were very similar (Extended Data Fig. 1).
Protective neutralization classification model
The above modeling approach assumed that neutralization levels were normally distributed. Here, we present a method for determining a protective threshold that is free of assumptions regarding the distribution of neutralization levels. This model assumes that there is a protective neutralization level, T, above which individuals will be protected from infection and below which individuals will be susceptible. The protective efficacies observed in phase 3 clinical trials of vaccinated individuals (and another large cohort study of convalescent individuals^{1}; Supplementary Table 2) are denoted by E_{s}. These represent the proportion of individuals in each study, s, who should possess a neutralization level above the protective threshold. It follows then that the number of individuals above the protective threshold in study s is a function of T, which we denote K_{s}(T). Therefore, the probability of observing K_{s}(T) individuals above the protective threshold, given that there were N_{s} individuals in the immunogenicity study (which are much smaller than the phase 3 studies), is given by
where Bi is a binomial distribution. Note that
and N_{s} = D_{s}, such that H is the heavyside step function taking the value 1 when n_{i} − T > 0, or 0 otherwise, and D_{s} denotes the size of set D_{s} (that is, the number of neutralization levels measured). To determine one protective threshold using the results of all efficacy studies in this paper, we construct a likelihood function
where D is the set of vectors of the neutralization levels from each study. Note that this likelihood function is discontinuous as the threshold T is varied. Therefore, we evaluate this likelihood function with the threshold T set equal to all observed neutralization levels n_{i} across all studies, and find the value of T that maximizes this likelihood (Extended Data Fig. 3). This method determines a protective level at which the proportion of individuals with neutralization levels above the threshold is in greatest agreement with the observed protective efficacy of that vaccine.
Equation (8) is the likelihood function that should be adopted when neutralization measurements are not affected by an LOD. In the case that some neutralization levels are below the LOD, the likelihood function is adjusted as follows:
where J_{s} is an index that takes the value 1 when the LOD of study s is above the threshold T and at least one value is censored, or 0 otherwise. C_{s} is the number of censored values in study s and Q is the cumulative binomial distribution function. This later term considers the probability that as many as all of the censored values were below the threshold T given the protective efficacy of the study E_{s}.
To determine the 95% CIs for the estimated protective neutralization level, a bootstrapping approach was used in which the neutralization levels were resampled 1,000 times at random with replacement. Resampling was performed so as to preserve the total number of neutralization levels in each study. These randomly generated samples of the original data were then fitted in the same way as described above, which generated 1,000 corresponding estimates of the protective neutralization level. The 95% CI was calculated as the 2.5 and 97.5 percentiles of these 1,000 estimates of the protective neutralization level.
Assessment of the predictive ability of the model
To determine the ability of the model to predict the efficacy of a vaccine, we performed a leaveoneout analysis in which we systematically excluded one of the vaccine studies (or the convalescent study) and performed the same modelfitting procedure described above. Using the model fitted on the subset of the studies we then estimated the efficacy of the vaccine that was left out from the fitted model. This leaveoneout analysis was performed for all versions of the logistic model (that is, using the six methods of estimating the mean neutralization level and standard deviation outlined above). The predicted efficacy for each vaccine and convalescence obtained while leaving the study out are plotted against the reported efficacy in Fig. 1c. Also, the 50% protective level that was estimated each time a study was left out of the analysis provides a metric of the sensitivity of the model to the inclusion of each study. Note that the exclusion of any of the studies did not greatly influence the estimate of the 50% protective level (Extended Data Fig. 1).
Error bars and regions in efficacy and neutralization
In Fig. 1a there are both horizontal and vertical error bars as well a 95% predictive interval for the fitted model. The vertical error bars indicate the 95% CIs for the efficacy estimates for each study, these were calculated using the Katzlog method specified in supplementary table 1 of ref. ^{57}. The horizontal error bars indicate 95% CIs in the difference of the mean of the (log_{10}) neutralization titers for vaccinated and convalescent individuals in each study. That is, these represent
where σ_{v} and σ_{c} are the standard deviation in the log_{10} of the neutralization titers for vaccinated and convalescent individuals, respectively, and were estimated as described in the section on estimating the standard deviation of neutralization titers above. n_{v} and n_{c} are the number of vaccinated and convalescent individuals who were included for each study, respectively. The 95% predictive interval in Fig. 1a was calculated using the delta method^{58}.
Comparison of the protective level in mild and severe infection
We also tested if the protective neutralization level was different between mild and severe infection by fitting the combined dataset with two different mathematical models. The simplest model assumes that we could use the same protective level in both severe and mild infection (that is, the shared steepness parameter (k) and 50% protective level (n_{50}) in the logistic model (above)); and the alternative model uses different protective level parameters while we constrained the model to have the same k in severe and mild infection (equation 3). We used both the Akaike information criterion and a likelihood ratio test to determine which model provided the best fit of the dataset for both severe COVID19 cases and all COVID19 cases reported (Supplementary Table 5).
Modeling of the decay of neutralization and the effects of antigenic variation
Comparison of decay in convalescence and vaccination
A number of studies have analyzed the decay in neutralization titer in convalescent subjects. These studies have generally shown a rapid early decay that slows with time^{3,4,5,59,60}. We identified one study by Widge et al.^{34} in which the time course of neutralization titer after mRNA vaccination was able to be analyzed. That study measured the decay of neutralization titer out to 115 d. To compare the halflife of decay of the neutralization titer in vaccinated versus convalescent cohorts we analyzed the decay in that vaccination study compared with a previously published study of convalescent neutralization titer^{3}, restricting the convalescent data to data collected within 115 d (Extended Data Fig. 2a).
We compared the decay rates using linear mixedeffects modeling and censoring values below the LOD^{61}, by treating ‘vaccination group’ (vaccinated or convalescent) as a binary variable. The statistical significance of ‘vaccination group’ as a covariate (whether it was significantly different from zero) was calculated using the likelihood ratio test. Note that this restricted convalescent time course was used only so that we could compare decay in vaccination and convalescence over a similar time course (timelimited by the vaccination data).
The halflife of neutralization titers in convalescent subjects was estimated over a longer time course of 240 d using the same mixedeffects modeling approach described above, but applied to the neutralization data reported by Dan et al.^{5} in supplementary data file 1 of their paper. This is the estimate that was used in the predictive model presented in Figs. 2 and 3.
Predicting the loss of efficacy as neutralization wanes and that due to variants
The decay in efficacy with time (Fig. 2a,b) was modeled by reducing the neutralization levels at a rate corresponding to a halflife of 108 d and recalculating the efficacy given this new distribution of neutralization levels, using equation (3). The efficacy against a SARSCoV2 variant, given an associated decrease in neutralization (Fig. 2c), was calculated by reducing the mean neutralization level by a factor of 2, 5 or 10 and using equation (3). Efficacy against severe infection was similarly calculated using the same approach as above, but using the 50% protective level associated with the severe threshold (Supplementary Table 5), which was a factor of 0.15 lower than that for mild infection (95% CI = 0.036–0.65) (Fig. 3a,b). We also extrapolated the decay of the neutralization level beyond the current data, assuming that neutralization levels decay with a halflife of 108 d up until d 250 and that this decay rate slows linearly to a 10year halflife over 1, 1.5 or 2 years (Fig. 3c).
Ethics statement
This work was approved by the University of NSW Sydney Human Research Ethics Committee (approval HC200242).
Reporting Summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
All data are freely available from the corresponding author upon request.
Code availability
All code is freely available from https://github.com/InfectionAnalytics/COVID19ProtectiveThreshold.
References
Lumley, S. F. et al. Antibody status and incidence of SARSCoV2 infection in health care workers. N. Engl. J. Med. 384, 533–540 (2021).
Kim, J. H., Marks, F. & Clemens, J. D. Looking beyond COVID19 vaccine phase 3 trials. Nat. Med. 27, 205–211 (2021).
Wheatley, A. K. et al. Evolution of immune responses to SARSCoV2 in mildmoderate COVID19. Nat. Commun. 12, 1162 (2021).
Gaebler, C. et al. Evolution of antibody immunity to SARSCoV2. Nature 591, 639–644 (2021).
Dan, J. M. et al. Immunological memory to SARSCoV2 assessed for up to 8 months after infection. Science 371, eabf4063 (2021).
Wang, P. et al. Antibody resistance of SARSCoV2 variants B.1.351 and B.1.1.7. Nature https://doi.org/10.1038/s41586021033982 (2021).
Hobson, D., Curry, R. L., Beare, A. S. & WardGardner, A. The role of serum haemagglutinationinhibiting antibody in protection against challenge infection with influenza A2 and B viruses. J. Hyg. (Lond.) 70, 767–777 (1972).
Coudeville, L. et al. Relationship between haemagglutinationinhibiting antibody titres and clinical protection against influenza: development and application of a bayesian randomeffects model. BMC Med. Res. Methododol. 10, 18 (2010).
Black, S. et al. Hemagglutination inhibition antibody titers as a correlate of protection for inactivated influenza vaccines in children. Pediatr. Infect. Dis. J. 30, 1081–1085 (2011).
Laurie, K. L. et al. International laboratory comparison of influenza microneutralization assays for A(H1N1)pdm09, A(H3N2), and A(H5N1) influenza viruses by CONSISE. Clin. Vaccine Immunol. 22, 957–964 (2015).
Khoury, D. S. et al. Measuring immunity to SARSCoV2 infection: comparing assays and animal models. Nat. Rev. Immunol. 20, 727–738 (2020).
Rogers, T. F. et al. Isolation of potent SARSCoV2 neutralizing antibodies and protection from disease in a small animal model. Science 369, 956–963 (2020).
McMahan, K. et al. Correlates of protection against SARSCoV2 in rhesus macaques. Nature 590, 630–634 (2021).
Regeneron Reports Positive Interim Data with REGENCOV™ Antibody Cocktail used as Passive Vaccine to Prevent COVID19 (Regeneron press release, 26 January 2021). https://newsroom.regeneron.com/newsreleases/newsreleasedetails/regeneronreportspositiveinterimdataregencovtmantibody (2021).
Mattiuzzo, G., et al. Establishment of the WHO International Standard and Reference Panel for AntiSARSCoV2 Antibody (WHO/BS/2020/2403) https://www.who.int/publications/m/item/WHOBS2020.2403 (WHO, 2020).
Jackson, L. A. et al. An mRNA vaccine against SARSCoV2: preliminary report. N. Engl. J. Med. 383, 1920–1931 (2020).
Keech, C. et al. Phase 1–2 trial of a SARSCoV2 recombinant spike protein nanoparticle vaccine. N. Engl. J. Med. 383, 2320–2332 (2020).
Walsh, E. E. et al. Safety and immunogenicity of two RNAbased Covid19 vaccine candidates. N. Engl. J. Med. 383, 2439–2450 (2020).
Logunov, D. Y. et al. Safety and efficacy of an rAd26 and rAd5 vectorbased heterologous primeboost COVID19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet 397, 671–681 (2021).
Folegatti, P. M. et al. Safety and immunogenicity of the ChAdOx1 nCoV19 vaccine against SARSCoV2: a preliminary report of a phase 1/2, singleblind, randomised controlled trial. Lancet 396, 467–478 (2020).
Sadoff, J. et al. Interim results of a phase 12a trial of Ad26.COV2.S Covid19 vaccine. N. Engl. J. Med. https://doi.org/10.1056/NEJMoa2034201 (2021).
Zhang, Y. et al. Safety, tolerability, and immunogenicity of an inactivated SARSCoV2 vaccine in healthy adults aged 18–59 years: a randomised, doubleblind, placebocontrolled, phase 1/2 clinical trial. Lancet Infect. Dis. 21, 181–192 (2021).
Baden, L. R. et al. Efficacy and safety of the mRNA1273 SARSCoV2 vaccine. N. Engl. J. Med. 384, 403–416 (2020).
Polack, F. P. et al. Safety and efficacy of the BNT162b2 mRNA Covid19 vaccine. N. Engl. J. Med. 383, 2603–2615 (2020).
Voysey, M. et al. Safety and efficacy of the ChAdOx1 nCoV19 vaccine (AZD1222) against SARSCoV2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 397, 99–111 (2021).
Reuters. Sinovac Says COVID19 Vaccine Effective in Preventing Hospitalization, Death https://www.reuters.com/article/ushealthcoronavirussinovacbiotechidUSKBN2A52Q6 (2021).
Janssen. Johnson & Johnson Announces SingleShot Janssen COVID19 Vaccine Candidate Met Primary Endpoints in Interim Analysis of its Phase 3 ENSEMBLE Trial https://www.janssen.com/johnsonjohnsonannouncessingleshotjanssencovid19vaccinecandidatemetprimaryendpoints (2021).
Novavax COVID19 Vaccine Demonstrates 89.3% Efficacy in UK Phase 3 Trial (Novavax press release 28 January 2021) https://ir.novavax.com/ (2021).
Mahase, E. Covid19: Novavax vaccine efficacy is 86% against UK variant and 60% against South African variant. BMJ 372, n296 (2021).
Qin, L., Gilbert, P. B., Corey, L., McElrath, M. J. & Self, S. G. A framework for assessing immunological correlates of protection in vaccine trials. J. Infect. Dis. 196, 1304–1312 (2007).
Dunning, A. J. A model for immunological correlates of protection. Stat. Med. 25, 1485–1497 (2006).
Bharat Biotech Announces Phase 3 Results of COVAXIN®: India’s First COVID19 Vaccine Demonstrates Interim Clinical Efficacy of 81% (Bharatbiotech press release 3 March 2021) https://www.bharatbiotech.com/press_releases.html (2021).
Ella, R. et al. Safety and immunogenicity of an inactivated SARSCoV2 vaccine, BBV152: interim results from a doubleblind, randomised, multicentre, phase 2 trial, and 3month followup of a doubleblind, randomised phase 1 trial. Lancet Infect. Dis. https://doi.org/10.1016/S14733099(21)000700 (2021).
Widge, A. T. et al. Durability of responses after SARSCoV2 mRNA1273 vaccination. N. Engl. J. Med. 384, 80–82 (2021).
Rodda, L. B. et al. Functional SARSCoV2specific immune memory persists after mild COVID19. Cell 184, 169–183 (2021).
Pradenas, E. et al. Stable neutralizing antibody levels 6 months after mild and severe COVID19 episodes. Med (NY) 2, 313–320 (2021).
Chia, W. N. et al. Dynamics of SARSCoV2 neutralising antibody responses and duration of immunity: a longitudinal study. Lancet Microbe https://doi.org/10.1016/S26665247(21)000252 (2021).
Chen, R. E. et al. Resistance of SARSCoV2 variants to neutralization by monoclonal and serumderived polyclonal antibodies. Nat. Med. 27, 717–726 (2021).
Wang, Z. et al. mRNA vaccineelicited antibodies to SARSCoV2 and circulating variants. Nature 592, 616–622 (2021).
Wu, K. et al. Serum neutralizing activity elicited by mRNA1273 vaccine. N. Engl. J. Med. 384, 1468–1470 (2021).
Liu, Y. et al. Neutralizing activity of BNT162b2elicited serum. N. Engl. J. Med. 384, 1466–1468 (2021).
Zhou, D. et al. Evidence of escape of SARSCoV2 variant B.1.351 from natural and vaccineinduced sera. Cell 184, 2348–2361 (2021).
Sette, A. & Crotty, S. Adaptive immunity to SARSCoV2 and COVID19. Cell 184, 861–880 (2021).
Rydyznski Moderbacher, C. et al. Antigenspecific adaptive immunity to SARSCoV2 in acute COVID19 and associations with age and disease severity. Cell 183, 996–1012 (2020).
Juno, J. A. et al. Humoral and circulating follicular helper T cell responses in recovered patients with COVID19. Nat. Med. 26, 1428–1434 (2020).
Cromer, D. et al. Prospects for durable immune control of SARSCoV2 and prevention of reinfection. Nat. Rev. Immunol. https://doi.org/10.1038/s4157702100550x (2021).
Amanna, I. J., Carlson, N. E. & Slifka, M. K. Duration of humoral immunity to common viral and vaccine antigens. N. Engl. J. Med. 357, 1903–1915 (2007).
Antia, A. et al. Heterogeneity and longevity of antibody memory to viruses and vaccines. PLoS Biol. 16, e2006601 (2018).
Coronavirus (COVID19) Update: FDA Issues Policies to Guide Medical Product Developers Addressing Virus Variants. Suite of Guidances Addresses Vaccines, Diagnostics and Therapeutics (FDA news release 22 February 2021) https://www.fda.gov/newsevents/pressannouncements/coronaviruscovid19updatefdaissuespoliciesguidemedicalproductdevelopersaddressingvirus (2021).
Monto, A. S. et al. Antibody to influenza virus neuraminidase: an independent correlate of protection. J. Infect. Dis. 212, 1191–1199 (2015).
Memoli, M. J. et al. Evaluation of antihemagglutinin and antineuraminidase antibodies as correlates of protection in an influenza A/H1N1 virus healthy human challenge model. mBio 7, e0041716 (2016).
Cheng, L. W. et al. Comparison of neutralizing and hemagglutinationinhibiting antibody responses for evaluating the seasonal influenza vaccine. J. Virol. Methods 182, 43–49 (2012).
Callow, K. A., Parry, H. F., Sergeant, M. & Tyrrell, D. A. The time course of the immune response to experimental coronavirus infection of man. Epidemiol. Infect. 105, 435–446 (1990).
Memoli, M. J. et al. Influenza A reinfection in sequential human challenge: implications for protective immunity and ‘universal’ vaccine development. Clin. Infect. Dis. 70, 748–753 (2020).
Ferdinands, J. M. et al. Intraseason waning of influenza vaccine protection: evidence from the US Influenza Vaccine Effectiveness Network, 2011–12 through 2014–15. Clin. Infect. Dis. 64, 544–550 (2017).
Aitchison, J. & Shen, S. M. Logisticnormal distributions: some properties and uses. Biometrika 67, 261–272 (1980).
Aho, K., Bowyer, R. T. & O’Hara, R. B. Confidence intervals for ratios of proportions: implications for selection ratios. Methods Ecol. Evolution 6, 121–132 (2015).
Ver Hoef, J. M. Who invented the delta method? Am. Statistician 66, 124–127 (2012).
Crawford, K. H. D. et al. Dynamics of neutralizing antibody titers in the months after severe acute respiratory syndrome coronavirus 2 infection. J. Infect. Dis. 223, 197–205 (2021).
Ibarrondo, F. J. et al. Rapid decay of antiSARSCoV2 antibodies in persons with mild Covid19. N. Engl. J. Med. 383, 1085–1087 (2020).
Vaida, F. & Liu, L. Fast implementation for normal mixed effects models with censored response. J. Comput. Graph. Stat. 18, 797–817 (2009).
Acknowledgements
This work is supported by Australian Government Medical Research Future Fund awards GNT2002073 (to S.J.K., M.P.D. and A.K.W.), MRF2005544 (to S.J.K., A.K.W., J.A.J. and M.P.D.) and MRF2005760 (to M.P.D.), an NHMRC program grant GNT1149990 (to S.J.K. and M.P.D.), NHMRC CRE 1153493 (to J.A.T.), and by the Victorian Government (S.J.K., A.K.W., J.A.J.). J.A.J., D.S.K. and S.J.K. are supported by NHMRC fellowships. A.K.W., D.C., K.S. and M.P.D. are supported by NHMRC Investigator grants. This study was supported by the Jack Ma Foundation (K.S.) and the A2 Milk Company (K.S.). The Melbourne WHO Collaborating Centre for Reference and Research on Influenza is supported by the Australian Government Department of Health.
Author information
Authors and Affiliations
Contributions
All authors contributed to the data collection, design of the study, writing of the manuscript and revision of the manuscript. D.S.K., D.C., A.R., T.E.S. and M.P.D. contributed to the modeling and statistical analysis of the data.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Medicine thanks Joshua Schiffer and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Alison Farrell was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Estimates of the 50% protective neutralization level with different models.
Two models, the logistic model (purple and orange shaded regions) and the protective neutralization classification model (PNC Model, black), were fit to the data on neutralization levels in each study and protective efficacy of vaccination (and convalescence) from 8 studies. Due to the limitations of reconciling data across the 7 vaccines (and studies of convalescent individuals) a number of approaches were employed to estimate the 50% protective neutralization level (described in Methods). Additionally, variation in estimates of the 50% protective neutralization level was explored when studies were excluded as a sensitivity analysis (‘leave one out analysis’). All of the approaches applied produced similar estimates of the 50% protective level. The logistic model was fit 6 times using a mean and standard deviation of the distribution in neutralization levels for each study that was estimated under different methods described above (Cens. = mean estimated using censoring regression, Rep. = mean as reported in the original study with no censoring, Study = using standard deviation (SD) estimated for each individual study, Conv. = using SD from the convalescence study^{3}, and Pool = using SD from pooling all neutralization data from all studies). Central horizontal bars indicate the estimated 50% protective neutralization level (for the logistic model), or protective threshold (for the PNC model) from each model. The error bars indicate the 95% confidence interval on these estimates. The 8 crosses overlaid on each model indicate the estimates of the protective level obtained when the same model is fit to only 7 of the 8 studies (that is excluding one study), and highlights that no single study strongly influences the estimate of protective neutralization level. *Note that in this figure we present nonlog transformed protective neutralization levels to maintain consistency with the main text.
Extended Data Fig. 2 Neutralization titers reported in vaccinated and convalescent individuals over time.
The decay halflives for convalescent and vaccinated individuals were estimated using a linear mixed effects model with censoring of neutralization titers below the limit of detection (dashed lines). a, The decay halflives were not significantly different between vaccinated^{34} and convalescent^{3} individuals (P=0.88, likelihood ratio test). b, The decay halflife in convalescent individuals out to approximately 240 days was estimated using data from Dan et al.^{5}.
Extended Data Fig. 3 Fitting the Protective Neutralization Classification Model.
Estimating the This figure shows the estimated loglikelihood (log of equation 9 of Methods) as a function of protective neutralization threshold. To estimate the protective neutralization threshold that is most consistent with the reported efficacy of each vaccine (and convalescence), the loglikelihood function (log of equation 9) was evaluated at all neutralization levels observed in any of the studies listed in Supplementary Table 1, and the best fitting protective level was determined as the neutralization level that maximized the likelihood function.
Supplementary information
Supplementary Information
Supplementary Tables 1–5.
Rights and permissions
About this article
Cite this article
Khoury, D.S., Cromer, D., Reynaldi, A. et al. Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARSCoV2 infection. Nat Med 27, 1205–1211 (2021). https://doi.org/10.1038/s41591021013778
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41591021013778
This article is cited by

Kinetics of COVID19 mRNA primary and booster vaccineassociated neutralizing activity against SARSCoV2 variants of concern in longterm care facility residents: a prospective longitudinal study in Japan
Immunity & Ageing (2023)

Frailty impacts immune responses to Moderna COVID19 mRNA vaccine in older adults
Immunity & Ageing (2023)

Prevalence and characteristics of COVID19 vaccine breakthrough infection in Upper Egypt
The Egyptian Journal of Bronchology (2023)

Efficacy and risk of mRNA vaccination in patients with autoimmune inflammatory rheumatic diseases
Inflammation and Regeneration (2023)

Comparison of antispike IgG, antispike IgA levels and neutralizing antibody activity induced by CoronaVac and BNT162b2 vaccines in patients with inflammatory rheumatic diseases receiving immunosuppressive therapy
BMC Rheumatology (2023)