Abstract
Advances in flow cytometry and other singlecell technologies have enabled highdimensional, highthroughput measurements of individual cells as well as the interrogation of cell population heterogeneity. However, in many instances, computational tools to analyze the wealth of data generated by these technologies are lacking. Here, we present a computational framework for unbiased combinatorial polyfunctionality analysis of antigenspecific Tcell subsets (COMPASS). COMPASS uses a Bayesian hierarchical framework to model all observed cell subsets and select those most likely to have antigenspecific responses. Cellsubset responses are quantified by posterior probabilities, and human subject–level responses are quantified by two summary statistics that describe the quality of an individual's polyfunctional response and can be correlated directly with clinical outcome. Using three clinical data sets of cytokine production, we demonstrate how COMPASS improves characterization of antigenspecific T cells and reveals cellular 'correlates of protection/immunity' in the RV144 HIV vaccine efficacy trial that are missed by other methods. COMPASS is available as opensource software.
Main
Recent technological advances in flow and mass cytometry assays have transformed the field of immunology by enabling large numbers of parameters to be quantified at the singlecell level in a highthroughput fashion. Many studies and clinical trials now rely on these assays to provide multiparameter, singlecell measurements, which enable a more comprehensive understanding of cellular function than methods that measure one parameter at a time. In particular, singlecell analyses by intracellular cytokine staining (ICS)—a type of flow cytometry assay (Fig. 1)—have become important tools to characterize subsets of antigenspecific T cells capable of simultaneously producing multiple effector cytokines and other functional markers, known as polyfunctional T cells^{1}. Polyfunctional T cells have an important role in protective immunity and nonprogression of diseases, and their presence correlates with favorable clinical outcomes^{2,3,4}. Vaccination in humans can generate broad Tcell cytokine responses^{5,6}, making polyfunctional Tcell subsets attractive as potential biomarkers of protection against disease. However, statistical tools for analyzing the complexity of these immune responses are lacking.
Although many analytic tools exist for cytometrybased assays^{7,8,9}, very few tools have been developed specifically for highdimensional ICS data analysis. Existing strategies are basic and low dimensional, ranging from ad hoc rules based on foldchanges^{10}, Hotelling's T^{2} statistics^{11} and 2 × 2 contingency tables^{12,13} to simple graphical displays of summary statistics^{7}. In most ICS assays, the frequencies (and thus cell counts) of antigenspecific subsets are very small (<0.1% of total T cells), making robust statistical analysis difficult. This difficulty increases as the cells are further partitioned based on coexpression of multiple cytokines. In addition, multiple comparisons across subjects (persons in the clinical study) and cell subsets must be adjusted for and can further reduce statistical power. Given the lack of analytical tools, studies have been limited to formal statistical comparisons of only a few functions (secreted cytokines), and most of the work on polyfunctionality has been phenomenological.
To address some of these limitations, Finak et al.^{14} proposed a framework called mixture models for singlecell assays (MIMOSA). MIMOSA is based on mixtures of betabinomials and rigorously analyzes count data derived from ICS assays but was mainly developed for univariate analysis of cell subsets, such as cells expressing a single function or a specified combination of functions. Although MIMOSA includes a multivariate version, Finak et al.^{14} were solely interested in identifying positive immune responses irrespective of the qualitative aspect of the response, and as a result, the output is still univariate (probability of response). Furthermore, to apply MIMOSA to multivariate data, the authors assumed that a measurable antigenspecific response exists across all functional subsets. However, this assumption is incorrect in practice. Different antigens usually induce very different functional profiles, and many of the possible functional cell subsets are not expected to be associated with antigen specificity. MIMOSA cannot jointly model all subsets to identify distinct antigenspecific responses, and this is particularly important to do as the number of definable subsets grows exponentially with the number of cytokines analyzed. As an example, an ICS experiment measuring seven functions can define 128 Boolean cell subsets, but only a fraction of those are expected to signify responsiveness to a specific antigen. A possible solution would be to model and test each subset separately, but this is computationally intensive, ignores the dependence between subsets and leads to multiple testing problems. In the interest of decreasing the number of variables while taking into account the degree of functionality, Larsen et al.^{15} introduced a polyfunctionality index (PI) that summarizes the polyfunctional profile into a single number. However, the PI uses empirical proportions, which are known to be extremely noisy when cell counts are small, and combines information from all cell subsets, including nonspecific ones, thus masking real signal. Indeed, many functional subsets that are not antigenspecific can have substantial background, that is, they can be present in substantial numbers in nonstimulated cells, and their inclusion makes the defined index more variable. In addition, by weighting the subsets by their magnitude, lowmagnitude (but potentially important) subsets will be downweighted significantly. Thus, the PI falls short of describing the true breadth of polyfunctionality, limiting its clinical utility. An ideal framework for the analysis of T cells should identify and quantify changes in possibly rare, antigenspecific cell subsets and permit the definition of different qualities of a polyfunctional response, such as through summary statistics that can be correlated with outcomes of interest.
To address these needs, we have developed COMPASS, which uses a Bayesian hierarchical mixture model to identify antigenspecific changes across all observable Tcell subsets simultaneously, enabling the definition of subject and cohortlevel antigenspecific Tcell profiles that can be summarized and correlated with outcomes of interest. This method jointly models all cell subsets, regularizes small cell counts and allows information sharing across all subjects through the calculation of subsetspecific posterior probabilities that can be used to automatically detect and quantify antigenspecific subsets. COMPASS uses a computationally efficient Markov Chain Monte Carlo algorithm to explore the space of all possible functional Tcell subsets and compute the subsetspecific posterior probabilities. These probabilities naturally account for multiple testing^{16} and permit the derivation of a falsediscoveryrate estimate through a direct posterior probability approach^{17}. COMPASS provides two scores—functionality and polyfunctionality—that can be correlated with any clinical outcome of interest. These scores summarize, as a single number for each subject, the posterior probabilities of antigenspecific response across cell subsets. We apply this method to three ICS data sets—the RV144 HIV vaccine casecontrol study^{18,19}, a crosssectional study of South African adolescents infected with Mycobacterium tuberculosis (MTB)^{20} and a phase 1b trial from the HIV Vaccine Trials Network^{21}—and demonstrate the ability of COMPASS to identify specific polyfunctional responses that are correlated with clinical outcomes of interest.
Results
COMPASS enables unbiased quantification of all cell subsets
We reanalyzed the ICS data generated through the RV144 HIV vaccine casecontrol study with 92TH023Env peptide pool ex vivo stimulation, as described in Online Methods and in a prior publication^{18}. Expression of a set of six functions (interferon (IFN)γ, TNFα, interleukin (IL)2, IL4, IL17 and CD40L) was measured in CD4^{+} T cells by ICS (n = 226 vaccine and n = 36 placebo recipients). We used COMPASS to define 64 functional cell subsets from the Boolean combinations of the individual cytokines expressed in each cell. Only 24 of these had nonnegligible (over five cells in more than two subjects) cell counts and were considered for analysis. Arguably, if there are only two subjects with no more than five cells in any cell subset then it would be difficult to detect antigenspecific responses, and we feel that these values should be reasonable for most ICS data sets. In addition, changing these cutoffs to be less stringent had no impact on the significance of the results or conclusions presented here and the values were selected purely for computational convenience to reduce the time it takes to fit a COMPASS model to cell subsets that contain no signal. A heatmap of posterior probabilities of antigenspecific responses from COMPASS for these Tcell subsets shows that, whereas some antigenspecific subsets were universally present in almost all individuals vaccinated (e.g., IL2, CD40L), others exhibited heterogeneity across subjects (e.g., the four and fivefunction subsets) (Fig. 2a). These findings support the need for unbiased polyfunctionality analyses that can provide insights on the possible association between polyfunctional antigenspecific T cells and vaccine efficacy.
ICS assays are often used to identify vaccine responders based on the magnitude of antigenspecific Tcell responses, although current approaches tend to be univariate. COMPASS can identify vaccine responders by computing response probabilities for each subject. We compared COMPASS with the two standard approaches for designating positivity—Fisher's exact test^{12} and MIMOSA^{14}—using the primary endpoint in the original RV144 analysis^{18} as well as the multivariate MIMOSA and the PI. A receiver operating characteristic (ROC) analysis shows that COMPASS substantially increased sensitivity and specificity when discriminating between vaccine and placebo recipients compared to all other approaches (Supplementary Fig. 1).
In addition to subjectlevel response probabilities, COMPASS defines two scores that summarize a subject's entire antigenspecific polyfunctional profile into a single numerical value (Fig. 2b). The functionality score (FS) is defined as the proportion of antigenspecific subsets detected among all possible ones. The polyfunctionality score (PFS) is similar, but it weighs the different subsets by their degree of functionality, naturally favoring subsets with higher degrees of functionality, motivated by the observations that a higherdegree function has been correlated with good outcomes in certain vaccine studies^{5,6}. The FS was well correlated with the number of functions, when measured by multiplex bead array on the same set of samples from the RV144 casecontrol study (Fig. 3a; correlation ρ = 0.69, P < 2.2 × 10^{−16}, 95% confidence interval (CI) = (0.62, 0.75)), suggesting that COMPASS detects true antigenspecific responses in Tcell subsets. Out of the 12 cytokines measured by a multiplex bead array, only four were also measured by ICS (IFNγ, TNFα, IL2, IL4). Subjects with a high FS also expressed many other cytokines not measured in the ICS assay. We also tried to restrict the analysis to the four most common cytokines, but it did not improve the correlation; in fact, the correlation decreased slightly to 0.65. The FS was also significantly correlated with HIV Envspecific antibody binding in plasma (Fig. 3b; correlation ρ = 0.41, P = 9.87 × 10^{−11}, 95% CI = (0.30, 0.52)). The placebo recipients had a much smaller FS and PFS compared to those of vaccine recipients, and the PI was noisier than the FS and PFS (Supplementary Fig. 2). All together, these data indicate that our FS and PFS were good at capturing and summarizing a subject's response to vaccination. These results were also confirmed with the RV144 pilot data, used to inform the selection of assays to be used for the casecontrol study, thus providing validation from an independent set of subjects (Supplementary Figs. 3,4,5,6). The FS and PFS for RV144, broken down by infection status (Fig. 2b), varied greatly by subject, and, on average, noninfected subjects had higher scores than infected individuals; these effects were stronger for PFS (Wilcoxon test P = 0.03 (FS), P = 0.01 (PFS)).
COMPASS identifies correlates of infection risk in RV144
Among the 17 different types of immune assays and their 152 component variables used to assess correlates of infection risk in RV144, 6 assays (including ICS CD4^{+} Tcell responses) were chosen as primary variables for having optimal statistical power when adjusting for multiple comparisons^{18}. The major findings in a multivariate model including all six primary variables were that gp70V1V2specific plasma IgG inversely correlated with infection rate (odds ratio 0.57, P = 0.02, q = 0.08), and Envspecific plasma IgA was a direct correlate of infection (odds ratio 1.54, P = 0.03, q = 0.08). No CD4^{+} Tcell cellular correlates were identified in the primary analysis in the original study^{18}.
In contrast, using the same method as in the original correlate analysis^{18} but including the FS and PFS rather than the primary CD4^{+} Tcell endpoint variable, the PFS was inversely correlated with infection (P = 0.005, q = 0.05, Table 1), as was the FS but to a lesser extent (P = 0.01, q = 0.06, Table 1). This correlation was also supported by the fact that the number of functions (secreted cytokines) as measured by multiplex bead array was also inversely correlated with infection (P = 0.013, Supplementary Table 1). However, the multiplex bead array data do not permit polyfunctionality analysis as it is an assay performed on bulk cells, and as such, it cannot measure the coexpression of multiple cytokines at the singlecell level. Our PFS suggests that polyfunctional antigenspecific CD4^{+} T cells might have played a role in vaccine efficacy.
To identify the specific cell subsets that contributed most to this correlation, we evaluated each subset of interest as a correlate for infection risk (Fig. 2a). Two antigenspecific Tcell subsets were significantly correlated with infection (q ≤ 0.1, Table 1). It can be seen that the polyfunctional subset expressing CD40L, IL2, IL4, IFNγ and TNFα shows the most significance (odds ratio (OR) = 0.58, P = 0.006, q = 0.05) followed by a threefunction subset (CD40L, IL2 and IL4; OR = 0.62, P = 0.01, q = 0.06). The fivefunction subset was as significant as the previously reported gp70V1V2 correlate. Notably, these subsets were identified in an unbiased way rather than by limiting our analysis to very specific subsets based on expected biological relevance as was done in the original study^{18}. Both subsets included IL4 and CD40L, a cytokine and functional marker, respectively, both important for the CD4^{+} T cell–B cell interaction. Thus, these particular CD4^{+} Tcells subsets may contribute the Tcell help necessary for the antibody production detected as a correlate in the primary analysis.
All subsets, including nonsignificant ones, are presented in Supplementary Table 2. In addition, we have also assessed the magnitude of response for all functional subsets (Supplementary Table 3) as well as the PI and MIMOSA scores (Supplementary Table 1) as individual correlates of risk, and none of these variables were significant. The superiority of COMPASS was also supported by an ROC analysis, using the predicted infection probabilities from a logistic regression model (Supplementary Fig. 7).
Mtb infection induces highly polyfunctional responses
We used COMPASS to investigate polyfunctionality in CD4^{+} T cells obtained from a South African study of 18 MTBuninfected and 22 MTBinfected subjects. Subjects were classified as MTBinfected, as described in the Online Methods section and in the original study^{20}. T cells were stimulated with MTBspecific^{22} and MTBnonspecific peptide pools, and a set of seven cytokines and functional markers was measured by ICS, with 22 Boolean combinations defined for COMPASS analysis.
As expected, CD4^{+} Tcell responses to MTBspecific proteins were largely absent from MTBuninfected subjects (Fig. 4). Ex vivo stimulation with MTBspecific antigens induced strong antigenspecific responses spread across polyfunctional subsets of degree three and five in MTBinfected subjects. In contrast, MTBuninfected subjects had weak responses in these subsets. The appearance of the fivefunction subset (IL17α, CD40L, IFNγ, TNFα, IL2) in some MTBuninfected persons may represent a more sensitive marker of infection or may simply be a falsepositive result. Further study will be required to distinguish among these possibilities. In contrast, the response to nonspecific mycobacterial peptides was concentrated over one subset expressing IFNγ, IL2, TNFα and CD40L, and was independent of subject's MTB status. This response to nonspecific mycobacterial proteins was likely due to Bacillus CalmetteGuérin (BCG) vaccination or environmental exposure to nontuberculous mycobacteria.
COMPASS detected MTBspecific responses for some cell subsets that did not include IFNγ, suggesting that such analysis may add value to standard Quantiferon test (QFT) analyses in identifying patients with MTB infection (Fig. 4a). Given this observation, we assessed whether COMPASS could be used to classify MTBuninfected and MTBinfected subsets and how this would compare to a classification based on marginal IFNγ response alone, essentially a surrogate for the QFT. Using COMPASS's polyfunctional profile to classify subjects as MTBinfected versus uninfected provided increased sensitivity and specificity compared to using IFNγ alone or the PI, as shown by a ROC analysis comparing COMPASS to various summary statistics based on the IFNγ response (Supplementary Fig. 8). The lack of total concordance between the QFT and ICS IFNγ response may be explained by differences between fresh whole blood and frozen peripheral blood mononuclear cells (PBMCs) used for the two assays, the lack of TB7.7 in the ICS assay and measurement of IFNγ expression (ICS) versus secretion (enzymelinked immunosorbent assay, ELISA).
COMPASS identifies functional differences between vaccine regimens
As a third example, we applied COMPASS to ICS data from a clinical trial of HIV vaccine candidates (HVTN078)^{23} to determine if a polyfunctionality analysis could unveil differences missed in the primary analysis. The combination of heterologous vectors has proven to be a good strategy to increase immune responses to vaccination^{24}, which is particularly relevant for HIV where there is not yet an effective, licensed vaccine. Bart et al.^{21} showed that priming with rAd5 followed by a boost with NYVAC resulted in an increased percentage of antigenspecific T cells producing IL2 and/or IFNγ compared to NYVAC followed by rAd5, but that higher doses of the rAd5 primer did not lead to any increased response. A heatmap of the posterior probabilities for all CD4^{+} Tcell Boolean subsets considered (Fig. 5a) supports the findings of Bart et al.^{21} and also shows that increasing dosage of rAd5 led to a decrease in overall response, which is nicely summarized by the FS and PFS (Fig. 5b). Although two of the experimental groups T3 and T4, have comparable FS and PFS, it is notable that very few subjects in T4 produced antigenspecific T cells coexpressing IL4, IFNγ, TNFα and CD40L compared to T3 (Fig. 5a). This was confirmed when the probability of response for this subset was plotted separately (Supplementary Fig. 9). This suggests that a difference in dosage led to substantial qualitative differences in immune responses after vaccination (P = 0.004 for Fisher's exact test on proportions of responders between T3 and T4).
Discussion
These three examples highlight the ability of COMPASS to reveal differences in the quality of an immune response that were not evident using standard approaches. It remains uncertain as to whether specific cellular subsets of unique polyfunctional profiles have clinical significance, for example, in terms of mediating protection from disease. The RV144 data set provided an opportunity to investigate the association between subset detection and a clinical endpoint, HIV infection. Indeed, two vaccineinduced polyfunctional CD4^{+} Tcell subsets not identified in the prior analyses^{18,19} were shown to be associated with decreased risk of HIV infection. Although the original cellular endpoint in RV144 was not designed to detect polyfunctional responses, current tools used to assess polyfunctionality are inadequate and miss this association. In the MTB study, COMPASS was used to detect associations between cellular subsets and the presence or absence of MTB infection. In this case, the diagnostic methods for determining infection are not definitive and our approach has the potential for augmenting diagnostic methods. The third example, applied in the context of a phase I clinical trial, did not have a clinical endpoint, but the difference in one fourfunction subset between two identical vaccine regimens that differed only in dose raises questions about the functional significance of this subset, or at least, the potential of this subset as a future correlate of vaccine efficacy.
COMPASS analysis of the tuberculosis data set revealed two findings that would likely have been missed using standard methods of analyzing multiparameter FACS (fluorescenceactivated cell sorting) data. First, we demonstrated the presence of a polyfunctional CD4^{+} Tcell subset recognizing MTBspecific proteins that did not express IFNγ, yet was preferentially detected among MTBinfected subjects. IFNγ release assays are the clinical standard for diagnosis and are still imperfect^{25}, so this observation may have implications for the design of improved diagnostics. Second, we demonstrated the presence of a Tcell subset that simultaneously produced IL17a, IFNγ, CD40L, IL2 and TNFα in response to MTBspecific proteins among MTBinfected subjects. To our knowledge, this has not been previously reported, and it is not clear if such highlypolyfunctional T cells are important for protection from tuberculosis, even though the analysis of RV144 data presented here would suggest this is possible. The availability of COMPASS will make it easier to analyze phase 2 studies of TB vaccines currently underway for the association of these subsets with clinical outcomes. Finally, only a small proportion of MTBinfected individuals will eventually develop active tuberculosis, so the stratification of MTBinfected subjects into different Tcell response categories could be used to predict which individuals are at highest risk of progression to active TB disease.
The FS and PFS introduced in this paper provide unique Tcell response summaries that can be used to quantify a subject's immune response and be used as a biomarker to be correlated with a given outcome. Although the score definitions are related, and thus the scores are expected to be correlated, we believe that the two scores are complementary and provide more information than one score alone. For example, even though the correlation between the two scores in the RV144 casecontrol study is 0.95, we have shown that whereas the FS was better correlated with other cellular and antibody functions, as measured by multiplex bead array assay, the PFS provided a better correlate of protection. In addition, multivariate response profiles, based on the posterior probabilities returned by COMPASS, were helpful in singling out specific subsets that were associated with specific treatment groups or clinical variables. Other published studies have reported similar findings^{26,27,28}, reinforcing the need for unbiased, multivariate analyses. It should be noted, however, that as COMPASS analyzes and reports probabilities for all functional subsets, the results could be prioritized and/or summarized in reference to cytokinebased functionalities (e.g., Th1, Th2, Th17).
These findings reinforce the idea that the quality rather than the magnitude of Tcell responses is more important for determining the outcome of infection or response to vaccination^{29}. We think that for the purpose of summarizing the functionality or polyfunctionality over all subsets, it is preferable to define a variable that does not include the magnitude of the response; otherwise, the largemagnitude subsets would mask the lowmagnitude ones. This is particularly important as the magnitude of highdegree (i.e., polyfunctional) subsets is typically smaller. This being said, our FS and PFS do take into account the magnitude of response up to a certain point, after which the subsets are treated equally (when the model judges that there is a significant increase upon stimulation and we are certain of the specificity of the response). Also, because our approach is based on a model, the relationship between the magnitude and the probability of antigen specificity takes into account the uncertainty in the observed cell counts (Supplementary Fig. 10), such that a cell subset with greater background will have lower response probabilities, even at comparable magnitude. Although we could use the actual proportion of antigenspecific cells in our calculation, we have found that it did not add any predictive value (Supplementary Figs. 11 and 12 and Supplementary Table 4). This could be explained by the fact that either the quality of the antigenspecific response has a more important effect on clinical outcome than the quantity of antigenspecific T cells, or that the estimates of proportions are too noisy to be meaningful when the overall proportion of antigenspecific cells is small, which is clearly the case for HIV and TB. However, if one wants to look at the magnitude, we think that it might be better quantified at the subset level, which can easily be done with COMPASS as it models all subsets jointly.
The development of statistically sound methods for the characterization of antigenspecific T cells from singlecell assays is becoming increasingly important, particularly as singlecell technologies improve and the number of functionally distinct subsets that can be defined from an experiment increases exponentially. As we have shown, standard approaches that ignore the multivariate nature of ICS data may miss rare but important signals. This problem will be exacerbated as new highdimensional, singlecell technologies such as CyTOF, multiplexedqPCR and RNAseq become more widely used. Although our approach was developed primarily for flow cytometry–based ICS assays, it is directly applicable to CyTOFbased ICS experiments and should be generalizable to other assay technologies including multiplexedqPCR and singlecell RNAseq.
Methods
Data for these studies derive from three clinical protocols that were approved by the relevant institutional review boards. All study participants provided written informed consent for immune response exploratory analyses.
Online Methods
RV144 casecontrol data set.
HIVnegative healthy volunteers enrolled in the RV144 trial (https://www.clinicaltrials.gov/ registration number NCT00223080) and were vaccinated at weeks 0, 4, 12 and 24; their immune responses at week 26 were evaluated as immune correlates of infection risk through a casecontrol analysis^{18,19}. A total of 246 vaccinated subjects were used for this analysis: 41 subjects who acquired HIV1 infection (cases) after week 26 and 205 frequencymatched subjects who did not become infected over the followup period (controls). A total of 17 types of immune assays were run on the casecontrol samples. In addition, these assays were also performed on random samples from 40 placebo recipients (20 cases and 20 controls). The analysis presented here focused on assessing polyfunctional HIV1 envelope protein (Env)specific Tcell response using ICS, one of the 17 available assays. As a means to validate our analysis and explore relationships between cellular and antibody functions, we also correlated our ICS responses with two other assays: a multiplex cytokine bead array measuring antigenspecific cytokine secretion in PBMCs, and a binding antibody multiplex assay measuring IgG binding to HIV1 envelope proteins. After removing missing data due to assay failure, we ended up with 226 vaccinated subjects (38 infected, 188 noninfected) with complete data across the three assay types.
ICS.
In this data set, a set of 6 functions—TNFα, IFNγ, IL4, IL2, CD40L and IL17a—were measured at the singlecell level in CD4^{+} T cells in the presence and absence of stimulation with a peptide pool matching one of the HIV1 envelope proteins contained in the vaccine (92TH023). Cell level data were extracted from raw data files using the analysis described in Haynes et al. (2012)^{18}, leading to the definition of count data for the 2^{6} = 64 theoretical Boolean subsets for each subject and stimulation condition. However, the actual number of observed (i.e., nonempty) cell subsets is 59, and we further filtered out subsets that had fewer than six cells in fewer than three subjects, reducing that number to 24.
Multiplex cytokine bead array.
A set of 12 cytokines (IFNγ, IL4, IL2, IL5, TNFα, IL10, TNFβ, IL13, MIP1β, GMCSF, IL3, IL9) were measured using multiplex bead array technology. For each subject and cytokine, the response is defined as the difference in log concentration between the stimulated (92TH023Env) and unstimulated samples. For each subject, individual cytokines were called positive/negative using the thresholds defined in the original study^{18}.
IgG total binding.
Binding IgG antibodies to the envelope (Env) protein of the HIV virus were measured using a binding antibody multiplex assay. Here, we used the IgG total binding response to Env defined as the mean binding IgG to multiple Env proteins^{18}.
Pilot data.
As part of the RV144 casecontrol study, the 17 assay types used in the casecontrol study were selected from 32 pilot assay types on the basis of reproducibility, ability to detect postvaccine responses, and uniqueness of responses detected, from which six primary assay variables were selected for the correlate analysis. ICS and multiplex bead array were two of the assays types that were evaluated during the pilot phase. All samples used in the pilot phase were from noninfected subjects. For ICS, 36 placebo and 119 vaccine recipient (60 before and 59 after vaccination) samples were used. For multiplex bead array data, 30 placebo and 98 vaccine recipient (57 before and 41 after vaccination) samples were used.
TB data set.
We obtained cryopreserved PBMC from an epidemiologic study of South African adolescents who were screened for the presence of latent tuberculosis infection (LTBI) using tuberculin skin testing and QuantiferonTB GOLD in tube testing of whole blood^{20}. This data set includes 40 subjects, 22 MTBinfected and 18 MTBuninfected. Subjects were classified as MTBinfected using both TB skin testing and the Quantiferon test intube gold (QTFgold) that measures ILγ release in wholeblood after stimulation with ESAT6, CFP10 and TB7.7 peptides^{25}. PBMCs were plated at a density of 2 × 10^{5} per well and stimulated for 6 h with either DMSO or peptide pools consisting of 15 mers overlapping by 12 peptides for the following mycobacterial proteins: ESAT6, CFP10, TB10.4, Ag85A and Ag85B at a final concentration of 1 μg/ml. Cells were stained using a published panel in which we replaced MIP1β and CD107a with IL17a Alexa 700 and IL22 PE Cy7 (refs. 12,30). Analysis of CD3^{+}CD4^{+} events was performed in FlowJo (TreeStar Inc.) after first gating on single cell events, CD14^{−} events, live cells and lymphocytes. For the purposes of analysis, we pooled counts obtained after stimulation with ESAT6 and CFP10 and defined this as “MTBspecific” because these proteins are known to be absent in Bacillus CalmetteGuérin (BCG) and many environmental mycobacteria. By extension, counts obtained after stimulation with Ag85A, Ag85B and TB10.4 were pooled and defined as 'MTBnonspecific' because these proteins are present in M. tuberculosis as well as BCG and many environmental mycobacteria. Seven functions were measured at the singlecell level in CD4^{+} T cells: TNFα, ILγ, IL4, IL2, CD40L, IL17a and IL22, leading to 128 theoretical subsets. However, the actual number of observed cell subsets was 79, and using the filtering criteria described above we reduced that number to 22 (MTBspecific) and 19 (MTBnonspecific).
HVTN078.
HVTN078 (refs. 21,23) is a randomized, doubleblind phase 1b clinical trial (ClinicalTrials.gov registration number NCT00961883) to evaluate the safety and immunogenicity of heterologous primer/boost vaccine regimens (NYVACB/rAd5 versus rAd5/NYVACB) in healthy, HIV1 uninfected, Ad5 seronegative adult participants. Eighty participants were enrolled into one of four groups receiving different combinations of NYVACB (New York Vaccinia (NYVAC) vector containing HIV1BX08 gp120 and HIV1IIIB gagpolnef at a dose of 1 × 10^{7} PFU (plaqueforming unit)) and rAd5 (HIV1 recombinant adenoviral serotype 5 (rAd5) vector vaccine VRCHIVADV01400VP ((HIV1HXB2/NL43 GagPol fusion; HIV192RW020, HIV1HXB2/Bal and HIV197ZA012 Env), at three increasing doses (1 × 10^{8}, 1 × 10^{9}, 1 × 10^{10} PFU)). We refer to the four different groups as T1, T2, T3 and T4. In the T1 group, NYVAC was the prime with rAd5 as the boost, whereas subjects in T2T4 received rAd5 as the prime and NYVAC as the boost with the three increasing doses of the prime. Here we used a subset of the ICS data generated through the trial measuring seven functions: ILγ, IL2, IL4, TNFα, MIP1β, CD107a and CD40L in CD4^{+} T cells in the presence and absence of stimulation with HIV1 peptides, leading to 128 theoretical subsets in 71 subjects (T1:29, T2:15, T3:13, T4:14). However, the actual number of observed cell subsets is 107, and using the filtering criteria described above we reduced that number to 26 subsets.
Statistical framework for modeling count data.
Without loss of generality, we assume cell counts are obtained from I subjects under two conditions: (antigen)stimulated and unstimulated as depicted in Figure 1. Let M denote the number of markers measured, then in theory, there are K_{M} = 2^{M} possible Boolean combinations defining functional cell subsets, depending on whether the marker is expressed or not. We let K (K ≤ K_{M}) denote the actual number of cell subsets considered for statistical analysis (i.e., after filtering empty and very sparse cell subsets). The observed counts for the K cell subsets in the stimulated and unstimulated samples are represented by n_{sik} and n_{uik}, k = 1,..., K, i = 1, ..., I, respectively, or represented as vectors we have n_{si} = (n_{si1},...,n_{siK})′ and n_{ui} = (n_{ui1},...,n_{uiK})′. Then N_{si} = Σ_{k} n_{sik} and N_{ui} = Σ_{k} n_{uik} are the total number of cells for subject i in the stimulated and unstimulated samples, respectively (Fig. 1). Without loss of generality, we order the cell subsets such that the K^{th} category represents the subset where no cytokines are expressed, that is, the degree of functionality is zero.
For a given subject i, we jointly modeled the cell counts under the two conditions using multinomial distributions: (n_{si}  p_{si}) ∼MN(N_{si}, p_{si}) and (n_{ui}  p_{ui}) ∼MN(N_{ui}, p_{ui}), where p_{si} and p_{ui} are the unknown proportions for the stimulated and unstimulated paired samples, respectively. To detect a responding subject, as well as antigenspecific subsets within a subject, we considered two competing hypothesis: H_{0}: p_{ui} = p_{si}, and H_{a}: ∃k∈{1, ..., K − 1} such that p_{sik} > p_{uik}. Under the null hypothesis H_{0}, there is no difference in the proportion of cytokine producing cells between the stimulated and unstimulated samples, and thus the proportion vector parameter is shared across the two multinomial models. Under the alternative hypothesis H_{a}, some subsets show an increase in their proportions. We define the cell subsets that express at least one function and are different under H_{a} as antigenspecific subsets as the change in proportion is induced by the antigen stimulation. The K^{th} null category is not considered here, as a change there would only reflect a change in some other category as the proportion vector has to sum to one. This framework allows each subject to be responding in none, some or all of the subsets, and jointly models all the subjects and subsets to allow information sharing to improve the power in detecting rare signals. The ultimate objective is to automatically identify antigenspecific cell subsets for each subject.
Statistical model for detecting antigenspecific polyfunctional Tcell subsets.
To allow the automatic identification of antigenspecific cell subsets for each subject, we introduced a binary indicator, γ_{ik} associated with each subject i and each category k, such that if the category is antigenspecific, γ_{ik} = 1, otherwise γ_{ik} = 0. In other words, when γ_{ik} = 1, the (unknown) cell proportions (p_{sik}, p_{uik}) for the stimulated and unstimulated samples are different, otherwise they are equal. The distribution of cell proportions can easily be specified conditional on the latent indicators, which is shown in Supplementary Methods, “Priors.” Our implementation of the COMPASS model uses an optimized Markov chain Monte Carlo (MCMC) algorithm allowing full exploration of the joint posterior distribution (Supplementary Methods, “Posterior sampling”). Then statistical inference about responding subsets across subjects, subsets or polyfunctionality degree would be based on the posterior summaries of the latent indicators, such as using the posterior mean, and/or FDRthresholded posterior probabilities. Our approach uses a Bayesian variableselection prior, with a subset specific—but common across subjects—prior weight shared and estimated across all subjects. As a result, the inference automatically takes into consideration multiple comparisons^{16} across subjects.
Subjectlevel response probabilities.
ICS assays are often used for determining vaccine responders based on the magnitude of antigenspecific Tcell responses. Current approaches for positivity tend to be univariate; a single cellsubset needs to be defined as an endpoint (e.g., cells expressing IL2 and/or ILγ). Because COMPASS uses a Bayesian approach to jointly model all cell subsets, any posterior summary of interest is readily available. COMPASS can classify vaccine responders or MTBinfected individuals by computing subjectlevel response probabilities, defined as the probability that at least two (disjoint) cell subsets exhibit an antigenspecific response in that subject. The rationale is that antigen stimulation in subjects with antigenspecific cells should induce changes in a variety of cell subsets, whereas nonspecific responses are expected to be sporadic and noisy. Note that even though our subjectlevel response probabilities are computed based on two or more responding cell subsets, we still define the alternative hypothesis H_{a} (described above) as at least one difference, as COMPASS can return posterior probabilities for each subset, that is, we really consider all possible alternative models.
Polyfunctionality and functionality score definition.
We introduce the functionality score (FS) and polyfunctionality score (PFS) to summarize the response across the different cell subsets for each subject. The benefits of a singlenumber summary of subjectlevel response have been outlined elsewhere^{10}. Primarily, it greatly facilitates statistical analysis, comparisons across treatment groups and correlation with outcome measures. We define the FS as the posterior mean of the average number of antigenspecific cell subsets among all measured subsets, irrespective of the degree of functionality:
where is the posterior mean of γ_{ik} estimated using MCMC (Supplementary Methods, “Posterior sampling”). The FS ranges from zero to one and measures the proportion of distinct cell subsets that are expressed for a given subject among all possible subsets.
The PFS is the mean of the posterior probabilities, weighted by the degree of functionality of the corresponding subset, and normalized by the total number of possible cell subsets that could be observed, given the number of markers considered:
where d(k) is the degree of functionality for cell subset k. The PFS ranges from zero to one. The normalization by the theoretical number of possible cell subsets facilitates the comparison of FS and PFS across different data sets, provided they utilize the same markers. It should be noted that the FS and PFS—based on the posterior probabilities—do not directly take into account the frequencies (i.e., magnitudes) of antigenspecific cells. The posterior probability summarizes all the evidence that a cell subset is antigenspecific by comparing the proportion of cytokinepositive cells in the stimulated sample to the corresponding proportion in the control sample. Once there is enough evidence that the subset is indeed antigenspecific (i.e., the probability is one), the actual proportion of antigenspecific cells, which can be estimated as the difference in proportion between the stimulated and unstimulated samples, is no longer relevant. In other words, two cell subsets with a probability of one will be treated equally in our approach, except for the degree of functionality that factors into the PFS even though they may have differing magnitude. Although, we could use the actual proportion of antigenspecific cells in our calculation (Supplementary Methods, “Posterior summary based on magnitude of response”), we have found that it did not improve our analysis, and that in fact it decreased some of the correlation reported here (Supplementary Figs. 11 and 12 and Supplementary Table 4).
The PFS will assign a higher score to subjects with antigenspecific cell subsets of higher degree, whereas the FS will assign a higher score to subjects that exhibit antigen specificity in more cell subsets irrespective of their degree of functionality. The two scores are complementary in the sense that the PFS emphasize the quality (i.e., the polyfunctionality) of antigenspecific cell subsets, whereas, the FS looks at the quantity of antigenspecific cell subsets. As an example, if we are considering six markers, two subjects may have the same PFS if one has a single, degreesix, antigenspecific cell subset (PFS = 0.286), whereas the other has antigen specificity in all subsets with degree < 4 (PFS = 0.286). The FS would distinguish between these, assigning a score of 0.016 to the first subject and a score of 0.651 to the latter.
Polyfunctionality index.
The polyfunctionality index was calculated as described in Larsen et al.^{15} using both uncorrected and backgroundcorrected cell frequencies with the tuning parameter q set to 1, which is the equivalent of the PFS score where all subsets are weighted by their degree of functionality. Basically, for the uncorrected PI score we simply used the stimulated frequencies, p_{sik}, whereas for the corrected PI score, we used the backgroundcorrected frequencies defined as max(p_{sik}p_{uik},0). The two scores are referred to as “PI” and “PI corrected” in Supplementary Figures 1,2,3,4,5,6,7,8,9,10,11,12. We have also tried other tuning parameter values including q = 1.2 and q = 2, and the results did not improve.
Correlate of risk analysis.
All immune variables identified here were assessed as correlates of infection risk (CoR) by using the statistical methods specified in the original correlates study^{18}. Briefly, for each immune biomarker, logistic regression accounting for the sampling design was used to estimate the odds ratio (OR) of infection, controlling for gender and baseline behavioral risk, and IgA levels.
RV144 primary analysis results.
Six assays from the 17 different types of immune assays in the original RV144 study were selected as primary endpoints for optimal statistical power when adjusting for multiple comparisons. These selected primary variables included Envspecific plasma IgA, Envspecific plasma IgG binding avidity, gp70V1V2specific plasma IgG, neutralizing antibodies, ADCC (antibody dependent cellmediated cytotoxicity) and Envspecific CD4^{+} Tcells. The major findings in a multivariate model including all six primary variables were that gp70V1V2–specific plasma IgG inversely correlated with infection rate (odds ratio 0.57, P = 0.03, q = 0.08), and Envspecific plasma IgA was a direct correlate of infection (odds ratio 1.54, P = 0.03, q = 0.08).
Accession codes.
COMPASS is available as an R package (http://github.com/RGLab/COMPASS) and provides an interactive, webbased interface for visualizing all data and results. All results presented in this paper can be visualized through a webtool available at: http://rglab.github.io/COMPASS/. All data are available here: https://zenodo.org/record/17500#.VVEvB9NViko.
References
 1.
De Rosa, S.C. et al. Vaccination in humans generates broad Tcell cytokine responses. J. Immunol. 173, 5372–5380 (2004).
 2.
Attig, S. et al. Simultaneous infiltration of polyfunctional effector and suppressor T cells into renal cell carcinomas. Cancer Res. 69, 8412–8419 (2009).
 3.
RodrigueGervais, I.G. et al. Dendritic cell inhibition is connected to exhaustion of CD8+ T cell polyfunctionality during chronic hepatitis C virus infection. J. Immunol. 184, 3134–3144 (2010).
 4.
Ciuffreda, D. et al. Polyfunctional HCVspecific Tcell responses are associated with effective control of HCV replication. Eur. J. Immunol. 38, 2665–2677 (2008).
 5.
Precopio, M.L. et al. Immunization with vaccinia virus induces polyfunctional and phenotypically distinctive CD8(+) T cell responses. J. Exp. Med. 204, 1405–1416 (2007).
 6.
Darrah, P.A. et al. Multifunctional TH1 cells define a correlate of vaccinemediated protection against Leishmania major. Nat. Med. 13, 843–850 (2007).
 7.
Roederer, M., Nozzi, J.L. & Nason, M.X. SPICE: Exploration and analysis of postcytometric complex multivariate datasets. Cytometry A 79, 167–174 (2011).
 8.
Aghaeepour, N. et al. Critical assessment of automated flow cytometry data analysis techniques. Nat. Methods 10, 228–238 (2013).
 9.
Lo, K., Brinkman, R.R. & Gottardo, R. Automated gating of flow cytometry data via robust modelbased clustering. Cytometry A 73, 321–332 (2008).
 10.
Trigona, W.L. et al. Intracellular staining for HIVspecific IFNgamma production: statistical analyses establish reproducibility and criteria for distinguishing positive responses. J. Interferon Cytokine Res. 23, 369–377 (2003).
 11.
Nason, M. Patterns of immune response to a vaccine or virus as measured by intracellular cytokine staining in flow cytometry: hypothesis generation and comparison of groups. J. Biopharm. Stat. 16, 483–498 (2006).
 12.
Horton, H. et al. Optimization and validation of an 8color intracellular cytokine staining (ICS) assay to quantify antigenspecific T cells induced by vaccination. J. Immunol. Methods 323, 39–54 (2007).
 13.
Proschan, M.A. & Nason, M. Conditioning in 2 × 2 tables. Biometrics 65, 316–322 (2009).
 14.
Finak, G. et al. Mixture models for singlecell assays with applications to vaccine studies. Biostatistics 15, 87–101 (2014).
 15.
Larsen, M. et al. Evaluating cellular polyfunctionality with a novel polyfunctionality index. PLoS ONE 7, e42403 (2012).
 16.
Scott, J.G. & Berger, J.O. Bayes and empiricalBayes multiplicity adjustment in the variableselection problem. Ann. Stat. 38, 2587–2619 (2010).
 17.
Newton, M.A., Noueiry, A., Sarkar, D. & Ahlquist, P. Detecting differential gene expression with a semiparametric hierarchical mixture method. Biostatistics 5, 155–176 (2004).
 18.
Haynes, BF. et al. Immunecorrelates analysis of an HIV1 vaccine efficacy trial. N. Engl. J. Med. 366, 1275–1286 (2012).
 19.
RerksNgarm, S. et al. Vaccination with ALVAC and AIDSVAX to prevent HIV1 infection in Thailand. N. Engl. J. Med. 361, 2209–2220 (2009).
 20.
Mahomed, H. et al. Predictive factors for latent tuberculosis infection among adolescents in a highburden area in South Africa. Int. J. Tuberc. Lung Dis. 15, 331–336 (2011).
 21.
Bart, P. et al. rAd5 prime/NYVACB boost regimen is superior to NYVACB prime/rAd5 boost regimen for both response rates and magnitude of CD4 and CD8 Tcell responses. Retrovirology 9 (suppl. 2), O72 (2012).
 22.
Reddy, T.B.K. et al. TB database: an integrated platform for tuberculosis research. Nucleic Acids Res. 37, D499–D508 (2009).
 23.
Bart, P.A. et al. HIVspecific humoral responses benefit from stronger prime in phase Ib clinical trial. J. Clin. Invest. 124, 4843–4856 (2014).
 24.
McElrath, M.J. & Haynes, B.F. Induction of immunity to human immunodeficiency virus type1 by vaccination. Immunity 33, 542–554 (2010).
 25.
Pai, M. et al. Gamma interferon release assays for detection of mycobacterium tuberculosis infection. Clin. Microbiol. Rev. 27, 3–20 (2014).
 26.
Ewer, K.J. et al. Protective CD8+ Tcell immunity to human malaria induced by chimpanzee adenovirusMVA immunisation. Nat. Commun. 4, 2836 (2013).
 27.
Overstreet, M.G., Cockburn, I.A., Chen, Y.C. & Zavala, F. Protective CD8 T cells against Plasmodium liver stages: immunobiology of an “unnatural” immune response. Immunol. Rev. 225, 272–283 (2008).
 28.
Derrick, S.C., Yabe, I.M., Yang, A. & Morris, S.L. Vaccineinduced antituberculosis protective immunity in mice correlates with the magnitude and quality of multifunctional CD4 T cells. Vaccine 29, 2902–2909 (2011).
 29.
Hawkridge, T. et al. Safety and immunogenicity of a new tuberculosis vaccine, MVA85A, in healthy adults in South Africa. J. Infect. Dis. 198, 544–552 (2008).
 30.
De Rosa, S.C., Carter, D.K. & McElrath, M.J. OMIP014: validated multifunctional characterization of antigenspecific human T cells by intracellular cytokine staining. Cytometry A 81, 1019–1021 (2012).
Acknowledgements
This work was funded by National Institute of Health grants (R01 EB008400), and grants (UM1 AI068635) and (UM1 AI068618) to the HIV Vaccine Trials Network (HVTN) and the Statistical Data Management Center (SDMC), the Human Immunology Phenotyping Consortium (U19 AI089986), and the Collaboration for AIDS Vaccine Discovery (OPP1032325). The work was also supported in part by an Interagency Agreement Y1AI264212 between the US Army Medical Research and Material Command and the National Institutes of Allergy and Infectious Diseases and by a cooperative agreement (W81XWH0720067) between the Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc., and the US Department of Defense. We also acknowledge G. Peterson for technical assistance with all assays related to the TB data set.
Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Department of Defense.
Author information
Affiliations
Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, USA.
 Lin Lin
 , Greg Finak
 , Kevin Ushey
 , Nicole Frahm
 , Peter Gilbert
 , Stephen C De Rosa
 , M Juliana McElrath
 & Raphael Gottardo
Division of Allergy and Infectious Diseases, University of Washington, Seattle, Washington, USA.
 Chetan Seshadri
 , Thomas R Hawn
 & M Juliana McElrath
South African Tuberculosis Vaccine Initiative, Institute of Infectious Disease and Molecular Medicine and School of Child and Adolescent Health, University of Cape Town, Cape Town, South Africa.
 Thomas J Scriba
 , Hassan Mahomed
 & Willem Hanekom
Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland.
 PierreAlexandre Bart
 & Giuseppe Pantaleo
Duke Human Vaccine Institute, Duke University Medical Center, Durham, North Carolina, USA.
 Georgia D Tomaras
Department of Disease Control, Ministry of Public Health, Nonthaburi, Thailand.
 Supachai RerksNgarm
Data Management Unit, Faculty of Tropical Medicine, Mahidol University, Ratchathewi, Bangkok, Thailand.
 Jaranit Kaewkungwal
Thai Component, Armed Forces Research Institute of Medical Sciences, Bangkok, Ratchathewi, Bangkok, Thailand.
 Sorachai Nitayaphan
Vaccine Trials Center, Faculty of Tropical Medicine, Mahidol University, Ratchathewi, Bangkok, Thailand.
 Punnee Pitisuttithum
US Military HIV Research Program, Walter Reed Army Institute of Research, Silver Spring, Maryland, USA.
 Nelson L Michael
 & Jerome H Kim
US Army Military HIV Research Program, Walter Reed Army Institute of Research; Henry M. Jackson Foundation, Bethesda, Maryland, USA.
 Merlin L Robb
US Army Medical Component, Armed Forces Research Institute of Medical Sciences, Ratchathewi, Bangkok, Thailand.
 Robert J O'Connell
 & Nicos Karasavvas
Department of Laboratory Medicine, University of Washington, Seattle, Washington, USA.
 Stephen C De Rosa
 & M Juliana McElrath
Authors
Search for Lin Lin in:
Search for Greg Finak in:
Search for Kevin Ushey in:
Search for Chetan Seshadri in:
Search for Thomas R Hawn in:
Search for Nicole Frahm in:
Search for Thomas J Scriba in:
Search for Hassan Mahomed in:
Search for Willem Hanekom in:
Search for PierreAlexandre Bart in:
Search for Giuseppe Pantaleo in:
Search for Georgia D Tomaras in:
Search for Supachai RerksNgarm in:
Search for Jaranit Kaewkungwal in:
Search for Sorachai Nitayaphan in:
Search for Punnee Pitisuttithum in:
Search for Nelson L Michael in:
Search for Jerome H Kim in:
Search for Merlin L Robb in:
Search for Robert J O'Connell in:
Search for Nicos Karasavvas in:
Search for Peter Gilbert in:
Search for Stephen C De Rosa in:
Search for M Juliana McElrath in:
Search for Raphael Gottardo in:
Contributions
R.G. directed the project. L.L., G.F. and R.G. conceived the research method. L.L. and R.G. developed the statistical model COMPASS. L.L., G.F. and K.U. developed the COMPASS R package, analyzed the data and contributed to interpretation. L.L., G.F. and R.G. wrote the manuscript. C.S. and S.C.D. generated the MTB dataset. C.S. also contributed to the interpretation of the MTB data. T.J.S., T.R.H., H.M. and W.H. contributed to the TB study design and provided overall manuscript feedback. N.F., S.C.D. and M.J.M. acquired and contributed all RV144 cellular data, and contributed to study design and data interpretation. G.D.T. acquired and contributed RV144 IgG data, contributed to study design and data interpretation. P.G., S.R.N., J.K., S.N., P.P., N.L.M., J.H.K., M.L.R., R.J.O. and N.K. contributed to the RV144 study design and data interpretation. P.G. also contributed to the statistical analysis of the RV144 casecontrol data and data interpretation. N.F., S.C.D., P.G., P.A.B., M.J.M. and G.P. contributed to the HVTN078 study design, helped acquire ICS data and contributed to data interpretation. All authors discussed the results and commented on the manuscript.
Competing interests
The authors declare no competing financial interests.
Corresponding author
Correspondence to Raphael Gottardo.
Integrated supplementary information
Supplementary figures
 1.
Receiver Operating Characteristic (ROC) analysis for the RV144 casecontrol data differentiating vaccinees from placebos.
 2.
Comparison of COMPASS scores and PIs on the RV144 casecontrol data to examine ability in summarizing subjects' response to vaccination.
 3.
Heatmap of COMPASS posterior probabilities for the RV144 pilot data (including both pre and post vaccination samples).
 4.
Receiver Operating Characteristic (ROC) analysis for the RV144 pilot data differentiating vaccinees at the postvaccination time point from placebos or vaccinees at the prevaccination time point (baseline).
 5.
Correlation of FS with the number of functions (secreted cytokines) as measured by multiplex bead array from the RV144 pilot study.
 6.
Comparison of COMPASS scores and PIs on the RV144 pilot data to examine ability in summarizing subjects' response to vaccination.
 7.
Receiver Operating Characteristic (ROC) analysis for the RV144 casecontrol data using the predicted infection probabilities from a logistic regression model based on different immune response variables, adjusted for IgA level, gender and baseline behavioral risk score
 8.
Receiver Operating Characteristic (ROC) analysis for the TB dataset discriminating MTBinfected subjects from noninfected subjects using the MTBspecific stimulation response.
 9.
Histograms of posterior mean of response probabilities for all the subjects and the specific category expressing IL4+/IFNγ+/ TNFα+/CD40L+ broken down by treatment groups in HVTN078 data.
 10.
Volcano plots for the RV144 casecontrol data on two selected cell subsets showing the relationship between the empirical magnitude (pspu) and the probability of antigenspecificity.
 11.
Heatmap of background corrected posterior mean of proportions (equation 11) for the RV144 data.
 12.
Boxplots of functionality and polyfunctionality scores computed from background corrected posterior mean proportions.
Supplementary information
PDF files
 1.
Supplementary Text and Figures
Supplementary Figures 1–12, Supplementary Tables 1–4 and Supplementary Methods
Rights and permissions
To obtain permission to reuse content from this article visit RightsLink.
About this article
Further reading

1.
Nature Medicine (2018)

2.
npj Vaccines (2018)

3.
Prospecting for an HIV vaccine
Tropical Diseases, Travel Medicine and Vaccines (2017)

4.
Scientific Reports (2017)

5.
Scientific Reports (2017)