Benchmarking of T cell receptor repertoire profiling methods reveals large systematic biases

Abstract

Monitoring the T cell receptor (TCR) repertoire in health and disease can provide key insights into adaptive immune responses, but the accuracy of current TCR sequencing (TCRseq) methods is unclear. In this study, we systematically compared the results of nine commercial and academic TCRseq methods, including six rapid amplification of complementary DNA ends (RACE)-polymerase chain reaction (PCR) and three multiplex-PCR approaches, when applied to the same T cell sample. We found marked differences in accuracy and intra- and inter-method reproducibility for T cell receptor α (TRA) and T cell receptor β (TRB) TCR chains. Most methods showed a lower ability to capture TRA than TRB diversity. Low RNA input generated non-representative repertoires. Results from the 5′ RACE-PCR methods were consistent among themselves but differed from the RNA-based multiplex-PCR results. Using an in silico meta-repertoire generated from 108 replicates, we found that one genomic DNA-based method and two non-unique molecular identifier (UMI) RNA-based methods were more sensitive than UMI methods in detecting rare clonotypes, despite the better clonotype quantification accuracy of the latter.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Performance statistics and VDJ rearrangement model of each method for Experiments A and B.
Fig. 2: TRBV usage comparison between flow cytometry and TCRseq.
Fig. 3: The reproducibility of detection of major TCR clonotypes by different methods.
Fig. 4: Sensitivity of TCR sequence detection by different methods.
Fig. 5: Sharing with robust and representative meta-repertoire.

Data availability

All the fastq data obtained in this study, including the Jurkat Clone E6-­1 (ATCC TIB­152) cell line TCR α and β sequences, were deposited in the National Center for Biotechnology Information Sequence Read Archive repository following MiAIRR standard recommendations47 under the BioProject ID PRJNA548335. The aligned sequence data will be stored in an iReceptor Repository at Sorbonne Université as a repository in the AIRR Data Commons and can be explored and downloaded through the iReceptor Gateway65 (https://gateway.ireceptor.org). Source data for TCRVβ flow cytometry data are provided as Supplementary Fig. 4a,b. All other data are available from the corresponding author upon reasonable request.

Code availability

All software packages and programs are publicly available and open source. Scripts used to analyze the data with MiXCR are available from https://mixcr.milaboratory.com. Decombinator is available from https://github.com/innate2adaptive/Decombinator. MiGEC is available from https://github.com/mikessh/migec. Detailed VDJ rearrangement statistics scripts are available from https://github.com/antigenomics/repseq-protocol-comparison. There is no restriction on the use of the code or data.

References

  1. 1.

    Cui, J.-H. et al. TCR repertoire as a novel indicator for immune monitoring and prognosis assessment of patients with cervical cancer. Front. Immunol. 9, 2729 (2018).

  2. 2.

    Davis, M. M. The αβ T cell repertoire comes into focus. Immunity 27, 179–180 (2007).

    CAS  PubMed  Google Scholar 

  3. 3.

    Lindau, P. & Robins, H. S. Advances and applications of immune receptor sequencing in systems immunology. Curr. Opin. Syst. Biol. 1, 62–68 (2017).

    Google Scholar 

  4. 4.

    Miles, J. J., Douek, D. C. & Price, D. A. Bias in the αβ T-cell repertoire: implications for disease pathogenesis and vaccination. Immunol. Cell Biol. 89, 375 (2011).

    CAS  PubMed  Google Scholar 

  5. 5.

    Schrama, D., Ritter, C. & Becker, J. C. T cell receptor repertoire usage in cancer as a surrogate marker for immune responses. Semin. Immunopathol. 39, 255–268 (2017).

    CAS  PubMed  Google Scholar 

  6. 6.

    Heather, J. M. et al. Dynamic perturbations of the T-cell receptor repertoire in chronic HIV infection and following antiretroviral therapy. Front. Immunol. 6, 644 (2016).

  7. 7.

    Howson, L. J. et al. MAIT cell clonal expansion and TCR repertoire shaping in human volunteers challenged with Salmonella Paratyphi A. Nat. Commun. 9, 253 (2018).

    PubMed  PubMed Central  Google Scholar 

  8. 8.

    Pogorelyy, M. V. et al. Precise tracking of vaccine-responding T cell clones reveals convergent and personalized response in identical twins. Proc. Natl Acad. Sci. USA 115, 12704–12709 (2018).

    CAS  PubMed  Google Scholar 

  9. 9.

    Sycheva, A. L. et al. Quantitative profiling reveals minor changes of T cell receptor repertoire in response to subunit inactivated influenza vaccine. Vaccine 36, 1599–1605 (2018).

    CAS  PubMed  Google Scholar 

  10. 10.

    Hogan, S. A. et al. Peripheral blood TCR repertoire profiling may facilitate patient stratification for immunotherapy against melanoma. Cancer Immunol. Res. 7, 77–85 (2019).

    CAS  PubMed  Google Scholar 

  11. 11.

    Jin, Y. et al. TCR repertoire profiling of tumors, adjacent normal tissues, and peripheral blood predicts survival in nasopharyngeal carcinoma. Cancer Immunol. Immunother. 67, 1719–1730 (2018).

    CAS  PubMed  Google Scholar 

  12. 12.

    Wieland, A. et al. T cell receptor sequencing of activated CD8 T cells in the blood identifies tumor-infiltrating clones that expand after PD-1 therapy and radiation in a melanoma patient. Cancer Immunol. Immunother. 67, 1767–1776 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Six, A. et al. The past, present, and future of immune repertoire biology – the rise of next-generation repertoire analysis. Front. Immunol. 4, 413 (2013).

  14. 14.

    Chien, Y. H., Gascoigne, N. R., Kavaler, J., Lee, N. E. & Davis, M. M. Somatic recombination in a murine T-cell receptor gene. Nature 309, 322–326 (1984).

    CAS  PubMed  Google Scholar 

  15. 15.

    Davis, M. M. & Bjorkman, P. J. T-cell antigen receptor genes and T-cell recognition. Nature 334, 395–402 (1988).

    CAS  PubMed  Google Scholar 

  16. 16.

    Lefranc, M.-P. Nomenclature of the human T cell receptor genes. Curr. Protoc. Immunol. 40, A.1O.1–A.1O.23 (2000).

    Google Scholar 

  17. 17.

    Dupic, T., Marcou, Q., Walczak, A. M. & Mora, T. Genesis of the αβ T-cell receptor. PLoS Comput. Biol. 15, e1006874 (2019).

    PubMed  PubMed Central  Google Scholar 

  18. 18.

    Robins, H. S. et al. Comprehensive assessment of T-cell receptor β-chain diversity in αβ T cells. Blood 114, 4099–4107 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Warren, R. L. et al. Exhaustive T-cell repertoire sequencing of human peripheral blood samples reveals signatures of antigen selection and a directly measured repertoire size of at least 1 million clonotypes. Genome Res. 21, 790–797 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Qi, Q. et al. Diversity and clonal selection in the human T-cell repertoire. Proc. Natl Acad. Sci. USA 111, 13139–13144 (2014).

    CAS  PubMed  Google Scholar 

  21. 21.

    Greiff, V., Miho, E., Menzel, U. & Reddy, S. T. Bioinformatic and statistical analysis of adaptive immune repertoires. Trends Immunol. 36, 738–749 (2015).

    CAS  PubMed  Google Scholar 

  22. 22.

    Wang, C. et al. High throughput sequencing reveals a complex pattern of dynamic interrelationships among human T cell subsets. Proc. Natl Acad. Sci. USA 107, 1518–1523 (2010).

    CAS  PubMed  Google Scholar 

  23. 23.

    Zhang, W. et al. IMonitor: a robust pipeline for TCR and BCR repertoire analysis. Genetics 201, 459–472 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Douek, D. C. et al. A novel approach to the analysis of specificity, clonality, and frequency of HIV-specific T cell responses reveals a potential mechanism for control of viral escape. J. Immunol. 168, 3099–3104 (2002).

    CAS  PubMed  Google Scholar 

  25. 25.

    Eugster, A. et al. Measuring T cell receptor and T cell gene expression diversity in antigen-responsive human CD4+ T cells. J. Immunol. Methods 400–401, 13–22 (2013).

    PubMed  Google Scholar 

  26. 26.

    Mamedov, I. Z. et al. Preparing unbiased T-cell receptor and antibody cDNA libraries for the deep next generation sequencing profiling. Front. Immunol. 4, 456 (2013).

  27. 27.

    Shugay, M. et al. Towards error-free profiling of immune repertoires. Nat. Methods 11, 653–655 (2014).

    CAS  PubMed  Google Scholar 

  28. 28.

    Oakes, T. et al. Quantitative characterization of the T cell receptor repertoire of naïve and memory subsets using an integrated experimental and computational pipeline which is robust, economical, and versatile. Front. Immunol. 8, 1267 (2017).

  29. 29.

    Liu, X. et al. Systematic comparative evaluation of methods for investigating the TCRβ repertoire. PLoS ONE 11, e0152464 (2016).

    PubMed  PubMed Central  Google Scholar 

  30. 30.

    Rosati, E. et al. Overview of methodologies for T-cell receptor repertoire analysis. BMC Biotechnol. 17, 61 (2017).

  31. 31.

    Dunn-Walters, D., Townsend, C., Sinclair, E. & Stewart, A. Immunoglobulin gene analysis as a tool for investigating human immune responses. Immunol. Rev. 284, 132–147 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Doenecke, A., Winnacker, E.-L. & Hallek, M. Rapid amplification of cDNA ends (RACE) improves the PCR-based isolation of immunoglobulin variable region genes from murine and human lymphoma cells and cell lines. Leukemia 11, 1787–1792 (1997).

    CAS  PubMed  Google Scholar 

  33. 33.

    Nielsen, S. C. A. & Boyd, S. D. Human adaptive immune receptor repertoire analysis—past, present, and future. Immunol. Rev. 284, 9–23 (2018).

  34. 34.

    Thomas, N., Heather, J., Ndifon, W., Shawe-Taylor, J. & Chain, B. Decombinator: a tool for fast, efficient gene assignment in T-cell receptor sequences using a finite state machine. Bioinformatics 29, 542–550 (2013).

    CAS  PubMed  Google Scholar 

  35. 35.

    Taylor, S., Yasuyama, N. & Farmer, A. A SMARTer approach to profiling the human T-cell receptor repertoire. J. Immunol. 196, 209.5 (2016).

    Google Scholar 

  36. 36.

    Bolotin, D. A. et al. MiXCR: software for comprehensive adaptive immunity profiling. Nat. Methods 12, 380–381 (2015).

    CAS  PubMed  Google Scholar 

  37. 37.

    Yokota, R., Kaminaga, Y. & Kobayashi, T. J. Quantification of inter-sample differences in T-cell receptor repertoires using sequence-based information. Front. Immunol. 8, 1500 (2017).

  38. 38.

    Gudikote, J. P. & Wilkinson, M. F. T-cell receptor sequences that elicit strong down-regulation of premature termination codon-bearing transcripts. EMBO J. 21, 125–134 (2002).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Schulze-Koops, H. Lymphopenia and autoimmune diseases. Arthritis Res. Ther. 6, 178–180 (2004).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. 40.

    Miho, E. et al. Computational strategies for dissecting the high-dimensional complexity of adaptive immune repertoires. Front. Immunol. 9, 224 (2018).

  41. 41.

    Kivioja, T. et al. Counting absolute numbers of molecules using unique molecular identifiers. Nat. Methods 9, 72–74 (2012).

    CAS  Google Scholar 

  42. 42.

    Britanova, O. V. et al. Dynamics of individual T cell repertoires: from cord blood to centenarians. J. Immunol. 196, 5005–5013 (2016).

    CAS  PubMed  Google Scholar 

  43. 43.

    Brüggemann, M. et al. Standardized next-generation sequencing of immunoglobulin and T-cell receptor gene recombinations for MRD marker identification in acute lymphoblastic leukaemia; a EuroClonality-NGS validation study. Leukemia 33, 2241–2253 (2019).

  44. 44.

    Knecht, H. et al. Quality control and quantification in IG/TR next-generation sequencing marker identification: protocols and bioinformatic functionalities by EuroClonality-NGS. Leukemia 33, 2254–2265 (2019).

  45. 45.

    Friedensohn, S. et al. Synthetic standards combined with error and bias correction improve the accuracy and quantitative resolution of antibody repertoire sequencing in human naïve and memory B cells. Front. Immunol. 9, 1401 (2018).

  46. 46.

    Breden, F. et al. Reproducibility and reuse of adaptive immune receptor repertoire data. Front. Immunol. 8, 1418 (2017).

    PubMed  PubMed Central  Google Scholar 

  47. 47.

    Rubelt, F. et al. Adaptive immune receptor repertoire community recommendations for sharing immune-repertoire sequencing data. Nat. Immunol. 18, 1274–1278 (2017).

  48. 48.

    Vander Heiden, J. A. et al. AIRR community standardized representations for annotated immune repertoires. Front. Immunol. 9, 2206 (2018).

  49. 49.

    Zhang, Y. et al. Tools for fundamental analysis functions of TCR repertoires: a systematic comparison. Brief. Bioinform. https://doi-org.eres.qnl.qa/10.1093/bib/bbz092 (2019).

  50. 50.

    Weber, C. R. et al. immuneSIM: tunable multi-feature simulation of B- and T-cell receptor repertoires for immunoinformatics benchmarking. Bioinformatics 36, 3594–3596 (2020).

  51. 51.

    Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer, 2016).

  52. 52.

    Marcou, Q., Mora, T. & Walczak, A. M. High-throughput immune repertoire analysis with IGoR. Nat. Commun. 9, 561 (2018).

  53. 53.

    Murugan, A., Mora, T., Walczak, A. M. & Callan, C. G. Statistical inference of the generation probability of T-cell receptors from sequence repertoires. Proc. Natl Acad. Sci. USA 109, 16161–16166 (2012).

    CAS  PubMed  Google Scholar 

  54. 54.

    Horn, H. S. Measurement of “overlap” in comparative ecological studies. Am. Nat. 100, 419–424 (1966).

    Google Scholar 

  55. 55.

    Jaccard, P. The distribution of the flora in the Alpine zone. New Phytol. 11, 37–50 (1912).

    Google Scholar 

  56. 56.

    Sadee, C., Pietrzak, M., Seweryn, M. & Rempala, G. divo: Tools for Analysis of Diversity and Similarity in Biological Systems (Diversity and Overlap Analysis Package). https://rdrr.io/cran/divo/ (2017).

  57. 57.

    Kolde, R. pheatmap: pretty heatmaps. https://cran.r-project.org/web/packages/pheatmap/index.html (2019).

  58. 58.

    Renyi, A. On measures of information and entropy. In Proc. 4th Berkeley Symposium on Mathematical Statistics and Probability 547–561 (University of California Press, 1961).

  59. 59.

    Hill, M. O. Diversity and evenness: a unifying notation and its consequences. Ecology 54, 427–432 (1973).

    Google Scholar 

  60. 60.

    Chaara, W. et al. RepSeq data representativeness and robustness assessment by shannon entropy. Front. Immunol. 9, 1038 (2018).

    PubMed  PubMed Central  Google Scholar 

  61. 61.

    Hausser, J. & Strimmer, K. Estimation of Entropy, Mutual Information and Related Quantities. (2014).

  62. 62.

    Colwell, R. K. et al. Models and estimators linking individual-based and sample-based rarefaction, extrapolation and comparison of assemblages. J. Plant Ecol. 5, 3–21 (2012).

    Google Scholar 

  63. 63.

    Hsieh, T. C., Ma, K. H. & Chao, A. Package iNEXT: Interpolation and Extrapolation for Species Diversity (2019).

  64. 64.

    Chao, A. et al. Rarefaction and extrapolation with Hill numbers: a framework for sampling and estimation in species diversity studies. Ecol. Monogr. 84, 45–67 (2014).

    Google Scholar 

  65. 65.

    Corrie, B. D. et al. iReceptor: a platform for querying and analyzing antibody/B-cell and T-cell receptor repertoire data across federated repositories. Immunol. Rev. 284, 24–41 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We are grateful to M. Barbié for providing the human samples. This work benefited from equipment and services from the iGenSeq core facility at ICM. This work was supported the ERC-Advanced TRiPoD (322856), LabEx Transimmunom (ANR-11-IDEX-0004-02) and RHU iMAP (ANR-16-RHUS-0001) grants to D.K. E.M.F. is funded by the European Research Area Network-Cardiovascular Diseases (JCT2018 and ANR-18-ECVD-0001) and iReceptorPlus (H2020 Research and Innovation Programme 825821) grants. M.S. and D.M.C. were supported by a grant from the Ministry of Science and Higher Education of the Russian Federation (075152019-1789). This work was funded, in part, by the intramural program of the National Institute of Allergy and Infectious Diseases (to D.C.D.). B.C. was supported by the National Institute for Health Research UCL Hospitals Biomedical Research. A.E. was supported by DFG CRTD (FZ 111). A.N.D. was supported by the Ministry of Education, Youth and Sports of the Czech Republic under project CEITEC 2020 (LQ1601). K.K. and P.P.R. were supported by the European Research Area Network-Cardiovascular Diseases (JCT2018 and AIR-MI Consortium) program.

Author information

Affiliations

Authors

Contributions

P.B., V.Q., E.S.E., A.N.D., I.U., M.I., T.O., A.E., S.D., A.R., K.K. and P.R. performed the experiments and raw data pre-processing. P.B., V.Q., M.S. and M.I. analyzed the data. E.M.F., D.M.C., B.C., D.C.D. and D.K. designed the experiments. P.B., D.K. and E.M.F. wrote the manuscript with input from all authors. D.K. and E.M.F. conceived the study, which was supervised by E.M.F. D.K., B.C., A.E., D.C.D., D.M.C., K.K. and M.S. obtained funding for the study.

Corresponding author

Correspondence to Encarnita Mariotti-Ferrandiz.

Ethics declarations

Competing interests

D.M.C. and M.S. are cofounders of MiLaboratory LLC. A.E., A.N.D., A.R., B.C., D.C.D., D.K., E.M.F., E.S.E., I.U., K.K., M.I., P.B., P.R., S.D., T.O. and V.Q. declare no conflicts of interest.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs 1–10, Tables 1 and 2, and Methods

Reporting Summary

Supplementary Data 1

Source data used for rank calculations displayed in Table 1. (Raw data sheet corresponds to raw data used for rank values calculation; rank summary values sheet corresponds to details of rank calculations and values displayed in Table 1.)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Barennes, P., Quiniou, V., Shugay, M. et al. Benchmarking of T cell receptor repertoire profiling methods reveals large systematic biases. Nat Biotechnol (2020). https://doi.org/10.1038/s41587-020-0656-3

Download citation

Search

Quick links

Sign up for the Nature Briefing newsletter for a daily update on COVID-19 science.
Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing