Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Benchmarking of T cell receptor repertoire profiling methods reveals large systematic biases

Nature Biotechnology volume 39, pages 236–245 (2021)Cite this article

Subjects

Abstract

Monitoring the T cell receptor (TCR) repertoire in health and disease can provide key insights into adaptive immune responses, but the accuracy of current TCR sequencing (TCRseq) methods is unclear. In this study, we systematically compared the results of nine commercial and academic TCRseq methods, including six rapid amplification of complementary DNA ends (RACE)-polymerase chain reaction (PCR) and three multiplex-PCR approaches, when applied to the same T cell sample. We found marked differences in accuracy and intra- and inter-method reproducibility for T cell receptor α (TRA) and T cell receptor β (TRB) TCR chains. Most methods showed a lower ability to capture TRA than TRB diversity. Low RNA input generated non-representative repertoires. Results from the 5′ RACE-PCR methods were consistent among themselves but differed from the RNA-based multiplex-PCR results. Using an in silico meta-repertoire generated from 108 replicates, we found that one genomic DNA-based method and two non-unique molecular identifier (UMI) RNA-based methods were more sensitive than UMI methods in detecting rare clonotypes, despite the better clonotype quantification accuracy of the latter.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Performance statistics and VDJ rearrangement model of each method for Experiments A and B.
Fig. 2: TRBV usage comparison between flow cytometry and TCRseq.
Fig. 3: The reproducibility of detection of major TCR clonotypes by different methods.
Fig. 4: Sensitivity of TCR sequence detection by different methods.
Fig. 5: Sharing with robust and representative meta-repertoire.

Data availability

All the fastq data obtained in this study, including the Jurkat Clone E6-­1 (ATCC TIB­152) cell line TCR α and β sequences, were deposited in the National Center for Biotechnology Information Sequence Read Archive repository following MiAIRR standard recommendations47 under the BioProject ID PRJNA548335. The aligned sequence data will be stored in an iReceptor Repository at Sorbonne Université as a repository in the AIRR Data Commons and can be explored and downloaded through the iReceptor Gateway65 (https://gateway.ireceptor.org). Source data for TCRVβ flow cytometry data are provided as Supplementary Fig. 4a,b. All other data are available from the corresponding author upon reasonable request.

Code availability

All software packages and programs are publicly available and open source. Scripts used to analyze the data with MiXCR are available from https://mixcr.milaboratory.com. Decombinator is available from https://github.com/innate2adaptive/Decombinator. MiGEC is available from https://github.com/mikessh/migec. Detailed VDJ rearrangement statistics scripts are available from https://github.com/antigenomics/repseq-protocol-comparison. There is no restriction on the use of the code or data.

References

  1. Cui, J.-H. et al. TCR repertoire as a novel indicator for immune monitoring and prognosis assessment of patients with cervical cancer. Front. Immunol. 9, 2729 (2018).

  2. Davis, M. M. The αβ T cell repertoire comes into focus. Immunity 27, 179–180 (2007).

    CAS  PubMed  Google Scholar 

  3. Lindau, P. & Robins, H. S. Advances and applications of immune receptor sequencing in systems immunology. Curr. Opin. Syst. Biol. 1, 62–68 (2017).

    Google Scholar 

  4. Miles, J. J., Douek, D. C. & Price, D. A. Bias in the αβ T-cell repertoire: implications for disease pathogenesis and vaccination. Immunol. Cell Biol. 89, 375 (2011).

    CAS  PubMed  Google Scholar 

  5. Schrama, D., Ritter, C. & Becker, J. C. T cell receptor repertoire usage in cancer as a surrogate marker for immune responses. Semin. Immunopathol. 39, 255–268 (2017).

    CAS  PubMed  Google Scholar 

  6. Heather, J. M. et al. Dynamic perturbations of the T-cell receptor repertoire in chronic HIV infection and following antiretroviral therapy. Front. Immunol. 6, 644 (2016).

  7. Howson, L. J. et al. MAIT cell clonal expansion and TCR repertoire shaping in human volunteers challenged with Salmonella Paratyphi A. Nat. Commun. 9, 253 (2018).

    PubMed  PubMed Central  Google Scholar 

  8. Pogorelyy, M. V. et al. Precise tracking of vaccine-responding T cell clones reveals convergent and personalized response in identical twins. Proc. Natl Acad. Sci. USA 115, 12704–12709 (2018).

    CAS  PubMed  Google Scholar 

  9. Sycheva, A. L. et al. Quantitative profiling reveals minor changes of T cell receptor repertoire in response to subunit inactivated influenza vaccine. Vaccine 36, 1599–1605 (2018).

    CAS  PubMed  Google Scholar 

  10. Hogan, S. A. et al. Peripheral blood TCR repertoire profiling may facilitate patient stratification for immunotherapy against melanoma. Cancer Immunol. Res. 7, 77–85 (2019).

    CAS  PubMed  Google Scholar 

  11. Jin, Y. et al. TCR repertoire profiling of tumors, adjacent normal tissues, and peripheral blood predicts survival in nasopharyngeal carcinoma. Cancer Immunol. Immunother. 67, 1719–1730 (2018).

    CAS  PubMed  Google Scholar 

  12. Wieland, A. et al. T cell receptor sequencing of activated CD8 T cells in the blood identifies tumor-infiltrating clones that expand after PD-1 therapy and radiation in a melanoma patient. Cancer Immunol. Immunother. 67, 1767–1776 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Six, A. et al. The past, present, and future of immune repertoire biology – the rise of next-generation repertoire analysis. Front. Immunol. 4, 413 (2013).

  14. Chien, Y. H., Gascoigne, N. R., Kavaler, J., Lee, N. E. & Davis, M. M. Somatic recombination in a murine T-cell receptor gene. Nature 309, 322–326 (1984).

    CAS  PubMed  Google Scholar 

  15. Davis, M. M. & Bjorkman, P. J. T-cell antigen receptor genes and T-cell recognition. Nature 334, 395–402 (1988).

    CAS  PubMed  Google Scholar 

  16. Lefranc, M.-P. Nomenclature of the human T cell receptor genes. Curr. Protoc. Immunol. 40, A.1O.1–A.1O.23 (2000).

    Google Scholar 

  17. Dupic, T., Marcou, Q., Walczak, A. M. & Mora, T. Genesis of the αβ T-cell receptor. PLoS Comput. Biol. 15, e1006874 (2019).

    PubMed  PubMed Central  Google Scholar 

  18. Robins, H. S. et al. Comprehensive assessment of T-cell receptor β-chain diversity in αβ T cells. Blood 114, 4099–4107 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Warren, R. L. et al. Exhaustive T-cell repertoire sequencing of human peripheral blood samples reveals signatures of antigen selection and a directly measured repertoire size of at least 1 million clonotypes. Genome Res. 21, 790–797 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Qi, Q. et al. Diversity and clonal selection in the human T-cell repertoire. Proc. Natl Acad. Sci. USA 111, 13139–13144 (2014).

    CAS  PubMed  Google Scholar 

  21. Greiff, V., Miho, E., Menzel, U. & Reddy, S. T. Bioinformatic and statistical analysis of adaptive immune repertoires. Trends Immunol. 36, 738–749 (2015).

    CAS  PubMed  Google Scholar 

  22. Wang, C. et al. High throughput sequencing reveals a complex pattern of dynamic interrelationships among human T cell subsets. Proc. Natl Acad. Sci. USA 107, 1518–1523 (2010).

    CAS  PubMed  Google Scholar 

  23. Zhang, W. et al. IMonitor: a robust pipeline for TCR and BCR repertoire analysis. Genetics 201, 459–472 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Douek, D. C. et al. A novel approach to the analysis of specificity, clonality, and frequency of HIV-specific T cell responses reveals a potential mechanism for control of viral escape. J. Immunol. 168, 3099–3104 (2002).

    CAS  PubMed  Google Scholar 

  25. Eugster, A. et al. Measuring T cell receptor and T cell gene expression diversity in antigen-responsive human CD4+ T cells. J. Immunol. Methods 400–401, 13–22 (2013).

    PubMed  Google Scholar 

  26. Mamedov, I. Z. et al. Preparing unbiased T-cell receptor and antibody cDNA libraries for the deep next generation sequencing profiling. Front. Immunol. 4, 456 (2013).

  27. Shugay, M. et al. Towards error-free profiling of immune repertoires. Nat. Methods 11, 653–655 (2014).

    CAS  PubMed  Google Scholar 

  28. Oakes, T. et al. Quantitative characterization of the T cell receptor repertoire of naïve and memory subsets using an integrated experimental and computational pipeline which is robust, economical, and versatile. Front. Immunol. 8, 1267 (2017).

  29. Liu, X. et al. Systematic comparative evaluation of methods for investigating the TCRβ repertoire. PLoS ONE 11, e0152464 (2016).

    PubMed  PubMed Central  Google Scholar 

  30. Rosati, E. et al. Overview of methodologies for T-cell receptor repertoire analysis. BMC Biotechnol. 17, 61 (2017).

  31. Dunn-Walters, D., Townsend, C., Sinclair, E. & Stewart, A. Immunoglobulin gene analysis as a tool for investigating human immune responses. Immunol. Rev. 284, 132–147 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Doenecke, A., Winnacker, E.-L. & Hallek, M. Rapid amplification of cDNA ends (RACE) improves the PCR-based isolation of immunoglobulin variable region genes from murine and human lymphoma cells and cell lines. Leukemia 11, 1787–1792 (1997).

    CAS  PubMed  Google Scholar 

  33. Nielsen, S. C. A. & Boyd, S. D. Human adaptive immune receptor repertoire analysis—past, present, and future. Immunol. Rev. 284, 9–23 (2018).

  34. Thomas, N., Heather, J., Ndifon, W., Shawe-Taylor, J. & Chain, B. Decombinator: a tool for fast, efficient gene assignment in T-cell receptor sequences using a finite state machine. Bioinformatics 29, 542–550 (2013).

    CAS  PubMed  Google Scholar 

  35. Taylor, S., Yasuyama, N. & Farmer, A. A SMARTer approach to profiling the human T-cell receptor repertoire. J. Immunol. 196, 209.5 (2016).

    Google Scholar 

  36. Bolotin, D. A. et al. MiXCR: software for comprehensive adaptive immunity profiling. Nat. Methods 12, 380–381 (2015).

    CAS  PubMed  Google Scholar 

  37. Yokota, R., Kaminaga, Y. & Kobayashi, T. J. Quantification of inter-sample differences in T-cell receptor repertoires using sequence-based information. Front. Immunol. 8, 1500 (2017).

  38. Gudikote, J. P. & Wilkinson, M. F. T-cell receptor sequences that elicit strong down-regulation of premature termination codon-bearing transcripts. EMBO J. 21, 125–134 (2002).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Schulze-Koops, H. Lymphopenia and autoimmune diseases. Arthritis Res. Ther. 6, 178–180 (2004).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Miho, E. et al. Computational strategies for dissecting the high-dimensional complexity of adaptive immune repertoires. Front. Immunol. 9, 224 (2018).

  41. Kivioja, T. et al. Counting absolute numbers of molecules using unique molecular identifiers. Nat. Methods 9, 72–74 (2012).

    CAS  Google Scholar 

  42. Britanova, O. V. et al. Dynamics of individual T cell repertoires: from cord blood to centenarians. J. Immunol. 196, 5005–5013 (2016).

    CAS  PubMed  Google Scholar 

  43. Brüggemann, M. et al. Standardized next-generation sequencing of immunoglobulin and T-cell receptor gene recombinations for MRD marker identification in acute lymphoblastic leukaemia; a EuroClonality-NGS validation study. Leukemia 33, 2241–2253 (2019).

  44. Knecht, H. et al. Quality control and quantification in IG/TR next-generation sequencing marker identification: protocols and bioinformatic functionalities by EuroClonality-NGS. Leukemia 33, 2254–2265 (2019).

  45. Friedensohn, S. et al. Synthetic standards combined with error and bias correction improve the accuracy and quantitative resolution of antibody repertoire sequencing in human naïve and memory B cells. Front. Immunol. 9, 1401 (2018).

  46. Breden, F. et al. Reproducibility and reuse of adaptive immune receptor repertoire data. Front. Immunol. 8, 1418 (2017).

    PubMed  PubMed Central  Google Scholar 

  47. Rubelt, F. et al. Adaptive immune receptor repertoire community recommendations for sharing immune-repertoire sequencing data. Nat. Immunol. 18, 1274–1278 (2017).

  48. Vander Heiden, J. A. et al. AIRR community standardized representations for annotated immune repertoires. Front. Immunol. 9, 2206 (2018).

  49. Zhang, Y. et al. Tools for fundamental analysis functions of TCR repertoires: a systematic comparison. Brief. Bioinform. https://doi-org.eres.qnl.qa/10.1093/bib/bbz092 (2019).

  50. Weber, C. R. et al. immuneSIM: tunable multi-feature simulation of B- and T-cell receptor repertoires for immunoinformatics benchmarking. Bioinformatics 36, 3594–3596 (2020).

  51. Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer, 2016).

  52. Marcou, Q., Mora, T. & Walczak, A. M. High-throughput immune repertoire analysis with IGoR. Nat. Commun. 9, 561 (2018).

  53. Murugan, A., Mora, T., Walczak, A. M. & Callan, C. G. Statistical inference of the generation probability of T-cell receptors from sequence repertoires. Proc. Natl Acad. Sci. USA 109, 16161–16166 (2012).

    CAS  PubMed  Google Scholar 

  54. Horn, H. S. Measurement of “overlap” in comparative ecological studies. Am. Nat. 100, 419–424 (1966).

    Google Scholar 

  55. Jaccard, P. The distribution of the flora in the Alpine zone. New Phytol. 11, 37–50 (1912).

    Google Scholar 

  56. Sadee, C., Pietrzak, M., Seweryn, M. & Rempala, G. divo: Tools for Analysis of Diversity and Similarity in Biological Systems (Diversity and Overlap Analysis Package). https://rdrr.io/cran/divo/ (2017).

  57. Kolde, R. pheatmap: pretty heatmaps. https://cran.r-project.org/web/packages/pheatmap/index.html (2019).

  58. Renyi, A. On measures of information and entropy. In Proc. 4th Berkeley Symposium on Mathematical Statistics and Probability 547–561 (University of California Press, 1961).

  59. Hill, M. O. Diversity and evenness: a unifying notation and its consequences. Ecology 54, 427–432 (1973).

    Google Scholar 

  60. Chaara, W. et al. RepSeq data representativeness and robustness assessment by shannon entropy. Front. Immunol. 9, 1038 (2018).

    PubMed  PubMed Central  Google Scholar 

  61. Hausser, J. & Strimmer, K. Estimation of Entropy, Mutual Information and Related Quantities. (2014).

  62. Colwell, R. K. et al. Models and estimators linking individual-based and sample-based rarefaction, extrapolation and comparison of assemblages. J. Plant Ecol. 5, 3–21 (2012).

    Google Scholar 

  63. Hsieh, T. C., Ma, K. H. & Chao, A. Package iNEXT: Interpolation and Extrapolation for Species Diversity (2019).

  64. Chao, A. et al. Rarefaction and extrapolation with Hill numbers: a framework for sampling and estimation in species diversity studies. Ecol. Monogr. 84, 45–67 (2014).

    Google Scholar 

  65. Corrie, B. D. et al. iReceptor: a platform for querying and analyzing antibody/B-cell and T-cell receptor repertoire data across federated repositories. Immunol. Rev. 284, 24–41 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We are grateful to M. Barbié for providing the human samples. This work benefited from equipment and services from the iGenSeq core facility at ICM. This work was supported the ERC-Advanced TRiPoD (322856), LabEx Transimmunom (ANR-11-IDEX-0004-02) and RHU iMAP (ANR-16-RHUS-0001) grants to D.K. E.M.F. is funded by the European Research Area Network-Cardiovascular Diseases (JCT2018 and ANR-18-ECVD-0001) and iReceptorPlus (H2020 Research and Innovation Programme 825821) grants. M.S. and D.M.C. were supported by a grant from the Ministry of Science and Higher Education of the Russian Federation (075−15−2019-1789). This work was funded, in part, by the intramural program of the National Institute of Allergy and Infectious Diseases (to D.C.D.). B.C. was supported by the National Institute for Health Research UCL Hospitals Biomedical Research. A.E. was supported by DFG CRTD (FZ 111). A.N.D. was supported by the Ministry of Education, Youth and Sports of the Czech Republic under project CEITEC 2020 (LQ1601). K.K. and P.P.R. were supported by the European Research Area Network-Cardiovascular Diseases (JCT2018 and AIR-MI Consortium) program.

Author information

Authors and Affiliations

  1. Sorbonne Université, INSERM, Immunology-Immunopathology-Immunotherapy (i3), Paris, France

    Pierre Barennes, Valentin Quiniou, David Klatzmann & Encarnita Mariotti-Ferrandiz

  2. AP-HP, Hôpital Pitié-Salpêtrière, Biotherapy (CIC-BTi) and Inflammation-Immunopathology-Biotherapy Department (i2B), Paris, France

    Pierre Barennes, Valentin Quiniou, David Klatzmann & Encarnita Mariotti-Ferrandiz

  3. Center of Life Sciences, Skoltech, Moscow, Russia

    Mikhail Shugay & Dmitriy M. Chudakov

  4. Genomics of Adaptive Immunity Department, Shemyakin and Ovchinnikov Institute of Bioorganic Chemistry, Moscow, Russia

    Mikhail Shugay, Evgeniy S. Egorov & Dmitriy M. Chudakov

  5. Center for Precision Genome Editing and Genetic Technologies for Biomedicine, Pirogov Russian National Research Medical University, Moscow, Russia

    Mikhail Shugay & Dmitriy M. Chudakov

  6. Adaptive Immunity Group, Central European Institute of Technology, Brno, Czechia

    Alexey N. Davydov & Dmitriy M. Chudakov

  7. Division of Infection and Immunity, University College London, London, UK

    Imran Uddin, Mazlina Ismail, Theres Oakes & Benny Chain

  8. DFG-Centre for Regenerative Therapies Dresden, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany

    Anne Eugster

  9. Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria

    Karl Kashofer

  10. Division of Cardiology, Medical University of Graz, Graz, Austria

    Peter P. Rainer

  11. BioTechMed-Graz, Graz, Austria

    Peter P. Rainer

  12. Vaccine Research Center, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, MD, USA

    Samuel Darko, Amy Ransier & Daniel C. Douek

Authors
  1. Pierre Barennes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valentin Quiniou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mikhail Shugay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Evgeniy S. Egorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexey N. Davydov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dmitriy M. Chudakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Imran Uddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mazlina Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Theres Oakes
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Benny Chain
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Anne Eugster
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Karl Kashofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Peter P. Rainer
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Samuel Darko
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Amy Ransier
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Daniel C. Douek
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. David Klatzmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Encarnita Mariotti-Ferrandiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.B., V.Q., E.S.E., A.N.D., I.U., M.I., T.O., A.E., S.D., A.R., K.K. and P.R. performed the experiments and raw data pre-processing. P.B., V.Q., M.S. and M.I. analyzed the data. E.M.F., D.M.C., B.C., D.C.D. and D.K. designed the experiments. P.B., D.K. and E.M.F. wrote the manuscript with input from all authors. D.K. and E.M.F. conceived the study, which was supervised by E.M.F. D.K., B.C., A.E., D.C.D., D.M.C., K.K. and M.S. obtained funding for the study.

Corresponding author

Correspondence to Encarnita Mariotti-Ferrandiz.

Ethics declarations

Competing interests

D.M.C. and M.S. are cofounders of MiLaboratory LLC. A.E., A.N.D., A.R., B.C., D.C.D., D.K., E.M.F., E.S.E., I.U., K.K., M.I., P.B., P.R., S.D., T.O. and V.Q. declare no conflicts of interest.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs 1–10, Tables 1 and 2, and Methods

Reporting Summary

Supplementary Data 1

Source data used for rank calculations displayed in Table 1. (Raw data sheet corresponds to raw data used for rank values calculation; rank summary values sheet corresponds to details of rank calculations and values displayed in Table 1.)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barennes, P., Quiniou, V., Shugay, M. et al. Benchmarking of T cell receptor repertoire profiling methods reveals large systematic biases. Nat Biotechnol 39, 236–245 (2021). https://doi.org/10.1038/s41587-020-0656-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41587-020-0656-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing