Genetics and Genomics

Impact of panel design and cut-off on tumour mutational burden assessment in metastatic solid tumour samples

Abstract

Tumour mutational burden (TMB) has emerged as a promising biomarker to predict immune checkpoint inhibitors (ICIs) response in advanced solid cancers. However, harmonisation of TMB reporting by targeted gene panels is lacking, especially in metastatic tumour samples. To address this issue, we used data of 2841 whole-genome sequenced metastatic cancer biopsies to perform an in silico analysis of TMB determined by seven gene panels (FD1CDx, MSK-IMPACT™, Caris Molecular Intelligence, Tempus xT, Oncomine Tumour Mutation Load, NeoTYPE Discovery Profile and CANCERPLEX) compared to exome-based TMB as a golden standard. Misclassification rates declined from up to 30% to <1% when the cut-point for high TMB was increased. Receiver operating characteristic analysis demonstrated that, for correct classification, the cut-point for each gene panel may vary more than 20%. In conclusion, we here demonstrate that a major limitation for the use of gene panels is inter-assay variation and the need for dynamic thresholds to compare TMB outcomes.

Background

Tumour mutational burden (TMB) has emerged as a promising biomarker to predict response to immune checkpoint inhibitors (ICIs) in a number of solid cancers. The reason TMB was proposed as predictive biomarker is the assumption that more mutations would possibly generate a higher number of new epitopes, called “neoantigens”, also referred to as mutational load (ML). A higher ML increases the probability that a tumour cell expresses a true neoantigen rendering a tumour cell more prone to T cell-mediated immune destruction.1,2

TMB is defined as the number of mutations (somatic single variant (SNV) and multinucleotide variant (MNV) and small insertions and deletions (indels)) per megabase pair (Mb) of sequence examined and can be measured by genome, exome or gene panel sequencing. This introduces a challenge because mutations are not randomly distributed throughout the genome and therefore the sequencing design will introduce a bias.3 Since whole-exome sequencing or whole-genome sequencing (WGS) techniques are not yet routinely used in clinical practice, panel-based sequencing methods have become feasible alternatives ready to be implemented in routine diagnostics. Still, how gene panel-based TMB relates to exome-based TMB and whether outcomes of different gene panel platforms are translatable, remains unknown. Budczies et al.4 recently described the limitations of panel-based TMB measurements by simulating TMB in publicly available datasets of primary tumours. However, effectiveness of ICI treatment is primarily described in metastatic disease and all Food Drug Administration approvals so far are granted to ICIs for the treatment of advanced stage disease. Since we know that mutational patterns may differ between primary tumour and metastases, selection of patients for ICI treatment is ideally based on biomarker detection in samples from metastatic tumours.5,6,7 Therefore, analysis of the concordance between different panels on metastatic samples is a relevant analysis.

Methods

Since a comprehensive comparison between available targeted gene panels and exome- or genome-based TMB in metastatic tumour samples is lacking, we here assessed the variety of TMB measured by seven different panels based on WGS as a reference. We used TMB based on exome as a reference standard because the coding sequence is most frequently examined in the context of TMB. We hypothesised that classifying patients in two categories of high versus intermediate or low TMB by seven different gene panels would not necessarily result in the same outcome as panel sizes and gene content differ and typically only a limited number of mutations are measured in these assays. By using data of 2841 whole-genome-sequenced metastatic cancer biopsies8 as a reference, we performed an in silico analysis of TMB determined by seven gene panels (FD1CDx by Foundation Medicine, MSK-IMPACT™ by Memorial Sloan Cancer Centre, Caris Molecular Intelligence by Caris Life Sciences, Tempus xT by Tempus, Oncomine Tumour Mutation Load by ThermoFisher, NeoTYPE Discovery Profile by NeoGenomics and CANCERPLEX by KEW) compared to exome TMB as a golden standard. For TMB determination, the number of variants (SNVs, MNVs and indels) within the panel design were divided by the panel footprint (0.78–1.48 Mb) or the size of the exome (30 Mb). Panel designs were retrieved from the manufacturer’s website and as exact designs are not available, panel footprint was assumed to encompass the longest open reading frame of each gene as based on Ensembl (GRCh38), although the real panel design will in most cases be smaller.

Results

First, we analysed how patients would have been classified by each gene panel compared to exome-based TMB for different TMB cut-offs. As expected, the misclassification rate (sum of the percentages of false positives and false negatives) declines from up to 30% to <1% when the cut-off is increased from 5 to 40 mutations per Mb (Table 1). The high percentages of false positives at lower thresholds of high TMB indicate that panels are generally over-calling. This is most certainly due to the fact that panels consist of oncogenic driver genes that will have a disproportionate effect on the mutation count. Second, we dichotomised our data with a cut-point of 10 mutations per Mb to define high TMB, as this cut-off is most frequently used in recent trials as a threshold for high TMB.9,10,11 At this cut-off, misclassification rates range between 4 and 10%, with largest panels typically performing best. Next, we performed a receiver operating characteristic (ROC) analysis for the different gene panels to determine the threshold that each gene panel should set in order to classify most patients in the right TMB category compared to a 10/Mb exome-based cut-off for high TMB (Fig. 1a). By adjusting the thresholds for each panel, a high correct pan-cancer classification of patients could be obtained, with area under the curves (AUCs) ranging from 0.97 to 0.98. However, larger differences (AUC 0.911–0.998) appeared when panel reliability was assessed for different tumour types (Fig. 1b), which is likely due to tumour-type-specific differences in TMB distribution.

Table 1 TMB determined by variants (SNVs, MNVs and indels) in targeted genes in various gene panels compared to TMB determined by the measurement of all variants (SNVs, MNVs and indels) in the exome.
Fig. 1: Performance of the 7 different gene panels in exome-based TMB determination and the relation between TMB and ML for different types of cancer.
figure1

a Receiver operating characteristic (ROC) curves for each gene panel compared to exome-based TMB for all tumour types in the cohort. Exome-based TMB was dichotomised at a 10/Mb cut-point. b ROC curves for each gene panel compared to exome-based TMB for colorectal cancer, skin cancer, lung cancer and breast cancer, respectively. Exome-based TMB was dichotomised at a 10/Mb cut-point. c For each tumour biopsy sequenced, mutational load is plotted against exome-based TMB. Linear regression lines are fitted on the colorectal cancer, skin cancer, lung cancer and breast cancer datasets, respectively. Goodness of fit (R2) and equations for the regression lines of these four tumour types are depicted in the graph.

Discussion

Here we show that, because of design differences, it is crucial to adjust the cut-off for each panel design as, for example, for correct classification at 10 mutations per Mb this may vary more than 20% (from 7.8 to 11.7) between commonly used test panels (Supplementary Table 1). It should be noted that in practice differences might even be larger due to experimental and variant calling differences between platforms. Nevertheless, a major limitation for the use of targeted sequencing platforms is the inter-assay variation due to design and the need for dynamic thresholds to compare TMB outcomes—for different platforms and cancer types. Specifically, trials that use TMB determined by a specific panel as a prospective selection biomarker can only result in platform- and disease-specific patient selection.

Several other factors impact the reliability of the number of mutations counted in the tumour genome. First, it should be realised that TMB does not reflect the number of neoantigens in a tumour cell that can be acted upon by the immune system. ML, the total number of non-synonymous SNVs, MNVs and indels in the tumour, would actually be a more relevant measurement because these mutations result (theoretically) in a change in amino acid(s) and may thus lead to potential neoantigens. Interestingly, ML and TMB do have a clear linear relationship, but in a tumour-type-specific manner. For example, a TMB of 10 mutations per Mb corresponds to a ML of ~20 × 10 in skin cancer and ~23 × 10 in lung cancer (Fig. 1c).

Second, this in silico analysis does not take into account the quality of the sample used for sequencing. In the studied cohort, WGS was performed on fresh frozen tumour tissue, but in daily clinical practice, formalin-fixed, paraffin-embedded tumour tissue is primarily used as a template for targeted sequencing.

Third, matching blood samples for determination of germ line variants are crucial for a valuable TMB assessment in both WGS and gene panel sequencing, since germline polymorphisms can easily contaminate the mutation count.12 Most gene panel platforms use tumour-only sequencing and filter germline variants out by using large germline variant datasets.

In conclusion, we would like to underscore the importance of whole-exome- or whole-genome-based mutation measurements of metastatic tumour samples for benchmarking TMB-based diagnostic biomarker platforms. Both of these platforms do potentially detect all mutations and all coding mutations, respectively, and lack the variability that comes with panel-based diagnostics. Thus, comprehensive tumour sequencing would be the most optimal strategy for the development of tumour-type agnostic, reproducible and reliable genetic biomarkers for immunotherapy.

References

  1. 1.

    McGranahan, N., Furness, A. J. S., Rosenthal, R., Ramskov, S., Lyngaa, R., Saini, S. K. et al. Clonal neoantigens elicit T cell immunoreactivity and sensitivity to immune checkpoint blockade. Science (New York, NY) 351, 1463–1469 (2016).

    CAS  Article  Google Scholar 

  2. 2.

    Schumacher, T. N. & Schreiber, R. D. Neoantigens in cancer immunotherapy. Science 348, 69–74 (2015).

    CAS  Article  Google Scholar 

  3. 3.

    Schuster-Böckler, B. & Lehner, B. Chromatin organization is a major influence on regional mutation rates in human cancer cells. Nature 488, 504–507 (2012).

    Article  Google Scholar 

  4. 4.

    Budczies, J., Allgäuer, M., Litchfield, K., Rempel, E., Christopoulos, P. Kazdal D. et al. Optimizing panel-based tumor mutational burden (TMB) measurement. Ann. Oncol. 30, 1496–1506 (2019).

  5. 5.

    van Dessel, L. F., van Riet, J., Smits, M., Zhu, Y., Hamberg, P., van der Heijden, M. S. et al. The genomic landscape of metastatic castration-resistant prostate cancers using whole genome sequencing reveals multiple distinct genotypes with potential clinical impact. bioRxiv https://doi.org/10.1101/546051 (2019).

  6. 6.

    Vignot, S., Lefebvre, C., Frampton, G. M., Meurice, G., Yelensky, R., Palmer, G. et al. Comparative analysis of primary tumour and matched metastases in colorectal cancer patients: evaluation of concordance between genomic and transcriptional profiles. Eur. J. Cancer 51, 791–799 (2015).

    CAS  Article  Google Scholar 

  7. 7.

    Angus, L., Smid, M., Wilting, S. M., van Riet, J., Van Hoeck, A., Nguyen, L. et al. The genomic landscape of metastatic breast cancer highlights changes in mutation and signature frequencies. Nat. Genet. 51, 1450–1458 (2019).

    CAS  Article  Google Scholar 

  8. 8.

    Priestley, P., Baber, J., Lolkema, M. P., Steeghs, N., de Bruijn, E., Shale, C. et al. Pan-cancer whole-genome analyses of metastatic solid tumours. Nature 575, 210–216 (2019).

    CAS  Article  Google Scholar 

  9. 9.

    Samstein, R. M., Lee, C. H., Shoushtari, A. N., Hellmann, M. D., Shen, R., Janjigian, Y. Y. et al. Tumor mutational load predicts survival after immunotherapy across multiple cancer types. Nat. Genet. 51, 202–206 (2019).

    CAS  Article  Google Scholar 

  10. 10.

    Hellmann, M. D., Ciuleanu, T.-E., Pluzanski, A., Lee, J. S., Otterson, G. A., Audigier-Valette, C. et al. Nivolumab plus ipilimumab in lung cancer with a high tumor mutational burden. N. Engl. J. Med. https://doi.org/10.1056/NEJMoa1801946 (2018).

  11. 11.

    Rizvi, H., Sanchez-Vega, F., La, K., Chatila, W., Jonsson, P., Halpenny, D. et al. Molecular determinants of response to anti-programmed cell death (PD)-1 and anti-programmed death-ligand 1 (PD-L1) blockade in patients with non-small-cell lung cancer profiled with targeted next-generation sequencing. J. Clin. Oncol. 36, 633–641 (2018).

    CAS  Article  Google Scholar 

  12. 12.

    Goodman, A. M., Kato, S., Bazhenova, L., Patel, S. P., Frampton, G. M., Miller, V. et al. Tumor mutational burden as an independent predictor of response to immunotherapy in diverse cancers. Mol. Cancer Ther. 16, 2598–2608 (2017).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank all nurses, medical specialists and patients in particular for their contribution to this study.

Author information

Affiliations

Authors

Consortia

Contributions

CPCT consortium members were involved in patient enrolment and clinical data collection in the study headed by M.P.J.K.L. WGS was performed under supervision of E.C. and P.R. S.S., M.P.J.K.L., W.N.M.D. and E.C. designed the idea for this specific study. M.P.J.K.L., E.C., M.S.P., F.H.G., P.R. and J.M.M conducted the data analysis and wrote the manuscript. All authors were involved in interpretation of the results and critically reading and correcting the manuscript.

Corresponding author

Correspondence to Marthe S. Paats.

Ethics declarations

Ethics approval and consent to participate

This study was conducted to the standards of Good Clinical Practice, in full conformance with the “Declaration of Helsinki” (latest amendment), the Dutch laws and regulations and with the Medical Research Involving Human Subjects Act (WMO) in particular. The study was approved by the local independent ethics committee of the University Medical Centre (UMC) Utrecht (MEC-12-023). All subjects enrolled in this study provided their informed consent to participate.

Consent to publish

Not applicable.

Data availability

All data described in this study is freely available from the Hartwig Medical Foundation for academic research within the constraints of the consent given by the patients. Standardised procedures and request forms can be found at https://www.hartwigmedicalfoundation.nl/en. All bioinformatic analysis tools and scripts used are available at https://github.com/hartwigmedical/.

Competing interests

M.P.J.K.L. has advisory board memberships for Pfizer, Bayer, Roche, JnJ and received research grants from Astellas, JnJ, Sanofi and MSD. H.J.D. is an advisory board member for AstraZeneca. The other authors declare no competing interests.

Funding information

Clinical studies and WGS analyses were financially supported by Hartwig Foundation and Barcode for Life. Implementation of the data portal was supported by a grant from KWF Kankerbestrijding (HMF2017-8225, GENONCO).

Additional information

Note This work is published under the standard license to publish agreement. After 12 months the work will become freely available and the license terms will switch to a Creative Commons Attribution 4.0 International (CC BY 4.0).

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mankor, J.M., Paats, M.S., Groenendijk, F.H. et al. Impact of panel design and cut-off on tumour mutational burden assessment in metastatic solid tumour samples. Br J Cancer 122, 953–956 (2020). https://doi.org/10.1038/s41416-020-0762-5

Download citation

Further reading

Search

Quick links