Introduction

The past decade has seen dramatic and rapid changes in the management of advanced-stage non-small cell lung cancer (NSCLC). Immunotherapies, having received their first FDA approval only in 2015, are now widely used in clinical practice1. At the same time, the development of targeted therapies has continued apace, with numerous agents now available that target many common NSCLC driver mutations, including hotspot mutations in EGFR and BRAF, and fusions in ALK, RET, and ROS12,3. Meanwhile, additional agents, such as those targeting mutations in KRAS, are under development or being investigated in clinical trials3,4,5. The use of targeted treatments has significantly improved clinical outcomes in patients with the corresponding genetic alterations, and, as a result, they are now frequently used as first-line therapy6,7,8.

A key requirement of targeted therapies is that molecular testing is needed to confirm the presence of any given target. With a growing number of gene mutations and fusions to consider and a multitude of testing modalities available, the process can be confusing and difficult to navigate for both patients and physicians. In current clinical practice, patients are frequently undertested and consequently many receive suboptimal therapy9,10,11. Use of expansive multigene next generation sequencing (NGS) panels performed on solid tissue can simplify the testing process by assessing numerous potential targets simultaneously, saving time and reducing costs12. Indeed, both the National Comprehensive Cancer Network and the College of American Pathologists advocate for the use of broad molecular profiling, such as a large NGS panel, with the goal of identifying both common and rare mutations associated with targeted treatments or clinical trials13,14.

However, solid tissue-based NGS assays also have significant disadvantages. Patients must be able to tolerate the invasive procedure required to obtain tumor tissue. Even when tissue is obtained, a significant proportion of specimens may be insufficiently cellular or yield inadequate nucleic acids for molecular testing15. Furthermore, a single biopsy may not account for possible spatial molecular heterogeneity in the tumor. Nor can this method easily be used to monitor for molecular changes over time, such as the development of resistance mutations. Due to these limitations, a non-invasive method for detecting molecular targets would be highly desirable.

Circulating tumor DNA (ctDNA) assays seek to address many of the limitations inherent to solid tissue-based NGS testing. Liquid biopsies use tumor DNA found in the patient’s plasma, rather than in tumor tissue, as the substrate for NGS and require only a blood draw rather than an invasive biopsy16,17,18. Many commercial and hospital laboratories are now utilizing ctDNA testing to detect actionable mutations in patients with various cancers, including NSCLC. Potentially if a specific mutation profile is identified, serial assays could be used to monitor for the development of resistance mutations, as well to track the overall tumor burden over the course of treatment19.

Prior studies have shown that integrating ctDNA testing into routine clinical care for NSCLC results in an increased number of patients receiving molecular testing, an increased number of clinically important mutations detected, and ultimately an increased number of patients receiving appropriate targeted therapy10,20. However, much of this work has been narrowly focused on individual driver mutations or else has compared multigene ctDNA testing against a patchwork of individual biomarker studies, and many include patients for whom tissue testing was not performed at all10,20,21,22,23,24. As such, it remains unclear what the precise clinical sensitivity and specificity of liquid biopsy is in a head-to-head comparison with a multigene tissue-based NGS panel. Furthermore, it remains unexplored whether testing both tissue and ctDNA, either in parallel or sequentially, provides any additional clinical value. In this study, we sought to directly address these points by analyzing a cohort of patients with lung adenocarcinoma who underwent concurrent testing with a tissue-based NGS assay and a ctDNA assay, and evaluating the ability of each test to detect clinically relevant mutations.

Materials and methods

We retrospectively identified and reviewed the records of 100 consecutive patients at NYU Langone Health with histologically confirmed lung adenocarcinoma who underwent concurrent testing with both our tumor tissue-based NGS platform (“Tissue-NGS”) and a commercially available ctDNA assay (“Plasma-NGS”) between November 29, 2018 and June 30, 2020 (IRB S16-00122). Testing was considered concurrent if tissue and blood samples were obtained within 16 weeks, without new intervening targeted therapy. For patients who obtained multiple tissue-NGS or plasma NGS tests, the results closest in time were used for comparison.

Tissue-NGS was performed by clinically validated NYU Oncomine Focus (Thermo Fisher Scientific, Waltham, MA, USA), an amplicon based assay performed on IonTorrent. This assay evaluates 52 genes and allows concurrent analysis of DNA and RNA to simultaneously detect multiple types of variants, including single nucleotide variants, insertions/deletions, copy number variants (CNVs), and gene fusions, in a single workflow. The variant calling is performed using Ion Torrent™ suite while annotations are performed using the Ion Reporter™ software.

Plasma-NGS was performed by Guardant 360, a commercially available assay developed and performed by Guardant Health (Redwood City, CA, USA), using the procedure previously described18. Briefly, this test uses cell-free DNA from whole blood to evaluate for hotspot mutations in 74 genes, CNVs in 18 genes, and fusions in 6 genes. Over the course of this study, the Guardant 360 panel was expanded from 73 genes to 74 genes. Both versions of the assay covered all mutations that were defined as clinically relevant for the purposes of this study.

Clinically relevant mutations were defined as a broad category that included mutations or fusions for which an approved targeted therapeutic was available (“targetable mutations”, e.g. EGFR exon 19 deletions, EGFR L858R, ALK fusions), resistance mutations (e.g. EGFR T790M), and also other driver mutations for which no approved targeted therapies are currently available (e.g. KRAS G12C), following the list published by Aggarwal et al.20. Despite the lack of associated therapy for the latter, these mutations should still be considered clinically important, as driver mutations are generally mutually exclusive and identification of one may reduce the need for further testing25.

Samples were considered true positives if at least one of the two assays identified a mutation and true negatives if both assays gave a negative result. Sensitivity and accuracy were calculated accordingly. The sensitivity and accuracy of tissue-NGS and plasma-NGS, both overall and within various subgroups, were compared using a one-sided paired t-test with the Welch modification26. The accuracy of plasma-NGS between different subgroups was compared using a one-sided t-test with the Welch modification. Overall % concordance was calculated as (number of concordant positive cases + number of concordant negative cases)/total cases × 100.

Results

Over the course of our study period, 226 lung adenocarcinoma specimens were submitted for tissue-NGS analysis at our institution. Of these, four samples (1.7%) were rejected due to insufficient tumor content. The remaining 222 samples represented 218 patients, of which 100 (45.9%) ultimately underwent concurrent testing with plasma-NGS (Supplementary Table 1). While we defined concurrent testing as up to 16 weeks apart, the median time between sample collections was much shorter at only 12 days, with 82% of samples collected within 4 weeks and 96% within 8 weeks. Tissue was collected first in 57 cases (57%), plasma first in 40 cases (40%), and in 3 cases (3%) the samples were collected on the same day.

Within the cohort, 44 patients (44%) were male and 56 (56%) were female; ages ranged from 35 to 92 years (median: 68 years). The majority of patients had stage IV disease (83, 83%), with the remaining patients classified as stage III (15, 15%) or stage II (2, 2%). Seventy-one patients (71%) were being tested at initial diagnosis and had not received any prior treatment, while 29 patients (29%) had been receiving treatment and were tested following disease progression (Table 1). A clinically relevant mutation was identified by either tissue-NGS or plasma-NGS or both in 78 cases (78%), including 55 cases with therapeutically targetable mutations (Fig. 1A). No mutations were detected by both assays in 22 cases (22%).

Table 1 Cohort demographics.
Full size table
Fig. 1: Presence of mutations identified by next generation sequencing.
figure 1

A Among entire cohort with breakdown of targetable mutations. B Previously treated patients. C Untreated patients.

Full size image

Of the 78 clinically relevant mutations, 37 (47.4%) were detected by both tissue-NGS and plasma-NGS, 37 (47.4%) were detected by tissue-NGS only, and 4 (5.1%) were detected by plasma-NGS only, with an overall concordance of 59% (Fig. 2A and Table 2). Across the entire cohort, the accuracy of tissue-NGS was 96% and the sensitivity was 94.9%, while plasma-NGS had an accuracy of 63% and a sensitivity of only 52.6% (p < 0.001 for both comparisons). Overall, the negative predictive value of plasma-NGS was 37.3%, compared to 84.6% for tissue-NGS. Similar findings were seen when focusing the analysis on mutations that were therapeutically targetable, with sensitivities of 94.5% and 52.7% for tissue-NGS and plasma-NGS respectively (p < 0.001).

Fig. 2
figure 2

Comparison of test performance in A entire cohort, B treated patients, C untreated patients. Tissue-NGS testing shows a higher detection rate of clinically relevant mutations compared to plasma-NGS testing in the entire cohort, the treated patients and the untreated patients.

Full size image
Table 2 Comparison of tissue-NGS and plasma-NGS across different patient subgroups.
Full size table

We hypothesized that the assays’ performance might be reduced in patients who had previously received treatment, as tumor burden would be expected to be decreased after treatment. To test this, we repeated our analyses in the treated and untreated patient subgroups. Of the 29 patients who previously received treatment, 25 (86.2%) had clinically relevant mutations (Fig. 1B) of which 10 (40%) were detected by both tissue and plasma-NGS, 15 showed discrepant results (14 detected by tissue-NGS only and 1 detected by plasma-NGS only) (Fig. 2B). In comparison, for the 71 patients tested at initial diagnosis, 53 (74.6%) had clinically relevant mutations (Fig. 1C), of which 27 (38%) were detected by both tissue and plasma-NGS, and 26 (37%) cases had discrepant results (23 detected by tissue-NGS only and 3 detected by plasma-NGS only) (Fig. 2C). The accuracy and sensitivity of tissue-NGS was significantly higher compared to plasma-NGS in both the treated and untreated subgroups (p < 0.001 for both comparisons) (Table 2). Although there were no significant differences, plasma-NGS in treated group showed a trend of decreased accuracy compared to untreated group (51.7% vs 67.6%; p = 0.08). The accuracy of tissue-NGS remained unchanged across the two groups (95.8% vs. 96.6%, p = 0.43).

A reduction in accuracy of plasma-NGS was seen among patients with nonmetastatic disease. Of the 17 patients in our cohort who did not have stage IV disease, 13 were found to have clinically relevant mutations, but only 4 were identified by plasma-NGS. The accuracy of plasma-NGS in this group was only 41.2%, a significant decrease from the 67.5% accuracy in patients with stage IV disease (p = 0.031).

Given the variability of plasma-NGS performance across these different subgroups, we sought to compare plasma-NGS to tissue-NGS solely in an “ideal” testing population. We repeated our analysis, this time including only the 59 patients in our cohort with stage IV disease who had not previously received treatment. Clinically relevant mutations were identified in 43 cases (72.9%). While the accuracy and sensitivity of plasma-NGS were improved (72.9% and 62.8% respectively), they were still significantly lower than tissue-NGS (94.9% and 93.0% respectively, p = 0.001 for both comparisons).

Clinically relevant mutations identified on tissue-NGS testing but not plasma-NGS testing were predominantly hotspot mutations in EGFR, BRAF, and KRAS. Additional cases included MET exon 14 skipping mutations, and fusions in ALK and ROS1 (Fig. 3). One particularly notable case involved the identification of a novel KIF5B-NTRK1 fusion by tissue-NGS. This patient was originally diagnosed with stage II lung adenocarcinoma in 2015, for which she was treated with surgical segmentectomy and adjuvant chemotherapy with pemetrexed and carboplatin. She was then without evidence of disease until August 2019, when surveillance imaging detected recurrent disease with metastasis to bone and liver. A liver biopsy showed metastatic lung adenocarcinoma and was sent for tissue-NGS testing, while concurrently blood was drawn and sent for plasma-NGS testing. Although plasma-NGS failed to detect any clinically relevant mutations, tissue-NGS identified the novel NTRK1 rearrangement, which was further confirmed by both targeted RNA sequencing (ArcherDx) and Sanger sequencing using specific probes. Finally, sequencing was also performed on the initial segmentectomy specimen, which confirmed the presence of the NTRK mutation in the original tumor. As a result of tissue-NGS testing, this patient is now eligible for targeted therapy with Larotrectinib or Entrectinib.

Fig. 3: Detection of clinically relevant mutations by gene.
figure 3

The 37 clinically relevant mutations identified on tissue-NGS testing but not plasma-NGS testing were predominantly in EGFR, KRAS, ALK and BRAF genes. Conversely, the 4 mutations identified in plasma-NGS testing but not tissue-NGS testing were in EGFR and KRAS genes.

Full size image

There were four clinically relevant mutations detected by plasma-NGS that were not also identified by tissue-NGS: one KRAS G12C mutation, one EGFR exon 19 deletion, and two EGFR L858R mutations. In three cases, the false negative was attributed to low tumor cellularity (below 20%) in the provided biopsy specimens, causing the mutations to fall below the limit of detection, a well-known pitfall of tissue-based NGS assays. In one of these cases (an EGFR L858R mutation) the targetable mutation had been identified previously, and in fact the patient had been receiving erlotinib for nearly a year. In the fourth case, tissue-NGS failed to detect an EGFR L858R mutation due to a simultaneous amplification of the wild-type allele, resulting in the variant allele frequency of the EGFR L858R mutation falling below the assay’s limit of detection. In this case, the discrepancy was realized almost immediately, as the patient’s biopsy specimen had been positive for EGFR L858R-specific immunohistochemistry, and a subsequent gene-specific PCR was able to confirm the presence of the mutation.

Finally, there were also several mutations identified by plasma-NGS, but not tissue-NGS, which did not appear to be related to the patients’ primary lung cancer, raising concerns about the assay’s specificity. Six patients in our cohort (6%) had concurrent JAK2 V617F mutations identified by plasma-NGS. These mutations are rare in NSCLC, though the plasma-NGS report notes that the result “raises the possibility of an incidental finding of a second myeloproliferative neoplasm”. While one of these patients did have a history of monoclonal gammopathy of undetermined significance, none of them had any clinical or laboratory findings suggestive of a myeloproliferative neoplasm and the mutations were ultimately attributed to clonal hematopoiesis (CH). In a separate case, plasma-NGS detected not only an EGFR exon 19 mutation (L757_A750delinsP) at 9.1% of cfDNA in addition to a KRAS G12D mutation at 0.04% cfDNA. Tissue-NGS on this case identified the EGFR exon 19 mutation only, with no KRAS mutation detected on manual review. While both of these mutations can be seen in lung adenocarcinomas, they very rarely co-occur and are typically considered mutually exclusive. Tissue NGS was performed on a cytology cell block, with the tumor cells being morphologically homogenous, reducing the possibility of a collision tumor. A second tumor may be a consideration, but this patient did not have another known malignancy. Ultimately, the KRAS mutation in this case may also be attributable to CH, as has been described previously27.

Discussion

We assessed the clinical performance of a commercial ctDNA assay in comparison to a solid tissue-based multigene NGS panel in patients with lung adenocarcinoma. ctDNA testing was able to detect clinically relevant mutations in 41% of patients, a similar proportion to prior studies using the same assay10,24. However, tissue-based NGS testing was able to identify significantly more clinically relevant mutations than plasma-based NGS (sensitivity of 94.9% vs 52.6% respectively), including therapeutic targets such as EGFR mutations, and fusions in ALK, ROS1, and NTRK1. Based on our cohort, a testing strategy that used plasma-NGS as a supplement to tissue-NGS would have resulted in the detection of only four additional clinically relevant mutations while a strategy that used plasma-NGS alone would have missed 37 clinically relevant mutations, including most importantly 26 alterations with associated targeted therapies.

Plasma-NGS showed a trend toward reduced accuracy in the post-treatment setting, although the difference in our study did not reach statistical significance. Given that one of the main proposed uses of plasma-NGS is the identification of resistance mutations at disease progression, this is a concerning finding that warrants further investigation. Conversely, the sensitivity of tissue-based NGS was high across all subgroups, including in both the treated and untreated populations. It is worth noting that almost all of the previously treated patients had undergone prior molecular testing and, typically, the presence of a clinically relevant mutation was already known. Indeed, the majority of these patients were already being treated with molecularly targeted agents prior to their disease progression. Molecular testing in these cases was generally seeking to identify either possible resistance mechanisms or actionable mutations that may have been missed by prior tests that were more limited in scope. Nevertheless, the diminished sensitivity of plasma-NGS in this patient group is still of clinical importance, as it likely reflects the fact that ctDNA levels in these patients may be too low for accurate clinical assessment. As such, this assay would be expected to have a comparably low sensitivity for detecting a new resistance mutation or previously unidentified targets. Probably for similar reasons, the accuracy of plasma-NGS was significantly diminished in the small number of patients who did not have stage IV disease, consistent with prior studies that show decreased ctDNA concentrations in nonmetastatic disease28,29. However, even when restricting the comparison to “ideal” candidates for plasma-NGS testing (treatment naive stage IV disease), tissue-NGS continued to show significantly higher sensitivity and accuracy.

In addition to sensitivity, specificity of plasma-NGS testing is also a concern. Some mutations can be attributed to circulating lung cancer tumor DNA with a high degree of confidence, but findings from plasma-NGS may also reflect other processes such as concurrent malignancies or CH. Indeed, previous studies have found that almost all JAK2 mutations, some TP53 mutations, and rare KRAS mutations found by plasma-NGS can all be attributed to CH rather than an underlying malignancy27. In our cohort, plasma-NGS identified JAK2 V617F mutations in multiple patients who lacked any evidence of a myeloproliferative neoplasm, as well as detecting a low level KRAS mutation in patient with established EGFR-mutant lung cancer. In light of this clinical information and the previous literature, we feel all of these results are best explained by CH. The possibility of KRAS mutations due to CH is of particular concern; a highly plausible scenario is one where a KRAS mutation from CH is misinterpreted as a lung cancer driver mutation while a true targetable driver mutation goes undetected. In addition to CH, passenger mutations and post-chemotherapy response can also lead to misinterpretations in plasma-NGS. In any event, these cases demonstrate the critical need for the molecular pathologist to take an active role in the interpretation of ctDNA results in close collaboration with the treating oncologist. Interpretative guidance provided through comments or notes included in the final report may be of great assistance to the clinician, helping to avoid both overtreatment and unnecessary patient anxiety.

There are some inherent limitations to our study. This is a retrospective study focused only on patients who received concurrent testing with tissue-based and plasma-based NGS at the discretion of their treating oncologist. These patients were not randomly selected, and we do not consider those patients who received only tissue-NGS but not plasma-NGS. Nor do we investigate the utility of plasma-NGS in patients who did not have concurrent tissue-NGS, either because they were unable to undergo tissue biopsy or because their biopsy specimens were inadequate for molecular testing. Some studies suggest that a substantial proportion (up to 40%) of patients will have biopsies that are insufficient for molecular testing18,20. However, it is our experience that relatively few lung adenocarcinoma samples are inadequate. Over the course of the study period, tissue-NGS was requested on 226 such specimens at our institution, and only 4 (1.7%) were rejected as insufficient. We attribute this to the relatively small amount of input needed for this particular assay, as well as institutional experience in the procurement and processing of such specimens. It should also be noted that in the case of an inadequate tissue specimen, the clinician is notified that the sample was insufficient and that the biopsy should be repeated. This is in contrast to plasma-NGS, where an “insufficient” sample (undetectable ctDNA) is essentially indistinguishable from a true negative. In fact, spurious plasma-NGS findings from the aforementioned clonal hematopoiesis may lead an unsuspecting clinician to believe that ctDNA was present and sequenced when actually this was not the case. Nevertheless, for patients that truly cannot undergo biopsy, ctDNA testing can still play a crucial role in the management of these patients by providing increased access to molecular testing and the potential to receive targeted therapies, as previous studies showed high concordance of specific mutation detection30.

We also did not examine other potential benefits of ctDNA assays, such as increased patient comfort and faster turn-around time. Indeed, the relatively short turn-around time of this ctDNA test is often cited by ordering clinicians as one of its most significant advantages. However, at this time and based on our data, we would discourage clinicians from making treatment decisions based solely on a negative plasma-NGS test. If clinicians do choose to perform ctDNA testing for reasons of turnaround time, we would suggest tissue-NGS also be performed concurrently to avoid unnecessary delays if plasma-NGS returns a negative result. Should it be a major concern, turnaround time for tissue-NGS can be significantly improved with better laboratory and ordering workflow.

In conclusion, our data show that tissue-based NGS has significantly higher clinical sensitivity than plasma-based NGS in the detection of clinically relevant mutations in lung adenocarcinoma. These findings support the use of a tissue-based, multigene NGS panel as the preferred method to assess the molecular profile of lung adenocarcinoma when tissue is available. Simultaneous testing appears to provide only a small increase in accuracy. ctDNA testing should still be incorporated into clinical management of advanced NSCLC, as it can play an important role in cases where tissue biopsy is not feasible or yields an inadequate specimen. However, given its significantly lower clinical sensitivity, plasma-NGS should not be considered a replacement for tissue-based testing nor should the two methodologies be presented to patients as equivalent. Clinicians must interpret negative results with caution given the test’s low negative predictive value, and, ideally, confirm any negative results with a tissue-based assay.