Liquid biopsy is a valuable precision oncology tool that is increasingly used as a non-invasive approach to identify biomarkers, detect resistance mutations, monitor disease burden, and identify early recurrence. The Tempus xF liquid biopsy assay is a 105-gene, hybrid-capture, next-generation sequencing (NGS) assay that detects single-nucleotide variants, insertions/deletions, copy number variants, and chromosomal rearrangements. Here, we present extensive validation studies of the xF assay using reference standards, cell lines, and patient samples that establish high sensitivity, specificity, and accuracy in variant detection. The Tempus xF assay is highly concordant with orthogonal methods, including ddPCR, tumor tissue-based NGS assays, and another commercial plasma-based NGS assay. Using matched samples, we developed a dynamic filtering method to account for germline mutations and clonal hematopoiesis, while significantly decreasing the number of false-positive variants reported. Additionally, we calculated accurate circulating tumor fraction estimates (ctFEs) using the Off-Target Tumor Estimation Routine (OTTER) algorithm for targeted-panel sequencing. In a cohort of 1,000 randomly selected cancer patients who underwent xF testing, we found that ctFEs correlated with disease burden and clinical outcomes. These results highlight the potential of serial testing to monitor treatment efficacy and disease course, providing strong support for incorporating liquid biopsy in the management of patients with advanced disease.
Liquid biopsies are increasingly used as a non-invasive method for the genomic profiling of cancer. When tumor tissue is difficult or impossible to obtain, next-generation sequencing (NGS) of circulating tumor DNA (ctDNA) from blood plasma can provide valuable insights for oncologic decision making. Recently, ctDNA analysis accurately identified therapeutic biomarkers1,2,3,4,5, tumor burden6, resistance mechanisms7,8, and disease progression9,10.
While tissue biopsies remain the gold standard for diagnosis and biomarker identification, tumor heterogeneity can cause subclonal or emerging mutations to be overlooked, particularly in metastatic cases. Additionally, many tissue biopsies involve invasive surgical procedures that are not amenable to repeat testing or produce samples insufficient for comprehensive testing. In such cases, liquid biopsies offer several advantages, including real-time detection of emerging resistance mutations, serial testing throughout the course of treatment, and biomarker detection when tumor tissue is unavailable.
However, numerous technical limitations must be overcome to improve the clinical utility of ctDNA NGS assays. For example, many patients lack abundant ctDNA in early-stage disease and ctDNA variants may be below the limit of detection (LOD), resulting in false negatives. In addition, differentiating between germline and somatic variants in ctDNA is difficult, as is differentiating between mutations derived from clonal hematopoiesis (CH) and the solid tumor of interest. Several genes frequently mutated in CH are also important in solid tumors, including TP53, GNAS, IDH2, and KRAS11,12. In these cases, mutations in hematopoietic lineage cells may be mistaken for tumor-derived mutations. The ability to differentiate between germline variants, CH, and somatic tumor mutations in ctDNA will vastly improve clinical utility.
The Tempus xF assay is a 105-gene, hybrid-capture, NGS panel spanning a total of 270 kb that detects actionable oncologic targets in four variant classes: single-nucleotide variants (SNVs), insertions/deletions (indels), copy number variants (CNVs), and gene rearrangements. To establish robust clinical performance, we completed extensive validation studies that demonstrated high sensitivity and specificity, and determined a precise LOD to reduce false negatives. We show that the Tempus xF liquid biopsy detects actionable variants with high accuracy when compared to orthogonal methods, including a commercial ctDNA NGS kit, the Tempus xT NGS tissue assay, and digital droplet PCR (ddPCR).
To further evaluate the genomic landscape, we also developed a method for measuring circulating tumor fraction from targeted sequencing data. The method predicts overall tumor burden and helps distinguish germline from somatic variants in cell-free DNA (cfDNA). Many of our solid tumor clinical samples have matched normal blood samples from which buffy coat DNA is obtained. Analyzing matched samples allowed us to identify mutations resulting from CH, differentiate germline from somatic variants, and evaluate levels of tumor shedding, all of which significantly reduced false positives and false negatives.
Here, we present the analytical validation of the Tempus xF assay and characterize the potential clinical utility of estimating circulating tumor fraction to support oncologic treatment decisions.
Tempus xF validation summary
The Tempus xF liquid biopsy oncology assay detects SNVs and indels in all 105 genes, CNVs in 6 genes, and chromosomal rearrangements in 7 genes (Supplementary Table 1). To validate the xF liquid biopsy assay, 310 samples were sequenced. Data from 11 samples were analyzed in multiple experiments for a total of 321 samples (for a breakdown of samples used in each experiment, see Supplementary Table 2). The runs generated an average of 287.7 M ± 65.6 M total reads with 143.7 M ± 32.7 M read pairs and a unique median read depth of 4579.9 ± 2305.5. The average percent of mapped reads across all runs was 99.90% ± 0.08. The median sample had a conversion efficiency of 40% (Supplementary Fig. 1).
Analytical sensitivity for all SNVs, indels, CNVs, and rearrangements targeted in the reference samples are shown in Table 1. Overall, SNVs were reliably detected at 0.25% VAF with 30 ng of input DNA (93.75% [45/48] sensitivity), indels at ≥0.5% VAF with 30 ng (95.83% [23/24] sensitivity), CNVs at ≥0.5% VAF with 10 ng (100.00% [8/8] sensitivity), and rearrangements at ≥1% VAF with 30 ng (90% [9/10] sensitivity). At ≥0.25% VAF with 30 ng of input DNA, analytical specificity was 100% for SNVs, indels, and rearrangements, and 96.2% for CNVs (Table 2).
Overall, intra-assay and inter-assay concordance between the replicates in this study was 100% for SNVs, indicating a high degree of repeatability and reproducibility. The inter-instrument concordance was 96.70% for SNVs and 100% for indels, with a combined concordance of 96.83% across instruments. Additionally, interfering substances such as genomic DNA, ethanol, and isopropanol did not cause a change in the detection of variants. Concordance between controls and samples with interfering substances was 100% among samples that passed filtering and were above the LOD.
Accuracy of the Tempus xF assay compared to orthogonal assays
To evaluate analytical accuracy, we compared the Tempus xF assay to the Roche AVENIO ctDNA Expanded Kit. In 30 ng cfDNA samples analyzed by Tempus xF assay and AVENIO ctDNA Expanded Kit (n = 40), sensitivity for SNVs, indels, CNVs, and rearrangements was 94.8%, 100%, 100%, and 100%, respectively. Among the six SNVs that were not detected, five were identified but filtered out due to insufficient evidence. In 10 ng samples (n = 29), sensitivity for SNVs, indels, CNVs, and rearrangements was 91.9%, 100%, 80%, and 100%, respectively. Of the 7 SNVs that were not detected, 6 were identified but filtered out due to insufficient evidence (Table 3).
To further validate xF assay results, patients with reported KRAS G12D (n = 12), TERT c.−124 (n = 7), TERT c.−146 (n = 5), TP53 R273H (n = 7), and TP53 R175H (n = 7) variants were selected for analysis by ddPCR. Then, xF NGS VAF was compared with ddPCR VAF to determine concordance. We observed 100% positive predictive value and a high correlation between ddPCR results and xF VAF (R2 = 0.892). The high correlation was also observed for individual variants such as KRAS G12D (R2 = 0.970) (Fig. 1a, b). Overall, these results suggest the Tempus xF assay accurately identifies hotspot mutations.
Concordance between Tempus xF liquid biopsy and Tempus xT tissue assay
We compared analytical sensitivity and specificity in matched xT (tumor biopsy) and xF (blood biopsy) tests from 55 patients. Since xT matched samples contain both tumor tissue and buffy coat (normal comparator), we used the Tempus xT classification strategy to determine germline variants and exclude them from the analysis. After removing intronic, synonymous, benign, and likely benign variants, as well as variants below the LOD of both assays, we identified 145 concordant SNVs, 20 concordant indels, and 11 concordant CNVs. We also identified multiple discordant results, including 66 SNVs, 11 indels, and 8 CNVs that were reported in xT but not xF. Conversely, we observed 209 SNVs, 14 indels, and 7 CNVs that were reported in xF but not xT. Using the Tempus internal Bayesian dynamic filtering methodology, however, reduced this discordance by 11.45% (Supplementary Table 3). The overall sensitivity of xF relative to xT was 68.18% for SNVs and indels, and 57.89% for CNVs (Supplementary Table 4). However, when the analysis was limited to clinically actionable targets, we identified 107 concordant variants and 37 discordant variants for a final overall sensitivity of 74.31%. While CNVs are generally detected with high sensitivity by solid tumor assays, distinguishing true CNVs from noise in cfDNA is more difficult due to the relatively lower tumor fraction. Nevertheless, these values are similar to previously reported comparisons between liquid biopsy and solid tumor sequencing13.
Using a corresponding xT assay result, which includes germline sequencing data from buffy coat, the xF assay can also distinguish between germline and CH-derived variants. We compared the classification of reportable variants between samples with matched xF and xT testing. Variants were considered CH-derived if found in both the plasma and the xT normal sample, but not at levels consistent with germline variation (Fig. 1c). The percent ctDNA VAF (top) and number of reportable variants detected (bottom) for each individual patient were categorized by assay type and CH or germline status. Overall, we observed high concordance between the xF and xT assays (Fig. 1c and Supplementary Table 4). As expected, variants detected at higher VAFs by xF were generally detected as germline variants by xT or are likely CH variants. Notably, two samples had a large number of CH variants only detected in xF, many of which were at low VAFs. These samples were subsequently found to have very high tumor mutational burdens (TMBs) in their corresponding xT analyses. Taken together, the large number of xF variants at low VAFs and high TMBs suggest that these tumors were highly heterogeneous, and some variants are more easily detected in blood.
Lastly, the xF assay was used to assess microsatellite instability (MSI) status in samples previously classified by the xT solid tumor test or immunohistochemistry. Among the 65 samples included in the MSI validation cohort, 16 were deemed MSI-high and 49 microsatellite-stable. The xF assay reported MSI-high status in 37.5% (6/16) of orthogonally confirmed MSI-high patients at 100% (6/6) positive predictive value. Furthermore, the xF assay did not report MSI-high status for any of the 49 confirmed MSS patients (Fig. 1d). Overall, our comparison between the xT and xF assays demonstrate the strengths of the xF assay and the added value of using multiple assays to detect genomic drivers of cancer.
OTTER, a method for estimating tumor fraction
Accurate measures of tumor fraction improve the understanding of variants identified through liquid biopsy testing. We developed a method, Off-Target Tumor Estimation Routine (OTTER), for determining a more accurate circulating tumor fraction estimate (ctFE). We compared OTTER ctFEs with VAFs from the xF 1000 cohort and found xF ctFE correlates with max pathogenic VAF (Fig. 2a, b) and median VAF (Fig. 2c, d) after removing germline variants and amplified regions. These results show only a modest relationship between detected VAFs and OTTER ctFEs, which we believe is primarily because VAF values provide a poor estimate of the circulating fraction. While the absolute correlation value is low, the number of samples with consistent VAF values and ctFEs is quite high (Fig. 2c, d). For the xF ctFEs to be consistent with the VAF values, we expect them to be greater than or equal to the maximal/medial somatic VAF that is not on an amplified region in a sample. Overall, after removing germline variants and variants on amplified regions, 94.1% of median VAFs were less than or equal to the corresponding xF ctFEs. The distribution of xF ctFEs for the xF 1000 cohort is shown in Fig. 2e. Overall, the median ctFE was 0.07 with a mean ctFE of 0.12.
In addition to VAF, low-pass whole-genome sequencing (LPWGS) is increasingly utilized to estimate tumor fractions and is thought to be a more accurate measure than VAF14,15. We compared LPWGS ichorCNA-predicted circulating tumor fraction to the OTTER ctFE in matched patient samples (n = 375) and found a strong correlation between methods (ρ = 0.890, P = 1.45e−57, Fig. 2f). This correlation indicates that OTTER ctFEs are highly concordant with estimates using LPWGS. However, OTTER exhibits a distinct advantage by calculating ctFE directly from the targeted-panel sequencing without requiring additional LPWGS analysis.
Another advantage of OTTER compared to ichorCNA is its apparent increased sensitivity for detecting tumor fraction. For 210 samples, ichorCNA estimated a tumor fraction of zero while the OTTER ctFE was greater than zero (Fig. 2f). In the majority of these samples (60%) we also detected a variant using the xF assay, indicating that there was detectable tumor DNA and the ichorCNA estimate was a false negative. Furthermore, samples with detected variants showed the same overall trend whereby the VAF was consistent with the OTTER ctFE (Supplementary Fig. 2). Finally, when ichorCNA estimates were greater than zero, they were consistently lower than OTTER ctFEs (Supplementary Fig. 3). Taken together these results suggest that OTTER is able to detect ctDNA in more samples than LPWGS.
Retrospective clinical profiling of Tempus xF 1000 cohort
To evaluate the clinical utility of the Tempus xF liquid biopsy, de-identified molecular and clinical data from the xF 1000 cohort were analyzed. The median ctFE predicted by OTTER was 0.07 for all cancer types, with the exception of prostate, which was 0.06 (Fig. 3a). A total of 8099 mutations were reported, of which 2732 were pathogenic and 2238 were clinically actionable (Fig. 3b). Overall, the most frequently mutated gene was TP53 (51.1% of patients). Within cancer types, commonly mutated genes included TP53 (41.7%), PIK3CA (38.2%), ESR1 (29.1%), BRCA2 (18.5%), NF1 (17.3%), ATM (14.6%) and APC (11.8%) in breast cancer, TP53 (59.8%), KRAS (21.6%), EGFR (18.3%), and ATM (14.6%) in lung cancer, and TP53 (69.4%), APC (66.3%), and KRAS (36.7%) in colorectal cancer (Fig. 3b). These findings are consistent with the existing literature on mutated genes in each cancer type5,10,16,17,18,19, and indicate that the xF test accurately detects common variants of interest.
Advanced disease is associated with higher estimated tumor fraction
A goal of liquid biopsy assays is to efficiently monitor treatment response and predict disease progression in patients over time. Accordingly, we investigated the association of ctFE with advanced disease states and found a significant difference in ctFEs between stages (P = 2.97e−5, Fig. 4a). However, since the majority of patients had advanced disease at the time of testing, additional early-stage samples are necessary to further validate these findings. We also evaluated ctFEs in patients with metastatic disease and found ctFE increases when distant sites are affected (Fig. 4b). Indeed, patients with no metastatic lesions had significantly lower ctFEs than patients with one or more distant sites (P = 4.77e−7, Fig. 4c, d), further highlighting the potential of ctFE for disease monitoring.
Estimated tumor fraction correlates with response to treatment
To determine how ctFE changes in response to treatment, we compared ctFE with the most recent clinical response outcome. We found a significant difference (P = 2.34e−5 by Kruskal–Wallis) in ctFEs among patients with complete response (0.05), stable disease (0.06), partial response (0.06), and progressive disease (0.08, Fig. 5a). In addition, we found that many patients with multiple xF tests had marked differences in ctFE between test dates (Fig. 5b). While additional longitudinal studies of serial xF testing are necessary to further understand how ctFE changes in response to treatment or disease progression, these findings highlight how serial testing can benefit precision oncology.
Liquid biopsies are increasingly used in clinical care to identify biomarkers, detect resistance mutations, and monitor response to treatment or disease progression in real time without invasive procedures. While tissue biopsies remain the gold standard for diagnosis and identifying treatment biomarkers, liquid biopsy can add valuable insights, particularly in heterogeneous metastatic tumors with subclonal mutations. The possibility for serial testing via non-invasive liquid biopsies throughout the course of disease could prove beneficial for many patients. By monitoring emerging resistance mutations and indicators of progression, like tumor burden, clinicians are able to adjust treatment plans more nimbly.
Here, we report extensive validation studies of the Tempus xF assay that established robust technical performance, demonstrating high sensitivity and specificity for calling SNVs, indels, CNVs, and gene rearrangements. These results were highly reproducible between runs and instruments, and in the presence of interfering substances. To further establish the clinical utility of the Tempus xF assay, we evaluated the assay performance on a cohort of 1000 samples across different cancer types. We found that Tempus xF testing is clinically applicable to patients of all ages with a wide range of cancer types, mutations, disease states, and treatment histories. While it is commonly assumed that some cancer types shed little tumor DNA and are therefore less suited for liquid biopsy monitoring, we found that ctDNA was detectable at similar levels across all cancer types tested, with the exception of prostate cancer. While overall there was a correlation between metastatic lesions and circulating fraction, it is important to note that other factors such as physiology, vascular accessibility of metastatic site, and physical tumor size likely play a role in detection of ctDNA. This may also be confounded by cancer type-specific propensity for multi-organ metastatic spread.
We also present OTTER, an algorithm capable of estimating the fraction of tumor-derived circulating DNA from panel sequencing data. We demonstrated the accuracy of xF ctFEs calculated using OTTER by comparing them against the maximal and medial somatic VAFs and to estimates from LPWGS. To demonstrate the direct clinical utility of xF ctFEs, we evaluated differences in ctFEs calculated using OTTER in advanced metastatic disease. Because up to 30% of breast cancer patients and up to 55% of lung cancer patients relapse after initial treatment20,21,22,23, as well as a significant portion of patients in other cancer cohorts, the ability to detect metastasis and disease recurrence earlier in these patients could significantly improve outcomes. Indeed, recent retrospective and prospective studies show ctDNA measurements after completion of treatment or surgery can act as a biomarker for disease recurrence in many cancer types, including breast cancer, lung cancer, melanoma, bladder cancer, and colon cancer24,25,26,27,28. Our results are consistent with these findings, as higher ctFEs were associated with disease progression at radiographic evaluation and with an increased metastatic lesion count. Additionally, we demonstrated that ctFEs correlate with important clinical outcomes while providing a minimally invasive method to monitor patients for response to therapy, disease relapse, and disease progression.
While liquid biopsy is a promising tool for improving outcomes in precision oncology, there are challenges that must be overcome before it can replace large-panel NGS tissue genotyping. For example, in early-stage disease, when treatments have much higher success rates, many patients have low ctDNA fractions that may be below the LOD for liquid biopsies29,30,31,32, limiting clinical utility because of the risk of false negatives. Furthermore, while validation studies of existing liquid biopsy assays have shown high sensitivity and specificity33,34,35,36,37,38, few studies have corroborated results with orthogonal methods, or between NGS testing platforms. Kuderer et al. compared commercially available liquid and tissue NGS platforms and found only 22% concordance in genetic alterations39. Other reports of liquid biopsy-based studies are limited by comparison to non-comprehensive tissue testing algorithms including Sanger sequencing, small NGS hotspot panels, PCR, and FISH. These metrics may not contain all NCCN guideline genes in their reportable range, thus suffering in comparison to a more comprehensive liquid biopsy assay1. Since the 105-gene Tempus xF liquid biopsy assay is a subset of the 648-gene xT tissue-based assay40, the concordance data presented here (74.31% for actionable variants) represents a direct comparison to a comprehensive NGS test containing the entire reportable range of the liquid biopsy assay. While this concordance is high relative to previous reports, our results nonetheless show that actionable variants would have been missed if only one of the tests were performed. The xF assay covers more genes, detects more CNVs, and includes more translocations than other commercially available methods. Overall, these benefits enhance the detection of resistance mutations. Furthermore, the method for estimating circulating tumor fraction described in the paper, OTTER, is more reliable than the median VAF values typically used in other commercially available assays. Tumor tissue profiling minimizes analytical biases driven by variable tumor shedding and the tumor percentage of the sample can be determined pre-analytically to prevent false negatives. Thus, we believe that liquid biopsies provide the greatest value to patients when used in combination with standard tissue genotyping. Furthermore, combining data from both tests enabled additional analyses to exclude germline and CH variants, significantly improving specificity.
The Tempus xF assay displays clinical utility both as a method of monitoring disease burden and as a method of detecting and responding to emerging resistance mutations. When used in the course of care, serial ctDNA monitoring can predict objective measures of progression in at-risk individuals. Due to cost and convenience of sampling, these tests can be applied at shorter time intervals than radiographic methods and allow for more timely intervention in the case of disease progression.
In summary, we present the analytical and clinical validation of the Tempus xF liquid biopsy. We show high accuracy compared to orthogonal methods, including tissue biopsy, Avenio liquid biopsy, ddPCR, and LPWGS. We also improve upon existing methodology for estimating circulating tumor fraction. Notably, we use our improved methodology and real-world clinical data to demonstrate the value and suitability of xF testing for monitoring disease progression, predicting objective measures of response, and assessing treatment outcomes. These results strongly support the Tempus xF assay’s use in routine monitoring of cancer patients with advanced disease.
Sample collection, storage, nucleic acid isolation, and library prep
To validate the Tempus xF liquid biopsy, 321 total specimens were analyzed. These consisted of 10 blood specimens purchased from BioIVT run in triplicate to assess inter-instrument concordance, 4 clinical samples run in triplicate to assess intra-assay concordance, 12 samples to assess inter-assay concordance, 44 residual plasma samples to assess analytical specificity, 69 clinical samples at 10 ng DNA input and 30 ng DNA input to assess analytical accuracy, and 12 clinical samples to assess interfering substances. The validation studies also included 1 cfDNA reference standard isolate (100% Multiplex I Wild Type Reference Standard HD776), and 4 cfDNA reference standards set in synthetic plasma (Horizon Discovery’s Multiplex I cfDNA Reference Standards HD812, HD813, HD814 and Horizon Discovery’s Structural Multiplex cfDNA reference standard HD786) loaded at 10 ng, 30 ng, and 50 ng of DNA input and titrated to achieve 0.1%, 0.25%, 0.5%, 1%, and 5% VAF, totaling 170 samples for evaluating xF LOD.
An additional 55 blood samples with matched tumor samples were used to compare the Tempus xF liquid biopsy and xT solid tumor tests, 65 blood samples were used to validate a microsatellite instability (MSI) classification model, and 375 blood samples were evaluated by low-pass whole-genome sequencing (LPWGS) for comparative analyses. Finally, data from an additional 1000 patient samples previously sequenced at Tempus, referred to as the xF 1000 cohort, were used for retrospective and clinical analyses. An overview of all samples included in the validation and retrospective profiling experiments is presented in Supplementary Table 5.
All blood was received in Cell-free DNA BCT® blood collection tubes (Streck) within 36 h of collection and stored at RT or 4 °C until plasma separation. Plasma was prepared immediately after accessioning and stored at −80 °C until nucleic acid extraction and library prep. At this time, cfDNA was isolated from plasma using the Qiagen QIAamp MinElute ccfDNA Midi Kit (QIAGEN) according to manufacturer’s instructions. Automated library preparation was performed on a SciClone NGSx (Perkin Elmer). All cfDNA samples were normalized with molecular grade water to a maximum of 50 μL.
xF sequencing assay
This assay uses New England BioLab’s NEBNext® Ultra™ II DNA Library Prep Kit for Illumina®, IDT’s xGen CS Adapters, unique molecular indices (UMI), and 96 pairs of barcodes to prepare cfDNA sequencing libraries with unique sample IDs. Each sample is ligated to a dual unique index, which enables multiplexed sequencing of up to 7 patients and 1 positive control per SP NovaSeq flow cell, 16 patients and 1 positive control per S1 NovaSeq flow cell, 34 patients and 1 positive control per S2 NovaSeq flow cell, and 84 patients and 1 positive control per S4 NovaSeq flow cell. The library preparation protocol is optimized for ≥20 ng cfDNA input to maximize mutation detection sensitivity. The final library is sequenced on an Illumina NovaSeq sequencer and analysis is performed using Tempus’ bioinformatics pipeline and analysis server.
Adapter-trimmed FASTQ files are aligned to the 19th edition of the human reference genome build (hg19) using Burrows-Wheeler Aligner (BWA)41. Following alignment, reads are grouped by alignment position and UMI family, and collapsed into consensus sequences using fgbio tools (http://fulcrumgenomics.github.io/fgbio/). Bases with insufficient quality or significant disagreement among family members are transformed to N’s. Phred scores are scaled based on initial base calling estimates combined across all family members. Following single-strand consensus sequence generation, duplex consensus sequences are generated by comparing the forward- and reverse-oriented PCR products with mirrored UMI sequences. Consensus sequences are re-aligned to the human reference genome using BWA. BAM files are generated and indexed after the re-alignment.
SNV and indel variants are detected using VarDict42. SNVs are called down to 0.1% VAF for specified hotspot target regions and 0.25% VAF at all other base positions across the panel. Indels are called down to 0.5% VAF for variants within specific regions of interest. Any indels outside of these regions are called down to 5% VAF. All SNVs and indels are then sorted, deduplicated, normalized and annotated. Following annotation, variants are then classified as germline, somatic, or uncertain using a Bayesian model. The model is based on prior expectations informed by internal and external databases of germline and somatic variants. Uncertain variants are treated as somatic for filtering and reporting purposes. Following classification, variants are then filtered based on a set of quality metrics including coverage, VAF, strand bias, and genomic complexity. Additionally, variants are filtered with a Bayesian trinucleotide context-based model with position-level background error rates estimated from process-matched healthy controls. Known artifactual variants are also removed.
Copy number variants (CNVs) are analyzed using CNVkit43 plus a Tempus CNV annotation and filtering algorithm. CNVkit is used for genomic region binning, coverage calculation, bias correction, normalization to a reference pool, segmentation, and visualization. The log2 ratios between the tumor sample and a pool of process-matched healthy samples from the CNVkit output are then annotated and filtered using statistical models whereby the amplification status (amplified or not-amplified) of each gene is predicted and non-focal amplifications are removed.
Rearrangements are detected using the SpeedSeq analysis pipeline44. Briefly, FASTQ files are aligned to hg19 using BWA. Split reads mapped to multiple positions and read pairs mapped to discordant positions are identified and separated, then used to detect gene rearrangements by LUMPY45. Fusions are then filtered by the number of supporting reads.
Predicted functional effect and clinical interpretation for each variant are curated by automated software using information from both internal and external databases. The software uses a weighted-heuristic model with logic-based recommendations from the AMP/ASCO/CAP/ClinGen Somatic working group46 and ACMG guidelines47.
To detect MSI, the relative frequency and distribution are determined for any read containing repetitive sequences. To predict the probability of an unstable locus, a k-nearest neighbors model (with k = 100) is used along with normalized percent lower, mean lower, and mean log-likelihood metrics. The percentage of unstable loci is calculated from the probabilities of each sample, with >50% unstable loci considered MSI-high.
To establish robust technical performance, extensive validation studies were performed. LOD was determined by assessing analytical sensitivity in reference standards with 5%, 1%, 0.5%, 0.25%, and 0.1% VAF generated from the Horizon Discovery reference set. The Horizon Discovery set includes 160 bp cfDNA fragments from human cell lines in an artificial plasma matrix to closely resemble cfDNA extracted from human plasma. VAFs of SNVs and indels, including EGFR (ΔE746 - A750), EGFR (V769 - D770insASV), EGFR A767_V769dup, EGFR (L858R), EGFR (T790M), KRAS (G12D), NRAS (A59T), NRAS (Q61K), AKT1 E17K, PIK3CA (E545K), and GNA11 Q209L were measured in reference samples by the xF assay. The reference samples were also evaluated for CNVs and rearrangements, including CCDC6/RET, SLC34A2/ROS1, MET, MYC, and MYCN. Each measurement was run with a minimum of three replicates at 10 ng, 30 ng, and 50 ng of DNA. Sensitivity was calculated by the number of detected variants divided by the total number of variants present in the reference samples. Samples with an on-target rate <30% were excluded from this analysis and MET (4.5 copies) was not included in the CNV sensitivity calculation. Sensitivity >90% was considered reliable detection.
Analytical specificity was determined using 44 normal samples titrated at 1%, 2.5%, or 5% from a wild-type cfDNA reference standard (HD776) with a list of SNVs and indels that are known to be absent and were considered true-negative variants for the specificity calculation. False-positive variants were those identified as pathogenic or likely pathogenic after subtracting the background variants from HD776 from all reported variants in the 44 samples. Since there is no existing standard with CNVs and rearrangements that are known to be absent, the number of reportable CNVs and rearrangements that were not identified in HD776 was used as a true-negative count for CNV and rearrangement specificity. Specificity was calculated by the number of known true-negative variants divided by the number of true-negative variants plus false-positive variants identified by the xF assay.
To assess inter-instrument concordance between the sequencing instruments, 10 patient libraries were sequenced on each instrument (3 NovaSeqs). Variants observed below the lower limit of detection (LLOD) (0.25% for SNVs and 0.50% for indels) were excluded from concordance analysis.
To evaluate reproducibility, inter-assay variant concordance of 4 clinical samples with variants near the target sensitivity of the assay were analyzed in three separate runs. The three separate runs were prepared with different barcodes and run on separate days by three different technologists. The four clinical samples used for inter-assay concordance were previously used in inter-instrument concordance. Variant concordance was evaluated for those variants that were above the LOD.
To evaluate repeatability, the same four clinical samples used in the inter-assay concordance were analyzed in triplicate within the same run using different barcodes. These samples contained variants near the assay target sensitivity. Variant concordance was evaluated for those detected above the LOD.
To establish analytical accuracy, the xF results from 40 validation samples with 30 ng DNA input and 29 validation samples with 10 ng DNA input were compared to the results of an orthogonal reference method (Roche’s AVENIO ctDNA Expanded Kit, Cat# 8061076001). Analytical accuracy was calculated by the number of detected variants divided by the total number of variants present in the sample. Variants that were off-target or below the LLOD (0.25% for SNVs and 0.5% for indels) were excluded from the analysis.
The effect of ethanol, isopropanol, and genomic DNA interference on sequencing was evaluated using three clinical samples from the analytical accuracy experiment. The clinical samples were normalized to 45 µL with 5 µL of the interfering substance spiked in prior to sequencing. Concordance between control samples and those containing interfering substances was evaluated for samples that passed filtering criteria and for variants above the LOD.
Digital droplet polymerase chain reaction (ddPCR)
Five variants were validated on the ddPCR platform: KRAS G12D (Integrated DNA Technologies, IDT, published sequences), TERT promoter mutation c.−124C > T (C228T), TERT promoter mutation c.−146C > T (C250T) (Thermo Fisher Scientific), TP53 p.R273H, and TP53 p.R175H (Thermo Fisher Scientific). Each amplification reaction was performed in 25 μL and contained 1X Genotyping Master Mix (Thermo Fisher Scientific), 1X droplet stabilizer (RainDance), 1X of primer/probe mix for TERT and TP53 (for KRAS: 800 nM of each primer and 500 nM of each probe) plus template. To improve the LLOD, 4-cycle amplification was performed prior to droplet generation. Amplification for KRAS was performed using the following cycling conditions: 1 cycle of 95 °C (0.6 °C/s ramp) for 10 min, 4 cycles of 95 °C (0.6 °C/s ramp) for 15 sec and 60 °C for 2 min, followed by 1 cycle of 98 °C (0.6 °C/s ramp) for 10 min. Cycling conditions for the TP53 variants were the same as those for KRAS, except the annealing and extension occured at 55 °C for 2 min. Amplification for TERT followed Thermo Fisher’s recommendation: 1 cycle of 96 °C (1.6 °C/s ramp) for 10 min, 4 cycles of 98 °C (1.6 °C/s ramp) for 30 sec and 55 °C for 2 min, followed by 1 cycle of 55 °C (1.6 °C/s ramp) for 2 min. Droplets were then generated on the RainDance Source, and amplification was performed following the above cycling conditions with 45 cycles for both KRAS and TP53, and 54 for TERT. Droplets were placed on the RainDance Sense droplet reader, and data were acquired and analyzed using RainDrop Analyst II v1.1.0.
Concordance between Tempus xF and xT assays
Matched xF (liquid biopsy) and xT (solid tumor) sample pairs (n = 55) from colon (n = 28), breast (n = 18), and lung (n = 9) cancers were used to calculate analytical sensitivity and specificity. Solid tumor and matched normal samples obtained from peripheral blood buffy coat were analyzed with the Tempus xT assay, and corresponding blood plasma samples were analyzed with the Tempus xF liquid biopsy assay. Since the 105-gene Tempus xF liquid biopsy assay is a subset of the 648-gene xT tissue-based assay40, only variants in the reportable range of both the xT and xF panels were included in these analyses. Germline, intronic, and synonymous variants identified in xT and xF were excluded from analysis with the exception of intronic splice variants. To determine analytical sensitivity, the number of variants identified in both xF and xT (true positives) was divided by the sum of true positives and those identified only in xT. To determine analytical specificity, the number of positions reported in neither xF nor xT (true negatives) was divided by the sum of true negatives and variants only identified in xF.
To improve xF variant calling, we developed a strategy that dynamically calculates local sequence errors using Bayes Theorem and the likelihood ratio test. The dynamic threshold is calculated using a sample-specific error rate, the error rate from healthy control samples, and from our internal cohort of solid tumor samples. This method was tested on 55 matched xF/xT samples, with variants detected in the xT solid tumor assay as the source of truth. Using sensitivity thresholds defined by the LOD analysis, fixed post-test odds (equal to the P(post-test) / [1 - P(post-test)]) and pre-test odds (calculated using the same equation but with historical data from the xT assay), we developed the following formula:
The specificity was input to a beta-binomial function and yielded the minimum number of alternate alleles to call a variant at a given depth. The pre-test odds metric was specific to individual cancer cohorts and individual genes, allowing for cancer-specific pre-test odds to be applied to individual exons.
Estimation of circulating tumor fraction
Existing methods for calculating tumor purity either require a separate sequencing experiment14 or a large number of targets48 for accurate estimation. Therefore, circulating tumor fraction estimates (ctFEs) were determined using an Off-Target Tumor Estimation Routine (OTTER) from both on- and off-target reads distributed across the human reference genome. OTTER is distinct from other methods in its generation of a ctFE from a single targeted-panel sequencing experiment.
As described above, CNVkit was run on each sample and segments were assigned via circular binary segmentation (CBS)49. Segments were then fit to integer copy states via an expectation-maximization algorithm using the sum of squared errors of the segment log2 ratios (normalized to genomic interval size) to expected ratios given a putative copy state and tumor purity. Following these methods, ctFEs were generated for the xF 1000 cohort.
OTTER begins with the output of CNVkit that defines a set of copy ratio segments, S
where each segment, CRi is defined as
and can vary in length depending on the number of bins in the segment assigned by the CBS algorithm in CNVkit. The set of segments are filtered based on two criteria. A segment is removed if it is on a contig that is historically difficult to sequence, such as the sex chromosomes X and Y. A segment may also be removed if it is too short. In this case we removed segments with fewer than 100 bins.
Next, based on the user parameters for possible tumor fractions (T) and copy states (C) to check, OTTER calculates a t x c matrix of expected log2 copy ratios, E. In this implementation we used tumor fractions ranging from 1 to 99% in increments of 1%
And copy states
however, different sets of T and C can be specified if desired. Each expected log2 copy ratio Et,c for tumor fraction t and copy state c can therefore be calculated as
For example, the expected log2 copy ratio for a segment with 4 copies of the genome at a tumor fraction of 50% would be
Using this expectation matrix the distance of a calculated copy ratio segment i to each copy state in C, for a given tumor fraction t can be calculated as
Therefore, the closest copy state, ci,t, for the segment at the given tumor fraction, t, is determined as
the weighted error associated with that segment is
where li is number of bins on the segment. Finally, the loss at a tumor purity t is calculated as
and the final tumor purity estimated by OTTER is
Low-pass whole-genome sequencing and analysis
To confirm the accuracy of OTTER, blood samples from 375 patients with xF testing were also sequenced using LPWGS across four flowcells. Sequencing coverage metrics for these samples were calculated using Picard CollectWgsMetrics. The tumor fraction and ploidy values for each sample were estimated using ichorCNA14 with a Tempus-specific panel of 47 normal samples. This approach provides tumor fraction estimates from WGS to compare against ctFEs generated from xF targeted-panel sequencing data. Reported variants from the corresponding xF analysis of each sample were used to assess the accuracy of the ctFE.
Clinical profiling of the xF 1000 cohort
A cohort of 1000 de-identified patient health records was randomly selected from the Tempus clinicogenomic database for analysis. The cohort included non-hematologic malignancies of known cancer type and stage previously sequenced with the xF assay. All data were de-identified in accordance with the Health Insurance Portability and Accountability Act (HIPAA) using Safe Harbor guidelines. Dates used for analyses were relative to the first xF sequencing date of each patient, and year of the first sequencing date was randomly off-set. The cohort comprised 55.7% female and 44.3% male patients, with a median age of 66 years and interquartile range of 15. The cohort also spanned 24 cancer categories (Supplementary Table 2), with breast (n = 254), colorectal (n = 98), lung (n = 241), pancreatic (n = 83), and prostate (n = 96) being the most common. Only variants classified as pathogenic or likely pathogenic were included in the analyses. These variants were further classified as actionable if matched to diagnostic, prognostic and/or therapeutic evidence, or if considered biologically relevant. Outcomes were determined according to the most recent clinical response noted in patient records. The study protocol was submitted to the Advarra Institutional Review Board (IRB), which determined the research was exempt from IRB oversight and approved a waiver of HIPAA authorization for this study.
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
The data generated and analyzed during this study are described in the following data record: https://doi.org/10.6084/m9.figshare.1467267350. The de-identified clinical data that support findings from the retrospective profiling study have been deposited in the Vivli repository (https://vivli.org/) in the file ‘xf_1000_De-identified_Phenotypic_Data_and_ctFE.tsv’ under the following accession: T20.0151. Access is restricted and interested parties must make an authorized request to the Vivli repository. Supplementary Tables 1–4 are openly available in accessible format as part of the data record50. Raw data from the validation experiments were generated and analyzed as part of a CAP/CLIA validation. As such, they are not publicly available but have been thoroughly reviewed by those governing authorities.
Requests for source code are subject to review by Tempus and can be directed to the lead author.
Leighl, N. B. et al. Clinical utility of comprehensive cell-free DNA analysis to identify genomic biomarkers in patients with newly diagnosed metastatic non-small cell lung cancer. Clin. Cancer Res. 25, 4691–4700 (2019).
Aggarwal, C. et al. Clinical implications of plasma-based genotyping with the delivery of personalized therapy in metastatic non-small cell lung cancer. JAMA Oncol. 5, 173–180 (2019).
Kato, S. et al. Revisiting epidermal growth factor receptor (EGFR) amplification as a target for anti-EGFR therapy: analysis of cell-free circulating tumor DNA in patients with advanced malignancies. JCO Precis Oncol. https://doi.org/10.1200/PO.18.00180 (2019).
Lee, J. et al. Detection of ERBB2 (HER2) gene amplification events in cell-free DNA and response to Anti-HER2 agents in a large asian cancer patient cohort. Front Oncol. 9, 212 (2019).
van Helden, E. J. et al. RAS and BRAF mutations in cell-free DNA are predictive for outcome of cetuximab monotherapy in patients with tissue-tested RAS wild-type advanced colorectal cancer. Mol. Oncol. https://doi.org/10.1002/1878-0261.12550 (2019).
Hyun, M. H. et al. Clinical implications of circulating cell-free DNA quantification and metabolic tumor burden in advanced non-small cell lung cancer. Lung Cancer 134, 158–166 (2019).
Parikh, A. R. et al. Liquid versus tissue biopsy for detecting acquired resistance and tumor heterogeneity in gastrointestinal cancers. Nat. Med. 25, 1415–1421 (2019).
Pailler, E. et al. Acquired resistance mutations to ALK-inhibitors identified by single circulating tumor cell sequencing in ALK-rearranged non-small-cell lung cancer. Clin. Cancer Res. 1176, 2019 (2019).
Jensen, T. J. et al. Genome-wide sequencing of cell-free DNA identifies copy-number alterations that can be used for monitoring response to immunotherapy in cancer patients. Mol. Cancer Ther. 18, 448–458 (2019).
Dal Maso, A. et al. Clinical features and progression pattern of acquired T790M-positive compared with T790M-negative EGFR mutant non-small-cell lung cancer: catching tumor and clinical heterogeneity over time through liquid biopsy. Clin. Lung Cancer https://doi.org/10.1016/j.cllc.2019.07.009 (2019).
Mayrhofer, M. et al. Cell-free DNA profiling of metastatic prostate cancer reveals microsatellite instability, structural rearrangements and clonal hematopoiesis. Genome Med. 10, 85 (2018).
Hu, Y. et al. False-positive plasma genotyping due to clonal hematopoiesis. Clin. Cancer Res. 24, 4437–4443 (2018).
Razavi, P. et al. High-intensity sequencing reveals the sources of plasma circulating cell-free DNA variants. Nat. Med. 25, 1928–1937 (2019).
Adalsteinsson, V. A. et al. Scalable whole-exome sequencing of cell-free DNA reveals high concordance with metastatic tumors. Nat. Commun. 8, 1324 (2017).
Chen, M. & Zhao, H. Next-generation sequencing in liquid biopsy: cancer screening and early detection. Hum. Genomics 13, 34 (2019).
Savli, H. et al. TP53, EGFR and PIK3CA gene variations observed as prominent biomarkers in breast and lung cancer by plasma cell-free DNA genomic testing. J. Biotechnol. 300, 87–93 (2019).
Cheng, F. T. et al. Liquid biopsy detects relapse five months earlier than regular clinical follow-up and guides targeted treatment in breast cancer. Case Rep. Oncol. Med. 2019, 6545298 (2019).
Keup, C. et al. Targeted deep sequencing revealed variants in cell-free DNA of hormone receptor-positive metastatic breast cancer patients. Cell Mol. Life Sci. https://doi.org/10.1007/s00018-019-03189-z (2019).
Li, G. et al. Genomic profiling of cell-free circulating tumor DNA in patients with colorectal cancer and its fidelity to the genomics of the tumor biopsy. J. Gastrointest. Oncol. 10, 831–840 (2019).
Colleoni, M. et al. Annual hazard rates of recurrence for breast cancer during 24 years of follow-up: results from the international breast cancer study group trials I to V. J. Clin. Oncol. 34, 927–935 (2016).
Yates, L. R. et al. Genomic evolution of breast cancer metastasis and relapse. Cancer Cell 32, 169–184 e167 (2017).
Uramoto, H. & Tanaka, F. Recurrence after surgery in patients with NSCLC. Transl. Lung Cancer Res 3, 242–249 (2014).
Taunk, N. K. et al. Immunotherapy and radiation therapy for operable early stage and locally advanced non-small cell lung cancer. Transl. Lung Cancer Res. 6, 178–185 (2017).
Coombes, R. C. et al. Personalized detection of circulating tumor DNA antedates breast cancer metastatic recurrence. Clin. Cancer Res. 25, 4255–4263 (2019).
Tie, J. et al. Circulating tumor DNA analyses as markers of recurrence risk and benefit of adjuvant therapy for stage III colon cancer. JAMA Oncol. https://doi.org/10.1001/jamaoncol.2019.3616 (2019).
McEvoy, A. C. et al. Monitoring melanoma recurrence with circulating tumor DNA: a proof of concept from three case studies. Oncotarget 10, 113–122 (2019).
Christensen, E. et al. Early detection of metastatic relapse and monitoring of therapeutic efficacy by ultra-deep sequencing of plasma cell-free DNA in patients with urothelial bladder carcinoma. J. Clin. Oncol. 37, 1547–1557 (2019).
Isaksson, S. et al. Pre-operative plasma cell-free circulating tumor DNA and serum protein tumor markers as predictors of lung adenocarcinoma recurrence. Acta Oncol. 58, 1079–1086 (2019).
Bettegowda, C. et al. Detection of circulating tumor DNA in early- and late-stage human malignancies. Sci. Transl. Med. 6, 224ra224 (2014).
Xue, V. W., Wong, C. S. C. & Cho, W. C. S. Early detection and monitoring of cancer in liquid biopsy: advances and challenges. Expert Rev. Mol. Diagn. 19, 273–276 (2019).
Hennigan, S. T. et al. Low Abundance of circulating tumor DNA in localized prostate cancer. JCO Precis. Oncol. https://doi.org/10.1200/PO.19.00176 (2019).
Abbosh, C., Birkbak, N. J. & Swanton, C. Early stage NSCLC - challenges to implementing ctDNA-based screening and MRD detection. Nat. Rev. Clin. Oncol. 15, 577–586 (2018).
Cheng, J. et al. Clinical validation of a cell-free DNA gene panel. J. Mol. Diagn. 21, 632–645 (2019).
Hanibuchi, M. et al. Development, validation, and comparison of gene analysis methods for detecting EGFR mutation from non-small cell lung cancer patients-derived circulating free DNA. Oncotarget 10, 3654–3666 (2019).
Van Laar, R., Lincoln, M. & Van Laar, B. Development and validation of a plasma-based melanoma biomarker suitable for clinical use. Br. J. Cancer 118, 857–866 (2018).
Odegaard, J. I. et al. Validation of a plasma-based comprehensive cancer genotyping assay utilizing orthogonal tissue- and plasma-based methodologies. Clin. Cancer Res. 24, 3539–3549 (2018).
Clark, T. A. et al. Analytical validation of a hybrid capture-based next-generation sequencing clinical assay for genomic profiling of cell-free circulating tumor DNA. J. Mol. Diagn. 20, 686–702 (2018).
Plagnol, V. et al. Analytical validation of a next generation sequencing liquid biopsy assay for high sensitivity broad molecular profiling. PLoS One 13, e0193802 (2018).
Kuderer, N. M. et al. Comparison of 2 commercially available next-generation sequencing platforms in oncology. JAMA Oncol. 3, 996–998 (2017).
Beaubier, N. et al. Integrated genomic profiling expands clinical options for patients with cancer. Nat. Biotechnol. 37, 1351–1360 (2019).
Li, H. & Durbin, R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 25, 1754–1760 (2009).
Lai, Z. et al. VarDict: a novel and versatile variant caller for next-generation sequencing in cancer research. Nucleic Acids Res. 44, e108–e108 (2016).
Talevich, E., Shain, A. H., Botton, T. & Bastian, B. C. CNVkit: genome-wide copy number detection and visualization from targeted DNA sequencing. PLoS Comput. Biol. 12, e1004873–e1004873 (2016).
Chiang, C. et al. SpeedSeq: ultra-fast personal genome analysis and interpretation. Nat. Methods 12, 966–968 (2015).
Layer, R. M., Chiang, C., Quinlan, A. R. & Hall, I. M. LUMPY: a probabilistic framework for structural variant discovery. Genome Biol. 15, R84–R84 (2014).
Li, M. M. et al. Standards and guidelines for the interpretation and reporting of sequence variants in cancer: a joint consensus recommendation of the association for molecular pathology, American society of clinical oncology, and college of American pathologists. J. Mol. diagnostics: JMD 19, 4–23 (2017).
Kalia, S. S. et al. Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics. Genet. Med. 19, 249–255 (2017).
Shen, R. & Seshan, V. E. FACETS: allele-specific copy number and clonal heterogeneity analysis tool for high-throughput DNA sequencing. Nucleic Acids Res. 44, e131 (2016).
Olshen, A. B., Venkatraman, E. S., Lucito, R. & Wigler, M. Circular binary segmentation for the analysis of array‐based DNA copy number data. Biostatistics 5, 557–572 (2004).
Finkle, J. D. et al. Metadata record for the manuscript: Analytical validation of a targeted liquid biopsy assay with comprehensive molecular and clinical profiling of circulating tumor DNA from 1000 patients. figshare 10.6084/m9.figshare.14672673 (2021).
Vivli https://search.vivli.org/?search=T20.01 (2021).
We thank operations, product, engineering, and clinical data teams at Tempus Labs. We thank the pathology and lab teams for processing samples and collecting data. We thank M. Salazar for management and administrative support. We thank R. Kasat for xF validation support. We thank M. Kase and A. Bobe for proofreading, administrative support, and figure assembly. We thank E. Lefkofsky for support and discussions.
J.D.F., H.B., T.M.D., C.L., R.A.B., A.H., A.A.K., A.L., K.E.M., J.P., W.Z., K.P.W., R.T. and N.B are employees and shareholders of Tempus Labs, Inc. The remaining authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Finkle, J.D., Boulos, H., Driessen, T.M. et al. Validation of a liquid biopsy assay with molecular and clinical profiling of circulating tumor DNA. npj Precis. Onc. 5, 63 (2021). https://doi.org/10.1038/s41698-021-00202-2