Liquid biopsy mutation panel for non-small cell lung cancer: analytical validation and clinical concordance

Molecular testing for genomic variants is recommended in advanced non-small cell lung cancer (NSCLC). Standard tissue biopsy is sometimes infeasible, procedurally risky, or insufficient in tumor tissue quantity. We present the analytical validation and concordance study of EGFR variants using a new 17-gene liquid biopsy assay (NCT02762877). Of 144 patients enrolled with newly diagnosed or progressive stage IV nonsquamous NSCLC, 140 (97%) had liquid assay results, and 117 (81%) had both EGFR blood and tissue results. Alterations were detected in 58% of liquid samples. Overall tissue-liquid concordance for EGFR alterations was 94.0% (95% CI 88.1%, 97.6%) with positive percent agreement of 76.7% (57.7%, 90.1%) and negative percent agreement of 100% (95.8%, 100%). Concordance for ALK structural variants was 95.7% (90.1%, 98.6%). This assay detected alterations in other therapeutically relevant genes at a rate similar to tissue analysis. These results demonstrate the analytical and clinical validity of this 17-gene assay.


Supplementary Figures
Supplementary Figure 1. DNA yield in patients with EGFR mutations identified in tissue and liquid biopsy and in patients where EGFR mutations were identified in tissue only. It should be noted that although yields tended to be lower in patients with alterations identified only in tissue, many patients with alterations in liquid had yields at the same level.

Supplementary Tables
Supplementary Table 1 Percentages for percent positive agreement (PPA), negative percent agreement (NPV), and overall percent agreement (OPA) are given with two-sided 95% Clopper-Pearson confidence intervals in parentheses. TP = true positive (variant found in both liquid and tissue), FP = false positive (variant found in liquid but not tissue), FN = false negative (variant found in tissue but not liquid), TN = true negative (variant not found in liquid or tissue).

Inclusion criteria (all of the below)
• Patients must be 18 years or older.
• Patients with stage IV non squamous NSCLC who are either newly diagnosed or progressing on any treatment (progression defined as increasing tumor size or new metastatic lesions on clinical or imaging assessment). Patients with available tissue sample from a metastatic site or, if the patient presents with stage IV disease at diagnosis, from the primary tumor or a metastatic site. If a patient is progressing on EGFR targeted therapy (erlotinib, gefitinib, afatinib), tumor tissue sample is required only if available.
• No new systemic anti-tumor therapy administered in the interval between the tissue biopsy and collection of the blood sample (interval not to exceed eight weeks). Local radiation therapy is permitted.
• Able and willing to read, understand and sign an informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization where this is applicable, or equivalent in other geographies as required.
Exclusion criteria: • Patients who are currently receiving therapy (targeted, immune therapy or chemotherapy) without sign of progression.
• Patients with more than 8 weeks between collection of tumor specimen and collection of blood sample for analysis. (Not applicable for patients progressing on EGFR targeted therapy) • Patients changing EGFR therapy due to toxicity or preference without documented disease progression.
• Patients with brain metastases only.
• Patients who are unable to comply with study and/or follow-up procedures.
• Patients who are unable or unwilling to provide written informed consent.

Limit of blank (LOB):
cfDNA samples from 20-40 mL blood from 103 individual healthy controls were collected in either Streck or Roche blood collection tubes and processed through the standard workflow using a range of DNA inputs (10-50 ng). The sample size exceeded the recommended minimum of 60 as described in CLSI EP17A, Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline. In addition, 10 replicates of sheared/sized selected HapMap DNA were processed for input levels of 10 ng, 30 ng, and 50 ng. Within each variant type, LOBs were determined for individual variants or designated collections of variants (for example, SNV hotspots were divided into low background and high background). For any given collection, the maximum detection metric value across all member variants was computed for each sample and those values used to compute the LOB for the group.

Interfering substances:
A variety of conditions were tested to ensure specificity in the presence of potential interfering substances. High-molecular-weight DNA (HMW-DNA) from buffy coat was spiked into cfDNA and sheared/size selected reference standard DNA at 4 levels (0%, 17%, 33%, and 100%). Stability of blood in both Roche and Streck tubes was determined using blood collected from healthy controls (8 tubes per donor) and held at room temperature for 0, 3, 5, or 10 days. The ability to detect cross-contamination was evaluated either prior to index ligation or after index ligation. In addition, the impact of cross-talk (or "bleed-over") was evaluated by sequencing samples with an Indel at 25% allele fraction (AF) and a sample with an SV at 25% or 50% AF was sequenced with cfDNA samples from healthy controls.

Limit of detection (LOD): Study design was performed in accordance with CLSI EP17A, Evaluation of Detection Capability for Clinical Laboratory Measurement
Procedures; Approved Guideline, which suggests inclusion of 60 low-level samples. Variant positive reference standards containing SNVs, indels, and SVs (Horizon DX) were sheared and size-selected in multiple batches and diluted into sheared/sizeselected HapMap DNA to various AFs and run through the standard process as shown in Table 1. Digital droplet PCR (ddPCR) was used to verify achievement of targeted dilution levels prior to library preparation. For CNVs, LOD was established using pools of cell line DNA containing amplifications of MET, EGFR, and ERBB2 as shown in Table 2.
A suitable ALK control was not available; thus it was not possible to establish LOD for the ALK fusion. However, during a physician experience program with version 1 of the assay, an EML4-ALK fusion was found at approximately 0.1% AF in an NSCLC patient with >900 ng cfDNA, and thus 10 replicates of this samples were run as part of the LOD study.
LODs were established using probit regression with input sample AF or copy number equivalent as the independent variable, and detection rate as the response. The study included dilutions of three preparations which were processed with three different lots of reagents, and LOD estimates were calculated separately for each reagent combination. Where possible, data for each individual variant were fitted separately. Otherwise, all the variants for a given variant type were pooled and a single curve fitted. A = number where the assay is positive given that the variant is known to be present B = number where the assay is positive given that the variant is known not to be present C = number where the assay is negative given that the variant is known to be present 95% confidence intervals for the PPA and TPPV point estimates were calculated using the Jeffrey's and Clopper-Pearson methods.

Reproducibility and repeatability:
Reproducibility and repeatability of the end-to-end assay (extraction to sequencing) was assessed using contrived samples. Variant positive reference standards containing a total of 11 SNVs, two Indels, three SVs, and one CNV gain were spiked into pools of plasma from healthy individuals. Variant positive reference standards were first diluted into HapMap DNA to obtain AFs representative of patient samples (0.72%-6%). The study spanned 3 weeks and included two adapter lots, two bait lots, three pond kit lots, and two sequencers. See Table 3 for the list of runs completed.

Supplementary Note 1: Analytical validation results
Specificity: Specificity was established using data from the LOB and Interfering Substances Study. In the LOB, the signals from two variants (17:37872568:A and 10:43610128:T) in two different cfDNA samples appeared well above the background noise level and were anticipated to be true positives. ddPCR assays were therefore designed to these variants and verified. With the exception of these two variants and known SNPs, thresholds were set at or above the signal from all LOB samples, which represents >99% per-sample specificity.

Sensitivity:
The limit of detection (LOD) of an assay is the lowest amount of target that can be detected by a test system with a stated probability (CLSI). The LOD95 (lowest amount of target for 95% detection rated) was determined for each variant type.
For SNVs, the reference standard material contained 10 SNVs verified to be present by the vendor (Horizon DX). Data for all SNVs within a batch of reference standard material were pooled and the LOD95 calculated. As shown in Supplementary Table 4, the LOD95 varied between batch (0.32-0.43 AF), which is not unexpected given these are point estimates derived from probit models. The pooled batch average was 0.37% AF.
For indels, the LOD95 was determined for the two most common Indels in NSCLC (EGFR 9-bp insertion and 15-bp deletion). The overall LOD95 for each Indel was similar (0.13% and 0.07% for insertions and deletions, respectively) with the overall batch average for Indels being 0.10% AF (Supplementary Table  5).
For structural variants, the LOD95 was determined for two translocations with clinical actionability in NSCLC: SLC34A2→ROS1 and CCDC6→RET. The reference standard control used to determine LOD was also shown to harbor the reciprocal ROS1 translocation (ROS1→SLC34A2), so the LOD95 was also calculated for this translocation. The overall LOD95 for each translocation was similar (0.37%-0.54%) with the overall batch average for SVs being 0.44% AF (Supplementary Table 6). Given that the EML4-ALK translocation is the most common translocation in NSCLC, it was also important to determine the sensitivity of the assay for this variant. In the absence of a suitable reference standard, a patient sample containing the EML4-ALK translocation was run a total of 10 times (starting from cfDNA). One library failed quality metrics prior to sequencing, but the EML4-ALK translocation was detected in all nine libraries sequenced at an AF of 0.15%-0.26%. A ddPCR assay was designed to this specific EML4-ALK translocation and verified to be present at 0.12%.
For CNVs, three samples were used to estimate the LOD95; one sample was a mixture of two cell lines, containing both an EGFR and MET copy number (CN) gain. The second sample contained a ERRB2 CN gain, and the third contained a MET CN gain. Supplementary Table 7 represents the LOD95 point estimates for CNVs using these three samples.
Accuracy: Various types of samples were used to establish accuracy; blood from cancer patients previously shown to have a variant in their tissue, or detected by LSv1.0, variant positive and variant negative reference standards (sheared and size selected to represent cfDNA), and a plasma matrix sample containing known variants. The latter two sample types are representative "contrived" samples, which are defined in the MolDX guidance (M00135) as "those created from controlled materials designed to define the analytical limits of the assay." Eighty-one samples passed the standard quality metrics and are summarized in Supplementary Table 8.
The following represented exclusions from analysis: • No ddPCR results could be obtained for variant N1259K in cfDNA from cancer patient (ID#120338490), thus it was excluded.
• V769L in cfDNA from cancer patient (ID#120612937) was tested as negative by ddPCR; thus this variant was not included in the analysis because it was no longer an "expected variant" based on ddPCR data.
• Indel V237fs in sample HD786 is not targeted by the 17-gene panel and was therefore not included in the analysis.
• The commercially purchased plasma matrix sample failed a standard quality metric for low coverage and was excluded.
• No suitable ddPCR assay could be obtained for the ALK fusion in cfDNA from cancer patient (ID#120694318), therefore it was excluded.
PPA and TPPV are summarized in Supplementary Table 10.
There were two false negatives in this study (both SNVs), but these were both below the LOD95: G13D KRAS was found to be present in cfDNA at 0.16% AF by ddPCR but not by the 17-gene liquid biopsy assay. In addition, a de novo K117N NRAS which was present in stock cell line DNA at 50% AF was intended to be diluted to 1% AF. However, by ddPCR, this variant was shown to be at 0.07% AF and thus not detected by the 17-gene liquid biopsy assay.
Although there were two variants that counted as false positives in this study, one was likely a true positive. A purchased plasma sample was found to contain an EGFR Indel at 0.07% AF by the 17-gene liquid biopsy assay, and this matched the information provided by the vendor based on tissue testing. However, it was not found in the cfDNA by ddPCR, which was being used as the comparator/gold standard in this study. Therefore, the only probable true false positive in this accuracy study was a de novo Indel in EGFR, which was detected in one of three replicates in a NA24385 (HapMap) sample.
There were two other unexpected variants detected by the 17-gene liquid biopsy assay, but these were confirmed to be present by ddPCR, as shown in Supplementary Table 11.
Repeatability and reproducibility: All expected variants were observed, indicative of 100% PPA and reproducibility 1 . Supplementary Table 12 illustrates the frequency of observations as number of actual observations over number of expected observations.
Interfering substances: The impact of high-molecular-weight DNA (HMW-DNA) was assessed with regard to variant specificity and sensitivity. As expected, 100% HMW-DNA failed to generate sufficient pond library to continue processing. Yields were below the limit of quantitation, demonstrating that HMW-DNA fragments are not incorporated into pond libraries. There was no increase in CNV signal, nor any false positive CNVs detected, with the addition of HMW-DNA, indicating that HMW-DNA does not appear to be an interfering substance in terms of specificity. All expected variants were detected with the exception of one CNV in one replicate, and one fusion in another with no overall trend with increasing amounts of HMW-DNA addition, indicating it is unlikely that HWM-DNA impacts the sensitivity of the assay at the levels tested.
Blood stability: There was no significant impact on pond yield or quality metrics after up to 10 days of blood storage for either tube type. There was no increase in CNV signal, nor any false detection of CNV.
Blood storage for up to 10 days demonstrated no interference to assay performance.
Cross talk: All samples intentionally cross-contaminated (pre-and post-index ligation) were detected using the contamination module. No false positive calls as a result of crosstalk (i.e., bleed-over) were found when libraries containing 25% AF (Indel and SV) or 50% AF (SV) were sequenced with cfDNA from healthy controls.