Abstract

Molecular monitoring of chronic myeloid leukemia patients using robust BCR-ABL1 tests standardized to the International Scale (IS) is key to proper disease management, especially when treatment cessation is considered. Most laboratories currently use a time-consuming sample exchange process with reference laboratories for IS calibration. A World Health Organization (WHO) BCR-ABL1 reference panel was developed (MR1–MR4), but access to the material is limited. In this study, we describe the development of the first cell-based secondary reference panel that is traceable to and faithfully replicates the WHO panel, with an additional MR4.5 level. The secondary panel was calibrated to IS using digital PCR with ABL1, BCR and GUSB as reference genes and evaluated by 44 laboratories worldwide. Interestingly, we found that >40% of BCR-ABL1 assays showed signs of inadequate optimization such as poor linearity and suboptimal PCR efficiency. Nonetheless, when optimized sample inputs were used, >60% demonstrated satisfactory IS accuracy, precision and/or MR4.5 sensitivity, and 58% obtained IS conversion factors from the secondary reference concordant with their current values. Correlation analysis indicated no significant alterations in %BCR-ABL1 results caused by different assay configurations. More assays achieved good precision and/or sensitivity than IS accuracy, indicating the need for better IS calibration mechanisms.

Introduction

The development of BCR-ABL1 tyrosine kinase inhibitors, from the first-generation imatinib to newer agents such as nilotinib and dasatinib, has enabled progressively deeper molecular responses in chronic myeloid leukemia (CML) patients undergoing tyrosine kinase inhibitor therapy.1, 2 Deeper molecular responses are defined as BCR-ABL1 levels of 0.01% (MR4) and 0.0032% (MR4.5) on the international reporting scale (International Scale (IS)) and are important milestones for patients considering treatment cessation.3 Other landmarks on the IS also represent different treatment decision thresholds and prognostic outcomes.4 For example, patients who reach 10% IS or below at 3 months after treatment have significantly higher rates of MR4.5 by 5 years,5 and reaching 0.1% IS (major molecular response) by 12 months of treatment is predictive of subsequently achieving undetectable BCR-ABL1 levels.6 Thus, regular molecular monitoring using real-time reverse-transcription quantitative PCR (RT-qPCR) is recommended for optimal disease management, and treatment decisions rely on achieving milestone molecular responses in the first year of therapy and beyond.2, 7

As treatment decisions are directly impacted by test results, accuracy and precision of BCR-ABL1 assays across the entire measurement range is crucial for patient management, especially in patients with deep molecular responses when considering possible treatment cessation. It is well known that high variability exists between RT-qPCR methods used in different laboratories.8, 9 The first international standardization attempt occurred in 2003, when different BCR-ABL1 assays used in the International Randomised Study of Interferon versus STI571 (IRIS) trial established IS based on 30 CML patient samples.10 Subsequently, a process for establishing a test-specific IS conversion factor (CF) by exchanging 20–30 CML patient samples with a reference laboratory was developed.11 Although this process works well for laboratories with tests that show good stability over time, it is time consuming, expensive and difficult to access for smaller laboratories.12, 13

In 2010, the ‘first International Genetic Reference Panel for quantitation of BCR-ABL mRNA’ was developed as a primary standard for BCR-ABL1 assay IS calibration and accredited by the World Health Organization (WHO).14 The WHO panel is made of lyophilized K562 and HL-60 cell line mixtures, which allows the inclusion of cellular RNA extraction in the IS calibration against the two major BCR-ABL1 breakpoints (e13a2 and e14a2) and carries three sets of nominal %BCR-ABL1 values using ABL1, BCR and GUSB as reference genes. Owing to restricted access, the WHO panel is currently only available to manufacturers of BCR-ABL1 test kits and secondary standards.13 The commercial secondary standards available to date are made of RNA;15, 16 thus, RNA extraction is not included in the IS calibration process, except when the standards are artificially spiked into the cell samples. Furthermore, none of these are calibrated to the WHO panel against all three reference genes.

In this study, we describe the successful development of the first cell-based BCR-ABL1 secondary reference panel that is traceable to and faithfully replicates the WHO panel in both raw materials (lyophilized K562 and HL-60 cell mixes) and manufacturing process, with the addition of a MR4.5 level. Nominal %BCR-ABL1 IS values were assigned to the secondary panel using reverse-transcription droplet digital PCR (RT-ddPCR) against ABL1, BCR and GUSB. The secondary panel was successfully evaluated by 45 different BCR-ABL1 assays in a subsequent international multi-center evaluation study.

Materials and methods

Manufacturing and IS calibration of secondary reference panel

K562 (ATCC CCL-243) and HL-60 cells (ATCC CCL-240) (American Type Culture Collection, Manassas, VA, USA) were cultured, mixed and lyophilized following methods described by White et al.14 with minor modifications (Supplementary Information). Calibration to the WHO standards was performed as described.14 IS calibration using ABL1 as a reference gene was conducted using 10 sets of WHO ‘first International Genetic Reference Panel for quantitation of BCR-ABL mRNA’ panels (National Institute for Biological Standards and Control, South Mimms, UK). Calibration using BCR and GUSB was conducted in a second study using another 10 sets of WHO panels. On each day of 10 non-consecutive days, 1 WHO panel and 2–3 secondary panels were tested using RT-ddPCR in 4 replicates for the MR1 (10% BCR-ABLIS) to MR4 (0.01% BCR-ABLIS) samples and in 8 replicates for the MR4.5 (0.0032% BCR-ABLIS) sample, to enhance assay precision. Data analysis was performed using the statistical methods described by White et al.14

Reverse-transcription droplet digital PCR

RNA extraction from the secondary panel was performed using RNeasy mini kits (Qiagen, Hilden, Germany). Reverse transcription was performed using ABI High Capacity cDNA reverse-transcription kit (Thermo Fisher Scientific, Waltham, MA, USA) and ddPCR was performed using 2X ddPCR Supermix (Bio-Rad, Hercules, CA, USA) on the QX-100 or QX-200 ddPCR system (Bio-Rad). All primer and probe sequences are listed in Table 1. BCR-ABL1 and reference genes were run as singleplex reactions in separate wells. To achieve optimal assay precision and avoid signal saturation, cDNA input for BCR-ABL1 per 20 μl reaction was 80 ng for the MR1 sample, 400 ng for the MR2 sample and 675 ng for samples MR3. cDNA input for reference genes was 10 ng per 20 μl reaction for all three reference genes. For each RT-ddPCR run, wells with >9025 accepted droplets were considered valid as per the manufacturer’s recommendations.

Table 1: Primer and probe sequences for the RT-ddPCR assays

Results

Manufacturing and IS calibration of the WHO BCR-ABL1 secondary reference panel

We successfully manufactured >12 000 vials of secondary BCR-ABL1 lyophilized cell reference panel, using the same K562 and HL-60 cell lines and following similar manufacturing procedures as the primary WHO panel (Supplementary Information).14 An MR4.5 level was added to the secondary panel, to enable more accurate IS calibration at this critical level, as CML patients reaching this deep molecular response are increasingly being considered for treatment cessation. Quality-control assessments indicated that the secondary panel had minimal residual moisture, excellent vial-to-vial homogeneity and >2.5 years of real-time stability (Supplementary Information).

To calibrate the secondary panel to the WHO ‘first International Genetic Reference Panel for quantitation of BCR-ABL1 mRNA’, we followed the study design described by White et al.,14 except that the sample size was doubled to strengthen the statistical power (Supplementary Information). RT-ddPCR was chosen as the calibration method, owing to its superior sensitivity, precision and absolute quantification capability compared with RT-qPCR.17, 18 At the time of this study, no commercially available BCR-ABL1 test used BCR or GUSB as reference genes. We developed three sets of RT-ddPCR assays, including BCR-ABL1/ABL1, BCR-ABL1/BCR and BCR-ABL1/GUSB, to enable IS calibration of the secondary panel against all three reference genes. All RT-ddPCR assays were validated following a combination of industry best practices, Minimum Information for Publication of Quantitative Real-Time PCR Experiments guideline, and Clinical and Laboratory Standards Institute guideline, to ensure proper accuracy, precision, sensitivity and linearity were achieved (Supplementary Information).19, 20, 21, 22 Using methods described by White et al.,14 we determined the IS CF for the RT-ddPCR assays to be 0.93 for BCR-ABL1/ABL1, 1.85 for BCR-ABL1/BCR and 1.28 for BCR-ABL1/GUSB.

Each CF was subsequently applied to the empirical %BCR-ABL1 of the secondary panel measured by RT-ddPCR, to obtain the assigned %BCR-ABL1IS values (Figure 1 and Table 2a). We found that the mean %BCR-ABL1 for each level of the secondary panel met all targeted BCR-ABL1 levels and were within 1.3-fold of the WHO standards values. The assigned %BCR-ABL1IS of level E was 0.0038%, 0.0050% and 0.0029% for ABL1, BCR and GUSB, respectively, indicating that an MR4.5 level was successfully created. Moreover, the mean copy number of BCR-ABL1, ABL1, BCR and GUSB per ng of RNA measured using RT-ddPCR was highly similar between the WHO and secondary panels (Table 2b). This demonstrated that the secondary panel replicated the primary WHO panel faithfully, with the successful addition of an MR4.5 level.

Figure 1
Figure 1

The BCR-ABL1 secondary reference panel was calibrated to the WHO standards using RT-ddPCR against (a) ABL1, (b) BCR and (c) GUSB for IS conversion (n=40 from MR1 to MR4, n=80 for MR4.5). The IS CFs for the RT-ddPCR assays were determined to be 0.93 for the BCR-ABL1/ABL1 assay, 1.85 for the BCR-ABL1/BCR assay and 1.28 for the BCR-ABL1/GUSB assay. Blue dotted lines represent the nominal %BCR-ABL1 value of the WHO panel at different levels.

Table 2: The BCR-ABL1 secondary reference panel was highly concordant with the WHO panel

Laboratory evaluation of the WHO BCR-ABL1 secondary reference panel

Study design

The secondary panel was sent to 44 clinical laboratories from 24 countries worldwide for evaluation, including 34 laboratories from Europe (Supplementary Table 2). One laboratory tested the panel with two different BCR-ABL1 assays, resulting in a total of 45 BCR-ABL1 tests included in this report. The laboratories were asked to conduct two studies with the secondary panel. In Study 1, to determine the optimal sample input of the secondary panel specific for each BCR-ABL1 test, a standard curve experiment was run with Vial A (MR1) and Vial C (MR3) of the panel using 50, 100, 200 and 400 ng of RNA (for one-step assays) or cDNA (for two-step assays) input per PCR reaction; three replicates were run per sample at each input level (Figures 2a and b). In Study 2, to assess the usability of the secondary panel and performance of the BCR-ABL1 tests, laboratories used the optimal sample input determined in Study 1 to test three sets of the panel on six different days (Figure 2c).

Figure 2
Figure 2

Study design for the international multi-center evaluation of the secondary panel, including (a) Study 1 for one-step RT-qPCR tests, (b) Study 1 for two-step RT-qPCR tests and (c) Study 2.

Study 1: sample input optimization

The WHO panel did not offer recommendations on sample input for IS calibration. Nonetheless, sample input outside of the linear dynamic range of a RT-qPCR assay might potentially lead to inaccurate results. Thus, we designed a standard curve experiment to help laboratories determine the optimal sample input of the secondary panel for their BCR-ABL1 tests (Figures 2a and b). Surprisingly, approximately half of the tests showed different %BCR-ABL1 results against different sample inputs of the same sample (P<0.05), even when data from either the highest or lowest sample input were allowed to be removed based on auxiliary-pick-regression analysis (Figure 3 and Supplementary Information). Among the 42 assays that tested Vial A (MR1), 29% (12 of 42) showed decreasing %BCR-ABL1 with increasing sample input (Figure 3d) and 24% (10 of 42) showed increasing %BCR-ABL1 (Figure 3g and Supplementary Figure 2). Among the 45 assays that tested Vial C (MR3), 22% (10 of 45) showed decreasing %BCR-ABL1 with increasing sample input (Figure 3d) and 18% (8 of 45) showed increasing %BCR-ABL1 (Figure 3g and Supplementary Figure 3).

Figure 3
Figure 3

Examples of different assay results from Study 1 of the multi-center evaluation study. A properly optimized assay should yield similar %BCR-ABL1 results at different sample inputs and both the BCR-ABL1 and reference gene assays should have a PCR efficiency close to 1 (ac). Some assays showed decreasing %BCR-ABL1 measurements with increasing sample input (df), whereas others showed increasing %BCR-ABL1 measurements instead (gi). The PCR efficiency of these assays tended to be suboptimal (<0.9 or >1.1), resulting in disproportional increase of BCR-ABL1 or reference gene copy number with increasing sample input (e, f, h and i), which subsequently led to the unstable %BCR-ABL1 measurements (d and g). Occasionally, BCR-ABL1 and reference gene assays that were suboptimal in a similar manner could cancel each other’s defects, to achieve artificially stable %BCR-ABL1 measurements (jl). Red lines represent the linear regression fit based on actual data and blue lines represent what ideal data should resemble.

In this study, we observed that the mean standard deviation (s.d.) in %BCR-ABL1 measurements from all 45 assays was 0.2log, which was mathematically equivalent to a 1.6-fold difference in the linear scale. Based on recommendations by Thiers et al.,23 1 s.d. (0.2log) was considered the optimal cutoff value for determining differences in measurements in this study. Selecting 1 s.d. as the cutoff value took into consideration the fact that a <1-s.d. cutoff value would require a substantially larger sample size to be considered statistically robust, whereas >1 s.d. would increase the number of misclassifications.23 Thus, in this study, a mean difference of 0.2log in %BCR-ABL1 value at different sample inputs by the same test was considered as beyond the inherent variability of an assay. We found that among the assays that showed changing %BCR-ABL1 against different sample inputs, 55% (12 of 22) at MR1 and 78% (14 of 18) at MR3 obtained results with 0.2log difference. Overall, these results showed that some BCR-ABL1 tests were nonlinear and might therefore yield statistically different %BCR-ABL1 results against different sample inputs. To mitigate the risk of inaccurate %BCR-ABL1 measurements from using an inappropriate sample input, it is highly recommended that laboratories standardize CML patient sample inputs by quantifying the extracted RNA before performing RT-qPCR.

To further investigate the cause of the unstable %BCR-ABL1 measurements across different sample inputs, we calculated the PCR efficiency and efficiency ratio for each individual BCR-ABL1 and reference gene assay using the formula ‘Efficiency=−1+10(−1/slope)’ (Supplementary Information).24 A well-optimized PCR assay should have PCR efficiency between 0.9 and 1.1,25 which would result in a PCR efficiency ratio of ~1 between the BCR-ABL1 and reference gene assays. Indeed, we found that most assays that successfully achieved stable %BCR-ABL1 across different sample inputs had a PCR efficiency close to 1 for both the BCR-ABL1 and reference gene assays, resulting in a mean efficiency ratio of 1.03 (n=43; abnormal efficiency ratios of <0 and >10 were excluded from the analysis) (Figures 3a–c). Assays that showed decreasing %BCR-ABL1 against increasing sample input had a mean PCR efficiency ratio of 1.51 (n=20) (Figures 3d–f), whereas assays that showed increasing %BCR-ABL1 had a mean PCR efficiency ratio of 0.63 (n=16) (Figures 3g–i). This indicated that the lack of stability in %BCR-ABL1 against sample input was directly correlated with suboptimal PCR efficiency. Surprisingly, BCR-ABL1 and reference gene assays that were suboptimal in a similar manner could artificially cancel each other’s defects to achieve artificially stable %BCR-ABL1 results (Figures 3j–l). Overall, these results illustrated that both the BCR-ABL1 and reference gene assays needed to be properly optimized and validated, in order to achieve good quality %BCR-ABL1 testing.26, 27, 28

As different assays had different linear dynamic ranges, we observed a >30-fold range of optimal sample input for the secondary panel calculated for the different BCR-ABL1 tests. Interestingly, among the 31 laboratories that routinely quantified their CML patient sample inputs, the reported patient sample input was on average 2.4-fold higher than the calculated optimal input of the secondary panel, after two extreme outlier values of 11.6- and 48.3-fold were identified using robust regression analysis and excluded from the calculation (Supplementary Information). This was concordant with the fact that although the mean copy number of ABL1, BCR and GUSB per ng of RNA in the secondary panel was 674, 1028 and 1245 (Table 2b), the mean copy per ng of human EDTA anticoagulated blood RNA was only 289 for ABL1 (n=40), 750 for BCR (n=6) and 420 for GUSB (n=5) (data not shown). These results indicated that the optimal sample input of the secondary panel was approximately half of the patient sample input typically used by the laboratory in terms of ng RNA or cDNA. Thus, when using the secondary panel, laboratories might consider using twofold less cDNA input per PCR reaction compared with CML patient samples, to achieve similar copy numbers and avoid exceeding the linear dynamic range of their assay. Nonetheless, it is recommended that laboratories perform a similar standard curve experiment to identify the optimal sample input specific for their assay before using the secondary panel for the first time. The optimized sample input should maximize copy number detection but minimize potential PCR inhibition caused by carryover from the reverse-transcription reactions.

Study 2: performance of clinical BCR-ABL1 tests on the secondary panel

A robust BCR-ABL1 test should demonstrate good IS accuracy, precision and sensitivity within statistical limits, especially at the lower disease levels. In Study 2, laboratories were asked to use the optimal sample input determined in Study 1 to test three sets of the secondary panel on six different days, following the design previously used by White et al. (Figure 2c).14 Results from each assay were subsequently analyzed to assess accuracy, precision and sensitivity. For IS accuracy, the overall mean %BCR-ABL1 from all 45 assays, calculated using robust regression analysis to minimize effects of outliers, were highly concordant with the assigned values of the secondary panel (Table 3 and Supplementary Figure 5). The mean ratios for all reference genes at all levels were within 1.5-fold or 0.16log of the panel’s assigned values, with the exception of Vial D (MR4) for GUSB, which differed by 1.7-fold/0.24log. For results from individual assays, 60% (27 of 45) obtained mean %BCR-ABL1 within twofold of the panel’s assigned values at all levels (Supplementary Figure 5). The observed concordance was most likely due to three factors. First, optimal sample input calculated in Study 1 was used for each assay, thus restricting the PCR reactions within the assay’s linear dynamic range. Second, even though many different assay configurations were used by the laboratories (Supplementary Table 2), 60% (27 of 45) followed the EAC recommendations for the PCR primer design26 and 18% (8 of 45) used commercial BCR-ABL1 kits (Supplementary Table 2 and Supplementary Figure 5), which are generally well optimized and validated. Lastly, 51% (23 of 45) of assays were IS calibrated via sample exchange with the reference laboratory in Mannheim, Germany, and 20% (9 of 45) with Adelaide, Australia. This demonstrated that BCR-ABL1 tests can be effectively harmonized by using the same PCR primer designs and by standardizing the IS calibration process. Thus, commercial availability of a common IS reference material could contribute to worldwide IS standardization.

Table 3: Summary results of the secondary panel international multi-center evaluation study

Most clinical samples are typically run in only one or two replicates, which requires a high degree of assay precision to ensure accuracy of each of the final BCR-ABL1 test results. To enable 95% confidence for the true value of a CML patient sample to be within 0.5log on each side of the measured value, an s.d. of 0.25log for the BCR-ABL1 test is required. Accordingly, if a sample is measured at MR4.5, there will be 95% confidence that the true value of the sample is not above MR4 or below MR5.29 We calculated the intra-lab s.d. for each BCR-ABL1 test and noted that 84% (38 of 45) successfully achieved an s.d. of 0.25 log from MR1 to MR4. For BCR-ABL1 tests that obtained >0.25log s.d., precision may be improved by performing further assay optimization and increasing the number of replicates per patient sample.28, 30

For monitoring deep molecular response, good BCR-ABL1 assay sensitivity and precision are key performance characteristics. In Study 2, we found that 93% (42 of 45) assays successfully detected all replicates at MR4 and 76% (34 of 45) detected all replicates at MR4.5. To further understand how sample input affected assay sensitivity and precision, we analyzed results from the 32 ABL1 assays, as this provided the largest sample size. Among the ABL1 assays, 56% (18 of 32) detected all replicates at MR4.5 and achieved 0.25log s.d., 19% (6 of 32) detected all replicates but had >0.25log s.d. and 25% (8 of 32) assays had at least 1 undetected replicate. Logistic regression analysis showed a strong positive correlation between increased sample input and increased detection rate (P=0.001). In addition, the median ABL1 copy number per PCR reaction was 98 202 for laboratories that achieved good detection rate and precision, 74 923 for those that achieved good detection rate but suboptimal precision and 29 717 for those that had undetected replicates at MR4.5, further illustrating that an increased sample input could improve the sensitivity and precision of MR4.5 detection.

The laboratories that participated in this study used diverse assay configurations for BCR-ABL1 testing (Supplementary Table 2). To determine whether assay configurations affected assay performance in terms of IS accuracy, precision and sensitivity, we performed Bayesian average analysis and found no statistically significant relationship between assay performance versus choice of reference gene, RNA extraction method and the usage of commercial versus laboratory-developed tests. Interestingly, among the assays that successfully achieved good accuracy (within twofold of assigned values), good precision (0.25log s.d.) and good MR4.5 sensitivity (no undetected replicate), 82% (14 of 17) showed stable %BCR-ABL1 measurements against sample inputs at MR3 in Study 1, compared with 46% (13 of 28) among assays with less optimal assay performance. This illustrated that although BCR-ABL1 assays of different designs can perform equally well, proper PCR optimization is required to ensure good clinical performance. Interestingly, the number of assays that achieved good precision (84%) and MR4.5 sensitivity (76%) exceeded the number that achieved good IS accuracy (60%), indicating that there remains an unmet need for a simple and broadly available IS calibration mechanism.

Calculating WHO IS CF from the secondary panel

Using the assigned BCR-ABL1 IS values of the secondary panel, it was possible to calculate an IS CF traceable to the WHO panel for each test. Before CF calculation, Bland–Altman analysis was performed for each assay as described by White et al.,14 to assess whether bias was linear across the %BCR-ABL1 range. Of all the assays, 69% (31 of 45) showed no statistically significant trend in the Bland–Altman analysis, indicating that a valid CF could be calculated for these assays and compared with the laboratory’s current CF, most of which were obtained from sample exchange. Mathematically, when the ratio between the assay’s current CF and the CF calculated from the secondary panel is <0.63 or >1.58, the resulting %BCR-ABL1 values will have a >0.2log difference (Supplementary Information). Among the 31 assays, 58% (18 of 31) achieved a CF ratio between 0.63 and 1.58, indicating that their current CF was equivalent to the CF from the secondary panel. This was concordant with the fact that 60% of assays obtained mean %BCR-ABL1 within twofold of the panel’s assigned values. This also indicated that the secondary panel can be effectively used to obtain IS CFs equivalent to sample exchange.

Discussions

In this study, we successfully created the first cell-based BCR-ABL1 secondary reference panel that faithfully replicated the WHO BCR-ABL1 International Genetic Reference Panel in both raw material and manufacturing methods,14 with an additional MR4.5 level. The secondary panel was manufactured under Good Manufacturing Practice and confirmed to have low residual moisture content, vial-to-vial homogeneity and >2.5-year stability. BCR-ABL1 IS values traceable to the WHO panel were assigned to the secondary panel using RT-ddPCR against ABL1, BCR and GUSB. Both the assigned IS values and absolute copy numbers of the secondary panel were found to be highly concordant with the primary WHO standards.

Many of the 44 laboratories that participated in the secondary panel evaluation currently act as a national reference laboratory for their country. The panel was successfully processed by all laboratories, indicating that it is compatible with many different BCR-ABL1 test configurations. Through a standard curve experiment, we found that close to half of the tests showed signs of inadequate PCR optimization such as poor linearity against different sample inputs and suboptimal PCR efficiency. Interestingly, when a customized optimal sample input was used, 60% (27 of 45) of assays achieved mean %BCR-ABL1 values within twofold of the panel’s assigned values, 84% (38 of 45) achieved good precision ( 0.25log s.d.) from MR1 to MR4 and 76% (34 of 45) achieved 100% detection rate down to MR4.5. Three factors probably contributed to these excellent results: usage of a validated optimized sample input specific to the assay, the fact that 78% (35 of 45) assays used either the EAC primer design or a commercial kit and that 71% (32 of 45) assays were IS calibrated via sample exchange with one of the two major international reference centers. This indicated that using published assay designs and a harmonized IS calibration approach may lead to BCR-ABL1 test standardization. Nonetheless, we noted that the number of assays that achieved good precision and sensitivity exceeded the number that achieved good IS accuracy, indicating that there remains an unmet need for a simple and broadly available calibration mechanism, such as this secondary panel, to ensure IS accuracy is maintained in laboratories over time.

We also showed that different assay designs including different reference genes, RNA extraction methods and usage of commercial kits versus laboratory-developed tests did not affect assay performance. Nonetheless, better PCR optimization correlated with better assay performance and increased sample input improved detection rate and precision at MR4.5. In addition to being a reference sample for IS calibration,14 the secondary reference material and its derivatives could also be used in assay analytical validation and optimization. For example, it can be used as standardized samples in External Quality Assessment programs for proficiency testing,31 especially as nominal values (that is, correct answers) are available for each member of the panel. Second, it can become a source of positive control samples to be run alongside patient samples for quality assurance. This can become especially powerful when multiple laboratories participate in a peer group quality-control monitoring program, in which results from such positive control samples are compared and monitored regularly. Lastly, the MR4.5 sample in the reference panel can be used to validate the sensitivity and MR4.5 detection capability of an assay.

In conclusion, a secondary reference panel traceable to the WHO panel with an additional MR4.5 level can provide easier access to IS calibration, as well as act as a tool for assay optimization, validation and quality assurance.

References

  1. 1.

    , , , , , et al. Chronic myeloid leukemia: an update of concepts and management recommendations of European LeukemiaNet. J Clin Oncol 2009; 27: 6041–6051.

  2. 2.

    , , , , , et al. European LeukemiaNet recommendations for the management of chronic myeloid leukemia: 2013. Blood 2013; 122: 872–884.

  3. 3.

    , , , , , et al. Deep molecular response is reached by the majority of patients treated with imatinib, predicts survival, and is achieved more quickly by optimized high-dose imatinib: results from the randomized CML-study IV. J Clin Oncol 2014; 32: 415–423.

  4. 4.

    , , , , , et al. Assessment of BCR-ABL1 transcript levels at 3 months is the only requirement for predicting outcome for patients with chronic myeloid leukemia treated with tyrosine kinase inhibitors. J Clin Oncol 2012; 30: 232–238.

  5. 5.

    , , , , , et al. Early molecular response predicts outcomes in patients with chronic myeloid leukemia in chronic phase treated with frontline nilotinib or imatinib. Blood 2014; 123: 1353–1360.

  6. 6.

    , , , , , et al. BCR-ABL messenger RNA levels continue to decline in patients with chronic phase chronic myeloid leukemia treated with imatinib for more than 5 years and approximately half of all first-line treated patients have stable undetectable BCR-ABL using strict sensitivity criteria. Clin Cancer Res 2007; 13: 7080–7085.

  7. 7.

    , , , , , et al. Chronic myelogenous leukemia, version 1.2015. J Natl Compr Canc Netw 2014; 12: 1590–1610.

  8. 8.

    , , , , , et al. Harmonization of BCR-ABL mRNA quantification using a uniform multifunctional control plasmid in 37 international laboratories. Leukemia 2008; 22: 96–102.

  9. 9.

    , , , , , et al. Harmonization of molecular monitoring of CML therapy in Europe. Leukemia 2009; 23: 1957–1963.

  10. 10.

    , , , , , et al. Frequency of major molecular responses to imatinib or interferon alfa plus cytarabine in newly diagnosed chronic myeloid leukemia. N Engl J Med 2003; 349: 1423–1432.

  11. 11.

    , , , , , et al. Monitoring CML patients responding to treatment with tyrosine kinase inhibitors: review and recommendations for harmonizing current methodology for detecting BCR-ABL transcripts and kinase domain mutations and for expressing results. Blood 2006; 108: 28–37.

  12. 12.

    , , , , , et al. Desirable performance characteristics for BCR-ABL measurement on an international reporting scale to allow consistent interpretation of individual patient response and comparison of response rates between clinical trials. Blood 2008; 112: 3330–3338.

  13. 13.

    , , . Molecular monitoring of chronic myeloid leukemia: principles and interlaboratory standardization. Ann Hematol 2015; 94 (Suppl 2): S219–S225.

  14. 14.

    , , , , , et al. Establishment of the first World Health Organization International Genetic Reference Panel for quantitation of BCR-ABL mRNA. Blood 2010; 116: e111–e117.

  15. 15.

    , , , , , et al. Establishment of a standardized multiplex assay with the analytical performance required for quantitative measurement of BCR-ABL1 on the international reporting scale. Blood Cancer J 2011; 1: e13.

  16. 16.

    ELITechGroup. Philadelphia P210 RNA reference package insert, 2013.

  17. 17.

    , , , . Comparison of next-generation droplet digital PCR (ddPCR) with quantitative PCR (qPCR) for enumeration of Cryptosporidium oocysts in faecal samples. Int J Parasitol 2014; 44: 1105–1113.

  18. 18.

    , , . Comparison of droplet digital PCR and quantitative real-time PCR for examining population dynamics of bacteria in soil. Appl Microbiol Biotechnol 2014; 98: 6105–6113.

  19. 19.

    , , , , , et al. The digital MIQE guidelines: minimum information for publication of quantitative digital PCR experiments. Clin Chem 2013; 59: 892–902.

  20. 20.

    Clinical and Laboratory Standards InstituteEvaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline, 2nd edn. Clinical and Laboratory Standards Institute: Wayne, PA, 2012.

  21. 21.

    Clinical and Laboratory Standards InstituteEvaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline, 2nd edn. Clinical and Laboratory Standards Institute: Wayne, PA, 2004.

  22. 22.

    Clinical and Laboratory Standards InstituteEvaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline. Clinical and Laboratory Standards Institute: Wayne, PA, 2003.

  23. 23.

    , , , . Sample stability: a suggested definition and method of determination. Clin Chem 1976; 22: 176–183.

  24. 24.

    . Quantification on the Light Cycler. Rapid Cycle Real-Time PCR. Springer, 2001; pp 21–34.

  25. 25.

    , , , . Checklist for optimization and validation of real-time PCR assays. J Clin Lab Anal 2009; 23: 145–151.

  26. 26.

    , , , , , et al. Standardization and quality control studies of 'real-time' quantitative reverse transcriptase polymerase chain reaction of fusion gene transcripts for residual disease detection in leukemia - a Europe Against Cancer program. Leukemia 2003; 17: 2318–2357.

  27. 27.

    , . Diagnosis and monitoring of chronic myeloid leukemia by qualitative and quantitative RT-PCR. Methods Mol Med 2006; 125: 69–92.

  28. 28.

    , , , , , Molecular Oncology Resource Committee of the College of American Pathologists. Design and analytic validation of BCR-ABL1 quantitative reverse transcription polymerase chain reaction assay for monitoring minimal residual disease. Arch Pathol Lab Med 2012; 136: 33–40.

  29. 29.

    Qiagen Inc. Summary of Safety and Effectiveness Data (SSED) of Artus CMV RGQ MDx Kit. U.S. Food and Drug Administration, 2014.

  30. 30.

    , , , , , et al. Laboratory recommendations for scoring deep molecular responses following treatment for chronic myeloid leukemia. Leukemia 2015; 29: 999–1003.

  31. 31.

    , , , , . BCR-ABL1 Quantitation: To IS or not to IS, that is the question? Haematologica 2011; 96 (Suppl 2): S82.

Download references

Acknowledgements

We thank Dr James Garrett (Novartis Pharmaceuticals Corporation) for assisting with the biostatistical analysis in this report and Dr Nancy Krunic (Novartis Pharmaceuticals Corporation) for reviewing and providing editorial assistance with this manuscript.

Author information

Affiliations

  1. Wessex Regional Genetics Laboratory, Salisbury NHS Foundation Trust, Salisbury, UK

    • N C P Cross
    •  & H E White
  2. Faculty of Medicine, University of Southampton, Southampton, UK

    • N C P Cross
    •  & H E White
  3. Department of Hematology/Oncology, Universitätsklinikum Jena, Jena, Germany

    • T Ernst
    •  & A Hochhaus
  4. Department of Genetics and Molecular Pathology, Centre for Cancer Biology, SA Pathology, Adelaide, SA, Australia

    • L Welden
    •  & S Branford
  5. III. Medizinische Klinik, Medizinische Fakultät Mannheim, Universität Heidelberg, Mannheim, Germany

    • C Dietz
    •  & M C Müller
  6. Department of Clinical and Biological Sciences, San Luigi Hospital, University of Turin, Orbassano, Italy

    • G Saglio
  7. Bergonie Institute Cancer Center Bordeaux, INSERM U1218, University of Bordeaux, Bordeaux, France

    • F-X Mahon
  8. Novartis Pharmaceuticals Corporation, East Hanover, NJ, USA

    • C C Wong
    • , D Zheng
    • , S Wong
    •  & S-S Wang
  9. West Midlands Regional Genetics Laboratory, Birmingham, UK

    • S Akiki
  10. Department of Hematology, University of Bari, Bari, Italy

    • F Albano
  11. Laboratory of Molecular Diagnostics, Hungarian National Blood Transfusion Service, Budapest, Hungary

    • H Andrikovics
  12. Department of Pathophysiology, Semmelweis University, Budapest, Hungary

    • H Andrikovics
  13. King’s College Hospital London, London, UK

    • J Anwar
  14. National Specialized Hospital for Active Treatment of Hematological Diseases, Sofia, Bulgaria

    • G Balatzenko
  15. Laboratorio de Biologia Tumoral, Disciplina de Hematologia do HC-FMUSP, São Paulo, Brazil

    • I Bendit
  16. PathWest Laboratory Medicine WA, Department of Haematology, Fiona Stanley Hospital, Perth, WA, Australia

    • J Beveridge
  17. Department of Laboratory Medicine, University Hospitals Leuven, Leuven, Belgium

    • N Boeckx
  18. Department of Oncology, KUL, Leuven, Belgium

    • N Boeckx
  19. Department of Genetics, Portuguese Oncology Institute, Porto, Portugal

    • N Cerveira
  20. Department of Hematology and Oncology, Quest Diagnostics Nichols Institute, San Juan Capistrano, CA, USA

    • S-M Cheng
  21. Hematopathology Unit, Hospital Clinic, IDIBAPS, Barcelona, Spain

    • D Colomer
  22. Division of Molecular Microbiology, Children’s Cancer Research Institute, Vienna, Austria

    • S Czurda
  23. Department of Clinical and Biological Sciences, San Luigi Hospital, University of Turin, Orbassano, Italy

    • F Daraio
  24. Laboratoire Hematologie, Centre Hospitalier Universitaire de Bordeaux, Universite Bordeaux, Bordeaux, France

    • S Dulucq
  25. Laboratory of Molecular Pathology, Oslo University Hospital, Oslo, Norway

    • L Eggen
  26. Clinique de Genetique Oncologique-Service de genetique, Hopital Erasme, Brussels, Belgium

    • H El Housni
  27. Imperial Molecular Pathology, Hammersmith Hospital, London, UK

    • G Gerrard
  28. Department of Hematology and Bone Marrow Transplantation, Poznan University of Medical Sciences, Poznan, Poland

    • M Gniot
  29. Department of Clinical Medicine and Surgery, University ‘Federico II’ of Naples, Naples, Italy

    • B Izzo
  30. CEINGE – Biotecnologie Avanzate, Naples, Italy

    • B Izzo
  31. Genoptix, Inc., Carlsbad, CA, USA

    • D Jacquin
    •  & H Mobtaker
  32. Department of Hematology and Molecular Diagnostics, VU University Medical Center, Amsterdam, The Netherlands

    • J J W M Janssen
  33. MLL Munich Leukemia Laboratory, Munich, Germany

    • S Jeromin
  34. Center of Molecular Biology and Gene Therapy, Department of Internal Medicine-Hematology and Oncology, Masaryk University and University Hospital Brno, Brno, Czech Republic

    • T Jurcek
  35. Seoul St Mary’s Hospital, Leukemia Research Institute, The Catholic University of Korea, Seoul, Korea

    • D-W Kim
  36. Department of Molecular Genetics, Institute of Hematology and Blood Transfusion, Prague, Czech Republic

    • K Machova-Polakova
  37. Department of Hematology, Hospital Universitario 12 de Octubre, Universidad Complutense, CNIO, Madrid, Spain

    • J Martinez-Lopez
  38. Division of Cancer Medicine, Department of Pathology, Peter MacCallum Cancer Centre, East Melbourne, VIC, Australia

    • M McBean
  39. Pathology Department, University Clinical Center Tuzla, Tuzla, Bosnia and Herzegovina

    • S Mesanovic
  40. Department of Laboratory Medicine, Division of Medical and Chemical Laboratory Diagnostics, Medical University of Vienna, Vienna, Austria

    • G Mitterbauer-Hohendanner
  41. Departement de Biopathologie, Institut Paoli-Calmettes, Marseille, France

    • M-J Mozziconacci
  42. Specialized Haematology Laboratory, Department of Haematology, University Medical Centre Ljubljana, Ljubljana, Slovenia

    • T Pajič
  43. Department of Surgical Pathology, Zealand University Hospital, Roskilde, Denmark

    • N Pallisgaard
  44. Hematology Unit, First Department of Internal Medicine, Laiko Hospital, University of Athens, Athens, Greece

    • P Panagiotidis
  45. Department of Pathology and Knight Cancer Institute, Oregon Health and Science University, Portland, OR, USA

    • R D Press
  46. Peking University People’s Hospital, Peking University Institute of Hematology, Beijing, China

    • Y-Z Qin
  47. Fred Hutchinson Cancer Research Center, Seattle, WA, USA

    • J Radich
  48. Chair and Department of Hematology, Jagiellonian University, Kraków, Poland

    • T Sacha
  49. Laboratory of Molecular Biology, Hematology Department and HCT Unit, G. Papanicolaou Hospital, Thessaloniki, Greece

    • T Touloumenidou
  50. Bristol Genetics Laboratory, Bristol, UK

    • P Waits
  51. HMDS, Leeds Teaching Hospitals, Leeds, UK

    • E Wilkinson
  52. Faculty of Pharmacy and Biochemistry and University Hospital Center Zagreb, University of Zagreb, Zagreb, Croatia

    • R Zadro
  53. School of Pharmacy and Medical Science, University of South Australia, Adelaide, SA, Australia

    • S Branford
  54. School of Medicine, University of Adelaide, SA, Adelaide, Australia

    • S Branford
  55. School of Molecular and Biomedical Science, University of Adelaide, Adelaide, SA, Australia

    • S Branford

Authors

  1. Search for N C P Cross in:

  2. Search for H E White in:

  3. Search for T Ernst in:

  4. Search for L Welden in:

  5. Search for C Dietz in:

  6. Search for G Saglio in:

  7. Search for F-X Mahon in:

  8. Search for C C Wong in:

  9. Search for D Zheng in:

  10. Search for S Wong in:

  11. Search for S-S Wang in:

  12. Search for S Akiki in:

  13. Search for F Albano in:

  14. Search for H Andrikovics in:

  15. Search for J Anwar in:

  16. Search for G Balatzenko in:

  17. Search for I Bendit in:

  18. Search for J Beveridge in:

  19. Search for N Boeckx in:

  20. Search for N Cerveira in:

  21. Search for S-M Cheng in:

  22. Search for D Colomer in:

  23. Search for S Czurda in:

  24. Search for F Daraio in:

  25. Search for S Dulucq in:

  26. Search for L Eggen in:

  27. Search for H El Housni in:

  28. Search for G Gerrard in:

  29. Search for M Gniot in:

  30. Search for B Izzo in:

  31. Search for D Jacquin in:

  32. Search for J J W M Janssen in:

  33. Search for S Jeromin in:

  34. Search for T Jurcek in:

  35. Search for D-W Kim in:

  36. Search for K Machova-Polakova in:

  37. Search for J Martinez-Lopez in:

  38. Search for M McBean in:

  39. Search for S Mesanovic in:

  40. Search for G Mitterbauer-Hohendanner in:

  41. Search for H Mobtaker in:

  42. Search for M-J Mozziconacci in:

  43. Search for T Pajič in:

  44. Search for N Pallisgaard in:

  45. Search for P Panagiotidis in:

  46. Search for R D Press in:

  47. Search for Y-Z Qin in:

  48. Search for J Radich in:

  49. Search for T Sacha in:

  50. Search for T Touloumenidou in:

  51. Search for P Waits in:

  52. Search for E Wilkinson in:

  53. Search for R Zadro in:

  54. Search for M C Müller in:

  55. Search for A Hochhaus in:

  56. Search for S Branford in:

Competing interests

This study was designed and funded by Novartis Pharmaceuticals Corporation. CW, DZ, SW and SSW are employed by Novartis Pharmaceuticals Corporation. None of the co-authors, participating laboratories or institutions received any payments for participating in this study. All authors have read and agreed with the contents of the manuscript. All other authors declare no conflicts of interest.

Corresponding author

Correspondence to C C Wong.

Supplementary information

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/leu.2016.90

Supplementary Information accompanies this paper on the Leukemia website (http://www.nature.com/leu)

Further reading