Widespread use of incorrect PCR ramp rate negatively impacts multidrug-resistant tuberculosis diagnosis (MTBDRplus)

The scale-up of rapid drug resistance testing for TB is a global priority. MTBDRplus is a WHO-endorsed multidrug-resistant (MDR)-TB PCR assay with suboptimal sensitivities and high indeterminate rates on smear-negative specimens. We hypothesised that widespread use of incorrect thermocycler ramp rate (speed of temperature change between cycles) impacts performance. A global sample of 72 laboratories was surveyed. We tested 107 sputa from Xpert MTB/RIF-positive patients and, separately, dilution series of bacilli, both at the manufacturer-recommended ramp rate (2.2 °C/s) and the most frequently reported incorrect ramp rate (4.0 °C/s). Mycobacterium tuberculosis-complex DNA (TUB-band)-detection, indeterminate results, accuracy, and inter-reader variability (dilution series only) were compared. 32 respondents did a median (IQR) of 41 (20–150) assays monthly. 78% used an incorrect ramp rate. On smear-negative sputa, 2.2 °C/s vs. 4.0 °C/s improved TUB-band positivity (42/55 vs. 32/55; p = 0.042) and indeterminate rates (1/42 vs. 5/32; p = 0.039). The actionable results (not TUB-negative or indeterminate; 41/55 vs. 28/55) hence improved by 21% (95% CI: 9–35%). Widespread use of incorrect ramp rate contributes to suboptimal MTBDRplus performance on smear-negative specimens and hence limits clinical utility. The number of diagnoses (and thus the number of smear-negative patients in whom DST is possible) will improve substantially after ramp rate correction.

The latest iteration of MTBDRplus (version 2) was designed to have improved sensitivity on specimens, irrespective of smear status, and culture isolates. MTBDRplus's follow-on test for second-line resistance (MTBDRsl; Hain Lifescience, Germany) is based on similar principles and also WHO-endorsed 9,16,21,22 .
MTBDRplus requires thermocycling to amplify DNA. The manufacturer recommends a ramp rate (speed of temperature change between cycles) of ≤ 2.2 °C/s 8 , which the thermocycler they sell (the GTC-cycler) is capable of. Laboratories can use their own thermocyclers, however, these thermocyclers may have different default ramp rates or, in cheaper models, may not permit ramp rate to be changed. None of the studies in a recent systematic review and meta-analysis of MTBDRplus report ramp rate and few studies reported rates of TUB-band positivity 16,23 . If an assay is TUB-band-negative, susceptibility results cannot, per the manufacturer's recommendation, be reported 8 and studies that do not report TUB-band positivity rates do not provide a complete characterisation of test performance.
We hypothesised that suboptimal sensitivities and high indeterminate rates reported for MTBDRplus on smear-negative specimens [12][13][14][15][16][17] were partly associated with incorrect ramp rate. If this phenomenon is widespread, it may explain a major limitation in the routine diagnosis of MDR-TB, for which MTBDRplus is the only commercially available molecular assay. This could result in large numbers of possible MDR-TB diagnoses being missed, exacerbate diagnostic delay, and will have implications for diagnostic algorithms (e.g., confirmation of Xpert-indicated rifampicin-resistance, detection of rifampicin or isoniazid mono-resistance), clinical practice (e.g., detection of acquired resistance during treatment monitoring), and research studies (e.g., MDR-TB drug trials that need to rapidly screen patients).

Ethics statement. This study was approved by the Health Research Ethics Committee of Stellenbosch
University (N09-11-296) and done in accordance with these relevant guidelines and regulations. Permission was granted by the institutional review board (IRB) to access anonymised residual specimens collected as part of routine diagnostic practice and thus patient informed consent was waived.
Survey of diagnostic and research laboratories. An invitation to an online survey was sent to 74 laboratories using MTBDRplus identified from a recent systematic review and meta-analysis 16 , expert consultation, the Global Laboratory Initiative, the Global Health Delivery network, and FIND. We placed no restrictions on the type of facility or country that could respond. Initial non-responders were emailed at least a further three times. Questions included country, average number of MTBDRplus assays per month, primary purpose of the assay, specimen smear status, models of thermocyclers, whether the thermocycler permitted ramp rate to be changed, and the MTBDRplus ramp rate used (the full questions are listed in the supplement). Permission was obtained from respondents to use their anonymised data for publication.
Specimen collection and decontamination. 107 de-identified sputa consecutively submitted to an accredited government quality-assured (South African National Accreditation System) laboratory in Cape Town, South Africa were collected. Sputa were from patients with symptoms of TB who were, using a separate paired specimen, Xpert MTB/RIF (Xpert)-positive for TB and rifampicin-susceptible or -resistant. Sputa were decontaminated with NaOH-N-Acetyl-L-Cysteine (1% final concentration) 24 . Each decontaminated sediment had ~50 µl used for Auramine-O 25 smear microscopy and, if the paired specimen was Xpert rifampicin-resistant, ~500 µl used for culture for DST. 52 sputa were smear-positive (26 Xpert-rifampicin resistant, 26 Xpert-rifampicin susceptible) and 55 smear-negative (39 Xpert-rifampicin resistant, 16 Xpert-rifampicin susceptible). The sediments were stored at 4 °C for 2-3 days prior to transport to Stellenbosch University for DNA extraction.
Impact of thermocycler ramp rate on MTBDRplus performance in clinical specimens. DNA was extracted from sediments using the GenoLyse kit (Hain Lifescience, Germany) 8 . DNA was amplified using two ramp rates: the manufacturer-recommended ramp rate (2.2 °C/s), and 4.0 °C/s, the most frequently used incorrect ramp rate in the survey, using a CFX96 (Bio-Rad, United States), which was the only machine available with a customisable ramp rate. This instrument undergoes annual servicing and calibration by the manufacturer. Hybridisation was done with the GT-Blot 48 (Hain Lifescience, Germany) 26 . An experienced reader interpreted bands in a blinded manner.
Impact of thermocycler ramp rate on MTBDRplus performance in a dilution series. A drug-susceptible strain (H37Rv, ATCC 25618) and a phenotypically-confirmed clinical MDR strain (with known rpoB, katG, and inhA promoter SNPs) were grown to mid-exponential phase in Middlebrook 7H9 media (Becton Dickinson, United States) supplemented with Middlebrook Oleic Albumin Dextrose Catalase supplement (Becton Dickinson, United States). Colony counts after incubation on Middlebrook 7H10 media (Becton Dickinson, United States) for 21 days at 37 °C were done. This experiment was done in triplicate. MTBDRplus was done on dilutions of 10 2 , 10 3 and 10 4 CFU/ml in phosphate buffer with 0.025% Tween 80. 10 4 CFU/ml corresponds approximately to smear-positivity 27 and the lower concentrations in the dilution series correspond to paucibacillary smear-negative disease (i.e., the patients we hypothesise ramp rate to impact the most). The CFX96 machine with ramp rates of 2.2 °C/s or 4.0 °C/s was used. An experienced reader interpreted bands in a blinded manner.

Assessment of inter-reader variability.
MTBDRplus strips from the dilution series were interpreted by two experienced technicians in a blinded manner. Variability between readers (individual banding patterns, final diagnostic classifications) was assessed. When a strip is interpreted, a banding call determination is made if a specific band is present or absent; whereas a diagnostic call (susceptibility or resistance to rifampicin and/or isoniazid) is based on the overall banding pattern. Hence, banding patterns may change but not the diagnostic call. as the presence of the TUB-band with the amplification and conjugation control bands. Sensitivity for M. tuberculosis-complex DNA was calculated using a paired MGIT960 liquid culture (Becton Dickinson, United States) result from the national laboratory as a reference standard. A strip was classified as indeterminate if the amplification or conjugate control bands were absent but any other bands were present. A drug indeterminate result was defined as the absence of any locus control band (rpoB, katG and inhA) on a TUB-band-positive strip. A result was classified as actionable if the strip was TUB-band-positive and not indeterminate for any drugs.

Statistical analyses. The two sample test of proportion was used for comparisons between proportions,
and McNemar's test was used to calculate differences in sensitivity or indeterminate rates across ramp rates for paired data. We used the percent improvement in actionable results (calculated from our clinical specimen experiment, 21%) to estimate the number of additional TUB-band-positive diagnoses (and MDR-TB diagnoses) in survey respondents who said they tested smear-negative specimens. For this calculation, we assumed 1) the volume of assays done by the respondent was evenly spread across input material types (e.g., a respondent doing 100 MTBDRplus assays per month does ~33 smear-positive, smear-negative, and isolates; we unfortunately did not retrieve specific data on the monthly volume of smear-negative specimens only), 2) the MDR-TB prevalence in smear-negative specimens corresponded to the overall WHO estimate for the respondent's country, and 3) ramp rate changes would equally affect resistance and susceptibility detection. We used GraphPad Prism version 6.0 (GraphPad Software) and Stata version 14 (StataCorp) software. All statistical tests are 2-sided at α = 0.05. Data availability. The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Performance of MTBDRplus at different ramp rates on dilution series of bacilli.
Each of the three technical replicates for each strain in the dilution series (10 2 , 10 3 and 10 4 CFU/ml) were TUB-band-positive and there were no indeterminate results, irrespective of ramp rate. At 4.0 °C/s, the drug-susceptible strain gave a false-positive rifampicin-resistance result in a 10 2 CFU/ml replicate, however, at higher concentrations all results were true-susceptible. Overall, bands at 2.2 °C/s were subjectively interpreted as being darker, clearer, and more distinct than those at 4.0 °C/s by the experienced readers.

Assessment of inter-reader agreement on dilution series. Banding and diagnostic calls differed
between readers and were most pronounced at 10 2 CFU/ml (Table 3)

Discussion
Our key findings are: 1) the vast majority of survey respondents, who are globally diverse and do a large volume of MTBDRplus assays, use an incorrect ramp rate and this, 2) decreases sensitivity for TB (and hence precludes resistance detection), 3) increases indeterminate rates in smear-negative specimens, and 4) likely increases false-resistance calls and banding pattern disagreement between readers. These findings are of clinical relevance as most respondents used this assay routinely, indicating that incorrect ramp rate usage is likely affecting patient diagnoses.
To the best of our knowledge, ours is the first evaluation of ramp rate on commercial assay performance in the clinical diagnostics literature. Ramp rate has been previously-documented to be important: techniques such as "slowdown PCR", which are optimised to amplify GC-rich regions with complex secondary structures, use different rates for heating and cooling to improve primer annealing and amplification. Here, ramp rate is critical for the performance of this technique 28 . As M. tuberculosis is GC-rich and rpoB can form secondary structures 29 , it is possible that slower ramp rates help reduce secondary structure formation (e.g., during the transition from denaturation to annealing phases) and thereby result in better detection.
Our survey found the majority of laboratories to use an incorrect ramp rate, despite a lower ramp rate being recommended. About half of respondents could change the ramp rate. Together, this illustrates that incorrect ramp rate usage is likely widespread but, importantly, easily fixable without the purchase of new thermocyclers, which may be prohibitively expensive in high burden settings.
TUB-band detection on smear-negative sputa failed more frequently at incorrect ramp rates. As this band is required before a susceptibility result is reported, drug resistant diagnoses are more likely to be missed at the incorrect ramp rate. Differences in ramp rate may hence partly explain previously reported variation in performance in smear-negative specimens [12][13][14][15]17 however, we only received responses to our queries regarding ramp rate from two studies in the systematic review who used smear-negative specimens.
Although Xpert is often the initial first-line test for rifampicin-resistance, MTBDRplus is used for MDR-TB in several high TB-burden countries and to confirm isoniazid-susceptibility. Isoniazid can be included in the new WHO-endorsed MDR-TB second-line regimen 30 . In response to the WHO's endorsement of the regimen, laboratories are scaling-up MTBDRsl capacity for second-line drug resistance testing. MTBDRsl is thus of increasing importance, however, we did not include MTBDRsl for reasons of cost and feasibility. MTBDRsl is nevertheless similar to MTBDRplus, has the same recommended ramp rate, and is hence likely similarly adversely impacted. We will validate this in future.
We did not assess the impact of several ramp rates or thermocyclers for reasons of cost and limited clinical specimens, but chose to use the most frequently reported incorrect ramp rate and a machine commonly used in our setting (the survey results showed a large diversity in thermocycler models used, with no predominant    . We did not spike sputa with bacilli as clinical specimens from patients, which we also included, are more suitable (bacilli from patients in sputum are suspended in a mucous matrix rather than bubbles as they are in spiked sputa). Furthermore, spiking was not done at very low concentrations ( < 10 2 CFU/ml), where the incorrect ramp rate might have more of an impact, however, such concentrations of bacilli are clinically rare and often Xpert-negative (and hence unlikely be tested by MTBDRplus). Examination of the impact of ramp rate at lower concentrations might be required for tests that succeed MTBDRplus and have higher sensitivity. Finally, despite repeated attempts to survey a wide range of laboratories, it is possible that non-respondents may have different ramp rate usage patterns (e.g., due to less TB or research expertise), which may limit generalisability. This implies our estimated extent of incorrect ramp rate usage is an underestimate.
Our study is the first to investigate ramp rate as a cause of suboptimal MTBDRplus performance. We recommend 1) laboratories switch to the manufacturer-recommended ramp rate, 2) the manufacturer makes the recommended ramp rate more prominent in the documentation accompanying the assay, and 3) studies on the line probe assays publish the ramp rate used. Furthermore, we suggest that diagnostic laboratories who have conducted pilot evaluations of MTBDRplus on smear-negative specimens and found MTBDRplus to have unsatisfactorily high rates of non-actionable results repeat the evaluation if an incorrect ramp rate was originally used.
In conclusion, incorrect ramp rate usage is a widespread problem that negatively affects the diagnostic accuracy of potentially thousands of MTBDRplus assays each month. New molecular tests for drug-resistance are critical, however, if they are not done using the correct manufacturer-recommended conditions, performance is compromised and recent promising technical advances (e.g., ability to test smear-negative specimens) will not be fully capitalised upon. Laboratories doing MTBDRplus should hence ensure they use the correct thermocycler ramp rate of ≤ 2.2 °C/s.