Improved performance of the artus Mycobacterium tuberculosis RG PCR kit in a low incidence setting: a retrospective monocentric study

Tuberculosis (TB) and the spread of Mycobacterium tuberculosis complex (MTBC) strains resistant against rifampin (RIF) and isoniazid (INH) pose a serious threat to global health. However, rapid and reliable MTBC detection along with RIF/INH susceptibility testing are challenging in low prevalence countries due to the higher rate of false positives. Here, we provide the first performance data for the artus MTBC PCR assay in a low prevalence setting. We analyze 1323 respiratory and 311 non-respiratory samples with the artus MTBC PCR assay as well as by mycobacterial culture and microscopy. We propose retesting of specimens in duplicate and consideration of a determined cycle-threshold value cut-off greater than 34, as this significantly increases accuracy, specificity, and negative predictive value without affecting sensitivity. Furthermore, we tested fourteen MTBC positive samples with the GenoType MTBDRplus test and demonstrate that using an identical DNA extraction protocol for both assays does not impair downstream genotypic testing for RIF and INH susceptibility. In conclusion, our procedure optimizes the use of the artus MTB assay with workload efficient methods in a low incidence setting. Combining the modified artus MTB with the GenoType MTBDRplus assays allows rapid and accurate detection of MTBC and RIF/INH resistance.


Materials and Methods
Sample collection and preparation. This retrospective monocentric study includes data from January 2011 to December 2014, gathered at the Institute for Hygiene and Microbiology, University of Wuerzburg, Germany. The institute provides diagnostic services predominantly for the University Hospital Wuerzburg, which is a tertiary hospital with 1430 hospital beds in lower Franconia, a region in northern Bavaria, Germany, with a population of around 1.3 million.
In addition to standard cultivation procedures, the artus MTB assay was only performed from respiratory and non-respiratory samples either if (a) AFB smears were positive or if (b) requested by the attendant clinician due to suspected TB. After collection, all samples were immediately transported to our laboratory and processed within 24 h. If needed, the samples were cooled at 4 °C for overnight storage, except cerebrospinal fluid (CSF), which was stored at room temperature for less than 24 hours. For the purpose of this validation study, all redundant samples from the same patients as well as all samples from patients under treatment with anti-TB drugs were excluded (Fig. 1). We note that analyses from multiple samples from the same patient of course raises the chance of finding MTBC, but the focus of this work was on a technical validation of the artus MTB assay and not on TB epidemiology.
Ethics statement. All data analyses were performed in a patient-blinded anonymous manner. All data included in this study were taken from specimens analyzed in the diagnostic routine, and no samples were taken exclusively for the purpose of this study. Likewise, neither were the standard operating procedures affected or changed for the sake of this study, nor was the reporting of the results affected by the results of this study. The ethics committee of the Julius-Maximilians-University Würzburg (no. 20170222 02) has approved the use of clinical specimens for the validation of diagnostic tests in a patient-blinded anonymous manner without further need for obtaining informed consent from the patients.
Microscopy, cultivation and phenotypic resistance testing. All samples except for urine, stool and genital tract specimens were examined via AFB smear using the Ziehl-Neelsen staining method (ZN) and cultivation on two solid and one liquid media (Table 1). Non-sterile material, e.g. respiratory specimens, stool, urine or gastric aspiration, were treated with N-acetyl-L-cysteine-sodium hydroxide (NALC-NaOH) before cultivation and further PCR procedures. Solid media as Stonebrink agar with polymyxin B, amphotericin B, carbapenicillin and trimethoprim (PACT) (Becton-Dickinson, Franklin Lakes, USA) and Loewenstein-Jensen-medium with glycerin and PACT (Becton-Dickinson) were used for the conventional cultivation as well as the automated liquid media system Bactec MGIT 960 (Becton-Dickinson) 1 . In line with common standards in TB diagnostics, a minimum cultivation time of eight weeks at 37 °C on solid media applied to every sample. In case of growth of AFB on solid or in fluid media PCRs and subsequent sequencing of the targets 16 S ribosomal DNA 24 and mycobacterial gyrase B subunit DNA, gyrB 25 , were performed to differentiate between the mycobacteria species of the MTB complex. Non-tuberculous mycobacteria were identified by sequencing the 16 S -23 S rRNA intergenic transcribed spacer region according to Roth et al. 26 . Phenotypic antibiotic susceptibility testing was performed according to the manufacturer's instructions with the Bactec MGIT 960 SIRE and PZA kits (Becton-Dickinson).

Molecular detection of MTB from clinical samples with the artus MTB assay. DNA extraction
and purification was performed with 250 µl of the clinical sample, after decontamination with NALC-NaOH if appropriate, using the QIAampDNA Mini kit (QIAGEN). A Rotor-Gene Q (QIAGEN) real-time PCR cycler was used for the molecular detection of MTB complex bacteria with the artus MTB assay which is validated by the manufacturer (QIAGEN) for sputum, bronchoalveolar lavage, bronchial secretion, cerebrospinal fluid, gastric aspiration and peritoneal fluids. Results are expressed in threshold cycle (C t ) values, expressed as an exponential increase of fluorescence compared to background fluorescence. Every C t value less than or equal to the maximum cycle number of 45 was considered positive.
Retesting of NAAT positive samples with the artus MTB. In order to improve the PPV all artus MTB positive specimens were retested twice in parallel ("NAAT 3x" in Table 2 and Table 3) and counted as positive only if the initial positive result could be confirmed at least once. Primarily negative materials were not retested ("NAAT 1x") and counted as "NAAT negative" (Fig. 2 and Tables 2 and 3).
SCIentIfIC REPORTS | 7: 14127 | DOI:10.1038/s41598-017-14367-z Genotypic resistance testing with the GenoType MTBDRplus kit. For direct detection of INH and RIF resistance we used the GenoType MTBDRplus line probe multiplex PCR kit (Hain Lifesciences) Version 2.0 which targets common resistance mutations in the RNA polymerase gene (rpoB) for RIF resistance as well as low mutations in the katalase-polymerase gene (katG) and the promoter region of the NADH-enoyl-ACP-reductase gene (inhA) leading to INH resistance. The GenoLyse kit (Hain Lifesciences) was used in parallel with the QIAampDNA Mini kit (QIAGEN) on a subset of sputum samples for DNA extraction and purification as the GenoType MTBDRplus PCR protocol is validated only for use with the GenoLyse kit (Hain Lifesciences). After amplification of the target sequences and reverse hybridization on test stripes containing specific probes, visualization of hybridized amplicons was carried out by an enzymatic reaction according to the manufacturer's instructions.
Comparison with evaluation studies using different platforms in other low incidence countries. All compared studies had to be performed with a common CE-IVD certified, commercially available platform. In addition, specimens had to be obtained in a comparable epidemiological setting: the standard used in this work is a low incidence or notification rate of <10 TB cases /100.000 population 23 . For a better comparability of study results only respiratory specimens were taken into account. A PubMed/NCBI research was performed using the search-terms: "evaluation" and "performance" with each of the chosen platforms' names: GeneXpert Statistical analyses. Unless stated otherwise, statistical analyses were performed with the R statistical software package (version 3.2.1) and with SPSS Statistics, Version 23 (IBM, Armonk, USA), respectively. P-values for comparing NAAT results and for comparing the performance of the DNA extraction methods were calculated using a 2-sample test for equality of proportions. Analysis of deviance of a logistic regression model was used to assess the impact of the ZN staining result ("positive" or "negative") and the sample type ("respiratory" or "non-respiratory") on the performance ("correct" or "false" with respect to the cultural result used as reference) of the artus M. tuberculosis assay. Calculation of Cohens Kappa was done with the http://vassarstats.net/kappa. html homepage by Richard Lowry. Binomial confidence intervals were determined using the http://statpages. info/confint.html homepage by John C. Pezzullo. We further used the diagnostic odds ratio to define a reasonable C t score cut-off value to minimize false positive results.
Data Availability Statement. The underlying primary datasets generated during the current study are not publicly available due to (medical) confidentiality but patient-blinded excerpts are available from the corresponding author on reasonable request. Most analysed data are included in the published article.

Results
Sample origin and ZN status. Since the purpose of this study was a per-sample analysis of the artus MTB assay performance we only investigated primary materials from each patient. The sample mix consisted of respiratory and non-respiratory materials. While bronchial secretions and lavages were the predominant specimen type for respiratory materials, pleural aspirations, urine and CSF held the majority of non-respiratory specimens. Since the majority of NAAT requests were filed by infectious diseases units, intensive or intermediate care units or medical units, only selective materials underwent direct NAAT investigation. Consequently, sputa were the most frequent respiratory specimen in overall TB requests but not for direct NAAT investigation requested by the attendant clinician ( Fig. 1) 1 .
Of the 1323 respiratory specimens analyzed by mycobacterial culture, microscopy and PCR, 26 were culture positive (2.0%) of which 16 were also ZN positive. The percentage of MTB-complex culture positive samples was significantly higher among the non-respiratory samples (n = 311, 5.5%, p < 0.01). However, there were no significant differences in the proportion of ZN positive samples between respiratory and non-respiratory specimens (29/1323 vs. 6/311, p > 0.1). Altogether, given the estimated TB point prevalence in the general German population of about ≤0.001% 22 , this suggests that clinical examination increased the pre-test TB prevalence over 1000-fold, which is largely in agreement with similar values found in other laboratories in low-prevalence countries 3 .   Table 2. Stratified results for sensitivity, specificity and accuracy of the artus MTB assay using culture as diagnostic reference. 1      i.e. either true positive or true negative results with the artus MTB PCR using cultural detection as diagnostic reference ( Fig. 2 and Table 2). Fitting the impact of the sample type (respiratory or non-respiratory) and ZN staining result (positive or negative) on the artus MTB test accuracy by a logistic regression model (p < 0.005 for both explanatory variables), analysis of deviance showed that both variables had an independent and significant impact on test accuracy (p < 0.05). The artus MTB assay showed a significantly higher accuracy for respiratory than for non-respiratory sample types and for ZN negative than for ZN positive ( Fig. 2 and Table 2). The artus MTB assay showed the best performance on ZN negative, respiratory samples with 97.0% correct results, which is also the most frequently encountered sample type in routine diagnostics as outlined above. Furthermore, while there was only a moderate overall concordance between the ZN staining and the artus MTB results (Cohens κ = 0.44, 95%-CI = [0.2979, 0.5735]), there was a yet significant negative correlation between the semi-quantitative number of acid-fast bacilli (ranging from "(+)" to "+++") and the respective C t value (Spearman's rank correlation coefficient ρ = − 0.58, p < 0.05) among the ZN positive specimens (Fig. 3A).
The overall sensitivity of the artus MTB assay was rather low with 51.2% (Fig. 2 and Table 2). As expected, the sensitivity was highest in the subgroup of ZN positive specimens (88.2%) and lowest in ZN negative non-respiratory samples (18,8%). The sensitivity was significantly higher for respiratory (69.2%) than for non-respiratory samples (23.5%).
The overall specificity of the artus MTB assay was quite high with 96.9% and was highest for ZN negative and significantly lower in ZN positive respiratory samples. Overall, the specificity was only slightly higher for respiratory than for non-respiratory samples ( Fig. 2 and Table 2).
In accordance with the low prevalence setting, the overall NPV was very high with 98.7% ( Fig. 2 and Table 3). It was highest in the clinically most frequent subgroup of ZN negative respiratory samples (99.5%), still 95.6% for non-respiratory specimens and 98.7% irrespective of sample origin or ZN staining result.
In contrast to the excellent NPV, the overall PPV was poor with 44.0%. It was highest in the clinically highly relevant subgroup of ZN positive respiratory samples (87.5%) but was only 18.2% in the ZN negative respiratory samples. The overall PPV was non-significantly higher for respiratory than for non-respiratory sample types (47.4% vs. 33.3%) but significantly higher for ZN positive compared to ZN negative samples (83.3% vs. 23.3%), respectively. Among the respiratory samples, this retesting approach resulted in a significant increase in the accuracy of the test to 98.2% (Fig. 2 and Table 2). A closer analysis revealed that 25 of the 50 retested samples were true positives when compared with the culture results. Twenty-two of these 25 samples considered as positives in the "NAAT 3x" approach were in turn also positive in the subsequent MTB culture, significantly increasing the overall PPV to 88% and the PPV for respiratory samples in particular to 94.7%, respectively. The increase in the PPV was most dramatic for the large group of ZN negative samples (87.5%). While there was also a concomitant significant increase in the overall specificity (98.5%), mainly stemming from the increase in the specificity in the two largest subgroups of respiratory (98.8%) and ZN negative samples (98.6%), respectively, there were no differences in the NPV and sensitivity between both approaches.

Retesting of NAAT positive samples with the artus MTB increases overall specificity and PPV
Application of a C t cut-off improves the overall specificity and PPV of the artus MTB PCR but at the expense of its sensitivity. Since there was no C t cut-off defined by the manufacturer, samples with large C t values were also considered as positive throughout this study.
As depicted in Fig. 3B and E, application of an increasing C t cut-off resulted in an increase in the sensitivity at the expense of the specificity and PPV. The sensitivity increases with increasing C t value, yet does not surpass approx. 50.0% at C t = 41. The specificity slightly decreases with increasing C t values, yet stayed above 98.5% at C t = 41. The PPV markedly decreases with increasing C t values, dropping to approx. 40.0% at C t = 41. The diagnostic odds ratio showed a maximum at a C t cut-off of 34, and most of the false positive test results could be found past the 34 th cycle (Fig. 3D and G). Applying a C t cut-off of 34, a significant increase in specificity (98.5%) and PPV (84.6%) could be observed, particularly in respiratory and ZN negative but not in non-respiratory or ZN positive samples ( Table 2 and Table 3). While leaving the NPV unaffected this increase came, however, at the expense of a significantly decreased sensitivity of only 25.6% at a C t cut-off of 34 (p < 0.05).
Combined application of a C t cut-off in the first round and confirmation of all positive results in the second round in test-retest approach results in a significantly enhanced artus MTB PCR performance. By applying the C t cut-off of 34 in the first round of artus MTB testing and confirming each positive result at least once by retesting two times in parallel ("NAAT 3x") without such a C t cut-off, the overall accuracy further increased to 97.2% when compared to the standard testing protocol as recommended by the manufacturer. As further shown in Fig. 2 and Table 2, while the sensitivity and NPV remained unaffected by this test modification, this combined approach resulted in a significant increase in the overall specificity (98.4%) and PPV (84.6%). The increase in the PPV was most dramatic for respiratory (90.0%) as well as for ZN negative (87.5%) samples.
At the same time, applying the C t cut-off in the first round of testing significantly reduced the number of positive test results to be confirmed among the total of 1634 samples from 50 to 25 (p < 0.01), corresponding to only 50 additional artus MTB PCRs. In consequence, compared with the test as originally implemented by the manufacturer our test modifications resulted in a reduction of the number of false-positive patients from 28 to only 4 without altering the number of false negatives (n = 20) among the 1634 patients tested.

Comparison of extraction methods for genotypic INH/RIF resistance testing.
Since the artus MTB PCR does not allow for a simultaneous genotypic resistance testing we combined this assay with the GenoType MTBDRplus test which detects the most frequent mutations conferring resistance against the two first line antibiotics rifampin and isoniazid. As the latter test was validated by the manufacturer only for use with the GenoLyse DNA extraction assay, we compared the GenoType MTBDRplus performance after DNA extraction with the GenoLyse and the QIAampDNA mini kit in fourteen MTB complex positive clinical samples using phenotypic resistance testing as reference. We obtained identical results with both assays in eleven cases (78.6%, p < 0.001, exact binomial test), comprising four cases with correct identified susceptibilities, two cases with correct identified resistances, two cases with false negative results compared to phenotypic resistance testing and another three cases that were non-determinable. Three materials showed differing results: one case with correct susceptibilities after DNA extraction with the GenoLyse kit and a non-determinable result after DNA extraction with the QIAampDNA mini kit; two cases in which only DNA extraction with QIAampDNA mini kit resulted in the detection of correct sensitivities while after DNA extraction with the GenoLyse protocol the resistance genotype could not be determined. Together, there was no significant difference in the performance of the GenoType MTBDRplus assay after DNA extraction with both kits (p = 0.71, 2-sample test for equality of proportions) and the concordance between the GenoType MTBDRplus test results after DNA extraction with either protocol was reasonably high (Kappa = 0.65, 95-CI = [0.18,1.00]).

Discussion
In recent years, PCR-based molecular assays have become a mainstay for the rapid detection and identification of MTB complex organisms in clinical specimens. The artus MTB is a real-time PCR-based test for which, however, performance data obtained in a low incidence/prevalence country have been lacking so far.
Using a collection of 1323 respiratory and 311 non-respiratory samples from patients with suspected TB we overall obtained 95.7% correct results using cultural detection as diagnostic reference. Whereas the overall sensitivity and PPV of the artus MTB performed once were rather poor with 51.2% and 44.0%, respectively, the specificity and NPV were quite high with 96.9% and 98.7% ( Table 2, Table 3 and Fig. 2), respectively. Both, the sample type as well as ZN staining, had an independent and significant impact on the test performance. As expected, the test performed slightly better with respiratory than with non-respiratory materials and with ZN negative than with ZN positive. Accordingly, the artus MTB test performed on ZN negative, respiratory samples with 97.0% correct results and a NPV of 99.5%. For ZN positive respiratory samples, the artus MTB assay showed a comparable accuracy of 86.2% and a PPV of 87.5%. Together, the high NPV suggests that the artus MTB PCR can be used to screen respiratory samples from patients with clinically suspected TB for DNA from MTB complex bacteria in order to exclude patients tested negative from additional confirmatory testing. For ZN positive samples, the artus MTB assay has also a reasonably high PPV although positive results need further confirmatory testing.
Given the lack of consistent clinical data, it remains elusive at present whether the false-positive results contributing to the low PPV were caused by residual MTB complex DNA, cross-reactivity of the artus MTB PCR with DNA from mycobacteria other than tuberculosis (MOTT) or due to other unspecific amplification reactions. In line with current recommendations 1 , we deliberately used mycobacterial culture as a reference method in order to have a consistent diagnostic "gold standard" for all 1634 patient samples and thus to avoid inconsistencies due to incomplete or missing clinical information regarding the (suspected) TB status. However, we note that this could contribute to a too high number of putatively "false" positive results of the artus MTB assay and thus the observed poor sensitivity and PPV of the molecular assay. In fact, most of the 28 artus MTB positive results that could not be confirmed by culture (Table 2 and Table 3) showed C t values past the 34 th cycle which might be indicative of trace amounts of MTB DNA (Fig. 3D).
We note, however, that a recent validation study by Hur et al. 21 performed in South Korea, having a higher TB prevalence than Germany, identified a higher C t threshold of 38 cycles and could not find any link between high cycle values and false positive NAAT results.
For 11 of the 28 presumably false positive patients we had also data from interferon-gamma release assays (TSPOT.TB, Oxford Immunotec, Abingdon, Oxfordshire, UK). Of these, four were positive, indicating that they had prior exposure to MTB and therefore either a latent or active TB infection.
On the other hand, since two of these presumably false positive samples were culture positive for M. chelonae and M. xenopi, respectively, unspecific reactions are in fact likely to contribute to the number of false positives, too, and the discrepancies between the artus MTB PCR and the cultural results are likely multifactorial.
Notwithstanding these caveats, the poor PPV of the artus MTB assay compared with other validation studies [27][28][29][30] prompted us to test whether modifications in the data analysis and/or a screening/re-testing approach would result in a significant improvement of the assay performance. In fact, by re-testing all those positive specimens in duplicate having a C t value greater than 34 we could significantly increase the overall specificity, PPV and accuracy of the artus MTB assay to 98.4%, 84.6% and 97.2%, respectively, with only a moderate increase in the number of additional test runs. This reduces the number of patients that would have received an anti-MTB chemotherapy in our data set from 28 to only four, i.e. by 86%. Accordingly, per 1000 patients tested this corresponds to a decrease from 17.1 to 2.4 false positives. Given that a negative test result obtained via mycobacterial culture takes eight weeks, the approximately 30 additional artus MTB PCRs per 1000 patients tested have consequently to be weighed against over 700 daily doses of an anti-MTB combination chemotherapy along with the notification and isolation of the 13 patients that would have been erroneously tested positive over this time period.
Furthermore, the combination of the artus MTB assay for MTB detection and the GenoType MTBDRplus assay for genotypic detection of RIF/INH resistance seems to be a possible alternative to fully automated platforms such as the GeneXpert MTB/RIF given that sufficient laboratory resources are available [11][12][13] . Whereas the GeneXpert MTB/RIF is a closed system and limited to genotypic resistance testing of RIF only, the artus MTB assay is an open platform and thus allows for the combination with, e.g., the GenoType MTBDRplus assay for the genotypic detection of high and low level INH as well as RIF resistance patterns. Further optimization of the diagnostic workflow by utilizing the same DNA extraction method for both assays would allow a seamless integration of molecular MTB detection with genotypic RIF/INH resistance testing in respiratory as well as non-respiratory specimens. This, in turn, will facilitate optimized patient care in low incidence settings. Accordingly, further studies in low incidence countries, best in a prospective multi-center setting and including also clinical data as reference, will be required to assess the impact of such a combined testing-retesting approach on patient care and public health in a comprehensive manner.