Introduction

Although curable, tuberculosis (TB) remains the deadliest infectious disease caused by a single aetiological agent, Mycobacterium tuberculosis (M. tb), among all infectious diseases. In 2017, 1.5 million people died of TB globally. In South Africa, HIV-negative TB deaths were estimated to be ~23000 (17000–29000) while HIV-positive mortalities totalled 101 000 (67 000-142 000) in 20171,2. The WHO has reported a decline in TB incidence and mortality rates, and this has been due to the introduction of rapid diagnostic assays and effective TB treatment. Despite the global decline in new infections, the emergence of multidrug-resistant TB (MDR-TB) is worsening the burden of TB. In 2017, 558 000 people had MDR-TB, an increase from 490 000 people in 2016 while in South Africa, a decline from 19 000 cases in 2016 to 15 000 cases in 2017 was reported1. In South Africa, this decline was due to major efforts by the government to manage the problem, including the rolling out of GeneXpert machines throughout the country for quick diagnosis, screening high-risk groups (HIV infected individuals) to prevent transmission, treating latent TB, and using quality M. tb drugs for treatment3.

In 2010, the WHO recommended the use of the GeneXpert MTB/RIF G4 (hereafter referred to as Xpert G4), which is an automated cartridge-based molecular test, as a primary test to increase TB detection and improve diagnosis of rifampicin (RIF) resistance in pulmonary and extra-pulmonary TB (EPTB) specimens4. However, the Xpert G4 assay’s sensitivity for TB detection is inadequate when few bacilli are present in the specimens, particularly in vulnerable groups such as HIV-infected patients, children and in extra-pulmonary TB samples5,6,7. Moreover, in terms of RIF resistance detection, the Xpert MTB/RIF G4 cartridge assay gives false-positive results for strains that carry phenotypically silent mutations (synonymous mutations), or for paucibacillary specimens8,9. This has led to the limited usefulness of the Xpert MTB/RIF G4 cartridge assay in diagnosing TB and RIF resistance.

A new GeneXpert MTB/RIF Ultra (hereafter referred as Xpert Ultra) assay has been developed to overcome the limitations of the old Xpert MTB/RIF G4 assay with improved sensitivity in the detection of TB and RIF resistance. The Xpert Ultra assay is a rapid assay that uses an improved assay chemistry and cartridge design. It incorporates two different multicopy amplification targets viz., IS6110 and IS1081, and the RIF resistance-determining region (RRDR) of the rpoB gene10,11. These changes resulted in a ten-fold improvement in the lower limit of TB detection. Additionally, analytical laboratory data also showed an improved differentiation of certain silent mutations, improved detection of RIF resistance in mixed infections and reduced false-positive results in detecting RIF resistance in paucibacillary specimens.

Currently, only one multicentre study has been conducted to evaluate the performance of the new Xpert Ultra assay10. In that study, adults presenting with pulmonary TB symptoms at primary healthcare centres and hospitals in eight countries i.e., South Africa, Uganda, Kenya, India, China, Georgia, Belarus, and Brazil, were recruited. Reported findings showed that the sensitivity of Xpert Ultra was superior to that of Xpert G4 in patients with paucibacillary disease and in patients with HIV. However, an increase in sensitivity led to a decrease in specificity10. Thus, more studies are needed to assess the performance of the new Xpert MTB/RIF Ultra assay on pulmonary and extra-pulmonary samples in different geographical settings.

Besides the Xpert G4, the Xpert Ultra, the auramine smear microscopy and Hain’s line probe assay (LPA), which are also WHO-approved diagnostic tests for TB detection12, were compared using the BACTEC MGIT 960 culture assay as a gold standard in all cases. To our knowledge, no evaluation study has been undertaken to compare the Xpert Ultra to the smear microscopy and LPA, making this the first. This study therefore aims to provide additional and informative data on the diagnostic efficiencies of these assays to inform TB diagnostic choices, particularly in less-resourced clinical microbiology laboratories.

Materials and Methods

Study design and setting

This study prospectively evaluated and compared the TB detection capabilities of the new GeneXpert MTB/RIF Ultra assay (Cepheid Inc, USA), the old GeneXpert MTB/RIF G4 (Cepheid Inc, USA), the GenoType MTBDRplus assay (Hain Lifescience, Nehren, Germany) and the auramine smear microscopy using BACTEC MGIT 960 (BD, Sparks MD, USA) culture as a gold standard (Fig. 1).

Figure 1
figure 1

Flow diagram of methodologies used in processing samples and evaluating the diagnostic efficacies of smear microscopy, Xpert Ultra, Xpert G4 and LPA tests in detecting Mycobacterium tuberculosis from pulmonary and extra-pulmonary samples.

Full size image

Remnants of pulmonary (bronchial alveolar lavage [BAL], trachea, trachea aspirate, and sputum)(n = 125) and extra-pulmonary (urine, superficial swabs, gastric aspirates, abscess, tissue, cerebrospinal fluid [CSF], and fluid aspirates [from non-pulmonary but unspecified/unknown anatomical site])(n = 80) clinical specimens were collected from the National Health Laboratory Services (NHLS), Tshwane Academic Division (NHLS/TAD) in Pretoria from patients who were suspected to be infected with TB and had come for their first TB screening. The volume of each specimen differed as remnant samples were used; hence, 800 µL of each specimen was used for the molecular assays. All remnant samples from persons suspected of having TB were included while those from doubtful sources, on treatment or had contaminations were discarded.

Sample processing

The 205 remnant samples were equally divided into three aliquots (volumes): one was processed by decontamination using 1% NALC-NaOH method for culture into MGIT 7H9 (BD, Sparks MD, USA); the second was used for the auramine smear microscopy; and the third sample (800 µL per sample) was transferred into 1.8 mL PrimeStore (PS) molecular transport media (MTM) tubes (Longhorn Vaccines and Diagnostics LLC, USA) for molecular assays (Xpert and MTBDRplus) (Fig. 1). Each 1.8 mL PS-MTM tube contained 1 mL of PS-MTM. All samples were vortexed using the Vortex Genie 2 mixer (Scientific Industries Inc, USA) for 5–10 seconds. A volume of 800 µL of specimen was aliquoted into the correctly labelled 1.8 mL PS-MTM tube, mixed by pipetting up and down thrice and the tip was discarded after use. Finally, the PrimeStore tubes were vortexed with the specimen using Vortex Genie 2 mixer (Scientific Industries Inc, USA) and stored at room temperature for later use. The specimens stored in PrimeStore were divided into three: one specimen (200 µL) was used for DNA extraction for the LPA assay and the other two were used for the Xpert Ultra assay (500 µL) and for the old Xpert G4 assay (500 µL) according to manufacturer’s instructions13,14.

Auramine smear microscopy

The second portion of the 205 specimens were subjected to the auramine smear microscopy by fixing the specimens unto slides, applying rhodamine auramine to the fixed slide for 5 minutes, and then rinsing it, applying the decolorizing agent, acid-alcohol, and then rinsing it, applying potassium permanganate for 2 minutes and then rinsing, drying and observing the slide with a fluorescent microscope as already described (Wanger et al., 2017).

GeneXpert MTB/RIF assays

The GeneXpert assays (Ultra and G4) were conducted on all 205 (125 pulmonary and 80 extra-pulmonary) samples. A 1.0 mL of GeneXpert MTB/RIF assay sample reagent (Cepheid Inc, USA) was added to 500 µL of each specimen (already mixed with the PS-MTM) using a sterile pipette (Tang et al., 2017).

DNA extraction

DNA was extracted from all 205 (125 pulmonary and 80 extra-pulmonary) specimens using PrimeXtract extraction kits (Long Horn Vaccines and Diagnostics, USA). A volume of 200 μL of 100% ethanol, 200 μL lysis buffer and 200 μL specimen (already mixed with the PS-MTM) were transferred into a 1.5 mL micro-centrifuge tube followed by thorough mixing and subsequent centrifugation. The entire supernatant was then pipetted into a micro extraction column, where it was centrifuged for 1 min at 13,000x g, and the flow-through material was discarded. A wash buffer (200 μL) was pipetted into the extraction column and centrifuged for 1 min at 13 000x g, followed by further addition of wash buffer (200 μL) to the extraction column, and subsequent centrifuging as described above, discarding the flow through material. The total M. tb DNA was eluted by centrifuging for 1 min at maximum speed using 50 μL of elution solution preheated at 75 °C. The DNA was stored at −20 °C for future use.

The GenoType MTBDRplus LPA test was conducted according to the manufacturer’s instructions. Briefly, amplification reactions were undertaken using a 35 µL primer nucleotide mix, 10 µL polymerase mix (Hain Lifescience, Germany), and 5 µL of genomic DNA. PCR and hybridization were performed and interpreted as per the manufacturer’s instructions using the GTBlot and GenoScan machines (Hain Lifescience, Germany).

Sensitivity, specificity, and negative and positive predictive values calculations

Sensitivity, specificity, negative-predictive values (NPV) and positive-predictive values (PPV) were calculated using equations reported by Carvajal and Rowe (2010)15.

Ethical approval

Ethical approval was obtained from the Human Research Ethics Committee (UPHREC) at the Faculty of Health Sciences, University of Pretoria (Ethics Reference Number: 153/2018) before the study commenced. All protocols and consent forms were executed according to the agreed ethical approval terms and conditions. All clinical samples were obtained from a reference laboratory and not directly from patients, who agreed (through informed consent forms) to our using their specimens for this research. The guidelines stated by the Declaration of Helsinki for involving human participants were followed in the study.

Results

MGIT 960 culture

Out of the 205 samples, 63/205 (30.73%) were culture positive, 139/205 (67.80%) were culture negative and three (1.46%) were contaminated. Fifty out of 125 pulmonary were culture positive while 72 were culture negative; three pulmonary specimen were contaminated. Furthermore, 13 extra-pulmonary samples (n = 80) were culture positive while 67 were culture negative (Table 1).

Table 1 General characteristics of specimens used in evaluating the smear microscopy, Xpert G4, Xpert Ultra and LPA assays.
Full size table

Auramine smear microscopy

The auramine smear microscopy resulted in 59/205 (28.78%) smear positives and 146/205 (70.73%) smear negatives. In pulmonary specimens, 51/125 (40.80%) were smear positive and 74/125 (59.20%) were smear negative while in extra-pulmonary specimens, 8/80 (10.00%) were smear positive and 72/80 (90.00%) were smear negative.

GeneXpert MTB/RIF assays

For Xpert Ultra, 76/125 (60.80%) specimens were positive for TB and 47/125 (37.60%) were negative in pulmonary samples, with two (2/125) samples being invalid while 43/80 (53.75%) specimens were positive for TB and 37/80 (46.25%) were negative in extra-pulmonary samples.

For the old XpertG4, 49/125 (39.20%) of specimens were TB-positive and 75/125 (60.00%) were negative, with one (1/125) having invalid results in pulmonary samples while 8/80 (10.00%) specimens were positive for TB and 71/80 (88.75%) were negative, one of which had invalid results in extra-pulmonary specimens.

GenoType MTBDRplus LPA assay

For LPA, 107/125 (85.60%) specimens were TB-positive and 18/125 (14.40%) were TB-negative in pulmonary samples while 59/80 (73.75%) specimens were TB-positive, and 21/80 (26.25%) were TB-negative in extra-pulmonary samples (Table 1).

Sensitivity, specificity, PPV and NPV

For all the tests, i.e. auramine smear microscopy, Xpert Ultra, Xpert G4 and LPA), the sensitivities were higher in pulmonary samples than extra-pulmonary samples. A similar trend was observed in the specificity of Xpert Ultra, for which pulmonary samples had a higher specificity than extra-pulmonary specimens: 58.57% and 49.25% respectively. However, the specificities of auramine smear microscopy, Xpert G4 and LPA were higher in extra-pulmonary samples than in pulmonary specimens (Table 2).

Table 2 Relative diagnostic efficiencies of smear microscopy, Xpert Ultra, Xpert G4 and LPA tests in detecting Mycobacterium tuberculosis from pulmonary and extra-pulmonary samples.
Full size table

The negative predictive value (NPV) was 100.00% for all tests and samples within this category while auramine smear microscopy had a specificity of 100.00% for all samples within this category. The specificity of Xpert G4 was very high, between 98.46% and 98.48% for both extra-pulmonary and pulmonary samples. The specificity of Xpert Ultra was between 50.00% and 53.49% while that of LPA was below 26% for both pulmonary and extra-pulmonary samples within this category (Table 3).

Table 3 Breakdown of the relative diagnostic efficiencies of smear microscopy, Xpert Ultra, Xpert G4 and LPA tests in detecting Mycobacterium tuberculosis from pulmonary and extra-pulmonary samples under different culture and smear results categories.
Full size table

In smear positive-culture positive (SP-CP) samples, sensitivity of auramine, Xpert Ultra, Xpert G4 and LPA was between 92–100% while the specificity and NPV were 0.00% for all the assays; PPV was 100% for all assays (Table 3). In smear-negative culture-positive (SN-CP) samples, specificity and NPV were both 0.00% for all samples and tests while PPV for auramine was 0.00% for both pulmonary and extra-pulmonary samples. However, the PPV for auramine Xpert Ultra, Xpert G4 and LPA was 100.00% for all samples, but that of Xpert G4 was 0.00% for extra-pulmonary while that of Xpert Ultra and LPA was 100.00% for extra-pulmonary samples. The sensitivity for auramine was 0.00% for all samples and less than 50.00% for all samples for Xpert Ultra, Xpert G4 and LPA, except for LPA sensitivity in extra-pulmonary samples, which was 66.67% (Table 3).

In smear-positive culture-negative (SP-CN) samples, the sensitivity and PPV and NPV were 0.00% for all four tests in both extra-pulmonary and pulmonary specimens. However, a 100.00% NPV was obtained for Xpert Ultra and Xpert G4 in the combined (both pulmonary and extra-pulmonary) and pulmonary samples (Table 3). Specificity was 0.00% for all four tests for extra-pulmonary samples, 0.00% for auramine and LPA for combined and pulmonary samples. Specificity of Xpert G4 and Ultra were both 12.50% for combined samples, and 14.21% and 14.29% for pulmonary samples respectively (Table 3; Supplementary Dataset).

Rifampicin resistance was detected in 19 isolates, with 16 and one being detected by Xpert Ultra in sputum from SN-CN and SN-CP samples respectively. While the 16 RIF-resistant isolates in SN-CN samples were detected only by Xpert Ultra, the single RIF-resistant isolates in SN-CP samples were detected by both Xpert Ultra and G4. LPA detected two RIF-resistant isolates, which was not confirmed by either Xpert Ultra or Xpert G4, in a single tissue SN-CP sample and in a sputum SP-CP sample (Supplementary Dataset). Other antibiotics resistance conferring-mutations were not determined.

Discussion

In this study, we compared the performance of the new Xpert Ultra assay with the old Xpert G4 assay, the LPA and auramine smear microscopy for detecting TB in pulmonary and EPTB clinical specimens. All these tests are used as initial rapid diagnostic tests in individuals suspected of having TB12. Our results show that the new Xpert Ultra assay has the highest sensitivity (88%) for detecting TB among both pulmonary and EPTB clinical specimens (Table 2), followed by the auramine smear microscopy (82%), LPA (80%) and the old Xpert G4 (79.59%), which had the lowest sensitivity. These results are consistent with the results from recent reports, which showed that Xpert Ultra had a superior sensitivity for TB case detection10,16. Chakravorty et al. (2017), reported an overall sensitivity of 87.50% for the Xpert Ultra versus a sensitivity of 81.00% for the old Xpert. However, this high sensitivity by the Xpert Ultra led to a lower specificity of 58.57% in pulmonary samples and 49.25% in extra-pulmonary samples as reported in other studies10,16. The low specificity could be due to the high rate of Xpert Ultra false positives in mostly pulmonary specimens and the fact that Xpert Ultra has increased limits of detection as compared to other assays11,16. Previous TB episodes may also cause false positives due to dead DNA in the sputum11,16.

In terms of specificity however, the auramine smear microscopy was higher in both pulmonary and EPTB specimens i.e., 90.28% and 98.51% respectively, than the Xpert G4 (90.28% and 97.01% respectively), Xpert Ultra (58.57% and 49.25% respectively), and the LPA (11.11% and 25.37% respectively), which recorded the lowest specificity among all the tests. It is evident from these results that all the tests are more sensitive to M. tb in pulmonary specimens than in EPTB specimens. Except for the Xpert Ultra, all the tests were more specific in detecting the absence of TB in EPTB specimens than in pulmonary clinical samples. The very close sensitivity of the auramine smear microscopy to the Xpert Ultra assay, particularly in pulmonary samples, and its better specificity in detecting the presence and absence of TB in both pulmonary and extra-pulmonary clinical specimens make it a good alternative to these nucleic acid tests. Coupled with its lower costs, lesser skills required and easier application in lower resource settings, our findings support the WHO’s recommendation for the use of the smear microscopy in detecting TB.

Although highly sensitive, the poorer specificity of the Xpert Ultra is a cause for concern as it can lead to high rate of false positives and subsequent placement of such patients on very toxic TB regimens for at least six months. Thus, there is the need to supplement the Xpert Ultra with either culture or the smear microscopy, preferably the latter due to its rapidity and simplicity, to augment for the lower specificity of the Xpert Ultra. Nevertheless, it is also worthy to note that the culture assay, which was used as the gold standard in this study, cannot identify dead, dormant or non-viable M. tb strains, a situation that can also lead to false positive results in NAATs that detect DNA in clinical samples. Although MGIT has an analytical sensitivity threshold of 10 bacilli (colony forming units per ml), which is equivalent to Xpert Ultra, the decontamination step reduces the number of bacilli in the sputum. Thus, the MGIT culture is an imperfect gold standard and final diagnosis can be done with other tests (i.e. symptom screen, X-Ray, microscopy). This is a major limitation of the culture tests, making it difficult to provide a true specificity of NAATs when the former is used as a gold standard for evaluating the latter. However, it should be noted that the samples included in this study were from patients who had come for their first TB screening. As a result, there should not be a substantial number of dead M. tb DNA in the specimens. Particularly when they had not been given any TB chemotherapy.

In addition, the findings support the use of pulmonary samples for diagnosing TB in suspected patients instead of EPTB ones as the difference in sensitivities for all the four tests between pulmonary and extra-pulmonary samples is substantially wide. It is notable, however, that certain conditions might make it impossible to obtain pulmonary samples for TB diagnosis, specifically from infants and children who cannot produce sputum. And for such cases, the Xpert Ultra and the LPA seem a better diagnostic option due to their higher sensitivity (69.23%) than the other two tests (Table 2). The less than 100% sensitivity of the LPA for smear-positive samples could be due to the presence of novel resistance mechanisms that are undetectable by the LPA probes, inability of culture to detect dead or dormant M. tb cells or the processing procedures used17.

The specificities and NPVs of all the tests were lowest when culture was positive (SP-CP and SN-CP samples) irrespective of the source of the sample: pulmonary or extra-pumonary. Furthermore, in smear positive-culture positive (SP-CP) samples, the sensitivity was even higher for all the tests, with auramine having the highest sensitivity (100%) followed by Xpert ultra (97.92%), Xpert G4 (93.48%) and LPA (89.58%). Notably, a low sensitivity and PPV were recorded by all the tests in SN-CN and SP-CN samples, albeit with variable specificities that were substantially higher in SN-CN samples (from 19.08% in LPA to 98.47% in Xpert G4) than in SP-CN samples (from 12.50% for both Xpert assays down to 0% for LPA). This shows that a joint absence of TB in both culture and smear microscopy enhances the veracity of the NAATs in predicting the absence of TB than when the two phenotypic tests disagree (in SP-CN samples), which was strongly shown in the Xpert G4 results (Table 3). Contrary to SP-CP samples, where both smear and culture agree and all four tests had higher sensitivies, SN-CP samples, for which smear and culture disagree, reported very low sensitivies for all four tests. These results are consistent with, albeit lower than, the results reported by Dorman et al. (2018) where a superior sensitivity of 63% for Ultra was seen vs 44% for the Xpert G4 in SN-CP specimen.

The highest sensitivity of the auramine stain in culture-positive specimens is not surprising as smear positivity normally correlates well with quantitative growth of the TB bacteria18. Moreover, because NAATs detect DNA, which are undetectable by culture and auramine, NAATs are more likely to report positive results in specimens for which auramine and culture, which detects cells, are negative. Smear microscopy is still the baseline diagnostic test in resource-limited, high TB-burden countries, specifically on the African continent and South East Asia12. However, this is not the case in South Africa as it has rolled out the highest number of Xpert matches than any other country in the world1. Smear microscopy is well known for its high specificity, which makes it a reliable test if the specimen does not have the TB bacteria. Our results are therefore consistent with the ones reported in the literature18,19.

Rapid diagnosis of EPTB is challenging because of the paucibacillary nature of the disease, causing rare positive smear microscopy results and long incubation time for culture growth20. In this study, the Xpert Ultra’s sensitivity was superior to the old Xpert and smear microscopy with pulmonary specimen, albeit the Xpert Ultra had the same sensitivity as the LPA (69.23%) in EPTB specimens. Our results are consistent with results from two studies that have evaluated the performance of Xpert Ultra in EPTB samples11,21. This detection improvement in the Xpert Ultra in EPTB samples is due to the larger PCR chamber, the incorporation of two different multicopy targets (IS1081 and IS6110 insertion sequences), and the optimization of PCR and thermal-cycling parameters16. The WHO strongly recommends the use of the Xpert assays in EPTB samples such as the cerebrospinal fluid over traditional smear microscopy and culture.

Ninan et al. (2016) reported similar sensitivity rates (67%) in smear negative samples for LPA in their study as observed herein with SN-CP specimens, with our LPA also showing a sensitivity of 66.67%22. However our LPA sensitivity was a little lower than that reported by Meaza et al.23. Our Xpert sensitivity in this group of specimens was within the range reported by Perez-Risco et al.11, with sensitivities ranging from 100% to 0% for different EPTB specimen types. In this study’s EPTB samples, smear microscopy demonstrated the highest specificity (98.51%), followed by the Xpert G4 (97.01%), Xpert Ultra (49.25%) and LPA (25.37%). The assays were all TB-negative in SN-CN extra-pulmonary specimens, which confirms that the patients were negative to TB but presented with other related disease symptoms that could complicate the differential diagnoses.

The Xpert Ultra detected more RIF resistance than the Xpert G4 and the LPA, which supports one of the reasons the Xpert Ultra assay was developed i.e., to overcome the limitations of the old Xpert assays with regards to differentiating certain silent mutations, improving the detection of RIF resistance in mixed infections, and avoiding false-positive results in detecting RIF resistance in paucibacillary specimens16.

The use of Primestore MTM (Longhorn Vaccines and Diagnostics, USA) to preserve the specimens used in this study also played a very big role in the high detection of TB in the three molecular assays used in this study. The PrimStore media has been shown to concentrate and enhance M. tb detection at low concentrations when using the Xpert assays24. The PrimeStore MTM is optimized to inactivate/kill infectious biological pathogens, including Haemophilus influenzae, M. tb, Gram-Positive/Negative bacteria, and viruses (Daum et al., 2015). It disrupts/lyses lipid membranes and inactivates nucleases14. Moreover, PrimeStore MTM preserves and stabilizes RNA and DNA at ambient temperature and is compatible with manual and high throughput RNA/DNA isolation kits14.

Major limitations to this study includes the relatively smaller sample size, the absence of samples from totally healthy patients to serve as negative controls, the use of samples from a single reference laboratory (although this laboratory serves about 200 clinics and at least six academic hospitals in three provinces in South Africa), and the inability of the MGIT to identify dormant or dead cells. In addition, as primestore MTM was used to concentrate our DNA from all samples, we suspect that it could have given an advantage to the molecular assays in terms of sensitivity as low bacilli load in specimens can be killed during decontamination step in culture assays. Notably, the sensitivity and specificity values of the NAATs might be impacted by the MGIT, which was used as a gold standard; specifically, when positive samples are identified as negative by the culture (MGIT) due to low bacillary load. This could explain the lower specificity of the LPA and Xpert Ultra.

Conclusion

A significant diagnostic performance of the GeneXpert Ultra assay in the detection of TB and RIF resistance was observed, confirming its efficiency for rapid diagnosis of TB in either pulmonary or extra-pulmonary specimens. Nevertheless, caution is advised in interpreting or using the Xpert Ultra’s specificity (TB-negative predictions) results clinically without a confirmation with culture or smear microscopy. The LPA showed relatively comparable diagnostic accuracy in detecting TB in all clinical specimens, albeit it was poorly specific than the Xpert Ultra; more studies are needed to confirm these findings on the two molecular assays using a better gold standard.