Main

One-step nucleic acid amplification (OSNA) assay is a molecular detection technique that might be introduced into clinical usage for intra-operative detection of sentinel lymph node (SLN) metastasis in breast cancer. Many studies focusing on OSNA assay have been published; however, so far, there is no clear consensus on the diagnosis performances of OSNA assay. The present study evaluates the performance of OSNA assay for intra-operative diagnosis of SLN metastasis in breast cancer on a per-patient basis. With high sensitivity, specificity, diagnostic odds ratio (DOR) and AUC, we figured out that OSNA assay is an accurate diagnostic tool with an advantage over traditional intro-operative methods of sentinel lymph node biopsy (SLNB).

Breast cancer is the most common malignancy in women. One-third of all breast cancer patients will suffer from metastasis, which is the main cause of mortality (Molloy et al, 2012). Axillary lymph node dissection (ALND) is a traditional clinical procedure to identify axillary lymph node metastasis in breast cancer; however, since ALND is an invasive method, patients who have undergone ALND might suffer from unpleasant complications (Wang et al, 2011). In order to provide evidence for the surgery plan without patients’ invasive pains, SLNB is widely recommended for intra-operative diagnosis of patients with early-stage breast cancers. By intra-operative evaluation of SLNB, patients with positive nodes are spared a second surgical procedure as well as patients with clinically node-negative avoid undesirable ALND (Kumar et al, 2012).

Essentially, SLN analysis includes intra-operative evaluation and post-operative pathological examination. Multistep formalin-fixed tissue sections stained by haematoxylin and eosin (H&E) with or without immunohistochemistry (IHC) is usually accepted in post-operative pathological detection (Tamaki et al, 2012), yet this method, based on formalin-fixed tissue sections, is not appropriate to intra-operative assessment. The frozen section (FS) and imprint cytology (IC) are the most commonly used techniques for intra-operative SLN evaluation. Unfortunately, the two methods suffered from low sensitivity ranging from 57 to 74% compared with post-operative pathological examination (Khaddage et al, 2011), in addition, they are hampered by variable accuracy and nonstandard operational approach.

With the advantage of standardisation, reliability and repetitiveness, molecular diagnostic techniques for intra-operative detection of SLN won the favour of many pathologists and clinicians. One-step nucleic acid amplification (Sysmex, Kobe, Japan) assay is established as an automatic technology for accurately identifying lymph node metastasis by quantitatively evaluating the number of cytokeratin 19 (CK19) mRNA copies (Osako et al, 2011). According to the cut-off levels defined previously by Tsujimoto et al (2007), the CK19 mRNA copy numbers larger than 5000 per μl of dilution homogenate are classified as (++), copy numbers between 250 and 5000 per μl as (+), and copy numbers less than 250 per μl as (−). The (++) and (+) results of OSNA assay, demonstrated by clinical trials, are well consistent with SLN macrometastasis and micrometastasis, while (−) judgement of OSNA assay represents that the lymph node has no tumour deposits or only isolated tumour cells.

So far there is no clear consensus on the diagnosis performance of OSNA for detecting SLN metastasis in patients with breast cancer. A meta-analysis performed by Tiernan et al suggested that the sensitivity and specificity of OSNA is 0.87 and 0.98 (Tiernan et al, 2014), whereas the sensitivity of 0.85 and specificity of 0.92 were observed by a systematic review conducted by Huxley et al (2015), included studies of them were published before December 2012 and were mainly based on a per-node analysis. Considering that numerous studies whose data analysis is obtained on a per-patient basis have been published and that the intra-operative detection is performed on each patient, evaluating the performance of OSNA assay on a per-patient basis is of great clinical significance (Chen et al, 2013); however, there is no meta-analysis with sufficient data to address this problem. This present meta-analysis was conducted aiming to investigate the intra-operative diagnosis performance of OSNA for detecting SLN metastasis based on a per-patient analysis, which will definitely complement previous researches.

Materials and methods

Search strategy and study selection

To retrieve all relevant papers published up to 31 December 2016, three databases including PubMed, Cochrane Library and Web of Science were searched by two independent reviewers (SHI F and ZHANG Q). Search terms were defined as follows: ‘one-step nucleic acid amplification’ or OSNA, and breast cancer or breast neoplasms or breast carcinoma or breast malignant tumour or mammary cancer or mammary neoplasm or mammary carcinoma or mammary malignant tumour. The detailed search strategies were shown in Supplementary Table 1. The references of all related papers were also checked for potential studies.

All articles were screened according to the inclusion and exclusion criteria by two independent reviewers (SHI F and ZHANG Q). The inclusion criteria were as follows: (1) patients included in the studies were diagnosed with breast cancer; (2) patients did not undergo any new adjuvant therapy or radiotherapy; (3) the diagnostic purpose was to detect SLN metastasis; (4) the specimens detected were fresh SLNs; (5) the reference standard method for the detection of SLN metastasis was post-operative histopathology; (6) the same patient was detected by OSNA and post-operative histopathology together; (7) detailed method of post-operative histopathology was described in detail; (8) data analysis of study was based on per-patient; and (9) extracted data were available for calculating true-positive, false-positive, false-negative, and true-negative values. The exclusion criteria of this study were set as follows: (1) Non-English literatures; (2) Non-clinical research literatures including basic experiments, reviews, conference abstracts and letters to journal editors; (3) The analysis of study was based on a per-node model; and (4) The data published were not sufficient to form 2 × 2 tables.

Data extraction and quality assessment

For each eligible study, the following information was extracted by two independent reviewers (SHI F and LIANG Z): first author, year of publication, type of study design, country, number of patients, mean age of patients, number of slices used for pathological examination, thickness of slices detected by pathological examination, interval of serial slices detected by pathological examination, reference standard method, whether IHC for CK19 was brought into the reference standard method, the turnaround time of OSNA assay for detecting one node, and whether the result was adjusted for excluding tissue allocation bias (TAB). Parameters such as true-positive, false-positive, false-negative, and true-negative values were placed into 2 × 2 tables. Conflicting results were evaluated by a third-party researcher (LIU X) and resolved consensually.

To assess the quality of included literature, Quality Assessment of diagnostic Accuracy Studies 2 (QUADAS-2) (Whiting et al, 2011) was conducted by two independent reviewers (SHI F and ZHANG M). QUADAS-2 was an evidence-based quality assessment tool that consisted of four domains: patient selection, index test, reference standard, and flow and timing.

Statistical analysis

First, threshold effect and heterogeneity were measured. If the P-value of Spearman correlation coefficient was more than 0.05, there was no threshold effect, whether a heterogeneity caused by non-threshold effect existed was further analysed. The heterogeneity was evaluated by P-value of Cochran-Q test and I-square statistic. If P>0.05 and I2⩽50%, studies were thought to be free from the significant heterogeneity, and a fixed effects model was performed to calculate the pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and DOR. If P<0.05 or I2>50%, which meant the existence of statistical heterogeneity, a random-effect model was constructed subsequently and a subgroup analysis was further performed to identify possible sources of heterogeneity. The eight covariates for subgroup analysis were as follows: (1) race of patients, (2) the number of patients, (3) the number of slices used for pathological examination, (4) the thicknesses of slices detected by pathological examination, (5) the interval of slices detected by pathological examination, (6) detailed approaches of reference pathology, (7) utilising IHC for CK19 in reference method, and (8) adjusting the result for TAB. If P-value was more than 0.05 and I2 was not greater than 50% in one subgroup, the according covariate was considered as source of heterogeneity. The pooled sensitivity, specificity, PLR, NLR and DOR, along with their 95% confidence intervals (CIs) were computed and graphically displayed using forest plots. An sROC curve was constructed to access the superior diagnostic accuracy for OSNA, and AUC was calculated; the higher the AUC was, the higher the diagnostic value was. Publication bias was assessed by Deeks’ funnel plot asymmetry test. The meta-analysis was undertaken using STATA13.0 (StataCorp, College Station, TX, USA) and Meta-Disc 1.4 (Unit of Clinical Biostatistics, Ramo e Cajal Hospital, Madrid, Spain). Quality assessments of included studies were carried out with RevMan 5.3 (Revman, the Cochrane Collaboration). P<0.05 was considered of statistical significance.

Results

Study selection

According to the search strategy, a total of 394 articles from PubMed, Cochrane Library and Web of Science databases were retrieved. No new articles were found after screening the references. Among these articles, 113 were excluded because of duplication and 269 articles were excluded for they did not meet the inclusion criteria: 117 articles were not related to the diagnosis effort of OSNA for breast cancer; 7 articles were with adjuvant therapy or radiotherapy; the subjects of 8 articles were non-SLNs rather than SLNs; 1 paper was non-English literature; 95 papers were non-clinical research literature; 36 articles were excluded because data analysis was not based on a per-patient analysis; 5 articles were excluded because data were not sufficient to form 2 × 2 tables. Finally, 12 articles were included in this meta-analysis (Figure 1).

Figure 1
figure 1

Flow diagram of the study selection process.

Full size image

Study characteristics

The 12 eligible articles (Choi et al, 2010; Khaddage et al, 2011; Snook et al, 2011; Le Frere-Belda et al, 2012; Vegue et al, 2012; Wang et al, 2012; Chen et al, 2013; Heilmann et al, 2013; Jimbo et al, 2013; Bettington et al, 2014; Pathmanathan et al, 2014; and Mokhtar et al, 2016) containing a total of 2833 patients with a mean sample size of 218 (range: 35–772) were included in this meta-analysis. All studies were designed by prospective methods. There were two adopted approaches of reference pathology: the detection of formalin-fixed tissue sections with H&E staining (8/12), as well as combined detection of H&E and IHC (4/12). Among the four studies taken with IHC, three studies utilised IHC for CK19 while one utilised IHC for anti-cytokeratin antibody AE1 and AE3. One whole node was commonly cut into four parts and divided equally between OSNA assay and pathology. The thicknesses of the slices used for pathological examination were mostly more than 1 mm, and the interval between the slices for pathological examination in most studies was 200 μm. Three studies adjusted the result for excluding TAB. The turnaround time of OSNA assay for detecting one node was all less than 40 min (Table 1).

Table 1 Major characteristics of included studies
Full size table

Risk of bias and quality assessment

Deek’s funnel chart was used to analyse publication bias (Supplementary Figure 1). The funnel chart was symmetrical and P>0.05, suggesting no significant publication bias. The results of the QUADAS-2 (Supplementary Figure 2) suggested that the risk of bias was low and the overall quality of included studies were of moderate to high.

Diagnostic effect

The pooled sensitivity and specificity (Figure 2) of OSNA assay for detecting SLN metastasis were 0.87(95% CI 0.81–0.91) and 0.92 (95% CI 0.86–0.95), respectively. The pooled PLR and NLR (Figure 3) were 10.65 (95% CI 6.18–18.34) and 0.14 (95% CI 0.10–0.20), respectively. The pooled DOR (Supplementary Figure 3) was 75.08 (95% CI 37.77–149.22) and AUC (Figure 4) was 0.94 (95% CI 0.91–0.95), respectively.

Figure 2
figure 2

Forest plots of pooled sensitivity and specificity.

Full size image
Figure 3
figure 3

Forest plots of pooled positive likelihood ratio and negative likelihood ratio.

Full size image
Figure 4
figure 4

SROC curve of OSNA assay for the diagnosis of SLN metastasis in breast cancer.

Full size image

Heterogeneity and subgroup analysis

In the threshold analysis, spearman correlation coefficient was −0.074 and P-value was 0.820, indicating no threshold effect. But a significant heterogeneity caused by non-threshold effect was observed. As shown in Figure 3, I2 of sensitivity was 59.26%, and I2 of specificity was 92.74%. A random-effects model was used since the existence of high heterogeneity. The subgroup analysis was further conducted to identify possible sources of heterogeneity. The result of subgroup analysis showed that the number and interval of slices used for pathology might be responsible for sensitivity heterogeneity, and the major source of specificity heterogeneity was not found (Table 2).

Table 2 Subgroup analysis of diagnostic effect
Full size table

Discussion

Numerous studies have focused on the diagnosis performance of OSNA assay for intra-operatively detecting SLN metastasis in breast cancer. However, no consistent conclusion has ever been made up to present. A meta-analysis based on a per-node analysis suggested the sensitivity and specificity of OSNA is 0.87 and 0.98 (Tiernan et al, 2014), whereas the sensitivity of 0.85 and specificity of 0.92 were observed by Huxley N et al (Huxley et al, 2015); however, as Huxley N et al only included five studies based on per-patient analysis, the small number of included studies limited the quality of evidence. Moreover, papers included in previous systematic reviews were all published before December 2012. The aim of this study is to investigate the diagnosis effort of OSNA assay on a per-patient basis and all related papers before 2017 were included in our work. The pooled sensitivity, specificity, PLR, NLR, DOR and AUC of OSNA for detecting SLN metastases are 0.87(95% CI 0.81–0.91), 0.92 (95% CI 0.86–0.95), 10.65 (95% CI 6.48–18.34), 0.14 (95% CI 0.10–0.20), 75.08 (95% CI 37.77–149.22) and 0.94 (95% CI 0.91–0.95). This meta-analysis demonstrates that OSNA assay is a reliable and accurate diagnosis method for SLNs.

Discordant cases between OSNA and reference pathology are analysed in detail. The false-positive results may be potentially due to TAB. TAB refers to a sample bias yielding because of the uneven distribution of metastatic foci throughout the node (Chen et al, 2013). In all studies included in our meta-analysis, one whole node is divided into several alternate slices and a same slice cannot be detected by both OSNA assay and pathological examination; thus a TAB inevitably yields. Conventional pathological method potentially underestimates the degree of tumour burden as only a few proportion of node sample is detected (Castellano et al, 2012). Considering that OSNA assay detects a greater portion or even a whole of node in clinical practice, the missed diagnosis of tumour burden is well avoided (Choi et al, 2010; Snook et al, 2011). Moreover, subgroup analysis of our study (sensitivity: 0.91 vs 0.83, specificity: 0.94 vs 0.90 and DOR: 160.21 vs 53.37) shows that the presence of TAB affects the result of research, suggesting more studies with rigorous design are needed in this area. The false-negative results usually exist owing to low CK19 expression. CK19 is an epithelial marker highly expressed in up to approximately 98% of breast cancers, which means approximately 2% of breast cancer patients lack CK19 expression; moreover, certain subtypes such as metaplastic cancers, lobular cancers, luminal (ER+) cancers and triple-negative phenotype breast cancer are more prone to the lack of CK19 expression (Abd El-Rehim et al, 2004; Bettington et al, 2014). As CK19 is the single molecular marker used in OSNA assay, low CK19 expression of nodes may result in a false-negative result. In our work, a better DOR of OSNA assay (160.21 vs 53.37) was observed when studies adopted IHC for CK19; so IHC for CK19 is recommended to be carried out to examine the CK19 expression in patients with the above subtypes and OSNA should not be omitted for patients lacking CK19 expression. Meanwhile, a greater sensitivity by using multiple markers together rather than a single marker has been observed by some findings (Snook et al, 2011; Le Frere-Belda et al, 2012); so the accuracy of intro-operative diagnosis of SLNs may be improved by using multiple markers such as mammaglobin and CK19.

The SLNB, a widely accepted clinical staging method for cancers, provides evidence for the appropriate selection of surgical treatment and reduces pain and surgical cost for each patient with breast cancer (Tamaki et al, 2012). Frozen section and TIC are the most commonly used intro-operative detection method of SLNB; however, the value of them is still controversial. These techniques are hampered by a rather low sensitivity and a no unified standardisation. A meta-analysis assessing intra-operative FS for SLNs showed that the pooled sensitivity of FS was 73% (Liu et al, 2011). The poor sensitivity is owing to a limited amount of tissue detected by FS, as well as samples suffering from destructive, freezing, and compression artefacts (Olson et al, 2007). Tew et al reported that the pooled sensitivity of IC was 63% (Tew et al, 2005). IC technique fails to provide information about size of a metastatic deposit that can assist intra-operative decision-making (Bettington et al, 2014). Our results show that the sensitivity of OSNA assay is 86%, indicating that OSNA assay is superior to FS and IC for intro-operative diagnosis of SLN metastasis.

With a shorter turnaround time, automated operating procedure and quantitative results, OSNA assay has obvious advantages in comparison with post-operative pathology with formalin-fixed tissue sections. Four SLNs can be detected by OSNA assay at the same time (Wang et al, 2012), whereas only a single SLN can be evaluated by pathology at once. The turnaround time of OSNA assay was all less than 40 min for detecting one node and ranges from 50 to 62 min for detecting four nodes, demonstrating that OSNA assay can meet the time demand for rapid diagnosis of SLNs and can be incorporated into intra-operative clinical setting. Moreover, the turnaround time can be reduced with an accumulating experience of technicians. Besides less turnaround time, OSNA assay is easier to operate compared with labour-intensive pathology. As RNA purification is not required and mixing of reagents or samples for CK19 mRNA amplification is performed by an automatic device, the manual processes in OSNA assay are quite easy (Khaddage et al, 2011); hence manipulation errors made by technicians are greatly reduced. The results of pathological examination depend on the subjective judgments of pathologist, which means accuracy of results may be influenced by experience and expertise of pathologists (Chen et al, 2013). As OSNA assay quantitatively evaluates the number of CK19 mRNA copies and outcomes correlate well with the number of cancer cells in a node as well as all procedure of OSNA follows a uniform protocol, the result is objective and repeatable.

Although OSNA assay has many advantages, it still has some inherent limitations. One-step nucleic acid amplification assay does not provide histological information, such as extra capsular extension and occult pathology, which is associated with the patient’s prognosis (Bettington et al, 2014). Besides, this method would be hampered by the lack of morphological correlation information such as the location of metastasis, size, pattern of spread and extra nodal spread, which is important information for predicting non-SLN involvement and can be used in guiding the selection of optimal therapeutic regimen (Bernet et al, 2011; Pathmanathan et al, 2014).

Considering the huge benefits derived from OSNA assay, OSNA is still proposed to introduce into intra-operative clinical setting. One-step nucleic acid amplification assay has been recommended by NICE guidance for diagnosis of SLN metastasis in breast cancer (Chaudhry et al, 2014). By introducing highly sensitive OSNA assay into intra-operative clinical usage, a recall for the second surgical procedure is avoided; thus the number of hospitalisations and related costs is reduced.

Our study includes 12 articles with OSNA based on a per-patient analysis and makes up for deficiencies in previous studies. However, there are two limitations in our work. Firstly, a statistical heterogeneity of specificity was observed in our study, however, the major source of specificity heterogeneity was not found. Secondly, non-English studies were excluded from this meta-analysis which might cause selection bias.

In conclusion, our meta-analysis adds the evidence that OSNA assay is a rapid and accurate molecular method for intra-operatively detecting SLN metastasis in breast cancer. One-step nucleic acid amplification assay can be introduced into routine clinical setting and replace FS as well as IC in intra-operative SLNB.