Introduction

The World Health Organization declared a global pandemic of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, a novel coronavirus disease 2019 (COVID-19), in March 2020. In accordance with global trends, the national situation of COVID-19 in Korea has changed dynamically. The Delta variant was disseminated in early August 2021, and the Omicron variant replaced the Delta variant in December 20211,2,3,4. The number of COVID-19 cases escalated in each event, with the appearance of variants of higher transmissibility; therefore, it is necessary to prepare a rapid and reliable diagnostic tool for detection of SARS-CoV-2. Real-time reverse transcription PCR (RT-PCR) has been the gold standard in detecting SARS-CoV-2 due to its high sensitivity and specificity5,6. Nevertheless, it is time-consuming and requires expert technicians to be performed accurately7,8,9. Rapid antigen detection tests could be an alternative option for diagnosis of COVID-19 infection.

Initial approval of a rapid SARS-CoV-2 Antigen Detection Test (RADT) was granted in Korea in November 2020. The RADT was approved for use in areas with urgent needs including emergency rooms, intensive care units, and long-term care facilities. The RADT exhibits lower sensitivity in detecting COVID-19 patients compared with PCR-based methods; however, the advantages of RADT are rapidity, simplicity, and point-of-care feasibility with a 15–20 min turnaround time10. There have been several studies on the diagnostic performance of RADTs compared with results obtained by RT-PCR; however, most of them were performed with small numbers of samples or were analyzed with inconsistent sample types, reagents, and test kits. Therefore, the clinical performance of RADTs has been reported in a variety of ranges with heterogeneity. In addition, there was a limitation in those prior studies that variants of SARS-CoV-2 could not be fully evaluated11,12,13,14. The aim of our study was to evaluate the clinical performance of the STANDARD Q COVID-19 Ag Test (SD Biosensor Inc., Gyeonggi-do, Korea) compared to RT-PCR with a large number of samples over a long period, covering the Alpha, Delta, and Omicron variant pandemics. We also analyzed the diagnostic performance of the STANDARD Q COVID-19 Ag Test according to the change of positive rates of SARS-CoV-2.

Materials and methods

Study population

To evaluate the clinical performance of RADT comparing with the results of RT-PCR, a retrospective chart review was performed. Inclusion criteria was the cases subjected for detection of SARS-CoV-2 by both RT-PCR and RADT in same day from January 2021 to March 2022. A total of 86,678 samples was tested for detection of SARS-CoV-2 by RT-PCR, and 14,194 cases simultaneously tested by RADT were included in this study (Fig. 1). The nasopharyngeal swab (NPS) and oropharyngeal swab (OPS) of the patients were collected and transported via viral transport medium (VTM) (AB Transport Medium (AB MEDICAL, Seoul, Korea)) and were used in performing RT-PCR and RADTs to detect SARS-CoV-2. All processes of this study were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and comparable ethical standards. This study was approved by the Institutional Review Board of Yonsei University Gangnam Severance Hospital (approval number: 3-2023-0243) to waive the requirement for informed consents about the collection of clinical data.

Figure 1
figure 1

Results of real-time PCR and rapid antigen detection test of SARS-CoV-2 during the study period.

Full size image

SARS-CoV-2 RNA detection using RT-PCR

RNA was extracted from 300 \(\mathrm{\mu L}\) of VTM using the TANBead Nucleic Acid Extraction Kit (Taiwan Advanced Nanotech Inc., Taiwan) according to the manufacturer’s instructions. RT-PCR of SARS-CoV-2 was performed with PowerChek SARS-CoV-2, Influenza A&B Multiplex Real-time PCR Kit (KOGENEBIOTECH Co., Seoul, Korea). The PowerChek assay simultaneously targets the envelope (E) and open reading frame 1ab (ORF1ab) genes of SARS-CoV-2. One drop of 5 μL of extracted template RNA was added to 15 μL of PCR reaction mixture (2X RT-PCR Master Mix 10 \(\mathrm{\mu L}\) & Primer/Probe Mix 5 \(\mathrm{\mu L}\)). A total volume of 20 μL was loaded and amplified 40 cycles by the CFX96 Real-time PCR Detection System (Bio-Rad Laboratories, Inc., Hercules, CA). Exponential fluorescence curves that crossed the threshold line at or before 38 cycles (cycle threshold [Ct] ≤ 38) for both the SARS-CoV-2 E gene and ORF1ab gene were considered a positive result. A negative result was determined when Ct values for both genes were above 40 (Ct > 40). The indeterminate results were defined when Ct value of only one of the E gene and the ORF1ab gene was below 38 and when Ct values of either E gene or ORF1ab gene was between 38 and 40.

Rapid SARS-CoV-2 Antigen Detection Test (RADT)

The STANDARD Q COVID-19 Ag Test was approved by the MFDS of Korea in November 2020. This test is a lateral flow immunochromatographic assay (ICA) that detects the nucleocapsid protein (NP) antigen of SARS-CoV-2 semi-quantitatively. It has two pre-coated lines on the nitrocellulose membrane: control (C) and test (T) lines. A sample of 350 μL of VTM was mixed with the extraction buffer solution, and 2–3 drops of the sample mixture were applied on the sample area of the device. After 15–20 min of incubation at room temperature, results were determined manually with the naked eye. For a positive result, two colored C and T lines were present within 30 min. The test was interpreted as ‘trace’ if the T line was fainter than the C line; ‘1+ (one positive)’ if the T line showed similar intensity to the C line; and ‘2+ (two positive)’ if the T line was thicker than the C line. For a negative result, only the C line was present.

Statistical analysis

Analyse-it\(\circledR \) (version 5.68; Analyse-it Software Ltd., Leeds, UK) was used for all statistical analyses. Differences between groups were analyzed using Kruskal–Wallis tests for continuous variables and Chi-square tests for categorical variables. Spearman’s rank test was performed to assess the correlation between results by RT-PCR and RADT. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the results obtained by the STANDARD Q COVID-19 Ag Test compared with those by RT-PCR. The 95% confidence intervals (CIs) were estimated by Wilcoxon sum rank test. To evaluate the diagnostic performance of the STANDARD Q COVID-19 Ag Test, receiver operating characteristics (ROC) curve analysis was performed, and the area under curve (AUC) was estimated. Statistical significance was determined when the P value was less than 0.05.

Results

Characteristics of cases, PCR results, and ORF1ab gene Ct values by period

Among the 14,194 cases, 500 positive and 13,694 negative results were obtained. The positive results consisted of 258 results of 2+, 128 results of 1+, and 114 results of trace (Fig. 1).

The evaluation period was divided into 5 periods (3 months per period), and the number of cases examined using the STANDARD Q COVID-19 Ag Test by period ranged from 1822 to 3217. The gender ratio, median age, PCR qualitative results, and ORF1ab gene Ct value distribution are shown in Table 1. The PCR positive rate was estimated to be 0.11% (2/1822) in Period 1, surged to 4.51% (145/3217) in Period 3 (the Delta variant epidemic), and peaked at 13.72% (429/3127) in Period 5 (the Omicron variant epidemic). The proportion of cases with Ct values less than 25.00, indicating a high viral load, was 0.05% in Period 1, increased to 2.55% in Period 3, and peaked at 6.78% in Period 5.

Table 1 Summary of SARS-CoV-2 rapid antigen detection test results.
Full size table

Correlation between Ct value of ORF1ab gene PCR and semiquantitative results obtained by STANDARD Q COVID-19 Ag Test

Table 2 summarizes the qualitative and semiquantitative results of the STANDARD Q COVID-19 Ag Test according to the Ct values of ORF1ab gene PCR, including indeterminate results. Most (329/348, 94.5%) of the cases with low CT values under 25.00 gave a positive result with the STANDARD Q COVID-19 Ag Test. In addition, most (137/157, 89.0%) of the cases with Ct values under 20.00 showed 2+, and more than half (102/191, 58.3) of the cases with Ct values of 20.00–24.99 also showed 2+ with the STANDARD Q COVID-19 Ag Test. When semiquantitative results of the STANDARD Q COVID-19 Ag Test were compared with Ct values obtained by RT-PCR, a significant and moderate negative correlation was observed (Spearman’s ρ = − 0.646, P < 0.0001) (Fig. 2).

Table 2 Correlation between RADT and ORF1ab gene Ct values of real-time RT-PCR including indeterminate results.
Full size table
Figure 2
figure 2

Correlation between Ct value (ORF 1ab gene) of real-time RT-PCR and RADT semi-quantitative results, including indeterminate results. The number of overlapping negative or indeterminate case dots with Ct value of ORF1ab over 40 were added in the graph.

Full size image

Clinical performance of the RADT

The PCR positive rates, clinical performance of the STANDARD Q COVID-19 Ag Test compared with the PCR results, and PPV and NPV by period are summarized in Table 3. The overall sensitivity, specificity, PPV, and NPV were 0.577 (95% CI 0.539–0.614), 0.991 (95% CI 0.989–0.993), 0.760 (0.724–0.795), and 0.980 (95% CI 0.978–0.981), respectively. The PPV was lowest in Period 1 (0.045; 95% CI 0.011–0.169) and high in Period 3 (0.951; 95% CI 0.890–0.979) and Period 5 (0.798; 95% CI 0.751–0.838). The specificity was greater than 0.970 in all periods. ROC curve analysis resulted in an AUC of 0.786 (P < 0.0001) and Youden’s index of 0.568 (Fig. 3). The monthly trends of sensitivity, specificity, PPV, NPV, and PCR positive rates are described in Supplementary Figure S1.

Table 3 Diagnostic performance of RADT compared to real-time RT-PCR (except indeterminate results).
Full size table
Figure 3
figure 3

PCR positive rate and analytical performance by period and receiver operating characteristics curve of RADT. (A) Sensitivity, specificity, positive predictive value, and negative predictive value by period, (B) PCR positive rate by period, and (C) receiver operating characteristic curve of RADT.

Full size image

Discussion

Antigen-based immunoassays for detecting SARS-CoV-2 were introduced for rapid diagnosis of COVID-19. In this study, the clinical performance of the STANDARD Q COVID-19 Ag Test, an immunochromatographic assay, was evaluated in comparison with results obtained by the RT-PCR method.

There have been previous studies evaluating the clinical performance of RADTs for diagnosing COVID-19 infection. In a German study, 2,028 samples collected from a variable population including patients with COVID-19 symptoms, individuals with SARS-CoV-2 exposure, and hospital staff members showed overall sensitivity and specificity of 42.86 and 99.89%, respectively15. In a study from Serbia, 120 symptomatic patients were tested by the RADT, and overall sensitivity was about 60%16. In another study from Egypt performed with COVID-19 suspected individuals, the sensitivity and specificity were 78.2% and 64.2%, respectively17. A previous study conducted in Korea with pre-hospital patients, emergency room visitors, and patients confirmed with COVID-19 by RT-PCR demonstrated an overall sensitivity of 17.5% and a specificity of 100%18. This study was performed with a large number of clinical samples over 15 months, provided more reliable results reflecting practical aspects compared with other evaluation studies, and minimized bias that might occur in selection of study target samples. Overall sensitivity and specificity of the RADT were 0.577 (95% CI, 0.539–0.614) and 0.991 (95% CI, 0.989–0.993) respectively, which are comparable to the results reported in previous studies16,17,18. In addition, three cases of false-negative results in the cases with Ct value < 20. Clerical errors, inappropriate specimen handling, or interference in specimens could be potential reasons19. A total 120 cases (0.89%) of false-positive results were identified in this study. According to the previous report in Canada, the false positive results were identified in 0.05% (462/903,408) of asymptomatic screening tests20, and a false positive rate in another study in Germany was reported to be 0.13% (1561/1,245,962)21. False positive results may lead to unnecessary quarantine and delay of appropriate procedure to patients. The AUC of RADT was suboptimal (0.781) due to relatively low sensitivity and confirmation test with RT-PCR assay should be performed for accurate diagnosis of SARS-CoV-2 infection.

Significant correlation was identified between the Ct values obtained by RT-PCR and semiquantitative results by RADT, and the accuracy of RADT was very high, particularly in cases with high viral loads, exhibiting low Ct values22,23. In the German study, higher sensitivities were noted as Ct values < 20, < 25, and < 30 produced results of 100%, 98.25%, and 88.64%, respectively15. Therefore, the RADT could be accurate in diagnosing SARS-CoV-2 infection in the early stage, in which a high viral load is usually identified.

PPV and NPV are the most important indexes to determine clinical relevance and could vary according to disease prevalence. During the study period, steep increases in prevalence were observed in Periods 3 and 5, which correspond to the Delta and Omicron variant epidemics, respectively. As the prevalence increased, PPVs were confirmed to increase in Period 3 (0.951). The high PPV in the COVID-19 epidemic indicates that an individual with a positive RADT result is highly suspected to be infected with SARS-CoV-2 and should be required to promptly quarantine rather than undergo additional RT-PCR testing. Meanwhile, the PCR positive rate in Period 5 (13.7%) was higher than that in Period 3 (4.5%), but PPV in Period 5 was lower than that in Period 3. During the Period 5, the number of positive cases were dramatically increased up to 4.1% (55/1336), and Ct value of more than half of these cases were ≥ 25.00.

Most RADTs target the nucleocapsid protein stably associated with RNA in the virion and is found at higher level than other viral structural proteins24. Prior to the introduction of VOCs, the majority of RADTs was developed with strains that lacked nucleocapsid mutations25, and there have only been a few clinical validation studies that have examined variants affecting antigen test performance. From our results, it can be inferred that the Omicron and Delta variants, which are characterized by many mutations in the spike protein, had little effect on the accuracy of the nucleocapsid targeting RADT. Nonetheless, Omicron and Delta variants also harbor nucleocapsid proteins involving mutations (i.e., Omicron; P13L, ∆31–33, R203K & Delta; D63G, R203M, D377Y), in conjunction with the spike protein's conspicuous alterations26,27. While there are some false-negative reports of specimens carrying nucleocapsid mutations that show a high viral load as confirmed by PCR28,29, it is considered not to affect the accuracy of the RADT since current tests target the C-terminus of the nucleocapsid protein, whereas the great majority of mutations is identified in the N-terminus. In our study, no substantial shift or inclination in specificity was identified according to variant, which indicates the variants did not affect the clinical performance of the RADT. However, we must remain cautious to detect changes in the sensitivity of RADT due to variations with specific mutation trends, and additional studies are necessary to validate this point.

One of the limitations of this study is that the samples were collected in a single center in a single country; therefore, ethnic or racial diversities could not be considered. In addition, samples tested for pre-hospitalization screening were included; therefore, some asymptomatic patients were inevitably included in our study, which would lower the positive rate. The lack of further investigation about false-positive and false-negative results due to the retrospective study design was another limitation of this study. Finally, STADNARD Q COVID-19 Ag was approved for testing the detection of SARS-CoV-2 by Ministry of Food and Drug Safety of the Republic of Korea in both specimens including nasopharyngeal swab and VTM. Even though the limit of detection of this diagnostic kit in both specimens were validated, possible dilution effect could be occurred when using VTM as a specimen.

In conclusion, the STANDARD Q COVID-19 Ag Test showed lower sensitivity compared with RT-PCR; however, this RADT kit showed high PPV in the pandemic situation with a high prevalence of COVID-19 infection, which suggests this RADT kit could be an alternative option for rapid detection of SARS-CoV-2 in a pandemic situation. Further investigation should be performed to compare the results of other RADTs to figure out the clinical usefulness of RADT in diagnosing SARS-CoV-2 infection.