key: cord-0826757-n8q0h9oc
authors: Jian, Ming-Jr; Perng, Cherng-Lih; Chung, Hsing-Yi; Chang, Chih-Kai; Lin, Jung-Chung; Yeh, Kuo-Ming; Chen, Chien-Wen; Hsieh, Shan-Shan; Pan, Pin-Ching; Chang, Hao-Ting; Chang, Feng-Yee; Ho, Ching-Liang; Shang, Hung-Sheng
title: Clinical assessment of SARS-CoV-2 antigen rapid detection compared with RT-PCR assay for emerging variants at a high-throughput community testing site in Taiwan
date: 2021-11-26
journal: Int J Infect Dis
DOI: 10.1016/j.ijid.2021.11.034
sha: 164f69adad83ee514d394bbae96909346c93e24a
doc_id: 826757
cord_uid: n8q0h9oc

Objectives : With the emergence of the B.1.1.7 lineage in the ongoing COVID-19 pandemic, Taiwan confronted a COVID-19 flare up in May 2021. Large-scale, accurate, affordable, and rapid diagnostic tests such as the lateral flow assay can help prevent community transmission, but their performance characteristics in real-world conditions and relevant subpopulations remain unclear. Methods : The COVID-19 Antigen Rapid Test kit (Eternal Materials, New Taipei City, Taiwan) was used in a high-throughput community testing site; the paired reverse-transcription polymerase chain reaction (RT-PCR) results served as a reference for sensitivity and specificity calculations. Results : Of 2,096 specimens tested using the rapid antigen tests, 70 (3.33%) were positive and 2,026 (96.7%) were negative. This clinical performance was compared with the RT-PCR results. The sensitivity and specificity of the rapid SARS-CoV-2 antigen detection test were 76.39% (95% CI, 64.91–85.60%) and 99.26% (95% CI, 98.78–99.58%), respectively, with high sensitivity in subjects with threshold cycle ≤24. Further, the COVID-19 Antigen Rapid Test kit effectively detected the SARS-CoV-2 B.1.1.7 lineage. Conclusions : Considering the short turnaround times and lower costs, this simple SARS-CoV-2 antigen detection test for rapid screening combined with RT-PCR as a double confirmatory screening tool can facilitate the prevention of community transmission during COVID-19 emergencies.

A cluster of pneumonia cases with unknown etiology was reported and confirmed as 2019-nCoV in 2019 , Zhu et al., 2020 . Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 , has since spread globally. Reverse transcriptase polymerase chain reaction (RT-PCR), which is performed to detects viral nucleic acids, is currently considered as the diagnostic gold standard for early diagnosis in patients with suspected SARS-CoV-2 infection (Russo et al., 2020 , Safiabadi Tali et al., 2021 . As of December 26, 2020, Taiwan had only 783 confirmed COVID-19 cases because of its effective centralized quarantine policies (Tsai et al., 2021) . Household transmission accounts for the major infection sources apart from community-acquired COVID-19 outbreaks (Hsu et al., 2021) . However, Taiwan confronted a COVID-19 flare-up in May 2021; the number of confirmed COVID-19 cases surged by over ten-fold compared to those in 2020, with confirmed cases reaching 15,000 by July 15, 2021 (https://covid19.mohw.gov.tw/en/sp-timeline0-206.html). In addition to the recent surge in the number of indigenous COVID-19 cases in northern Taiwan, new SARS-CoV-2 variants of concern (VOCs) with possibly enhanced transmissibility and/or severity, as well as diagnostic and/or treatment failure, are important issues (Boehm et al., 2021, ofNordling and Wu, 2021) . SARS-CoV-2 lineages carrying the 6 amino acid substitution N501Y spread rapidly in the United Kingdom during late autumn in 2020, and thereafter in Taiwan (Leung et al., 2021, Nordling and Wu, 2021) . Taiwan faced the third epidemic wave (May-July 2021), forcing laboratories to the maximum of their testing capacity despite limited reagent supplies. Therefore, there is an increasing need for more rapid and feasible assays, such as lateral flow assays. Several reports have described the performance of rapid antigen test kits, most of which were derived from COVID-19 symptomatic subjects (Caruana et al., 2021) .

Previous studies reported that the clinical sensitivities of these assays vary widely (Dinnes et al., 2020 , Hayer et al., 2021 . However, the role of rapid antigen test kits as a screening tool in community-based test sites to test subjects with or without the symptoms of COVID-19 remains unclear. Thus, these SARS-CoV-2 rapid antigen tests with Emergency Use Authorization require rigorous evaluation of their performance characteristics in different epidemiologic settings.

The aim of this study was to evaluate the clinical performance of the COVID-19

Antigen Rapid Test kit (Eternal Materials, New Taipei City, Taiwan) compared with nucleic acid amplification testing (NAAT) using a multiplex quantitative RT-PCR kit with dual-target genes (E, N1) on the LabTurbo AIO 48 system (LabTurbo, Taipei City, Taiwan). We also independently evaluated the performance characteristics of the rapid antigen test for detecting SARS-CoV-2, particularly the VOC B.1.1.7 lineage.

This study was performed at Wanhua District of Taipei City in Taiwan during May 17-22, 2021, when the COVID-19 prevalence was 3.5% because of a sudden increase in local COVID-19 cases; subsequently, the Central Epidemic Command Center of Taiwan announced an epidemic warning of Level 3 nationwide. This study included 2,096 eligible subjects (symptomatic or non-symptomatic with a contact history with a confirmed COVID-19 case); a flowchart of the screening and confirmatory strategy is shown in Figure 1 . Two simultaneous nasopharyngeal swabs were collected for symptomatic or asymptomatic subjects aged 8-99 years at community testing sites using a standard procedure. The first swab was analyzed using the COVID-19

Antigen Rapid Test kit (Eternal Materials) which has a turnaround time of 15 min.

Subjects with positive results were immediately quarantined and asked to wait for the nucleic acid amplification test (NAAT) confirmation, performed on the LabTurbo AIO 48 system (Taipei City, Taiwan), an automated high-throughput and sample-to-results diagnostic platform. Residents with negative results were asked to self-health monitor until they received the NAAT report. This study was approved by the Institutional Review Board of the Tri-Service General Hospital (TSGHIRB No.: C202005041) and was registered on February 8, 2021. Informed consent was obtained 8 for this study. We defined positive cases using NAAT as the reference method.

The COVID-19 Antigen Rapid Test kit device (Eternal Materials) is a membrane-based immunochromatographic assay that has been granted Emergency Use Authorization by the Taiwan FDA; it detects the nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. A healthcare professional first collected the nasopharyngeal swab samples using the material in the COVID-19 Antigen Rapid Test kit. The assay was performed and interpreted by on-site technicians according to the manufacturer's instructions. Positive results were scored as positive and weakly positive based on the intensity of the test line compared to the control line. Band intensities stronger than the control line were scored as positive and those stronger than the background but weaker than the control line were scored as weakly positive.

The rapid test results were compared with those of the multiplex real time RT-PCR kit with dual target genes (E, N1) performed on the LabTurbo AIO 48 system.

Another nasopharyngeal swab specimen was collected from residents suspected of having COVID-19, using LIBO specimen collection and transport swab kits in 2 mL Universal Transport Medium (LIBO MEDICAL PRODUCTS INC., New Taipei City, Taiwan). An automated sample-to-result SARS-CoV-2 RT-PCR assay for 9 high-throughput testing was performed using the LabTurbo AIO 48 system, with the LabTurbo AIO COVID-19 RNA testing kit for SARS-CoV-2 multiplex real-time RT-PCR, containing reverse transcriptase, primer/probe mixture, and 2X PCR master mix, according to the manufacturer's instructions.

SARS-CoV-2 positive samples detected by both the rapid antigen test and RT-PCR were analyzed. To rapidly screen for SARS-CoV-2 VOCs (B.1.1.7 lineage), we used VirSNiP SARS-CoV-2 Spike N501Y and Spike del H69/V70 (TIB Molbiol, Berlin, Germany) to detect the N501Y and del 69-70 mutations, respectively, in SARS-CoV-2-positive specimens. Briefly, an RT-PCR variant assay with melting curve analysis was used to detect the spike gene mutations on a LightCycler 480 (Roche, Basel, Switzerland) as described previously (Jian et al., 2021 , Ong et al., 2021 .

Specificity and sensitivity with 95% confidence intervals and positive and negative predictive values of the COVID-19 Antigen Rapid Test kit were calculated using the RT-PCR results as a reference method. Analyses were performed using Microsoft Excel and GraphPad Prism version 8.0 software (GraphPad, Inc., San Diego, CA, USA).

Individuals at the community testing site were more often male (61.7%) and aged from 8 to 99 years. Overall, 2,096 subjects from the community testing site were included in the study, of which 72 tested positive for SARS-CoV-2 by RT-PCR (Table 1) , and the prevalence was around 3.5%. The threshold cycle (Ct) values of the positive results ranged from 12 to 29.

Of the 2,096 nasopharyngeal specimens, 70 (3.3%) were scored as weakly positive or positive and 2,026 (96.7%) were negative by the rapid test detecting the SARS-CoV-2 nucleocapsid protein (Table 1) . Further investigation of the clinical performance of the test was compared with the RT-PCR assay results. The overall sensitivity and specificity of the rapid SARS-CoV-2 antigen detection test were 76.39% and 99.26%, respectively ( Table 2 ). The rapid SARS-CoV-2 antigen tests implemented in our study showed moderate sensitivity but high specificity. Further, these tests showed a moderate positive predictive value (78.90%) and high negative predictive value (99.14%) ( Table 2 ). The sensitivity of the antigen test was stratified by the Ct values, and the test sensitivity was related to the viral load; the highest sensitivity was observed for RT-PCR Ct values under 20, moderate sensitivity for Ct values between 11 20 and 25, and decreased dramatically at Ct values above 25 (Table 3) .

To determine the clinical performance of the COVID-19 Antigen Test kit in detecting VOCs, 55 specimens that yielded positive results in both the rapid antigen test and RT-PCR were evaluated. As VOC B.1.1.7 began circulating in early September 2020, we also chose previously stored 50 SARS-CoV-2-specific RT-PCR-positive specimens collected between June and July 2020 for further identification as wild-type (non-B.1.1.7) or VOC (B.1.1.7) strains. The COVID-19 Antigen Rapid Test kit showed 100% positive agreement with RT-PCR as a reference method for the 55 positive VOC (B.1.1.7) SARS-CoV-2-positive specimens (Table 4) . Thus, the antigen assays presented here can effectively detect the B.1.1.7 variant of this outbreak strain.

Across all 2,096 tested subjects, 17 false-negative results were observed with Ct values between 21 and 29, whereas 15 false-positive results were observed for RT-PCR-negative results (Figure 2 ). Further investigation of the 15 false-positive cases showed that five cases were scored as weakly positive with faint bands by two independent readers, and the remaining 10 were scored as positive with clearly visible bands ( Figure 2 ). When paired rapid antigen detection and molecular diagnostic methods were performed simultaneously, false-negative results were rapidly corrected 12 by the confirmatory RT-PCR results on the next day, and the subjects were quarantined or admitted to dedicated hospital wards. The 15 subjects with presumed positive results by the rapid antigen test were isolated until they received the confirmatory RT-PCR results on the next day. Following a negative RT-PCR confirmatory result, the subjects discontinued isolation and continued self-health management.

In this study, we comprehensively and systematically evaluated the clinical performance of the COVID-19 Antigen Rapid Test kit (Eternal Materials) in a community setting including symptomatic and asymptomatic subjects. In this real-life evaluation of the rapid antigen test kit in the community, SARS-CoV-2-infected subjects with low Ct values were identified by RT-PCR (Ct values < 20). As expected, the 100% specificity and 100% sensitivity mentioned in the manufacturer package inserts were not observed in the clinical evaluation in a community setting.

Our study revealed the actual test performance, particularly in symptomatic or asymptomatic subjects. Previous studies demonstrated the advantages and clinical performance of rapid antigen test devices for detecting circulating VOCs (Jungnick et al., 2021; Rodgers et al., 2021) ; however, these data showed favorable performance in detecting SARS-CoV-2 VOCs using rapid antigen testing with only limited clinical samples in a UK population or viruses derived from cell culture. We found that the COVID-19 Antigen Rapid Test could detect the SARS-CoV-2 B.1.1.7 variant, particularly during emergence of the Taiwan third wave B.1.1.7. As lateral flow assays suffer from subjective interpretation, which may lead to difficulties in analyzing weakly positive bands, we also addressed the weakly positive results along with the extended edge-to-edge of the test strip, which may pose major questions for technicians' visual interpretation. For example, 23 weakly positive results were reported by the rapid antigen test in our study, although only 56% of these results were true-positives using RT-PCR as the reference method. Subjective interpretation of uncertain results is a major limitation of lateral flow assays (Pegoraro et al., 2021) . Peeling et al. (2021) showed that most detected cases representing false-positives rather than true infections may require a two-tier approach for molecular confirmation. As the inherent lower sensitivity may be offset by combining this approach with NAAT methods, we performed paired RT-PCR to counter the limitation of rapid antigen tests because of its lower sensitivity in blocking COVID-19 community transmission. Lateral flow antigen detection diagnostics have long been used for various infectious diseases considering their convenience and short turnaround times of within 15 min (Dinnes et al., 2021) . In the current COVID-19 pandemic, rapid SARS-CoV-2 testing kits aid in the rapid assessment of infectiousness in densely populated areas affected by the epidemic and have been widely applied in many countries to contain the pandemic. In general, lateral flow assays are effective when subjects are symptomatic or have a contact history with confirmed COVID-19 cases. However, SARS-CoV-2 infection can be asymptomatic, and the viral load in the upper respiratory tract in these cases is comparably low to that of symptomatic cases (Han et al., 2020; Ra et al., 2021) .

In summary, the COVID-19 antigen rapid test kit evaluated in this study can detect SARS-CoV-2 infection with high viral loads in both asymptomatic and symptomatic individuals. Thus, the COVID-19 Antigen Rapid Test kit can serve as a rapid tool for blocking the community spread of SARS-CoV-2. The limitations of our study include its cross-sectional design and the fact that we did not distinguish symptomatic or non-symptomatic participants, which may influence the sensitivity of this rapid antigen assay. Additional studies of the field performance of this assay in different settings is needed to develop optimal use strategies.

The authors report no conflicts of interest associated with this work. 

Novel SARS-CoV-2 variants: the pandemics within the pandemic

The dark side of SARS-CoV-2 rapid antigen testing: screening asymptomatic patients

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

COVID-19: Insight into the asymptomatic SARS-COV-2 infection and transmission

Real-world clinical performance of commercial 17

SARS-CoV-2 rapid antigen tests in suspected COVID-19: A systematic meta-analysis of available data as of

Household transmission but without the community-acquired outbreak of COVID-19 in Taiwan

Clinical Comparison of Three Sample-to-Answer Systems for Detecting SARS-CoV-2 in B.1.1.7 Lineage Emergence

Detection of the new SARS-CoV-2 variants of concern B.1.1.7 and B.1.351 in five SARS-CoV-2 rapid antigen tests (RATs)

Early transmissibility assessment of the N501Y mutant strains of SARS-CoV-2 in the United Kingdom

Taiwan on track to end third COVID-19 community 18 outbreak

Rapid screening method for the detection of SARS-CoV-2 variants of concern

Scaling up COVID-19 rapid antigen tests: promises and challenges

Evaluation of three immunochromatographic tests in COVID-19 serologic diagnosis and their clinical usefulness

Upper respiratory viral load in asymptomatic individuals and mildly symptomatic patients with SARS-CoV-2 infection

Detection of SARS-CoV-2 variants by Abbott molecular, antigen, and serological tests

Current status of laboratory diagnosis for COVID-19: a narrative review

Tools and techniques for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)/COVID-19 detection

Taiwan can help-Community protection net: preventive effects of the Northern Taiwan Centralized Quarantine Center against SARS-CoV-2

The SARS-CoV-2 outbreak: What we know

A novel coronavirus from patients with pneumonia in China

Figure 1. Flowchart of the study using the rapid antigen test as a screening tool in combination with RT-PCR for SARS-CoV-2 infection in densely populated areas. #1 CECC: Central Epidemic Command Center in Taiwan