key: cord-0719325-9t3xkl9a
authors: Koeleman, Johannes G. M.; Brand, Henk; de Man, Stijn J.; Ong, David S. Y.
title: Clinical evaluation of rapid point-of-care antigen tests for diagnosis of SARS-CoV-2 infection
date: 2021-05-22
journal: Eur J Clin Microbiol Infect Dis
DOI: 10.1007/s10096-021-04274-7
sha: 5612aa3f43997d68d29e0d39ef12bad55652406a
doc_id: 719325
cord_uid: 9t3xkl9a

The RT-qPCR in respiratory specimens is the gold standard for diagnosing acute COVID-19 infections. However, this test takes considerable time before test results become available, thereby delaying patients from being diagnosed, treated, and isolated immediately. Rapid antigen tests could overcome this problem. In the first study, clinical performances of five rapid antigen tests were compared to RT-qPCR in upper respiratory specimens from 40 patients with positive and 40 with negative RTq-PCR results. In the second study, the rapid antigen test with one of the best test characteristics (Romed) was evaluated in a large prospective collection of upper respiratory specimens from 900 different COVID-19-suspected patients (300 emergency room patients, 300 nursing home patients, and 300 health care workers). Test specificities ranged from 87.5 to 100.0%, and test sensitivities from 55.0 to 80.0%. The clinical specificity of the Romed test was 99.8% (95% CI 98.9–100). Overall clinical sensitivity in the study population was 73.3% (95% CI 67.9–78.2), whereas sensitivity in the different patient groups varied from 65.3 to 86.7%. Sensitivity was 83.0 to 86.7% in patients with short duration of symptoms. In a population with a COVID-19 prevalence of 1%, the negative predictive value in all patients was 99.7%. There is a large variability in diagnostic performance between rapid antigen tests. The Romed rapid antigen test showed a good clinical performance in patients with high viral loads (RT-qPCR cycle threshold ≤30), which makes this antigen test suitable for rapid identification of COVID-19-infected health care workers and patients.

Accurate and early diagnosis of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is crucial for patient management and outbreak control of the coronavirus disease 2019 (COVID-19) pandemic. The quantitative reverse transcription polymerase chain reaction (RT-qPCR) assay for the detection of SARS-CoV-2 virus in respiratory specimens still remains the gold standard for diagnosing COVID-19 [1] .

A major drawback of this sensitive and specific molecular diagnostic method is the limited worldwide availability in combination with a relatively long turnaround time.

The development of rapid diagnostic tests allows faster identification of COVID-19 patients and enables the prompt implementation of infection prevention and control measures. Therefore, a large number of COVID-19 point-of-care (POC) antigen tests with rapid turnaround time and minimal need for instrumentation have been developed and introduced recently [2] [3] [4] [5] [6] [7] . These easy to perform and inexpensive POC tests based on lateral flow immunochromatographic assays (LFAs) can detect nucleocapsid protein from SARS-CoV-2 in nasopharyngeal specimens within 20 min, which makes them ideal for use in patient care settings but also in the community [8] .

Until now, little is known about the SARS-CoV-2 antigen test performance in different patient groups. In this study, we evaluated the clinical performance of different LFAs compared to RT-qPCR using upper respiratory specimens from several patient groups with suspected COVID-19.

For both studies described here, a prospective collection of upper respiratory specimens sent to the microbiology laboratory was used. In the periods from October 6 to October 12 (study 1) and October 24 to November 15, 2020 (study 2), samples were obtained from different COVID-19-suspected emergency room patients (ERP), nursing home patients (NHP), and health care workers (HCW) with acute onset of one or more respiratory symptoms (cough, sore throat, dyspnea, coryza, anosmia, ageusia) with or without fever. During the 19-day study period, the mean proportion of newly confirmed RT-qPCR-positive COVID-19 cases per day was 23%. The study was conducted in a teaching hospital (Franciscus Gasthuis & Vlietland, Rotterdam, the Netherlands).

Patients and HCW with suspected COVID-19 infection were sampled by one combined throat nasopharyngeal swab. After this, swabs were placed in 3 ml virus transport medium (VTM) and stored at 4°C until sample preparation after which positive samples were stored at −20°C. All specimens were examined for SARS-CoV-2 viral RNA by routine RT-qPCR on the day of collection and for antigen detection by three of five LFAs within 72 h and by two LFAs 1 month after collection.

The first study involved a comparison of five different COVID-19 rapid antigen tests for the detection of SARS-CoV-2 viral antigens. From October 6 to October 12, 2020, consecutive patient samples were included for a total of 40 RT-qPCR-negative and 40 RT-qPCR-positive samples with a proportional number of PCR-positive samples divided into different cycle threshold (C T ) categories (C T < 20, C T 20-30 and C T > 30). In the first part, three LFAs available at that time were evaluated: Certest SARS-CoV-2 (Certest Biotec S.L., Spain), Roche SARS-CoV-2 Rapid Antigen Test (Roche, Switzerland), and Romed Coronavirus Ag Rapid Test (Romed, The Netherlands). In the second part, the Romed Coronavirus Ag Rapid Test and two WHO-recommended LFAs, i.e., BD Veritor SARS-CoV-2 point-of-care test (Becton, Dickinson and Company, USA) and Panbio™ COVID-19 Antigen rapid test (Abbott, USA), were evaluated with 40 RT-qPCR-positive samples: 35 samples that were stored at −20°C from the first part, completed with five additional RT-qPCR-positive samples with corresponding C T values of the missing samples. The BD Veritor and Panbio™ LFAs were not available initially and were therefore added in the separate second comparison at a later moment.

In the second prospective study, the clinical performance of the Romed LFA was compared to RT-qPCR in an extended cohort of patients and HCW of which consecutive samples were included between October 24 and November 15, 2020. During the inclusion, patient categories (ERP, NHP, or HCW) and PCR results (positive and negative) were recorded. Inclusion of new samples in a particular patient category was stopped after a predefined number of 300 PCR-positive and 600 PCR-negative samples had been included divided among the three defined patient groups.

Samples from NHP and HCW were tested on two different RT-qPCR methods by either a validated in-house RT-qPCR assay or on the ELITe InGenius® (Elitech, France) platform [9] . Samples from ERP were tested with the GeneXpert Xpress SARS-CoV-2 PCR assay (Cepheid Inc., Sunnyvale, USA) according to the instructions of the manufacturer.

For antigen extraction, 350 μl of VTM was added to 300 μl of each respective extraction buffer and mixed for 10 s. Subsequently, a number of drops of the mixture were added to the sample port of the antigen assay according to the instructions of the manufacturer. The result was read visually after 15 min whereby any shade of color in the test line region was considered positive. All tests were independently assessed by two investigators who were blinded to all other test results, and in case of discrepancy, an additional assessment was performed by a third investigator.

The Institutional Review Board waived the need for informed consent because tests were performed on samples that had been required for routine microbiological investigation (IRB protocol number 2020-109). Also according to hospital procedure, all patients were informed about the possibility of an opt-out if they had objections against the use of leftover material for research to improve or validate diagnostic testing procedures. The study was performed in accordance with Helsinki Declaration as revised in 2013.

The primary outcome measures for both studies were clinical specificity and clinical sensitivity in relation to different C T values of the RT-qPCR. For the second study, positive predictive value (PPV) and negative predictive value (NPV) were also calculated as secondary outcomes in order to develop a diagnostic algorithm in different patient groups. All data were analyzed using Microsoft Excel, GraphPad Prism version 8, and R version 3.3.2 (R Foundation for Statistical Computing). Groups were compared by using Mann-Whitney U test for continuous variables and chi-square test or Fisher's exact test for categorical variables as appropriate. Values of p that were < 0.05 were considered to be statistically significant.

In part one of this study, three COVID-19 rapid antigen tests were compared to RT-qPCR in 80 selected specimens of which 40 were RT-qPCR negative and 40 were positive with different viral loads. The performance of the LFAs varied greatly as shown in Table 1 , with an overall sensitivity ranging from 55.0% (95% confidence interval (CI) 38.7-70.4) (Certest) to 72.5% (95% CI 55.9-84.9) (Romed). In specimens with a high viral load (C T 30 or lower), sensitivity of all the assays increased. The specificity was 87.5% or higher for all LFAs, and the Romed test showed 100% specificity.

Clinical sensitivity of the BD Veritor and the Panbio™ LFAs was 77.5% (95% CI 61.1-88.6) and 70.0% (95% CI 53.3-82.9), respectively. In order to compare the clinical sensitivity of all the LFAs tested, the Romed antigen test was also performed with these samples. The sensitivity of the Romed in the second comparison increased to 80.0% (95% CI 63. 8-90.4) indicating that storage of the samples did not affect LFA results and thus allows a sensitivity comparison of all LFAs tested. From all the five LFAs tested, the Romed LFA showed the highest clinical sensitivity. Based on these results, we decided to continue with the Romed antigen test in the second study as it had one of the best test characteristics.

A total of 900 throat nasopharyngeal swabs (300 in each patient category: ERP, NHP, and HCW) were prospectively selected. This resulted in 300 (33.3%) samples that were tested positive for SARS-CoV-2 by RT-qPCR. Compared to RT-qPCR, the clinical specificity of the Romed test was 99.8% (95% CI 98.9-100) ( negative patients (p < 0.01), and C T values were also statistically significantly different between LFA-positive and LFAnegative cases in each subgroup (all p < 0.01) (Fig. 1 ). In the group of ERP, the sensitivity of the antigen test on RT-qPCRpositive samples with a C T value lower than 30 was 90.6%

(95% CI 82.9-95.1). Clinical sensitivity of 100% was found in all patient categories with C T values below 20 which corresponds with high viral loads. The LFA results in ERP and HCW in this study showed a high sensitivity in samples obtained during the first week of symptoms. This was seen in HCW of whom the majority was tested within the first week after symptom onset and a clinical sensitivity of 86.7% was found. For ERP with symptoms less than 7 days and 7 days or more since onset, the sensitivity was 83.0% (95% CI 69.7-91.5) and 56.2% (95% CI 45.3-66.5) (p < 0.01), respectively, with significant lower C T values in the first group (p < 0.01) (Fig. 2) . Also, in this group, falsenegative LFA results were only seen in RT-qPCR-positive samples with high C T values (Fig. 3 ).

In this study, comparison of five different LFAs performed on patients suspected for COVID-19 showed a very high specificity and a moderate to high sensitivity ranging from 55 to 80%. Awareness of the lack of sensitivity in rapid antigen tests is important especially considering the criteria established by the European Centre for Disease Prevention and Control and the World Health Organization [10]. Sensitivities of these LFAs increased above 90% when considering specimens with C T values below 30 (i.e., higher viral loads), for the two best Specificity, sensitivity, PPV, and NPV are reported with 95% CI. The PPV was calculated for 4 scenarios: 1% and 10% prevalence in a general population and 25 and 50% prevalence in a high-risk population performing antigen tests. Compared to RT-qPCR, samples from the upper respiratory tract (nasal or nasopharyngeal swabs) have shown a highly variable sensitivity, ranging from 0 to 94% but with a high reported specificity (i.e., more than 97%) [7] . Another recent study showed differences in sensitivity of commercially available rapid antigen LFAs, which correlates with the ability to detect infectious COVID-19 patients [11] . In our study, we also observed a wide variation in the sensitivity of the five LFAs tested with a high sensitivity found for the Romed antigen test.

In our second study, the clinical performance of the Romed LFA antigen test was evaluated in 900 patients showing an overall specificity and sensitivity of 99.8% and 73.3% respectively, compared to RT-qPCR. This is the first study in which the clinical performance of the Romed LFA in a large proportion of samples collected from different patients has been evaluated. Our study shows that the sensitivity of this LFA test was moderate in patients suspected for COVID-19 presenting to the hospital, but increased in patients with symptoms for less than a week and also in HCW with short duration of symptoms. This is in accordance with the higher sensitivity found in samples with a low C T value corresponding to infected patients with a high viral load. Recently, two other reports have evaluated the performance of a rapid antigen test for COVID-19 community screening in individuals with COVID-19-like symptoms [12, 13] . In these studies, specificities of 100% were found whereas sensitivities of two studies with the Panbio™ test were 72.6% and 81.0%, and of the one with the BD Veritor 80.7%. Another study in 150 emergency room patients and 105 from primary health care centers sampled during the first week of symptoms showed a Panbio™ sensitivity of 73% [14] . These results are comparable with the sensitivities in HCW (86.7%) and NHP (76.0%) found in our study both reflecting individuals and patients with a recent COVID-19 infection. The majority of false-negative results were found in samples with high C T levels corresponding with low viral loads and longer durations of symptoms. In addition, several studies have shown a clear association between high C T values of RT-qPCR (i.e., above 30) and low COVID-19 patient infectivity [15, 16] . Therefore, the Romed LFA could be used in admitted COVID-19 patients in order to determine their infectivity which offers the opportunity for earlier discontinuation of isolation.

The clinical performance of COVID-19 rapid antigen tests largely depends on the prevalence of COVID-19 as well as the different patient populations in which they are used. The negative predictive value increases when the prevalence decreases. In settings where the pre-test probability of having COVID-19 is low and symptom duration is short, a negative LFA antigen result could exclude that person from the need of further testing, whereas positive LFA antigen results need to be confirmed by RT-qPCR because the PPV is only moderate when the prevalence is low and it may result in substantial false-positives. According to our findings, this is especially the case for HCW in a low prevalence setting (Table 2) . Based on our results, the application of Romed LFAs in HCW with short duration of symptoms can reliably identify COVID-19-positive contagious individuals. In contrast, in ERP with a high pre-test probability of having COVID-19, more false-negative results were observed which makes the use of these LFAs in this setting less suitable. Finally, in addition to RT-qPCR testing, Romed LFAs could be used for frequent and repeat screening of nursing home residents in outbreak situations for rapid identification and isolation of COVID-19 patients in order to prevent further transmission at an early stage. Our study has several limitations. First, all LFAs were performed on VTM and not on the originally collected throat nasopharyngeal swabs in order not to affect the routine COVID-19 diagnostics by RT-qPCR and to be able to compare 5 different LFAs head-to-head. This could theoretically have influenced test characteristics; however, the results of our study are in line with the results of other studies published [12] [13] [14] . Second, due to the late availability of two of the five LFAs tested, positive samples were frozen and used after thawing which also is not according to the manufacturers' instructions. Therefore, the Romed LFA was included in both the first and the second LFAs antigen comparison. Importantly, we did not observe a decrease in sensitivity when tested after freezing and thawing. Third, samples used in the second study were partially selected (i.e., in favor of RT-qPCR-positive samples) in order to obtain a high number of positive samples to allow adequate assessments of sensitivity characteristics in the different patient groups.

In our study, the Romed LFAs have a high sensitivity and specificity in throat nasopharyngeal samples with high viral loads which make them most suitable for rapid identification and isolation of COVID-19-infected and contagious HCW and patients. The worldwide spread of SARS-CoV-2 has resulted in a large number of COVID-19 patients. In order to reduce or prevent further spread in health care facilities and in the community, quick and accurate identification of COVID-19 patients followed by quarantine measures is essential. Therefore, in addition to RT-qPCR, COVID-19 rapid antigen tests, which are simple, rapid, inexpensive, and appropriate for wide-scale use, offer the opportunity to help with this containment strategy.

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Comparison of the SARS-CoV-2 Rapid antigen test to the real star Sars-CoV-2 RT PCR kit

Comparative evaluation of four SARS-CoV-2 antigen tests in hospitalized patients

Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19

Multicenter evaluation of the Panbio™ COVID-19 rapid antigendetection test for the diagnosis of SARS

CoV-2 infection

Performance of an antigen-based test for asymptomatic and symptomatic SARS-CoV-2 testing at two university campuses -Wisconsin

Van den Bruel A, Cochrane COVID-19 Diagnostic Test Accuracy Group (2020) Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection (Review)

2020) Fast, portable tests come online to curb coronavirus pandemic

Comparison of the GeneFinderTM COVID-19 Plus RealAmp Kit on the sampleto-result Platform ELITe InGenius to the national reference method: an added value of N gene target detection?

From more testing to smart testing: dataguided SARS-CoV-2 testing choices

Antigen Rapid Test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection

Evaluation of the test accuracy of a SARS-CoV-2 rapid antigen test in symptomatic community dwelling individuals in the Netherlands

Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms

Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards

Duration and key determinants of infectious virus shedding in hospitalized patients with coronavirus disease-2019 (COVID-19)

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Acknowledgements We would like to thank our laboratory technicians and team managers for their assistance in performing the molecular tests and LFAs.

Author contribution Johannes Koeleman and David Ong contributed to the study conception and design. Material preparation, data collection, and analysis were performed by all authors. The first draft of the manuscript was written by Johannes Koeleman and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.