key: cord-0908440-2zq1h5wg
authors: Gremmels, Hendrik; Winkel, Beatrice M.F.; Schuurman, Rob; Rosingh, Andert; Rigter, Nicolette A.M.; Rodriguez, Olga; Ubijaan, Johan; Wensing, Annemarie M.J.; Bonten, Marc J.M.; Hofstra, L.Marije
title: Real-life validation of the Panbio™ COVID-19 antigen rapid test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection
date: 2020-12-05
journal: EClinicalMedicine
DOI: 10.1016/j.eclinm.2020.100677
sha: eba994fe0c8d8dea769576b36e9cfc744ef2b338
doc_id: 908440
cord_uid: 2zq1h5wg

BACKGROUND: RT-qPCR is the reference test for identification of active SARS-CoV-2 infection, but is associated with diagnostic delay. Antigen detection assays can generate results within 20 min and outside of laboratory settings. Yet, their diagnostic test performance in real life settings has not been determined. METHODS: The diagnostic value of the Panbio™ COVID-19 Ag Rapid Test (Abbott), was determined in comparison to RT-qPCR (Seegene Allplex) in community-dwelling mildly symptomatic subjects in a medium (Utrecht, the Netherlands) and high endemic area (Aruba), using two concurrently obtained nasopharyngeal swabs. Findings: 1367 and 208 subjects were enrolled in Utrecht and Aruba, respectively. SARS-CoV-2 prevalence, based on RT-qPCR, was 10.2% (n = 139) and 30.3% (n = 63) in Utrecht and Aruba respectively. Specificity of the Panbio™ COVID-19 Ag Rapid Test was 100% (95%CI: 99.7–100%) in both settings. Test sensitivity was 72.6% (95%CI: 64.5–79.9%) in the Netherlands and 81.0% (95% CI: 69.0–89.8%) in Aruba. Probability of false negative results was associated with RT-qPCR Ct-values, but not with duration of symptoms. Restricting RT-qPCR test positivity to Ct-values <32 yielded test sensitivities of 95.2% (95%CI: 89.3–98.5%) in Utrecht and 98.0% (95%CI: 89.2–99.95%) in Aruba. INTERPRETATION: In community-dwelling subjects with mild respiratory symptoms the Panbio™ COVID-19 Ag Rapid Test had 100% specificity, and a sensitivity above 95% for nasopharyngeal samples when using Ct-values <32 cycles as cut-off for RT-qPCR test positivity. Considering short turnaround times, user friendliness, low costs and opportunities for decentralized testing, this test can improve our efforts to control transmission of SARS-CoV-2.

The SARS-CoV-2 pandemic has extensive impact on healthcare globally, with over 37 million confirmed cases and currently more than one million deaths [1] . Rapid diagnosis of SARS-CoV-2 infection and subsequent contact tracing are essential in the containment of transmission [2] .

The reference test for detection of acute SARS-CoV-2 infection is reverse transcriptase quantitative polymerase chain reaction (RT-qPCR) [3] . RT-qPCR requires the use of expensive laboratory instrumentation as well as dedicated lab supplies and trained personnel, which causes significant challenges to generate sufficient testing capacity and short turnaround times.

Lateral Flow Assay (LFA)-based point of care tests (POCT) for rapid antigen detection using antibodies are cheap, simple to perform, do not require laboratory instrumentation and generate results within 20 min [4] . Several different rapid antigen tests have been developed with usually high specificity but varying sensitivity [5À10] . These tests have the potential to alter testing strategies worldwide [8] .

Recently the WHO approved the first rapid diagnostic POCT and initiated a global partnership to pledge 120 million tests to lowand middle-income countries [11] . However, the diagnostic performance of individual POCTs in real-life community settings is unknown [10] .

We evaluated the Abbott Panbio TM COVID-19 Ag rapid test in community testing locations in both a medium-and high endemic population and compared results to RT-qPCR and determined associations with duration of symptoms and risk of exposure.

All individuals visiting COVID-19 community testing centers, located at the University Medical Center Utrecht (UMCU) in the Netherlands (September 22nd to October 6th 2020) and the Horacio Oduber Hospital on Aruba (September 23rd to October 9th 2020), aged 16 and over were asked to participate in this prospective evaluation. These settings were chosen based on the different SARS-CoV-2 prevalence at onset of the study (approximately 4% in Utrecht compared to approximately 30% in Aruba), while using the same platform for the reference RT-qPCR test.

In both study sites, subjects were first sampled for routine RT-qPCR testing, using a combined throat/nasopharyngeal swab. Study participants received an additional nasopharyngeal swab. Participants at the Utrecht study site were asked to fill out a questionnaire regarding (onset of) symptoms and risk of exposure to SARS-CoV-2. Inclusion of participants was continued until the target of 100 positive LFA results, as recommended by the National Institute for Public Health and the Environment (RIVM), was obtained.

PCR was conducted in a certified clinical laboratory and all procedures were validated according to the ISO 15,189 standard. After collection, swabs were transferred into 3 ml Universal transport medium until further processing. Nucleic acid extraction, RT-PCR and results interpretation were performed according the instructions of the manufacturer (Seegene, South-Korea). In short, RNA was isolated and purified using the MagC extraction kit (Seegene, South-Korea) on an automatic nucleic acid extractor Hamilton MicroLAB StartLET (Bonaduz, Switzerland). Subsequently, cDNA was generated and amplification was performed in a single tube assay using the Allplex 19-nCoV multiplex platform for detection of SARS-CoV-2 (Seegene, South-Korea), and results were interpreted with Seegene Viewer data analysis software. The assay uses fluorescent Taqman Ò probes for three SARS-CoV-2 genes (E [Envelope], N [Nucleocapsid]-, and RdRP [RNA dependent RNA Polymerase]genes). Amplification and detection were performed for 45 cycles on a Biorad CFX96 thermocycler (Biorad Laboratories, the Netherlands), the threshold Cycle (Ct) was automatically determined by the manufacturer's software. A positive result was defined as amplification of any of the three SARS-CoV-2 genes. If not all targets showed a positive result, this always corresponded with high Ct-values, suggesting low levels of SARS-CoV-2 RNA. If viral RNA levels are very low and around the limit of detection of the assay, amplification of these targets is more subject to stochasticity which can result in positive results in only one or two targets. Based on our experience within clinical practice and results from viral culture studies, [12] we use a cut-off Ct-value of 32 to determine clinically relevant levels of SARS-CoV-2 RNA.

The Panbio TM COVID-19 Ag rapid test device by Abbott (Lake Country, IL, U.S.A) is a membrane-based immunochromatography assay which detects the nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. Collected swabs were transferred into dedicated sample collection tubes containing a sampling buffer and transported to the laboratory. The laboratory is located within 5 min of walking distance from the sampling location. All samples were analysed within a maximum of 2 h after collection, and in practice the delay was much shorter (between 30 and 45 min on average), during which time the samples were kept at ambient temperature, which should have very little effect on relatively stable viral proteins. The PCR samples are processed similarly with no perceivable negative effect. Collected samples were subsequently processed in a level 2 biosafety cabinet in accordance with the manufacturer's protocol. Test results were recorded after 15 min of assay initiation, by two independent observers (blinded to each other and to the PCR results). Single lot LFA testing devices were used: lot 41ADF011A.

The medical research ethics committee (MREC) of Utrecht decided the study is not subject to the Medical Research Involving Human Subjects Act (WMO) and did not require full review by an accredited MREC. The ethical committee of the hospital board of Aruba approved the study. All participants have provided written informed consent.

Evidence before this study

We searched PubMed and MedRxiv for articles published with titles that included the search terms ("covid*"or "coronavirus"or "SARS-CoV-2 00 ) AND title or abstracts that included (antigen test" or "POCT"). Different rapid antigen tests have been developed with usually high specificity but varying sensitivity. The manufacturer of the Panbio TM COVID-19 Ag rapid test reported a very high sensitivity. Real-life data is only available from a very small Spanish cohort in a hospital setting. There is no data yet on the diagnostic performance of this test in individuals with mild symptoms within the community.

Testing and subsequent isolation and contact tracing is crucial for the global efforts to halt the ongoing SARS-CoV-2 pandemic. Rapid diagnostic tests have the potential to benefit testing strategies as they have short turnaround times, are cheap, and can be used in decentralized testing. Their potential hinges on their diagnostic performance, which has not been sufficiently determined to date. Our findings show that the Panbio TM COVID-19 Ag rapid test reliably identifies SARS-CoV-2 infected individuals with high viral load in nasopharyngeal samples, in a cohort of community-dwelling subjects with mild symptoms of respiratory tract infection. Specificity of this test was 100%. Although the sensitivity is lower than the RT-qPCR, false negative rapid test results were all due to low viral loads in nasopharyngeal samples.

Due to the lower sensitivity of the Panbio TM COVID-19 Ag rapid test, RT-qPCR would be the preferred diagnostic test of choice for clinical purposes in a hospital setting. However, for surveillance of SARS-CoV-2 within the community, this rapid antigen test reliably and rapidly identifies individuals with high potential of further transmission, and could therefore be an essential new tool in our testing strategies to control transmission of SARS-CoV-2.

Population characteristics are reported as mean (Standard Deviation, SD) or median [InterQuartile Range, IQR] values. Difference testing for comparisons of groups was performed by Chi-square testing for categorical variables, independent samples Student's t-tests with Welch's correction for continuous normally distributed variables and by using Mann-Whitney U tests for not non-normally distributed variables.

Specificity and sensitivity with 95% confidence intervals, and positive and negative predictive value of the LFA were calculated using the RT-qPCR results as reference test. Factors associated with LFA results were determined using logistic regression, using Nagelkerke's pseudo R 2 as a measure of goodness-of-fit. Data was analysed using the free, open-source software environment R [13] .

This study was investigator initiated. No external funding was received.

At the Utrecht study site 1369 subjects were included, of which 139 tested positive for SARS-CoV-2 by RT-qPCR (prevalence: 10.2%). Estimated participation rates were 50% at the Utrecht study site and 25% at the Aruba study site, based on the total number of samples processed at each site during the study period. Samples from two participants were excluded, due to inappropriate application of the nasopharyngeal swab and laboratory mislabelling. Symptom registration forms were absent from nine subjects and only incomplete data on duration of symptoms was available from 201 subjects (14.7% of total subjects, 11.9% of SARS-CoV-2 positive subjects, p = 0.439).

Individuals at the Utrecht study site were more often female (61.7%) and were largely between 20 and 50 years of age (Table 1) . Nearly all individuals reported symptoms (97.3%), most frequently coryza (69.0%), sore throat (66.3%) and cough (57.1%). Duration of symptoms were 1À3 days in 387 subjects (33.2%), 4À7 days in 560 subjects (48.0%), and more than a week in 191 subjects (16.4%). Of all individuals, 17% reported prior contact with a confirmed SARS-CoV-2 positive individual.

Compared to subjects who tested negative for SARS-CoV-2 in RT-qPCR, SARS-CoV-2 positive subjects were younger (p = 0.034), more likely male (p = 0.016), more frequently had prior contact with a confirmed SARS-CoV-2 positive individual (p<0.001), and more frequently reported fever, chills, an altered sense of smell or taste, or joint-or muscle ache (p<0.001 for all symptoms, see table 1). The most frequently reported symptoms, coryza and sore throat, were negatively associated with detection of SARS-CoV-2 by RT-qPCR (p<0.001).

At the Utrecht study site, 101 subjects tested positive by LFA yielding an overall sensitivity of 72.6% (95% Confidence Interval, CI: 64.5 À 79.9%) ( Table 2 ). False positive LFA results were not observed (specificity 100%, 95% CI: 99.7 À 100%). Similar results were obtained at the Aruba study site, with an overall sensitivity of 81.0% (95% CI: 69.0 À 89.8%) and specificity of 100% (95% CI: 97.5 À 100%) ( Table 2) . We observed no inter-rater variability in interpretation of test bands and no bands were classified as unclear by the independent observers. False negative LFA results were mostly observed in subjects with high RT-qPCR Ct-values, reflecting low viral load levels in nasopharyngeal material (Fig. 1) . Using logistic regression, the likelihood of a false negative LFA result was associated with the RT-qPCR Ct-value (R 2 = 0.77, p<0.0001, Fig. 2 ). When defining RT-qPCR Ct positivity on a cut-off Ct-value of 32 LFA sensitivity was 95.2% (95% CI: 89.3 À 98.5%) at the Utrecht site and 98.0% (89.2 À 99.95%) at the Aruba study site.

The duration of symptoms was not associated with Ct-values (p = 0.46) or with the occurrence of false negative LFA results (p = 0.30) (Fig. 3) . When including only symptoms positively associated with COVID-19 (i.e., fever, chills, loss of taste/smell, muscle-or joint ache), duration of symptoms was weakly associated with Ct-values (p = 0.02), but no such association was found with LFA results (p = 0.45). Restricting analyses to individuals with symptoms for less than 7 days did not change sensitivity of LFA neither for all subjects positive with RT-qPCR (74.3%; 95%CI 64.6 À 82.4%) or when applying a cut-off at Ct>32 (97.4%; 95%CI 90.9 À 99.7%).

In this real-life evaluation of the Panbio TM COVID-19 Ag rapid test in community-dwelling subjects with mild symptoms of respiratory tract infection, the assay reliably identified SARS-CoV-2 infected subjects with low Ct-values by RT-qPCR (i.e. infections with a high viral load in nasopharyngeal samples). In our study cohorts specificity was 100%, overall sensitivity was 72.6% and 95.2% when using a Ct-value of 32 as cut-off.

This new LFA test has not been evaluated extensively. The manufacturer reported a higher sensitivity (93.3%; 95CI 83.8À98.2), obtained in a high endemic setting in Brazil, testing individuals with symptoms for less than seven days only [14] . In another cohort of 257 patients (both symptomatic and asymptomatic) enrolled at the emergency department and primary health care setting in Spain, overall sensitivity was 73.3%, and 86.5% among individuals with symptoms for less than seven days [9] .

In our study cohort, false negative results were observed only at high Ct-values, i.e. with low viral load in nasopharyngeal material. This may occur very early in the infection (presymptomatic stage) before viral replication peaks, or in a late stage of infection when replication has decreased. Individuals with symptoms for more than 7 days may, therefore, be more likely to have a low viral load in nasopharyngeal swabs than those tested shortly after symptom onset. Indeed, a longer duration of symptoms associated with SARS-CoV-2 infection, was associated with higher Ct-values, but no such association was observed between false negative LFA results and duration of symptoms. As a result, test sensitivity did not substantially increase when analyses were restricted to individuals with symptoms for less than a week. It must be noted that the study period corresponded with high prevalence of seasonal rhinovirus and other respiratory infections in the Netherlands, [15, 16] thereby possibly obscuring the association of upper respiratory tract symptoms with SARS-CoV-2 infection.

From a public healthcare perspective, missed infections with the LFA in patients with high Ct-values in a late stage of infection may have limited impact, as these individuals are less likely to contribute to transmission. This is supported by findings that culturing of SARS-CoV-2 appeared not possible at Ct-values above 29 [12] .

Yet, false negative LFA results were also observed among individuals with a very short duration of symptoms and high Ct-values. A missed infection in the early stage may have consequences for Sensitivity and specificity are reported with 95% CI. transmission as these individuals may become infectious [17À19]. Furthermore, the majority of individuals in our cohort reported (mild) symptoms. Studies are needed to determine the sensitivity of this LFA test in asymptomatic subjects with detectable SARS-CoV-2 in RT-qPCR. These studies could also provide more insight in the performance of this LFA in the presymptomatic stage of infection. For symptomatic persons requiring hospital admission a diagnostic test with high sensitivity is needed to establish a definitive clinical diagnosis [20] . In this clinical context, we recommend to use RT-qPCR, as its sensitivity is superior to this LFA, allowing detection of individuals in the presymptomatic and late stage of infection which are both relevant in this context. However, in the context of community-based surveillance, tests need to identify symptomatic and asymptomatic infections in short time to stop onward spread. Transmission of SARS-CoV-2 is considered to occur mainly around symptom onset, when viral load peaks [21, 22] . This LFA, therefore, appears to reliably identify those patients that are most likely to contribute to onward transmission and could therefore be an essential new tool in our testing strategies. As the virus remains infectious in the assay sampling buffer (data not shown), biosafety should be taken into account when implementing the use of the Panbio TM COVIDÀ19 LFA outside a laboratory setting.

In the design of the present study many variables that may affect test outcome and interpretation were controlled as much as possible. For instance, all tests were conducted in laboratory conditions, with controlled temperature and lighting; interpreted by a limited pool of trained operators, and a single lot of assays was used throughout the study. This limits the generalizability of the results to some extent, as widespread, decentralized community testing will potentially occur under less well-controlled circumstances. A surprising finding is the perfect inter-rater agreement observed in this study, which occurred as there were no inconclusive bands observed. This finding could be due to the controlled laboratory conditions or the use of a single lot.

Considering the short turnaround time, user friendliness, opportunity for decentralized testing and low costs we believe that in the context of community-based surveillance of symptomatic individuals, these advantages outweighs the lower sensitivity of LFA compared to RT-qPCR. Modeling studies may provide further reassurance on the safety of such an approach.

Dr. Wensing reports grants from ViiV Healthcare, Gilead, grants from Janssen, Gilead, Merck, grants from Janssen, Gilead, Merck, ViiV Healthcare, grants from ARK diagnostics, outside the submitted work. All other authors report no conflict. 

An interactive web-based dashboard to track COVID-19 in real time

Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

SARS-CoV-2 coronavirus nucleocapsid antigen-detecting half-strip lateral flow assay toward the development of point of care tests using commercially available reagents

Access date

Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients

Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis

Fast coronavirus tests: what they can and can't do

Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms

point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Access date

Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants

A language and environment for statistical computing R foundation for statistical computing

Access date

A molecular epidemiological perspective of rhinovirus types circulating in Amsterdam from

Seasonal coronavirus protective immunity is short-lasting

Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2)

Presumed Asymptomatic Carrier Transmission of COVID-19

Transmission of 2019-NCOV infection from an asymptomatic contact in Germany

Rethinking Covid-19 Test Sensitivity -A Strategy for Containment

Virological assessment of hospitalized patients with COVID-2019

Temporal dynamics in viral shedding and transmissibility of COVID-19

The authors would like to thank all participating employees from the GGD Utrecht and the department of Public Health in Aruba and the participating students from the UMCU for their help with data collection, as well as all participating volunteers.

This study was investigator initiated. No external funding was received.

All de-identified participant available data can be made available upon request per email to the corresponding author.