key: cord-0768076-qs8fcfy5
authors: Landaas, Elisabeth Toverud; Storm, Margrethe Larsdatter; Tollånes, Mette Christophersen; Barlinn, Regine; Kran, Anne-Marte Bakken; Bragstad, Karoline; Christensen, Andreas; Andreassen, Trude
title: Diagnostic performance of a SARS-CoV-2 rapid antigen test in a large, Norwegian cohort
date: 2021-03-10
journal: J Clin Virol
DOI: 10.1016/j.jcv.2021.104789
sha: 3bca8a938da4b56f3bdeb34f03ee471b33e606f8
doc_id: 768076
cord_uid: qs8fcfy5

BACKGROUND: Rapid antigen tests (RATs) may be included in national strategies for handling the SARS-CoV-2 pandemic, as they provide test results rapidly, are easily performed outside laboratories, and enable immediate contract tracing. However, before implementation further clinical evaluation of test sensitivity is warranted. OBJECTIVES: To examine the performance of Abbott’s Panbio(TM) COVID-19 Ag Rapid Test Device for SARS-CoV-2 testing in a low to medium prevalence setting in Norway. STUDY DESIGN: A prospective study comparing the results of the Panbio RAT with PCR in 4857 parallel samples collected at a SARS-CoV-2 test station in Oslo, and from COVID-19 outbreaks in six Norwegian municipalities. RESULTS: A total of 4857 cases were included in the study; 3991 and 866 cases from the test station and the outbreak municipalities, respectively. The prevalence at the test station in Oslo was 6.3 %, and the overall sensitivity of the RAT was 74 %. Increased sensitivity was observed in patients who experienced symptoms (79 %) and when considering samples with viral loads above estimated level of infectivity (84 %), while it was lower in asymptomatic persons (55 %). In the outbreak municipalities, the overall prevalence was 6.9 %, and the total sensitivity of the RAT was 70 %. CONCLUSIONS: Our results indicate that the test correctly identified most infectious individuals. Nevertheless, the sensitivity is considerably lower than for PCR, and it is important that the limitations of the test are kept in mind in the follow-up of tested individuals.

The Norwegian strategy to fight the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is reliant upon rapid identification of infected individuals, isolation of cases, contact tracing and quarantine.

In this respect, SARS-CoV-2 rapid antigen tests (RATs) could represent a good alternative to polymerase chain reaction (PCR) given its short turnaround time, which enables earlier initiation of isolation and infection control measurements. However, further data on the clinical performance of RATs is needed. Most

RATs are based on lateral flow immunochromatography using antibodies to target the SARS-CoV-2 nucleocapsid protein. Compared to PCR, the sensitivity of RATs is lower. A key question is whether a positive RAT result may correlate with infectiousness and correctly identify infectious persons in specific settings. The

World Health Organization has suggested that RATs could play a significant role in patient management and surveillance of coronavirus disease 2019 (COVID-19) given a sensitivity of ≥ 80 % and specificity of ≥ 97 % (1).

Analytical sensitivity and specificity measures given by the manufacturers do not necessarily reflect the actual performance of the test. Furthermore, evaluation studies from other countries might not be transferable to national outbreak settings, and independent and setting-specific validations of RATs before their implementation are therefore recommended by the European Centre for Disease Prevention and Control (2) . Thus, a field evaluation of Abbott's Panbio TM COVID-19 Ag Rapid Test Device (Panbio RAT) in Norwegian settings was initiated. The aim of the evaluation was to study the RAT's performance compared to SARS-CoV-2 PCR in 1) a low to medium prevalence setting at the COVID-19 test station Aker in Oslo, and 2) in outbreaks in more rural areas.

The evaluation study of the Panbio RAT diagnostic performance was conducted from 30th October to 25th

November 2020. Individuals who signed up for a COVID-19 test at Aker test station in Oslo were given J o u r n a l P r e -p r o o f information about the study at the Oslo municipality website and invited to participate. Individuals were eligible to participate if they were aged 10 years or older, competent to consent, and able to read and understand Norwegian. Persons without a Norwegian personal identification number were excluded from participation. Specially trained personnel at Aker test station performed the testing. Two parallel combined throat/nasopharyngeal swabs were obtained from each person. The first swab was sent to Oslo University Hospital (OUH) for routine SARS-CoV-2 PCR. The other was examined with the Panbio RAT at the test station, according to the manufacturer's instructions. All study participants were surveyed about known exposure and symptom duration. Municipality physicians in charge of infection control, experiencing COVID-19 outbreaks during the study period, were asked to organize their municipality's participation in the study.

Testing was performed following the same procedure as Aker test station, except that the first swab was sent to the local microbiology laboratory instead of to OUH. 

The study was presented to the Data Protection Officer for Research at the Norwegian Directorate of Health, who gave advice on how to safeguard privacy aspects in the study. All persons who met the inclusion criteria at Aker test station or in the outbreak municipalities were asked to give oral consent before taking part in the study.

During the study period, a total of 5412 samples were collected for PCR analysis. Out of the 5412 cases, 4025

gave their consent to participate in the study, giving a participation rate of 74 %. Among the 4025 cases, 83 individuals were tested twice, and two individuals were tested three times. These were all considered J o u r n a l P r e -p r o o f independent cases in subsequent analysis. Of the 4025 samples, 3998 were successfully matched to their corresponding PCR results in the database. As one PCR result was inconclusive, and six antigen tests were either inconclusive or defective, a total of 3991 cases were successfully included in the study.

As shown in Table 1 At a prevalence of 6.3 %, the positive predictive value (PPV) of the RAT was 0.984, while its negative predictive value (NPV) was 0.983. Figure 1 illustrates how the PPV and NPV are affected by different prevalence rates, and a sharp decrease in PPV at prevalence rates below 1 % is depicted. Among cases that tested positive for SARS-CoV-2 by PCR, a comparison of RAT negative and RAT positive cases showed that the presence of COVID-19 symptoms was significantly associated with a positive RAT result (p < 0.001), while the duration of symptoms (≤ 5 days vs. > 5 days), was not ( Table 3 ). The mean and median ct values were significantly lower among the RAT positive cases compared to the RAT negative cases (p < 0.001), and as illustrated in Figure 2 , a significantly larger fraction of the RAT positive cases had ct values in the mid and lower range, while higher ct values were more prevalent among the RAT negative cases.

However, there was a considerable overlap between the two groups. 

In this study, we assessed the performance of the Panbio RAT for SARS-CoV-2 compared to PCR in 4857 parallel samples. In the 3991 cases from Aker test station in Oslo the overall sensitivity of the RAT was 74.4 Recent large prospective studies on the Panbio RAT have included from 913 to 4138 participants and were performed in settings with relatively high SARS-CoV-2 prevalence, ranging from 9.6 % to 40.1 % (6-12) . The prevalence in Oslo during the study period was low to medium with a prevalence rate in all samples from Aker test station at 5.7 % (unpublished data), which made it particularly important to conduct an independent evaluation. The overall sensitivity reported by the aforementioned studies varied between 60.5 % and 90.5 % (6-12). Studies comprising mainly symptomatic persons described higher sensitivity, all above 80 % (6, 8, 9, 12) , whereas the populations in the two studies reporting the lowest sensitivities (60.5 % and 71.4 %) both contained large fractions of asymptomatic individuals (7, 10). Our results also indicate a lower sensitivity among asymptomatic individuals, as we found the sensitivity of the RAT to be 55.3 % in this group.

The ct values were higher in the PCR positive samples from the asymptomatic persons compared to those with symptoms (median 28.2 versus 24.5, respectively), suggesting that lower viral loads could at least partly explain the lower sensitivity found in this group. In a recent study, sensitivity numbers (75-93 %) equal to that for symptomatic patients was found for patients with presymptomatic or early asymptomatic infections (13) . However, for asymptomatic patients late in the course of disease, the sensitivity was very low (26 %).

This may indicate that the Panbio RAT is equally sensitive for infectious individuals regardless of whether they have symptoms or not (13) .

The test principle of most RATs is based on antibodies targeting the SARS-CoV-2 nucleocapsid protein, while the reference standard PCR is based on amplification of RNA to millions of copies. The sensitivity of RATs is thus expected to be lower. It has, however, been proposed that RATs could be used as a test of infectiousness instead of clinical disease (14) , and modelling studies have shown that in this context, sensitivity is less important than test frequency and turn-around time (14, 15) . Viral loads around one million RNA copies per mL (or per swab) of respiratory secretions have been proposed as a reasonable cutoff for evaluating infectiousness, as replicating virus is rarely detected in samples with viral loads below this limit (4, (16) (17) (18) (19) . Using an RNA standard, we found that one million RNA copies/mL roughly corresponds to a ct value J o u r n a l P r e -p r o o f of 30 in our PCR. In the samples with ct values below 30, we found the RAT's sensitivity to be 83.8 %. This is in line with previous studies which also found an association between high viral load (ct values < 30 or viral load > 10 6 copies/ml) and increased sensitivity, with reports ranging from 80.0 % to 98.0 % (6) (7) (8) (9) (10) (11) (12) Several studies have shown that SARS-CoV-2 viral load peaks in the first week after symptom onset (20) .

After one week, viral loads in upper respiratory tract samples have been shown to drop below one million copies/swab (19) . Our data shows that for SARS-COV-2 infected patients with symptom duration less than five days, the sensitivity of the RAT was about 80 %, and when restricting the analysis to individuals with an expected contagious viral load (ct value < 30), the sensitivity was 87.6 %. This is in line with other studies of the Panbio RAT, where sensitivity ranged from 77.2 % to 95.8 % when only patients with symptom duration of less than one week were considered (6) (7) (8) (9) (10) (11) (12) .

In our study, the overall specificity of the antigen test was found to be very high (99.9 %). This is consistent with previous reports, which all showed specificity numbers close to 100 % (6) (7) (8) (9) (10) (11) (12) . False positive results are thus rare. However, in low prevalence settings (<1 %), the proportion of false positives still becomes notable. This is to our knowledge the largest prospective study comparing the performance of the Panbio RAT for SARS-CoV-2 with PCR. In addition, a major strength of our study is that most of the testing was conducted in routine testing facilities, which involved highly experienced testing personnel and performance of the RATs in realistic settings for future use. Moreover, the study was carefully planned for, thus securing wellfunctioning logistics, standardized routines and adequate training in performance of the RAT. The indication for testing in these facilities is liberal, and one limitation of the study is therefore that the reasons for testing may vary considerably between the participants. A poor indication for testing as well as a high proportion of asymptomatic individuals in the study population could partly explain why we find a lower sensitivity of the J o u r n a l P r e -p r o o f

Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays

European Centre for Disease Prevention and Control. Options for the use of rapid antigen tests for COVID-19 in the EU/EEA and the UK. ECDC: Stockholm

Rapid and simple method for purification of nucleic acids

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

A simple nomogram for sample size for estimating sensitivity and specificity of medical tests

Comparison of SARS

COV-2 nasal antigen test to nasopharyngeal RT-PCR in mildly symptomatic patients

Evaluation of the Panbio™ rapid antigen test for SARS-CoV-2 in primary health care centers and test sites

Real-life validation of the Panbio COVID-19 antigen rapid test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection

Evaluation of the accuracy and ease-of-use of Abbott PanBio -A WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for <em>SARS-CoV-2</em>

Nasopharyngeal Panbio COVID-19 antigen performed at point-of-care has a high sensitivity in symptomatic and asymptomatic patients with higher risk for transmission and older age

Multicenter evaluation of the Panbio™ COVID-19 Rapid Antigen-Detection Test for the diagnosis of SARS-CoV-2 infection

Antigen rapid tests, nasopharyngeal PCR and saliva PCR to detect SARS-CoV-2: a prospective comparative clinical trial

Screening for SARS-CoV-2 infection in asymptomatic individuals using the Panbio™ COVID-19 Antigen Rapid Test (Abbott) compared to RT-qPCR

Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening

Assessment of SARS-CoV-2 Screening Strategies to Permit the Safe 16

Longitudinal Surveillance for SARS-CoV-2 Among Staff in Six Colorado Long-Term Care Facilities: Epidemiologic, Virologic and Sequence Analysis

Temporal dynamics in viral shedding and transmissibility of COVID-19

Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards

Virological assessment of hospitalized patients with COVID-2019

SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis

We would like to express our thanks to the study participants for making this project possible. Further, we acknowledge the personnel involved in this study at Aker Test Station, Oslo Municipality, Noklus, and Oslo Agency for Health, for their contribution in the planning and performance of this study. We would also like to thank all contributing staff at the Department for Microbiology at OUH, and especially at the Section for Development, Ullevål and at the Unit for Large Scale SARS-CoV-2 Diagnostics. Finally, we would like to thank the technical and logistical skills of staff involved in this project at MSIS at NIPH, and at the Section of Influenza and Other Respiratory Viruses at NIPH.

We have conducted one of the largest studies to date comparing the Panbio RAT with PCR in a low to medium prevalence setting and found that the test results are in line with previous field evaluations in higher prevalence settings. It is important to be aware of the limitations of the RAT and to keep in mind that the sensitivity is lower compared to the well-established PCR tests. Nevertheless, in some situations this might be outweighed by the advantages of identifying infectious individuals faster and thus allowing for rapid isolation and contact tracing.

The study was funded by the Norwegian health authorities as part of the national strategy to increase the overall test capacity in Norway.

None.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.