key: cord-0885822-a1ivy85n
authors: Bulilete, O.; Lorente, P.; Leiva, A.; Carandell, E.; Oliver, A.; Rojo, E.; Pericas, P.; Llobera, J.; Group, COVID-19 Primary Care Research
title: Evaluation of the Panbio rapid antigen test for SARS-CoV-2 in primary health care centers and test sites.
date: 2020-11-16
journal: nan
DOI: 10.1101/2020.11.13.20231316
sha: da97bac3084bfa80b9c79b07683f4d58f88c057c
doc_id: 885822
cord_uid: a1ivy85n

Abstract Background Rapid antigen tests (Ag-RDT) are emerging as new diagnostic tools for COVID-19 and real-world evaluations are needed to establish their performance characteristics. Main objective To evaluate the accuracy of the Panbio Ag-RDT at primary health care (PHC) centers and test sites in symptomatic patients and close contacts, using the Reverse-Transcription Polymerase Chain Reaction (RT-PCR) test as the gold standard. Methods This was a prospective diagnostic study conducted in four PHC centers and two test sites in Mallorca, Spain. Consecutive patients older than 18 years, attending the sites for RT-PCR testing either for suggestive symptoms of infection or a close contact, were included. Two nasopharyngeal samples were collected, one for RT-PCR and the other was processed on-site using the Panbio rapid antigen test kit for SARS-CoV-2. The sensitivity and specificity were calculated using RT-PCR as the reference, and the predictive values using the pretest probability results for each analyzed group. Results A total of 1369 participants were included; mean age 42.5 {+/-} 14.9 years and 54.3% women. The overall prevalence was 10.2%. Most participants (70.6%) presented within 5 days of the onset of symptoms or close contact, and more than 70% had high viral loads. The overall sensitivity was of 71.4% (95% CI: 63.1%, 78.7%), the specificity of 99.8% (95% CI: 99.4%, 99.9%), the positive predictive value of 98.0% (95% CI: 93.0%, 99.7%) and a negative predictive value of 96.8% (95% CI: 95.7%, 97.7%). The sensitivity was higher in symptomatic patients, in those arriving within 5 days since symptom onset and in those with high viral load. Conclusion Ag-RDT had relatively good performance characteristics in suspected symptomatic patients within five days since the onset of symptoms. However, our results concludes that a negative Ag-RDT in these settings must be considered as presumptive. Keywords: COVID-19, rapid antigen test, SAR-COV-2, primary care.

Europe is currently experiencing a second wave of the COVID-19 pandemic and there is a greater demand for diagnostic testing. Fast diagnostic testing using an inexpensive, point-ofcare, easy-to-use, and rapid technique might help alleviate the burdens experienced by testing laboratories and caregivers in primary healthcare (PHC) centers and COVID-19 test sites (7) .

The increased demand for RT-PCR tests can cause delays in reporting of positive results and lead to delays in contact tracing, and thus have negative consequences for control of the COVID-19 pandemic.

Most previous studies that evaluated Ag-RDTs for COVID-19 examined symptomatic patients, were conducted in the setting of hospital emergency services, and examined patients who presented with moderate or severe symptoms of COVID-19. Thus, rigorous studies are needed before Ag-RDTs can be used for the diagnosis of SARS-CoV-2 in the setting of PHC centers or in the community.

The Panbio™ rapid antigen test kit for SARS-CoV-2 (Abbott Diagnostic GmbH, Jena, Germany) is a qualitative test using specimens from nasopharyngeal swabs. The manufacturer reported that the sensitivity for symptomatic patients was 93.3% overall and 98.2% in those RT-PCR cycle threshold (Ct) ≤ 33, and that the specificity was 99.4% (8) .

The main aim of this study was to evaluate the performance of the Panbio™ Ag-RDT at PHC centers and test sites in symptomatic patients and close contacts, using the RT-PCR test as the gold standard.

This prospective diagnostic study was conducted in Mallorca (Balearic Islands, Spain) from October 2-25, 2020. Two testing locations (COVID-EXPRESS) that cover the city of Palma and 4 PHC centers (Santa Ponça, Alcudia, Inca, and Coll d'en Rebassa) were included.

Individuals were invited to participate if they were older than 18 years, were not previously diagnosed with COVID-19, attended one of the above-named settings for RT-PCR testing, had symptoms suggestive of infection with referral by a general practitioner (GP), or had a close contact with another patient with an RT-PCR-confirmed infection.

All potentially eligible participants were asked to sign an informed consent document and to answer a short questionnaire that asked about the reasons for RT-PCR testing (referral by a GP . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint due to symptoms, close contact, others); socio-demographic information (sex and age); and the presence of symptoms, type of symptoms, and number of days since symptom onset or close contact to a positive SAR-CoV-2 patient.

Trained nurses collected two consecutive nasopharyngeal sample swabs for the RT-PCR test and the Ag-RDT and interpreted the results of the AG-RDT on-site.

Within 24 h of collection, one nasopharyngeal swab was sent for processing to Son Espases University Hospital, Microbiology Service without any additional information of the participants. RNA extraction was performed using the MagMAX™ Viral/Pathogen Nucleic Acid Isolation Kit (ThermoFisher) and amplification was performed using the TaqPath™ COVID-19 CE-IVD RT-PCR Kit and QuantStudio™ (ThermoFisher). The viral load was expressed as Ct for three genes (ORF, N, and S).

The other nasopharyngeal swab was processed on-site using the Panbio™ Ag-RDT (Abbott 

To evaluate the accuracy of the Panbio™ Ag-RDT, the initial prevalence of COVID-19 was estimated as 15%, the marginal error as 5%, and the sensitivity as 90%. Thus, it was necessary to test 927 participants.

The sensitivity, specificity, and their 95% CIs were calculated using RT-PCR as the reference. Sensitivity analysis was stratified by the declared reason for performing the RT-PCR test, symptoms, days since symptom onset or exposure, and Ct-value. Predictive values and 95% CIs were estimated using the pretest probability results for each analyzed group. Means and standard deviations were used to describe population characteristics and for descriptive analysis.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint All statistical calculations were performed using Stata 13 (StataCorp, College Station, TX, USA) with the Stata DIAGT module.

This study was conducted following the Declaration of Helsinki and was approved by the Balearic Research Ethics Committee (IB 4350/20 PI on 30/09/2020) and by the Mallorca Primary Care Research Commission. Each participant was asked to sign an informed consent agreement before inclusion. This study was promoted by the Balearic Public Health Service and no external funding was received.

No patients were involved in setting the research question and we did not seek public engagement in the design of the laboratory aspects of the study, nor were they involved in developing plans for the design of the study. No patients were asked to advise on interpretation or writing up of results.

We initially identified 1412 potentially eligible subjects visited consecutively in multiple PHC centers in Mallorca ( Figure 1 ). Twenty-seven individuals (1.9%) declined participation, mostly because of a lack of time or anticipation of discomfort from sample collection, and we were unable to retrieve Ag-RDT results for another 16 participants. The final sample consisted of 1369 participants, their mean age was 42.5 ± 14.9 years, and 54.3% were women. The overall prevalence of COVID-19 was 10.2%, and there were 140 positive RT-PCR tests and 102 positive Ag-RDTs. We excluded 7 RT-PCR results from Panbio performance characteristics analysis (3 because of incorrect labeling that could not be recovered and 4 because of inconclusive results).

We analyzed the characteristics of all participants (Table 1 ). Most appointments for RT-PCR testing were because of close contact with a confirmed positive COVID-19 individual (54.8%) or because of symptoms suggestive of COVID-19 and referral by a PHC professional (36.7%); the other 116 individuals (8.5%) were considered as unknown, because there were referred by PHC professionals without declaring the reason.

Almost half of the subjects (49.7%) reported symptoms within 7 days prior to testing, and the most frequently reported symptoms were headache (24.9%), sore throat (22.6%), cough . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint (18.4%), and tiredness (18.3%). Most participants (70.6%) presented within 5 days of the onset of symptoms or close contact, although these data were unavailable for 13.6% of the participants. The SARS-CoV-2 N gene viral load was not obtained for 6, 7 for S gene and 8 for . We considered a Ct of 25 to 29.9 as moderate viral load and a Ct above 30 as low viral load. More than 70% of the analyzed participants (n = 98) with available Ct values had high viral loads for the N gene.

Our analysis of the overall performance of the Panbio™ test ( 

As noted above, most patients (n = 963) received tests within 5 days since symptom onset or since the close contact (Table 3) . Nevertheless, the overall test sensitivity for these patients was only 77.2% (95% CI: 67.6%, 84.7%), below the minimal sensitivity recommended by WHO for Ag-RDTs (80%). However, the test sensitivity was acceptable for patients who reported symptoms (n = 556; 83.1%, 95% CI: 71.9%, 90.5%) and for patients referred by their GPs for symptoms (n = 418; 86.0%, 95% CI: 71.3%, 94.2%).

The test sensitivity was also unacceptable for patients tested within 5 days since the close contact, reporting symptoms (n = 117; 77.7%, 95% CI: 57.2%, 90.6%) or not at the moment of testing (n = 390; 65.5%, 95% CI: 45.6%, 82.0%). These general trends in test sensitivity and specificity were similar when the time period was prolonged from 5 to 7 days (Supplementary . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint Table S1 ). Analysis of test sensitivity according to patients' symptoms are provided in Supplementary Table S2 .

We assessed the sensitivity according to the viral gene load (Figure 2 ). The overall test sensitivity for patients with high viral loads of the N gene (n = 134; Ct < 25) was 87.7% (95% CI: 79.5%, 93.5%). Analysis of separate categories of patients with high viral loads indicated the test sensitivity was above 80% even in asymptomatic patients (86.2%, 95% CI: 68.3%, 96.1%). Notably, test sensitivity decreased considerably for patients with higher Ct values (lower viral loads). Analysis of S and ORF gene viral loads also indicated acceptable sensitivity of the test for patients with high viral loads, but not for patients with low viral loads (Supplementary Table S2 ).

There were false-negative Ag-RDT results in 40 of the 140 patients (28.6%) who had positive RT-PCR results. These individuals were mostly between 21 and 40 years-old (52.5%) and half of them were women. Among these 40 patients, the main reason for testing was for being in close contact with a SARS-CoV-2 confirmed patient within 5 days; 23 patients (57.3%) reported no symptoms when arriving at the testing site. Among those for whom data on N gene viral load were available (n = 36), 12 patients had high viral loads, 16 had moderate viral loads, and 8 had low viral loads.

There were false-positive Ag-RDT results in 2 of the 1222 patients (0.1%) who had negative RT-PCR results. Both of these patients were women, between 41 and 50 years-old, were tested because of close contact with a known patient, and had symptoms during the previous 5 days (headache, tiredness, and/or cough).

Two independent and blinded observers, one of whom was an expert evaluator, performed visual interpretations of 68 Ag-RDT results. The interobserver agreement was 100% (Supplementary Table 4 ).

No side effects were reported when collecting the nasopharyngeal swabs.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint We found that test sensitivity was higher in samples that had high viral loads (Ct < 25), even in patients who were asymptomatic at the moment of testing (86.2%, 95% CI: 68.3%, 96.1%). This is an important finding because some asymptomatic individuals with SARS-CoV-2 infections (i.e., "super spreaders") might have a substantial impact on the spreading of this virus 19 (9) . The Ag-RDT provides fast results, thus facilitating early identification, rapid isolation of the patient, and early contact tracing of highly contagious cases (10, 11) .

The present study is the largest evaluation of the Panbio Ag-RDT in samples from symptomatic and asymptomatic patients in real-world context. This is a strength of our study because this is the setting where the majority of the patients with mild symptoms (up to 75%) (12) and close contacts are visited and followed-up. We incorporated COVID-19 testing into the usual care and management of patients with suspected SARS-CoV-2 infection and less than 2% of participants denied taking part in the study.

Our study has several limitations. We only assessed one type of Ag-RDT that targeted the SARS-CoV-2 N (nucleocapsid) protein, although other tests are available. Also, we only examined two types of testing scenarios -referral by a GP based on symptoms and close contact with a confirmed case -and therefore cannot make inferences about the applicability of this test in other scenarios (screening of nursing-homes, workplaces, etc.). Additional rigorous studies are needed to establish the optimal performance characteristics for Ag-RDTs that have different protein targets, are employed in different specific settings, and that have different pretest probabilities.

There is no standardized method to establishing the infectiousness of a patient with confirmed SARS-CoV-2 based on viral load. Ct values can vary among studies according to the type of test and target gene. We considered low viral load as a Ct value above 30 in patients with positive RT-PCR results (13) , and assumed that such individuals can be considered non-. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. 

The test sensitivity in our patients was lower than that provided by the manufacturer (93.3% overall and 98.2% in those with Ct≤33), but in line with previous studies. Two studies conducted in Spain that included symptomatic patients who attended a PHC center and hospital emergency wards reported the overall test sensitivity was 79.6% and the sensitivity was 86.5% in those with high viral loads (14, 15) . Other studies of symptomatic patients in the Netherlands and Switzerland found similar test sensitivity (85.4%, 81.0%, and 72.6%) and a positive correlation between Ag-RDT positivity and SARS-CoV-2 viral load (16,17). A study in France that examined positive and negative frozen RT-PCR samples reported a much lower overall sensitivity of only 35.3% (18) ; however, test sensitivity was greater in samples collected within 3 days since the onset of the symptoms and in samples from patients with high viral loads. Two preprints (non peer-reviewed studies) reported test sensitivities of 91.7% (19) and 97.1% (10) .

An evaluation report in Spain performed in symptomatic patients who visited hospital emergency rooms reported a sensitivity of 98.2% (20) . All these previous studies reported excellent test specificity.

Our results support the use of the Ag-RDT for symptomatic patients within 5 days since the is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted November 16, 2020. ; 1 1 (14, 16, 22) . This might have a substantial impact from a public health point of view, because a negative Ag-RDT result especially in individuals with high pretest probabilities must be interpreted cautiously, and a confirmatory test should be considered (5) .

There are several new approaches used to overcome the low sensitivity of the Ag-RDT, in an effort to incorporate this test as a reliable diagnostic tool for massive testing to be used for monitoring and controlling outbreaks. One approach is the complementary use of the Ag-RDT with clinical diagnostic evaluations and another approach is the use of repeated testing (23) . A recent study using the enhanced epidemiological SIDHRE-Q model concluded that frequent Ag-RDT testing overcame the limitation of low test sensitivity, suggesting this might be an effective method to control SARS-CoV-2 transmission (24). However, future research must confirm these findings in real-world settings, as not all diagnostic tests are useful for screening (25) .

Our results show that a point-of-care Ag-RDT has good performance characteristics in suspected symptomatic patients within five days since the onset of symptoms in PHC. However, a negative Ag-RDT result must be considered presumptive when the pretest probability is high, and a confirmatory test might be required. Further studies are needed to examine the accuracy of the Ag-RDT in different settings with lower pretest probabilities.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint Notes 1 PL -protocol design, implemented the whole study, monitored the whole data collection, revised the .

It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review) . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; 1  9   T  a  b  l  e  2  :  O  v  e  r  a  l  l  s  e  n  s  i  t  i  v  i  t  y  ,  s  p  e  c  i  f  i  c  i  t  y  ,  a  n  d  p  r  e  d  i  c  t  i  v  e  v  a  l  u  e  s  o  f  t  h  e  P  a  n  b  i . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ; https://doi.org/10.1101/2020.11.13.20231316 doi: medRxiv preprint 0   T  a  b  l  e  3  :  A  c  c  u  r  a  c  y  o  f  t  h  e  P  a  n  b  i  o   T  M   t  e  s  t  i  n  p  a  t  i  e  n  t  s  t  e  s  t  e  d  5  o  r  f  e  w  e  r  d  a  y  s  (  t  o  p  )  o  r  m  o  r  e  t  h  a  n  5  d  a  y  s  (  b  o  t  t  o  m  )  s  i  n  c  e  s  y  m  p  t  o  m  o  n  s  e  t  o  r  c  l  o  s  e  c  o  n  t  a  c t . . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 16, 2020. ;

Considerations for diagnostic COVID-19 tests

Rapid Risk Assessment: Increased transmission of COVID-19 in the EU/EEA and the UK -thirteenth update

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev

Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays

Interim Guidance for Rapid Antigen Testing for SARS-CoV-2 CDC

Global partnership to make available 120 million affordable, quality COVID-19 rapid tests for low-and middle-income countries

Diagnostic Tests for Covid-19 [Internet]. FIND because diagnosis matters

Ag Rapid Test Device | Abbott Point of Care Testing

Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2. medRxiv

From more testing to smart testing: data-guided SARS-CoV-2 testing choices

Canadian Agency for Drugs and Technologies in Health. Rapid Point-of-Care Antigen Testing for SARS-CoV-2 Infection

Recommendations for use of antigenic tests in the diagnosis of acute SARS-CoV-2 infection in the second pandemic wave: attitude in different clinical settings. Rev Española Quimioter

Organización del diagnóstico de SARS-CoV-2 y estrategias de optimización

Field evaluation of a rapid antigen test (Panbio COVID-19 Ag Rapid Test Device) for the diagnosis of COVID-19 in primary healthcare centers

Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms

Évaluation de la performance diagnostique des tests rapides d ' orientation diagnostique antigéniques

Analytical and Clinical Performance of the Panbio COVID-19 Antigen-Detecting Rapid Diagnostic

European Commission. Commission Recommendation of 28.10.2020 on COVID-19 testing strategies, including the use of rapid antigen tests

Real-life validation of the Panbio COVID-19 Antigen Rapid Test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection. medRxiv

Rethinking Covid-19 Test Sensitivity -A Strategy for Containment

The impact of high frequency rapid viral antigen screening on COVID-19 spread and outcomes: a validation and modeling study 2 3

Covid-19 mass testing programmes