key: cord-0721136-f3smudd5 authors: Thirion-Romero, Dr. Ireri; Guerrero-Zúñiga, Dr. Selene; Arias-Mendoza, Dr. Alexandra; Cornejo-Juárez, Dr. Dora Patricia; Meza-Meneses, Dr. Patricia; Torres-Erazo, Dr. Darwin Stalin; Hernández, Dr. Thierry; Galindo-Fraga, Dr. Arturo; Villegas-Mota, Dr. Isabel; Sepúlveda-Delgado, Dr. Jesús; Ávila-Ríos, Dr. Santiago; Becerril-Vargas, Dr. Eduardo; Fernández-Plata, Rosario; Pérez-Kawabe, TIT Midori; Coeto-Cano, Dra. Ana; Vázquez-Pérez, Dr. Joel Armando; Kawa-Karasik, Dr. Simón; Reyes-Terán, Dr. Gustavo; Pérez-Padilla, Dr. José Rogelio title: EVALUATION OF PANBIO RAPID ANTIGEN TEST FOR SARS-CoV-2 IN SYMPTOMATIC PATIENTS AND THEIR CONTACTS: A MULTICENTER STUDY date: 2021-10-20 journal: Int J Infect Dis DOI: 10.1016/j.ijid.2021.10.027 sha: c43026bcb82198cb2814863383ddac5233564981 doc_id: 721136 cord_uid: f3smudd5 BACKGROUND Point-of-care rapid tests to identify SARS-CoV-2 can be of great clinical help. METHODS A cross sectional study in adults visiting emergency services or screening sites of referral hospitals for COVID-19, to define the diagnostic performance of a rapid antigen test for SARS-CoV-2 (Abbott´s Panbio™) performed by health personnel in a routine situation during an outbreak, compared with the RT-PCR for SARS-CoV-2. RESULTS A total of 1,060 participants (mean age of 47 years, 47% with a self-reported comorbidity) recruited from eight hospitals in Mexico provided 1060 valid rapid test-RT-PCR test pairs with a prevalence of a positive RT-PCR test of 45%. Overall sensitivity of the Panbio test was 54.2% (95%CI 51-57) and for patients during the first week of symptoms was 69.1% (95%CI 66-73). Sensitivity depended on viral load (Cycle threshold of RT-PCR, Ct), and the days of symptoms. With a Ct≤25, sensitivity was 82% (95%CI, 76-87%). On the other hand, specificity of the rapid test was above 97.8% in all groups. CONCLUSIONS The PanbioTM rapid antigen test for SARS-CoV-2 had a good specificity, but a low sensitivity in real life, and a negative test requires confirmation with RT-PCR, especially after the first week of symptoms. Dr. Ireri Thirion-Romero 1 Dr. Selene Guerrero-Zúñiga 1 , Dr. Alexandra Arias-Mendoza 2 , Dr. Dora Patricia Cornejo-Juárez 3 , Dr. Patricia Meza-Meneses 4 , Dr. Darwin Stalin Torres-Erazo 5 , Dr. Thierry Hernández 6 , Dr. Arturo Galindo-Fraga 6 , Dr. Isabel Villegas-Mota 7 , Dr. Jesús Sepúlveda-Delgado 8 , Dr. Santiago Ávila-Ríos 1 , Dr. Eduardo Becerril-Vargas 1 , Rosario Fernández-Plata 1 , TIT Midori Pérez-Kawabe 1 , Dra. Ana Coeto-Cano 1 ; Dr. Joel Armando Vázquez-Pérez 1 , Dr. Simón Kawa-Karasik 9 Dr. Gustavo Reyes-Terán 9 , Dr. José Rogelio Pérez-Padilla 1 and the Rapid COVID-19 Antigen Test Group. 1-Instituto Nacional de Enfermedades Respiratorias Ismael Cosío Villegas, Mexico City 2-Instituto Nacional de Cardiología Ignacio Chávez, Ciudad de México, México 3-Instituto Nacional de Cancerología, Ciudad de México, México 4-Hospital Regional de Alta Especialidad de Ixtapaluca, estado de México, México 5-Hospital Regional de Alta Especialidad de la Península de Yucatán, México Estado de Yucatán 6-Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, Ciudad de México, México 7-Instituto Nacional de Perinatología, Ciudad de México, México 8-Hospital Regional de Alta Especialidad Ciudad Salud Tapachula, Estado de Chiapas, México, México 9-Coordinación de Institutos Nacionales de Salud y Hospitales Regionales de Alta Especialidad, Ciudad de México, México The antigen test kits were donated by the WHO/PAHO, and the Mexican Health Secretariat purchased additional tests of the same type for the hospital care of the patients. No other additional support was available, and the expenses incurred were covered by the participating hospitals. Care of patients with COVID-19 was complimentary in all participating hospitals. Rapid tests to identify infectious agents can be highly useful since, in principle, they allow decisions to be made at the site of care (Point-of-Care, POC) for treatment selection, or for the separation of cohorts to avoid cross-infection. Rapid tests are especially useful in emergency situations, such as those currently being experienced in the context of the COronaVIrus Disease 19 pandemic. For influenza and other respiratory viruses, rapid tests are readily available and have shown clinical benefits (Benirschke et al., 2019; Rahamat-Langendoen et al., 2019; Shengchen et al., 2019; Wabe et al., 2019) , although not in all evaluations (Schechter-Perkins et al., 2019) . Rapid tests, at POC can be employed for screening purposes in asymptomatic people, for diagnostic purposes in persons with symptoms suggestive of the disease, or for contact tracing and epidemiological purposes in persons who had contact with suspect or confirmed cases. These situations in which the pre-test probabilities of SARS-CoV-2 infection are very different, lead to different demands (Watson et al., 2020) . A recent Cochrane review showed the urgent need for prospective and comparative evaluation of rapid tests in the context of COVID-19 (Dinnes et al., 2020) . Having a reliable rapid test would be highly desirable, especially in places with poor infrastructure or without easy access to standard laboratory tests, but also at reference sites, especially when faced with the possible arrival of patients with similar clinical manifestations but with infections with different virus (Kubina and Dziedzic, 2020) . Rapid tests could also be performed on the same subject on several occasions at low cost, for the purposes of detection and isolation of positive cases and for epidemiological surveillance, even when these usually have lower sensitivity than Reverse Transcription Polymerase Chain Reaction (RT-PCR)-based tests (Mina et al., 2020) . In the context of the COVID-19 pandemic, several international regulatory bodies have granted authorization for the emergency use of rapid tests for the presumptive diagnosis of Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2) infection based on the identification of antigens. While the overall recommendation is to confirm results with tests currently considered as gold standard, such as PCR-based tests, a readily available result obtained with a rapid antigen test can aid in making several important decisions in the clinical-care workflow. Here, we assessed the performance of Panbio rapid antigen test for SARS-CoV-2 as a diagnostic tool in symptomatic patients who arrived at the Emergency Rooms (ER) and outpatient clinics of referral hospitals for COVID-19, under real working conditions, by health personnel, in the midst of the pandemic and also in symptomatic or asymptomatic contacts of patients diagnosed with SARS-CoV-2. This is an observational, cross-sectional study performed in eight tertiary-care Patients ≥18 years of age, who presented at the emergency services or screening sites of the participating hospitals, with respiratory symptoms consistent with COVID-19 or influenza syndrome, and who provided written informed consent to participate in the study, were included, regardless of hospitalization status. Contacts of confirmed COVID-19 cases, who presented to the same sites for evaluation (mostly with respiratory symptoms but some asymptomatic), were also enrolled. Participants were recruited on weekdays, during the morning-afternoon shift, from the eight participating institutions. For the purpose of this work, the "Berlin" SARS-CoV-2 RT-PCR methodology recommended by the World Health Organization (WHO) (Corman et al., 2020) was considered the gold standard. All participating institutions implemented this test in situ and were accredited by the corresponding national authority, the National Epidemiological Reference Institute (InDRE) based on the detection of 4 SARS-CoV-2 markers: the N, E, ORF and RdRp genes. The RT-PCR test for SARS-CoV-2 was performed according to the Berlin protocol, on nasopharyngeal swab samples, taken with a synthetic fiber swab and flexible shaft, sent to the laboratory in a transport tube. In all cases Cycle threshold (Ct) for the different gene targets were requested. Laboratory personnel were blinded to results for the rapid tests. After explaining what the test consists of and the potential participant providing their written informed consent, the rapid test and RT-PCR were performed in the Emergency Room (ER), in triage, or at the usual reception site for probable patients with COVID-19, Samples were obtained with minutes of difference with different swabs following instructions for Panbio and RT-PCR. The Panbio test swab with the sample was inserted into the extraction tube with 300 uL of fluid buffer and processed immediately at the same place where it was taken. For RT-PCR the specimens were inserted inside the universal viral transport tube with 2 mL of media (Becton, Dickinson, NJ, USA) and sent to the laboratory. The Panbio TM COVID-19 Ag Rapid Test Device (nasopharyngeal) (Abbott Diagnostics Korea, Inc. Ref. 41FK10) was evaluated. This test does not require additional equipment and is approved by the corresponding regulatory agencies in the United States, Europe, and Mexico. The manufacturer reports a sensitivity of 93.3% (95% CI of 83-98%), and a specificity of 99.4% (95% CI, 95-99.3%) with a detection limit of 1.5x10 1.8 median Tissue Culture Infectious Dose (TCID 50 )/mL (Abbot 2020) . Healthcare personnel at the ER or the triage area, following the manufacturer's instructions, carried out the tests. A nasopharyngeal swab with a flexible shaft included with the kit was utilized to obtain the specimen, with a gentle rub and roll movement and afterwards the swab was inserted into the extraction tube and processed immediately where taken. All participating healthcare personnel were specifically trained in obtaining nasopharyngeal swabs and in the use of the rapid test. In all cases, a stopwatch was utilized to record the exact time of the reading. The result and a photograph of the cassette were incorporated into the database for control and later verification if necessary. Clinical information was retrieved using a REDCap database (REDCap Organization 2020), the use of respiratory support (oxygen by nasal prongs or high flow), mechanical ventilation, and the presence and duration of respiratory symptoms. For hospitalized patients, the results of routine laboratory tests and clinical data were subsequently collected, including a complete blood count, serum electrolytes, glucose, urea, creatinine, ElectroCardioGram (ECG), liver function tests, urinalysis, and Chest Computed Tomography. A sample number of 600 subjects, with at least 300 symptomatic patients, and 300 contacts were considered in the protocol (Hajian-Tilaki 2014, Bujang 2016), allowing for a study power of 0.8 even with a prevalence of positivity for the test of 0.2, and a sensitivity and specificity of 0.8. The laboratory that processes the RT-PCRs and the users of the rapid tests did not know the results of the other tests. The basic comparison was performed in a 2x2 table between the positive and negative tests by the gold standard and between the positive and negative tests by the rapid test, with which sensitivity, specificity, and positive and negative predictive value, as in a diagnostic test with an existing gold standard. We also evaluated concordance (kappa statistic) between gold standard and the Panbio TM tests. Patients and contacts and participating hospitals were considered as subgroups. The impact of the duration of symptoms in terms of the positivity of the rapid test was assessed with the positivity to the test as a dependent variable as a function of the duration of symptoms, taking into account the Cycles for the positivity of the test (Ct) and not only a dichotomic positivity or negativity, whose criteria can vary. Age, sex, days from the onset of symptoms (if they occurred), the symptoms that occurred, disease severity based primarily on the type of support needed, time of arrival at the ER, time of the test, time of the result, day and the time of obtaining the RT-PCR result were taken as covariates on order to observe modifying effects on sensitivity and specificity. Analysis was performed with STATA v13.0 statistical software, with summary statistics for diagnostic tests performed by DISGT and DIAGTEST procedures. The MIDAS procedure (Meta-analytical Integration of Diagnostic Accuracy Studies) was utilized to analyze diagnostic performance across recruiting sites. We obtained 1,069 rapid test results (from 1,069 patients) and a total of 1,060 rapid and RT-PCR test-pairs from 1060 patients (597 women From 1,060 valid test-pairs, one per recruited patient, 915 were obtained when the patient had any respiratory symptom (86.3%), and 145 (13.7%) were asymptomatic (usually contacts of a positive relative or coworker). Among participants that reported any symptom in 72% these had lasted for less than 1 week. About half (57%) of the participants requested attention in the ER or triage system, and 35% at an outpatient service. After initial screening 79% of all the participants were managed as outpatients. At the time of recruitment only 49 participants (5%) had received one dose of the SARS-CoV-2 vaccine (none had received a full vaccination scheme) and 46% of participants presented at least one comorbidity as follows: 18% had hypertension; 14% had diabetes; 10% were obese, and 10% were current smokers. Considering all of the test-pairs analyzed, 44.5% were positive in the RT-PCR test, and 25% were positive in the rapid test. Positivity of the rapid test was strongly associated with Ct (a surrogate for viral load) (Figure 1) , (Table S1 ) with a sensitivity of 82% (95%CI, 76-87%) in the presence of a Ct≤25 (Figure 2) . Positivity also depended on the days since the onset of symptoms (Figure 3) ( Table S2) , with an initial sensitivity of the rapid test of 69.1% (95%CI 63-74) during the first week of symptoms, exhibiting a progressive decline. Concordance (kappa statistic) between the rapid test and RT-PCR (Table S3) during the first week of symptoms was 0.72 (Standard Error, SE 0.04), but on considering all the participants it was 0.53 (SE 0.03). For participants presenting during the second week of symptoms, sensitivity was 0.31 (0.06) and 0.13 (SE 0.07) for participants with more than 2 weeks of symptoms. A positive result on the RT-PCR was predicted by a positive result on the rapid test (OR 98; 95%CI; 47-203), by days of symptoms (OR 1.08; 95%CI, 1.05-1.11), male sex (OR, 1.5; 95%CI, 1.03-2.08), and age (OR, 1.01; 95%CI, 1.00-1.03) in a logistic regression model with a Pseudo R 2 of 0.38, and these same variables were associated with the Ct. In a multivariate logistic regression, on modeling positivity for the rapid test, the most important predictor was the RT-PCR test and especially the Ct (aOR, 0.79; 95%CI, 0.75-0.84), but also the days of symptoms (aOR, 0.88; 95%CI, 0.83-0.94) adjusting for age, sex, and comorbidities (Pseudo R 2 0.34). The diagnostic characteristics of the rapid test varied across sites, with substantial heterogeneity in sensitivity (I 2 80.9; 95%CI, 68-93) and specificity (I 2 73.5; 95%CI, 54-92), with a relevant influence of centers contributing fewer participants (Figure 4 ). We present the results of a validation experiment of a commercial POC rapid antigen rapid test for SARS-CoV-2 infection, revealing high variability in sensitivity with respect to time from symptoms onset and to viral load. In our study, during the first week of symptoms, the sensitivity of the Panbio TM rapid antigen test was 69.6%, although the overall sensitivity was 54.2%, much lower than that reported by the manufacturer in the insert (Abbott, 2020). The test was performed in patients with a longer duration of symptoms, or with no respiratory symptoms, predominantly contacts of symptomatic cases, often common in ER or at screening sites, especially during outbreak peaks when accesses to hospitals were difficult. On the other hand, the specificity was consistently high in all subgroups. In studies conducted elsewhere with the Abbott Panbio TM test, lower sensitivities have also been found, ranging from 45% in pediatric patients with less than 5 days of symptoms compatible with COVID-19 (Villaverde et al., 2021) , 48% in household contacts of positive cases, 37% in non-domestic contacts (Torres et al., 2021) , and up to 73% in primary-care patients (Linares et al., 2020) . In a meta-analysis, days of symptoms were strongly associated with the Panbio rapid test result, with a positivity of 86.5% if symptoms were present for fewer than 7 days and 54% if these were present for more than 7 days (Linares et al., 2020) . In a study conducted under normal working conditions, the sensitivity was 73% in the Netherlands and 81% in Aruba, and > 95% if the Ct was <32 (Gremmels et al., 2021) . In all studies, the specificity has been 100% or close to 100%. In another study in symptomatic patients in primary care and in routine work, sensitivity was 80% and specificity 100% (Albert et al., 2021) . The Canadian authorities issued recommendations that took into account sensitivities incorporated into the annex to the test (Interim guidance, 2021). An update of a Cochrane review (Dinnes et al., 2020) , including 78 studies (20 preprints) and 24,087 samples (about one third positive for SARS-CoV-2), confirmed a substantial variation in sensitivity according to the presence or absence of symptoms (72% vs. 58%), first v. second week of symptoms onset (78% vs. 51%), a Ct of ≤25 v. >25 (94.5 vs. 40.7%) (Dinnes et al., 2020) . Sensitivities reported for rapid antigen tests for SARS-CoV-2 from different manufacturers ranged from 34% to 88%, but specificities were in general above 99% (Dinnes et al., 2020) . The pooled sensitivity of the Abbott Panbio TM test (11 studies) was 75.1% in symptomatic patients during the first week of symptoms, but dropped to 58% in asymptomatic individuals (Dinnes et al., 2020) . Specificity was 99.5%. In the first week of symptoms, the sensitivity in our study was 69.1%, and specificity was 98.5%. It Is relevant that only one study compiled by the recent Cochrane analysis (Alemany et al., 2021) had a test sensitivity with a lower limit of confidence >80%, the recommended cut-off point by the WHO for this type of tests. In our analysis, we observed a strong association between the result of the rapid test and viral load, estimated with the Ct, with no positive results in samples with a Ct of >39. It is also relevant that the test was performed in a group of persons with >5 days from the onset of symptoms, a group in which the positivity decreased significantly due to an expected lower viral load. Nevertheless, even in this group in which sensitivity is considerably lower, a positive test would be highly informative. It is noteworthy that, with symptoms of longer duration, the cost-effectiveness of applying POC rapid antigen tests would drop progressively. Thus, given that specificity is high, the greatest clinical advantage of utilizing a rapid antigen test would present when the result is positive, in which case, a confirmatory RT-PCR would not be needed. In the presence of symptoms compatible with COVID-19, or in a period of a high incidence rate of infection in the community, a negative RT-PCR test result would be unreliable, even during the first week of symptoms. In these cases, the RT-PCR test is usually repeated a second, or even a third time (Ramdas et al., 2020) . As the infection progresses, the viral and nucleic-acid load tends to decrease and the RT-PCR test tends to be negative, while antibody titers against SARS-CoV-2 begin to appear (Ghaffari et al., 2020; Kilic et al., 2020; Laureano and Riboldi 2020; Shyu et al., 2020) . Thus, confirming a SARS-CoV-2 infection ideally involves both the positivity of an RT-PCR test (not necessarily the first test), and the consideration of a combination of epidemiological variables, including the rate of community transmission, the presence of compatible symptoms and the presence of antibodies. False-positive RT-PCR test results have been reported rarely, and are attributed to contamination with viral genetic material in any of the steps between sampling and processing. On the other hand, in the case of a negative antigen test, confirmation with RT-PCR will be required, especially in the presence of compatible symptoms, a high rate of community infections, or in the case of persons with direct contact with confirmed COVID-19 patients. In any case, according to our observations, if a POC rapid antigen test were employed, 69.1% of the RT-PCR tests would be avoided during the first week of symptoms, which represents a considerable saving. The latter represents an enormous advantage in settings where decision-making is needed, and where the lack of infrastructure and high costs render it difficult to implement molecular testing. Our study was observational, subject to all possible biases or cross-sectional studies, performed predominately during daytime, the shift with more available staff and in-training personnel. Testing was carried out with the Mexican health system and personnel under stress, a very different situation from that of rapid tests performed under controlled circumstances, in a laboratory, by the same expert personnel, and this can explain, at least in part, the reduced sensitivity. On the other hand, our results are expected to be closer to what can be observed under outbreaks that saturate or overwhelm the screening sites and ER, that is, more demanding circumstances than those found under strictly controlled laboratory testing. Furthermore, overall positivity in participating hospitals to the RT-PCR was quite high (44.6%, ranging from 27 to 93% in different hospitals) allowing for a proper evaluation of sensitivity and specificity. Real conditions of use may be even more demanding than those present in our study for example if testing includes primary care, community hospitals, during all shifts including weekends and nights, characterized by the presence of fewer personnel, and especially if the overall positivity of the tests or community transmission is low. We have shown that the sensitivity of Panbio SARS-CoV-2 rapid antigen test is limited, with an overall estimated value of 69% in the present study. As expected, sensitivity was highly associated with time from the onset of symptoms and with viral load. The low sensitivity, and the high specificity observed in the test make it necessary to confirm all negative results, especially in the presence of symptoms compatible with COVID-19 and in settings with high community-infection rate. Nevertheless, the use of Panbio rapid antigen tests could be highly beneficial to screen for positive tests and to reduce the number of cases to be confirmed, permitting the making of quick clinical decisions. None known declared by the authors. The antigen test kits were donated by the Panamerican Health Organization, and the Mexican Health Secretariat purchased additional tests of the same type for the hospital care of the patients. No other additional support was available, and the expenses incurred were covered by the participating hospitals. Care of patients with COVID-19 was complimentary in all participating hospitals. The protocol was revised and approved (approval code C93-20) by "El Comité Unico, de Investigacion, de Etica en Investigación, y de Bioseguridad de la Comisión Coordinadora de Institutos Nacionales de Salud y Hospitales de Alta Especialidad": The Investigation, Ethics in investigation and Biosecurity Unique Committee from the Coordination of the National Institutes of Health and High Specialty Hospitals, a single Institutional Review Board appointed by the Mexican Institutes of Health authority for this multicentric study. All hospitals that were included in the protocol belong to the same organisation and share the same administrative coordination and adhered to the same protocol and consent form (adding only the principal investigator of the participating institutions). The study was registered with ClinicalTrials.gov The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. the use of the Abbott Panbio™ COVID-19 Antigen Rapid Test Abbot Panbio COVID-19 Ag Rapid Test Device Field evaluation of a rapid antigen test Analytical and clinical performance of the panbio COVID-19 antigen-detecting rapid diagnostic test Clinical Impact of Rapid Point-of-Care PCR Influenza Testing in an Urgent Care Setting: a Single-Center Study Requirements for Minimum Sample Size for Sensitivity and Specificity Analysis Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection Real-life validation of the Panbio™ COVID-19 antigen rapid test (Abbott) in communitydwelling subjects with symptoms of potential SARS-CoV-2 infection Sample size estimation in diagnostic test studies of biomedical informatics Molecular and Serological Tests for COVID-19 a Comparative Review of SARS-CoV-2 Coronavirus Laboratory and Point-of-Care Diagnostics Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms Impact of molecular point-of-care testing on clinical management and in-hospital costs of patients suspected of influenza or RSV infection: a modeling study Test, re-test, re-test': using inaccurate tests to greatly increase the accuracy of COVID-19 testing Point-of-care influenza testing does not significantly shorten time to disposition among patients with an influenza-like illness Evaluation of a molecular point-of-care testing for viral and atypical pathogens on intravenous antibiotic duration in hospitalized adults with lower respiratory tract infection: a randomized clinical trial. Clinical Microbiology and Infection: The Official Publication of the Laboratory Tests for COVID-19: A Review of Peer-Reviewed Publications and Implications for Clinical UIse Evaluation of a rapid antigen test (Panbio™ COVID-19 Ag rapid test device) for SARS-CoV-2 detection in asymptomatic close contacts of COVID-19 patients Diagnostic Accuracy of the Panbio Severe Acute Respiratory Syndrome Coronavirus 2 Antigen Rapid Test Compared with Reverse-Transcriptase Polymerase Chain Reaction Testing of Nasopharyngeal Samples in the Pediatric Population m1808. World Health Organization. A minimal common outcome measure set for COVID-19 clinical research FIGURE 1: Dot graph of Ct (red circles) with median value, and 25th and 75% percentiles (blue lines), in individuals with normal (left side) or abnormal tests (right side) FIGURE 4: Forest plot depicting combined sensitivity and specificity as a function of 2% (range 67-93), whereas specificity was consistent with an overall value of 0 Final diagnosis SARS-CoV-2 true negative=579 SARS-CoV-2 false negative=216