key: cord-0957326-dfobkg8l authors: Bornemann, Linus; Dähne, Theo; Fomenko, Alexey; Kaup, Olaf; Kleideiter, Johannes; Kulis-Horn, Robert; Ruprecht, Bertram; Wehmeier, Michael; Panning, Marcus title: Performance of the Sofia SARS-CoV-2 rapid antigen test as frontline test in a university hospital, Germany date: 2022-02-26 journal: Diagn Microbiol Infect Dis DOI: 10.1016/j.diagmicrobio.2022.115663 sha: c4ea98c28af8baefdc782dc56553e8f0953ddac2 doc_id: 957326 cord_uid: dfobkg8l BACKGROUND: The rapid and reliable detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is of high importance for individual patient care and hospital infection prevention. OBJECTIVES: To evaluate the performance of the Sofia SARS-CoV-2 antigen rapid diagnostic test (Ag-RDT) in comparison to real-time reverse-transcription polymerase chain reaction (RT-PCR). METHODS: We conducted a prospective, monocentric cross-sectional study in an emergency department of a German university hospital from November 2020 to March 2021. We tested all samples using both Sofia SARS-CoV-2 Ag-RDT and real-time RT-PCR. RESULTS: A total of 7877 patients were included. Overall sensitivity of the Ag-RDT was 62.9% and specificity was 99.4%. Sensitivity varied across study months, whereas specificity remained high. Sensitivity increased to 94.2% in samples with a cycle threshold (Ct)-value ≤25. CONCLUSIONS: The Sofia Ag-RDT proved to be a rapid tool to detect samples with high viral loads (Ct-value ≤25) and might thus help to identify infectious patients. A total of 7877 patients were included. Overall sensitivity of the Ag-RDT was 62.9% and specificity was 99.4%. Sensitivity varied across study months, whereas specificity remained high. Sensitivity increased to 94.2% in samples with a cycle threshold (Ct)-value ≤25. Conclusions The Sofia Ag-RDT proved to be a rapid tool to detect samples with high viral loads (Ctvalue ≤25) and might thus help to identify infectious patients. Keywords: SARS-CoV-2; rapid antigen testing; RT-PCR; clinical performance; real-world data Rapid and reliable diagnostics are instrumental in containing the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic. Hospitals are among the most vulnerable institutions and admission testing has become current standard of practice at many hospitals worldwide. RT-PCR is the current gold standard to diagnose SARS-CoV-2 infection but is hampered by rather long turnaround time and the dependency on reagents and instrumentation. In addition, trained laboratory personnel is needed, which limits its application in remote and/or resource-limited settings. Antigen rapid diagnostic tests (Ag-RDTs) can be done directly at the point of care within very short time, are less costly compared to RT-PCR and easy to perform. To enhance read-out, some Ag-RDT are equipped with a device to measure fluorescence. These characteristics allow for frequent diagnostic testing which is critical to contain community spread or for admission testing in the health care setting. A number of studies have been conducted to determine the clinical performance of Ag-RDT assays in different cohorts and study settings e.g. in outpatient SARS-CoV-2 testing facilities or to a lesser extent in hospitals [1, 2] . The World Health Organization (WHO) recommends a sensitivity of ≥80% of Ag-RDT in comparison to RT-PCR [1] [2] [3] [4] . In previous studies, the Sofia Ag-RDT demonstrated a sensitivity among asymptomatic patients between 35.7% and 78.6% and among symptomatic patients between 72.1% and 93.8%, while specificity of at least 96.9% have been reported [2, [5] [6] [7] . However, only a small number of studies using this assay were conducted in a hospital setting. We were able to show a sensitivity of 65% in a pilot study [8] . Of note, varying prevalences and the surge of novel variants of concern (VOC) may influence the performance of Ag-RDT [9] . Objectives To address this, we aimed to evaluate the Sofia Ag-RDT in a German university hospital in a real-world setting. The study was conducted over a period of five months beginning 1 November 2020 to 31 March 2021. This period was characterized by a substantial increase of COVID-19 cases and the introduction of the VOC B.1.1.7 (Alpha) in Germany in early 2021. Patients, study setting and sample collection: We prospectively enrolled all hospitalized patients irrespective of respiratory symptoms admitted to the emergency department (ED) at Bielefeld Hospital, a university hospital with 1300 beds in northern Germany from 1 November to 31 March 2021. The study was conducted over a time period of 5 months with 7-day incidence ranging between 73 and 974 per 100,000 inhabitants in Bielefeld, which peaked at the start of the study period and subsequently declined [10] . The end of the study period was characterized by the rise of the SARS-CoV-2 B.1.1.7 (Alpha) variant of concern (VOC), which was detected in 78.5% of cases by the end of March [11] . Trained medical personnel took two nasopharyngeal swab samples per patient using both nostrils. One nasopharyngeal swab was collected in universal transport medium for molecular testing within the next 24 hours, followed by another nasopharyngeal swab for immediate on-site antigen testing. We retrospectively extracted data on COVID-19 associated symptoms (i.e. cough, dyspnea, loss of smell/taste, and fever) from the medical charts of all RT-PCR positive patients. We used the Sofia antigen fluorescent immunoassay (FIA) (Quidel, Kornwestheim, Germany) according to the manufacturer's instructions in Europe [12] . This version also accepts nasopharyngeal swabs as sample type (Package Insert -DE, IT, FR, ES). Of note, the FDA EUA-authorized version recommends the use of anterior nasal swabs only. The Sofia FIA is a sandwich-based lateral flow assay and provides automated and user-independent read out using the Sofia 2 FIA analyzer. [13] [14] [15] . We used the INSTAND e.V. Bezugsprobe 1 (10.000.000 SARS-CoV-2 RNA copies/ml) and 2 (1.000.000 SARS-CoV-2 RNA copies/ml) to calibrate our assays. Assays were able to detect 1.000.000 RNA copies/ml at a Ct-value of approximately 25 [16] . The Ct-value is a surrogate measure for SARS-CoV-2 virus concentration and a Ct-value of ≤25 has been shown to correlate with infectivity [17] [18] [19] . Statistical analyses: Data were collected using SPSS software 24. Statistical analyses were performed using R (Version 4.1.0). We determined sensitivity, specificity, positive and negative predictive value (PPV, NPV) and 95% CI for Ag-RDT using real-time RT-PCR results as reference standard. Mann-Whitney U test and Tukey´s multiple comparison test were used for Ct value comparisons; p-values <0.05 were considered statistically significant. The performance measures of accuracy were compared using chi-squared test. Ethics: Ethical approval was obtained (Az 2020-870-f-S; AeKWL/WWU Muenster). A total of 7877 patients were included and tested by both Ag-RDT and RT-PCR. Median age was 67 years (range 1-107 years), and 3640 (46%) were males. Of note, only 127/7877 (1.6%) patients were To appreciate the dynamics of the pandemic, we calculated sensitivity and specificity, positive predictive value (PPV) and negative predicitve value (NPV) per study month (Table 2) . Lowest sensitivity was determined in January 2021 with 53.9%. Next, we plotted Ct-values separately for each month (Figure 2 ). We were able to show that median Ct-values were lower in December 2020 approximately corresponds to 1.000.000 SARS-CoV-2 RNA copies per ml indicating infectivity [20] . Here we report a clinical sensitivity of 62.9% (95% CI 57.8-67.8%) and specificity of 99.4% (95% CI, 99.2 -99.5%) for the Sofia Ag-RDT compared to RT-PCR among patients presenting to the ED of a large university hospital. The measures of accuracy varied by Ct-values, symptom status and test prevalence. The WHO defines a sensitivity of ≥80% and specificity of ≥97% as the acceptability criteria for SARS-CoV-2 Ag-RDT. While the overall sensitivity is lower than the accetable threshold, the sensitivity inversely increased with Ct-values and sensitivity at Ct-values ≤25 was 94% and 100% at Ct-values of ≤18. This is in concordance to other reports [2, 21, 22] . Several studies aimed to assess the sensitivities of Ag-RDTs compared to RT-PCR and cell culture [21, [23] [24] [25] . All studies observed an increase in sensitivities when cell culture was the reference standard, suggesting the use of Ag-RDT as a tool for detecting infectious patient samples. Pray et al. performed cell culture on 18 samples which were assessed by Sofia Ag-RDT as false negative, of which 2 viral isolations were obtained [25] . Taking this into consideration the here reported sensitivity of 94% at Ct-values ≤25 suggests that Sofia Ag-RDT might be suitable for identifying infectious patient samples in the ED setting. The dependence of sensitivity on viral loads is in line with previous studies [2, 23] . Stratified by months we found the largest difference in sensitivity in the months where Ct-values varied with statistical significance. By contrast, clinical specificity of the Sofia Ag-RDT was excellent throughout, which is a reassuring finding. Intriguingly, a number of studies evaluated the Sofia Ag-RDT and reported sensitivities ranging from 72%-80% in symptomatic persons [5, 25] . To the best of our knowledge only two studies tested the Sofia Ag-RDT in a clincal setting using nasopharnygeal swabs and obtained comparable results with sensitivities of 66% and 76%, respectively [2, 5] [6, 23] . In disconcordance to those studies we found differences in accuracy and Ct-values between symptomatic and asymptomatic patients. In part, this might be explained by differences in sampling types (nasopharyngeal versus anterior nasal swabs). False-positive results were observed but to a lesser extent than false-negative ones. A higher rate of false-positive results might be associated with a low prevalvence setting, which was not the case in our study. A total of 139 false negative results was seen, which is a concerning result for the hospital setting but needs to be interpreted in light of a high median Ct-value. Depending on test prevalence varying NPV and PPV were determined. As expected, test performance was better at higher test prevalence of SARS-CoV-2. Here, test prevalence never exceeded 10% despite covering a time frame of 5 months and the beginning of the third wave in Germany. WHO recommended acceptable PPVs of 62.8 and 78% for prevalences between 5% and 10%. Based on our study data the 95% confidence interval of the PPVs of November, January, February and March fall below these thresholds. This finding challenges the use of Ag-RDT irrespective of the course of the pandemic. Technically, the Sofia is equipped with a device for automated and improved read-out. The device can be connected to the hospital-based information system which facilitates documentation and reporting of test results. In our study, we did not encounter any technical problems with the system rendering it useful for routine testing. Reassuring, only a small fraction of Ag-RDT tests results returned invalid. This rate is lower than previously reported [25] . Importantly, most SARS-CoV-2 patients were identified using the Sofia Ag-RDT and the short time to result allows for optimized patient management and likely saves resources. For clinical practice e.g. in a hospital ED, it remains important to combine clinical data and results of the Ag-RDT. Current guidelines recommend the use of Ag-RDTs in patients with symptoms compatible with COVID-19 and re-testing of asymptomatic but Ag-RDT negative patients after 1-2 days [26] . This is in line with the procedures of the Infection Control Unit at Klinikum Bielefeld and our findings, where sensitivity of AG-RDT was higher in symptomatic compared to asymptomatic RT-PCR positive patients. As a caveat, false negative Ag-RDT results in a hospital setting are a concern as these patients might give rise to nosocomial transmissions. In these cases, the use of RT-PCR and adherence to infection prevention measures is recommended until final confirmation [27] . A strength of our study is the large number of unselected patients, which were all uniformly tested using both methods and its prospective nature. In addition, our study period covered times of varying SARS-CoV-2 test prevalence and the introduction of the SARS-CoV-2 Alpha VOC. Unlike previous studies we included patients of all age groups. However, now that the Delta VOC emerged worlwide follow-up studies are warranted. Of note, Bourassa et al. recently demonstrated drop outs of the Sofia 2 Ag-RDT due to a D399N nucleocapsid gene mutation of SARS-CoV-2. Reassuringly, preliminary data showed that the Delta VOC had no impact on the perfromance of Ag-RDT [28] . This study has several limitations. First, data of symptom status have been obtained retrospectively from the medical charts, thus rely on proper documentation and is prone for underestimation. Limited resources prevented us from reviewing the medical charts of every included patient. In addition, no systematic information on symptom onset in relation to date of testing was available to us and no information about symptom status of the test negative patients. Third, we did not perform cell culture due to a lack of resources to detemine infectivity but extrapolated this using Ct-values. Furthermore Ct-values have been obtained from six different assays and differences in the performance of the RT-PCR assays might have influenced the results of our study into an unknown direction. This study evaluated patients from the emergency department so test performance might differ in other test settings. In conclusion, the sensitivity of the Sofia Ag-RDT in comparison to RT-PCR in a real-world hospital setting was below the threshold recommended by WHO. However, the Sofia Ag-RDT proved to be a reliable tool to rapidly identify high viral loads (Ct-value ≤25), which might be promising in detecting infectious patients. Only a small fraction of Ag-RDT negative/RT-PCR positive samples yielded SARS-CoV-2 concentrations, which were likely to be infectious. Ethical approval: Ethical approval was obtained (Az 2020-870-f-S; AeKWL/WWU Muenster). Comparison of the Quidel Sofia SARS FIA Test to the Hologic Aptima SARS-CoV-2 TMA Test for Diagnosis of COVID-19 in Symptomatic Outpatients Sofia Rapid Antigen Assay for Detection of Severe Acute Respiratory Syndrome Coronavirus 2 among Emergency Department to Hospital Admissions COVID-19 Target product profiles for priority diagnostics to support response to the COVID-19 pandemic v.1.0. accessed 10th Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays Diagnostic Performance of an Antigen Test with RT-PCR for the Detection of SARS-CoV-2 in a Hospital Setting Surveillance testing for SARS-COV-2 infection in an asymptomatic athlete population: a prospective cohort study with 123 362 tests and 23 463 paired RT-PCR/antigen samples Comparative performance of SARS CoV-2 lateral flow antigen tests demonstrates their utility for high sensitivity detection of infectious virus in clinical specimens Real-life evaluation of the Sofia SARS-CoV-2 antigen assay in a large tertiary care hospital Nucleocapsid Variant that Affects Antigen Test Performance 7-Tage-Inzidenzen nach Bundesländern und Kreisen (fixierte Werte) sowie Gesamtübersicht der pro Tag ans RKI übermittelten Fälle und Todesfälle Berichte zu Virusvarianten von SARS-CoV-2 in Deutschland Quidel Corporation. Quidel Corporation. 2020 Comparison of commercial realtime reverse transcription PCR assays for the detection of SARS-CoV Multicenter Evaluation of the Cepheid Xpert Xpress SARS-CoV-2 Test Comparison of seven commercial RT-PCR diagnostic kits for COVID-19 Abschätzung der Viruskonzentration im Probenmaterial A generic, scalable, and rapid TR-FRET -based assay for SARS-CoV-2 antigen detection Assessment of SARS-CoV-2 infectivity by a Rapid Antigen Detection Test Predicting Infectious Severe Acute Respiratory Syndrome Coronavirus 2 From Diagnostic Samples Virological assessment of hospitalized patients with COVID-2019 Diagnostic accuracy of rapid antigen tests in asymptomatic and presymptomatic close contacts of individuals with confirmed SARS-CoV-2 infection: cross sectional study Performance of a Point-of-Care Test for the Rapid Detection of SARS-CoV-2 Evaluation of three rapid lateral flow antigen detection tests for the diagnosis of SARS-CoV-2 infection Evaluation of Abbott BinaxNOW Rapid Antigen Test for SARS-CoV-2 Infection at Two Community-Based Testing Sites -Pima County Performance of an Antigen-Based Test for Asymptomatic and Symptomatic SARS-CoV-2 Testing at Two University Campuses -Wisconsin Rapid Diagnostic Testing for SARS-CoV-2 SARS-CoV-2 antigen rapid immunoassay for diagnosis of COVID-19 in the emergency department Multidisciplinary assessment of the Abbott BinaxNOW SARS-CoV-2 point-of-care antigen test in the context of emerging viral variants and self-administration Box plot diagrams comparing the median, interquartile range (IQR) and range of Ct-values between true positive (TP) and false negative (FN) results for the AG-RDT samples (n=375) Acknowledgements We kindly acknowledge expert technical assistance of all laboratory technicians. The authors have no relevant competing interest to disclose in relation to this work.