key: cord-0734804-dn1ta7bl authors: Suliman, S.; Matias, W. R.; Fulcher, I. R.; Molano, F. J.; Collins, S.; Uceta, E.; Zhu, J.; Paxton, R. M.; Gonsalves, S. F.; Harden, M. V.; Fisher, M.; Meldrim, J.; Gabriel, S.; Franke, M. F.; Hung, D.; Smole, S. C.; Madoff, L. C.; Ivers, L. C. title: Evaluation of the Access Bio CareStartTM rapid SARS-CoV-2 antigen test in asymptomatic individuals tested at a community mass-testing program in Western Massachusetts date: 2021-06-20 journal: nan DOI: 10.1101/2021.06.17.21259109 sha: dd76de53bac751177c9e7679bd2306d81c4243f7 doc_id: 734804 cord_uid: dn1ta7bl Background: Point-of-care antigen-detecting rapid diagnostic tests (RDTs) for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) represent a scalable tool for SARS-CoV-2 infections surveillance. Data on their performance in real-world community settings is paramount for their implementation. Method: We evaluated the accuracy of CareStartTM COVID-19 Antigen test (CareStart) in a testing site in Holyoke, Massachusetts. We compared CareStart to a SARS-CoV-2 reverse transcriptase quantitative polymerase chain reaction (RT-qPCR) reference, using anterior nasal swab samples. We calculated the sensitivity, specificity, and expected positive and negative predictive values at different SARS-CoV-2 prevalence estimates. Results: We performed 666 tests on 591 unique individuals. 573 (86%) were asymptomatic. There were 52 positive tests by RT-qPCR. The sensitivity of CareStart was 49.0% (95% Confidence Interval (CI): 34.8 - 63.4) and specificity was 99.5% (95% CI: 98.5 - 99.9). Among positive RT-qPCR tests, the median cycle threshold (Ct) was significantly lower in samples that tested positive on CareStart. Using a Ct [≤] 30 as a benchmark for positivity increased the sensitivity to 64.9% (95% CI: 47.5 - 79.8). Conclusions: CareStart has a high specificity and moderate sensitivity. The utility of RDTs, such as CareStart, in mass implementation should prioritize use cases in which a higher specificity is more important. The Coronavirus disease 2019 pandemic caused by the Severe Acute 53 Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is the most significant infectious disease 54 pandemic in the last century (1, 2). In addition to preventive measures such as social distancing, 55 mask wearing, and vaccination, pillars of pandemic control rely on tools to rapidly identify cases in detecting SARS-CoV-2 infection in asymptomatic carriers. In fact, several independent 77 evaluations demonstrate the decreased sensitivity of antigen-detecting RDTs in asymptomatic 78 RT-qPCR positive individuals compared to those with symptoms (14-17). In the United States, 79 studies thus far have focused on 3 RDTs: Quidel Sofia™ (8, 18) , BD Veritor™ (13, 19) and Abbott 80 BinaxNOW™ (15, 16, 20) . On March 31 st , 2021, the FDA also authorized these tests for home 81 use, raising concerns about misinterpretation of false negative results (21). Therefore, evidence 82 to establish their performance characteristics to guide their implementation in real-world settings 83 is even more urgent now. In this study, we evaluated the Access Bio CareStart™ COVID-19 RDT (CareStart), a 85 chromatographic antigen-detecting lateral flow immunoassay that received EUA by the FDA (12, 86 17). We evaluated CareStart in asymptomatic and mildly symptomatic individuals presenting for 87 routine testing at one of the 'Stop the Spread' free community testing sites in Holyoke, Massachusetts. Public health messaging for testing at these community testing sites targeted 89 asymptomatic individuals. We evaluate the sensitivity, specificity, and positive (PPV) and negative week. Individuals who presented to the site during testing hours were approached by our research 101 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 staff who explained the nature of the study, risks, benefits, and answered any questions before 102 inviting individuals to participate in the study. Verbal consent was obtained from participants to 103 collect a second anterior nasal swab as well as from guardians of minors below 18 years of age, 104 from whom verbal assent was also obtained. The participants were treated in accordance with Both anterior nares were swabbed 2 times (5 rotations in each nostril), once for RT-qPCR testing 124 and once for the RDT sample. For practical reasons, the swabs for RT-qPCR and RDT were not 125 always collected in the same order. Both samples were placed inside closed test tubes. The RT- . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 qPCR sample was transported to the Broad Institute at the Massachusetts Institute of Technology. The second anterior nasal swab sample was transported to a nearby testing station and the RDT 128 was performed within an hour of sample collection. The RT-qPCR testing results were interpreted 129 according to the publicly available rubric for the Broad Institute COVID-19 testing program: CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10. 1101 /2021 We calculated sensitivity, specificity, PPV and NPV of the RDT from 2 x 2 contingency tables 152 using RT-qPCR as the gold standard reference. Tests with undetermined CareStart or RT-qPCR 153 results were excluded from these calculations (n=35; 5.2% of all tests). Sensitivity and specificity 154 were further stratified and compared by presence of symptoms and quantitative Ct values. Median Ct values were compared using the non-parametric unpaired Mann-Whitney U test. 95% Pearson- Clopper confidence intervals (CI) were calculated for sensitivity and specificity estimates. Since RDTs have been reported to have high accuracy among symptomatic individuals (8, 15-17), we 158 also tested whether presence of symptoms would increase the sensitivity of the CareStart RDT. Statistical analyses were conducted using R V3.6.0 (R Core Team 2020). We performed 666 CareStart RDTs from participants who provided verbal consent at the walk-up 163 testing site (Table 1 and Figure 1) The study staff evaluated the usability of the CareStart devices. All tests showed a positive 175 control band, indicating they were valid. However, plastic in some vial caps was slightly 176 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. To determine the accuracy of the CareStart RDT, we calculated the concordance between 181 the RDT and RT-qPCR ( CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint (24), these data suggest that applying a more stringent Ct value threshold moderately improves 202 the sensitivity of the CareStart RDT. Positive and negative predictive values of diagnostic tests depend on the prevalence of 204 infections in a population, where a higher prevalence increases the PPV at the expense of the 205 NPV (26). We calculated the PPV and NPV values as a function of prevalence rates up to 10%, where the PPV steeply dropped in prevalence rates lower than 5% (Figure 4) . At a sensitivity of 207 49% and specificity of 99.5% (Table 3) , the PPV of CareStart was 49.7% at a SARS-CoV-2 208 infection prevalence of 1%, and 91.6% at a prevalence of 10%. In contrast, the NPV was 99.5% 209 at a prevalence of 1%, and 94.6% at a prevalence of 10%. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. In laboratory studies, lower RT-qPCR Ct values were shown to correlate with increased recovery . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. Therefore, this limited sample supports the potential usefulness of serial rapid antigen testing in 286 detecting recent infections to implement containment measures and merits further study (38). Importantly, the public health benefits of serial testing with RDTs should be studied further. There are several advantages to rapid testing. CareStart has a lower turnaround time than . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 A few factors may have compromised the performance characteristics of CareStart in this 301 study. We had limited control over or ability to monitor the order by which the two bilateral nasal 302 swabs were collected because of embedding the study in a 'real world' testing program. It is 303 possible that performing the PCR swab first may decrease the available viral load for the antigen 304 test. However, a recent evaluation of the Abbott BinaxNOW suggested that the order of swabs CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint Repeat testers . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 20, 2021. ; No 2 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021. An interactive web-based dashboard to track COVID-367 19 in real time Clinical characteristics of novel coronavirus cases in tertiary 371 hospitals in Hubei Province Test, re-test, re-test': using inaccurate tests to greatly 375 increase the accuracy of COVID-19 testing Nucleic-acid-amplification tests from respiratory 380 samples for the diagnosis of coronavirus infections: systematic review and meta-analysis Epub 2020/11/15 The scientific and ethical feasibility 384 of immunity passports Prevalence of Asymptomatic SARS-CoV-2 Infection : A 387 Narrative Review Edmunds 390 WJ, Centre for the Mathematical Modelling of Infectious Diseases C-wg. Quarantine and 391 testing strategies in contact tracing for SARS-CoV-2: a modelling study Performance of an Antigen-Based Test for Asymptomatic and 400 Symptomatic SARS-CoV-2 Testing at Two University Campuses -Wisconsin Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid 407 immunoassays. Interim Guidance point-of-care antigen and 414 molecular-based tests for diagnosis of SARS-CoV-2 infection Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test 424 performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-425 of-care test Diagnostic performance of a SARS-CoV-2 rapid antigen test in a large, 429 Norwegian cohort Madoff 433 LC BinaxNOW Rapid Antigen Test in a High-throughput Drive-through Community Testing 435 Site in Massachusetts Performance characteristics of a rapid SARS-CoV-2 antigen detection assay at 441 a public plaza testing site in San Francisco Madoff 445 LC CareStart Rapid Antigen Test in a High-throughput Drive-through Community Testing Site 447 in Massachusetts Longitudinal assessment of 455 diagnostic test performance over the course of acute SARS-CoV-2 infection Evaluation of Performance of the BD Veritor 459 SARS-CoV-2 Chromatographic Immunoassay Test in COVID-19 Symptomatic Patients Performance Characteristics of BinaxNOW COVID-19 Antigen Card for 464 Screening Asymptomatic Individuals in a University Setting Counter and Other Screening Test Development. U S Food and Drug Administration Update to the standardized surveillance case definition 470 and national notification for 2019 novel coronavirus disease (COVID-19). infectious 471 Disease Committee Rapid Diagnostic Test for the 478 Detection of SARS-CoV-2 Antigen Accessed online La Scola 482 B. Correlation Between 3790 Quantitative Polymerase Chain Reaction-Positives 483 Samples and Positive Cell Cultures High Negative Predictive Value, Low Prevalence, and 486 Spectrum Effect: Caution in the Interpretation Guidelines on newly identified limitations of diagnostic tools for COVID-19 490 and consequences Disparities in SARS-CoV-2 Testing in Massachusetts During the COVID-19 Pandemic Community-Level 498 Factors Associated With Racial And Ethnic Disparities In COVID-19 Rates Health Aff (Millwood) COVID-19 patients: A systematic review and meta-analysis Koopmans 509 MPG, Geurtsvankessel C, van der Eijk AA. Duration and key determinants of infectious 510 virus shedding in hospitalized patients Diagnostic performance of CerTest and Panbio antigen rapid diagnostic tests 516 to diagnose SARS-CoV-2 infection 520 Predicting infectious SARS-CoV-2 from diagnostic samples. Clinical infectious diseases 521 : an official publication of the Infectious Diseases Society of America Initial report of 525 decreased SARS-CoV-2 viral load after inoculation with the BNT162b2 vaccine Postvaccination SARS-CoV-2 Infections Among Skilled Nursing 529 Facility Residents and Staff Members PubMed PMID: 33914721; PMCID: PMC8084122 Journal 532 Editors form for disclosure of potential conflicts of interest. No potential conflicts of interest 533 were disclosed Estimation of 538 secondary household attack rates for emergent spike L452R SARS-CoV-2 variants 539 detected by genomic surveillance at a community-based testing site in San Francisco. 540 Clinical infectious diseases : an official publication of the Infectious Diseases Society Antigen-Based Testing but Not Real-Time Polymerase 544 Chain Reaction Correlates With Severe Acute Respiratory Syndrome Coronavirus 2 Viral 545 Clinical infectious diseases : an official publication of the Infectious Diseases 546 Society of America Clarifying the 549 evidence on SARS-CoV-2 antigen rapid tests in public health responses to COVID-19 Exploring alternative swabs for use in SARS-CoV-2 detection from the oropharynx and 554 anterior nares Impaired performance of SARS-CoV-558 2 antigen-detecting rapid diagnostic tests at elevated and low temperatures The Holyoke Board of Health was instrumental in coordinating study logistics with Fallon 354 Ambulance Services who performed the swabs for RT-qPCR. The MA Department of Public