key: cord-1049251-mx63blmr
authors: McKay, Susannah L.; Tobolowsky, Farrell A.; Moritz, Erin D.; Hatfield, Kelly M.; Bhatnagar, Amelia; LaVoie, Stephen P.; Jackson, David A.; Lecy, K. Danielle; Bryant-Genevier, Jonathan; Campbell, Davina; Freeman, Brandi; Gilbert, Sarah E.; Folster, Jennifer M.; Medrzycki, Magdalena; Shewmaker, Patricia L.; Bankamp, Bettina; Radford, Kay W.; Anderson, Raydel; Bowen, Michael D.; Negley, Jeanne; Reddy, Sujan C.; Jernigan, John A.; Brown, Allison C.; McDonald, L. Clifford; Kutty, Preeta K.
title: Performance Evaluation of Serial SARS-CoV-2 Rapid Antigen Testing During a Nursing Home Outbreak
date: 2021-04-27
journal: Ann Intern Med
DOI: 10.7326/m21-0422
sha: 391c3efaee424f1728af0ab18207f83e1001cd27
doc_id: 1049251
cord_uid: mx63blmr

BACKGROUND: To address high COVID-19 burden in U.S. nursing homes, rapid SARS-CoV-2 antigen tests have been widely distributed in those facilities. However, performance data are lacking, especially in asymptomatic people. OBJECTIVE: To evaluate the performance of SARS-CoV-2 antigen testing when used for facility-wide testing during a nursing home outbreak. DESIGN: A prospective evaluation involving 3 facility-wide rounds of testing where paired respiratory specimens were collected to evaluate the performance of the BinaxNOW antigen test compared with virus culture and real-time reverse transcription polymerase chain reaction (RT-PCR). Early and late infection were defined using changes in RT-PCR cycle threshold values and prior test results. SETTING: A nursing home with an ongoing SARS-CoV-2 outbreak. PARTICIPANTS: 532 paired specimens collected from 234 available residents and staff. MEASUREMENTS: Percentage of positive agreement (PPA) and percentage of negative agreement (PNA) for BinaxNOW compared with RT-PCR and virus culture. RESULTS: BinaxNOW PPA with virus culture, used for detection of replication-competent virus, was 95%. However, the overall PPA of antigen testing with RT-PCR was 69%, and PNA was 98%. When only the first positive test result was analyzed for each participant, PPA of antigen testing with RT-PCR was 82% among 45 symptomatic people and 52% among 343 asymptomatic people. Compared with RT-PCR and virus culture, the BinaxNOW test performed well in early infection (86% and 95%, respectively) and poorly in late infection (51% and no recovered virus, respectively). LIMITATION: Accurate symptom ascertainment was challenging in nursing home residents; test performance may not be representative of testing done by nonlaboratory staff. CONCLUSION: Despite lower positive agreement compared with RT-PCR, antigen test positivity had higher agreement with shedding of replication-competent virus. These results suggest that antigen testing could be a useful tool to rapidly identify contagious people at risk for transmitting SARS-CoV-2 during nascent outbreaks and help reduce COVID-19 burden in nursing homes. PRIMARY FUNDING SOURCE: None.

Between 22 October and 3 November 2020, serial, facility-wide testing of all residents and staff was done 3 times over a 13-day period during an ongoing SARS-CoV-2 outbreak. Specimens were collected from all available and assenting residents and staff present on days of testing, including people identified as SARS-CoV-2-positive before 21 October. During the first round of facilitywide testing, trained project personnel collected paired bilateral swabs from the anterior nares (AN) of residents for antigen testing and RT-PCR and, from nursing home staff, an AN swab for antigen testing and a nasophary-ngeal swab from a single naris for RT-PCR. Because of patient intolerance, nasopharyngeal swabbing was discontinued during the second and third testing rounds and paired bilateral AN swabs were collected from both residents and staff (Appendix 2, available at Annals.org). All specimens were collected in accordance with CDC guidelines for specimen collection and handling (4) . Trained laboratory scientists tested 1 AN swab onsite using the BinaxNOW COVID-19 Ag Cards per manufacturer instructions for use (23) . The other was sent to the CDC for RT-PCR and virus culture reference testing.

The facility provided demographic characteristics and prior antigen testing results for residents and staff. During 7 to 21 October, the facility exclusively used BinaxNOW testing, and prior antigen positivity was defined as any positive result on a SARS-CoV-2 test during this time. At each visit, project personnel administered a standardized questionnaire assessing COVID-19like symptoms (24) . Able residents and staff self-reported symptoms at the time of testing. For residents who could not self-report, symptom information was obtained from nursing staff and electronic medical records and confirmed by residents, if possible. A symptomatic participant was defined as a resident or staff member who, at the time of collection, reported any new or worsening symptoms similar to those of in the 14 days before that round of testing.

Participant specimens were tested for SARS-CoV-2 RNA by RT-PCR using the CDC Influenza SARS-CoV-2 (Flu SC2) Multiplex Assay (25) on the Applied Biosystems 7500 Fast Dx Real-Time PCR Instrument (Thermo Fisher Scientific). Nucleic acid was extracted by either the QIAGEN EZ1 or the Roche MagNA Pure 96 extraction platforms. Cycle threshold (Ct) values were reported for the SARS-CoV-2 viral nucleocapsid protein gene target. Values less than 40 indicated that a specimen was positive for SARS-CoV-2 RNA. Previous experience showed an inability to detect culture-positive virus in samples with a Ct greater than 34. Therefore, virus culture was attempted on RT-PCR-positive specimens with Ct values of 34 or less and RT-PCR-negative, antigen-positive specimens. Culture was done using Vero CCL-81 cells, as previously described (26). Cells showing cytopathic 

Serial SARS-CoV-2 Rapid Antigen Testing During a Nursing Home Outbreak effect up to 8 days after culture inoculation were tested for the presence of SARS-CoV-2 by RT-PCR to confirm virus isolation and growth in culture (Appendix 2). Specimens were categorized into stages of infection using prior test results and Ct values. Stages were defined as early (low or decreasing Ct values), late (increasing or sustained high Ct values), resolved (negative test result in a person with a prior positive result), or uninfected (consecutive negative results in specimens from a person with no prior positive result). Table 1 gives full definitions.

Descriptive analyses were done using SAS, version 9.4 (SAS Institute). We determined PPA and PNA by comparing antigen test results with reference tests. Paired specimens with at least 1 invalid test result were excluded from analysis. We calculated PPA and PNA for all participants and stratified by resident or staff, symptom status, previous positivity by any test, specimen type, and stage of infection (27) . Clopper-Pearson exact binomial methods were used to calculate CIs.

This activity was reviewed by CDC, and its conduct was consistent with applicable federal law and CDC policy (28-32). This work did not receive any non-CDC funding support.

A total of 107 staff members participated in at least 1 round of paired testing; the median age was 39 years (range, 21 to 72 years), 81% (n = 87) were female, and 75% (n = 80) were Black. A total of 127 residents participated in at least 1 round of paired testing; the median age was 75 years (range, 35 to 101 years), 43% (n = 55) were female, and 60% (n = 76) were Black (Appendix Table 1 , available at Annals.org). Among 234 participants, 54% of residents (68 of 127) and 11% of staff (12 of 107) had at least 1 positive result on antigen or RT-PCR testing, including 43 of 68 residents and 5 of 12 staff who had tested positive at the facility between 7 and 21 October 2020.

During 3 facility-wide testing events between 22 October and 3 November 2020, a total of 532 paired specimens were collected, including 388 from people who had not previously tested positive (Appendix Table  2 , available at Annals.org) and 144 from those who had tested positive at least once since 7 October 2020. Details on the number of people tested during each facility-wide testing event are in Table 2 . No specimens tested positive for influenza.

Overall, 113 of 532 paired specimens (21%) were positive by antigen or RT-PCR testing. Of those that tested positive, 64% (72 of 113) were positive for both antigen and RT-PCR, 29% (33 of 113) were discordant RT-PCR-positive and antigen-negative, and 7% (8 of 113) were discordant RT-PCR-negative and antigen-positive (Appendix Table 2 ). The 8 discordant paired specimens that were RT-PCR-negative and antigen-positive were collected from 7 people who had previously tested positive, and 6 occurred 2 weeks or longer after the first positive test result (median, 18 days [range, 6 to 20 days] ). Across all 532 paired specimens, PPA between antigen test and RT-PCR was 69% (95% CI, 59% to 77%) and PNA was 98% (CI, 96% to 99%) ( Figure 1 , top). Among 388 specimens from people without a prior positive result, PPA between antigen test and RT-PCR was 63% (CI, 44% to 79%). Antigen test performance was similar to the overall results when limited to 1 test per person (at first test, and when stratified by round of facility-wide testing) (Appendix Table 3 , available at Annals.org). When stratified by symptom reports, PPA between antigen test and RT-PCR was 82% (CI, 48% to 98%) among specimens from symptomatic participants and 52% (CI, 30% to 74%) among those from asymptomatic participants. Between antigen test and RT-PCR, PNA remained close to 100% across all categories ( Figure 1 [top] and Appendix Table  2 ). Antigen test performance (that is, PPA and PNA) compared with RT-PCR was also similar for staff and residents (Appendix Table 4 , available at Annals.org) overall and stratified by symptom status. Antigen test performance compared with RT-PCR was also similar for nasopharyngeal and AN swabs (Appendix Table 5 , available at Annals.org).

Virus was recovered from 21% of positive specimens (21 of 101) where virus culture was attempted (Appendix Table 6 , available at Annals.org), including 29% (20 of 69) of concordant RT-PCR-positive and antigen-positive Tested at rounds 1, 2 23 (10) 18 (14) 5 (5) 1 Tested at rounds 1, 3 8 (3) 2 (2) 6 (6) 1

Tested at round 1 only 25 (11) 7 (6) 18 (17) 2 Tested at rounds 2, 3 17 (7) 4 (3) 13 (12) 2 Tested at round 2 only 18 (8) 3 (2) 15 (14) 3 Tested at round 3 only 18 (8) 2 (2) 16 (15) specimens and 4% (1 of 24) of specimens that were RT-PCR-positive and antigen-negative (Appendix Figure, available at Annals.org). Virus was not recovered from the 8 discordant paired specimens that were RT-PCRnegative and antigen-positive (Appendix Figure) . Using virus culture as the reference standard, PPA with antigen testing was 95% (CI, 86% to 100%; note that negative agreement with virus culture was not applicable because only specimens most likely to harbor infectious virus, including those with Ct ≤34 and antigen-positive specimens, were subjected to virus culturing) ( Figure 1 , bottom). Antigen test performance was similar to the overall results when limited to 1 test per person (Appendix Table 7 , available at Annals.org). The majority of culturepositive specimens (15 of 21 [71%] ) were collected 0 to 5 days from the first positive test result; 1 specimen was culture-positive at 13 days (Appendix Figure) . In the subset of 31 paired specimens from people without a prior positive result, PPA between antigen test and virus culture was 92% (CI, 62% to 100%) (Appendix Table 6 ).

Among 105 RT-PCR-positive specimens, we compared Ct values in relation to antigen test result and virus culture ( Figure 2 ). The median Ct value was significantly lower for antigen-positive paired specimens (median, 28.0 [range, 15.4 to 36.4] ) than for antigen-negative paired specimens (median, 33.2 [range, 21.3 to 38.7] ) (Wilcoxon P < 0.001) (Figure 2) . Similarly, among the 93 paired specimens that were RT-PCR-positive and had a Ct value of 34 or less for which virus culture was attempted, the median Ct that resulted in positive virus culture was significantly lower (median, 21.3 [range, 15.4 to 26.7] ) than that for culture-negative specimens (median, 30.2 [range, 22.5 to 35 .0]; Wilcoxon P < 0.001).

Among 173 people who were tested in more than 1 round of testing between 22 October and 3 November, 56 (32%) tested positive by RT-PCR at least once. Of Table 8 , available at Annals.org).

Among specimens categorized as early infection, PPA was 86% (CI, 74% to 94%) with RT-PCR (Appendix Table 8 ) and 95% (CI, 76% to 100%) with virus culture (Appendix Table 9 , available at Annals.org). Among specimens categorized as late infection, PPA was 51% (CI, 36% to 66%) with RT-PCR and none were positive by virus culture. Among paired specimens categorized as early infection, the median Ct value was significantly lower for antigen-positive pairs (median, 25.1 [range, 15.4 to 36.4] ) than for antigen-negative pairs (median, 28.6 [range, 15.4 to 36.4] ) (Wilcoxon P = 0.049) (Figure 3) . The median Ct value among paired specimens categorized as late infection was significantly lower for antigen-positive pairs (median, 31.7 [range, 30.1 to 36.4] ) than for antigen-negative pairs (median, 34.8 [range, 30.1 to 38.7] ) (Wilcoxon P = 0.006) (Figure 3 ).

Although highly sensitive RT-PCR can be an effective tool for thorough case finding during a nursing home outbreak, using RT-PCR to provide actionable results requires rapid turnaround and is likely to identify noninfectious people in addition to infectious ones. Despite low overall PPA compared with RT-PCR, in this evaluation antigen testing performed well in identifying early infections and specimens with replication-competent virus (that is, culture-positive). Further, consensus test analysis of test-positive individuals with more than 1 test result suggested that repeated testing produced similar PPA for antigen testing compared with RT-PCR regardless of the presence of symptoms. Our data suggest that early and frequent antigen testing during a SARS-CoV-2 outbreak can effectively identify infectious people with the greatest potential to transmit the virus.

Previous studies have shown that people with asymptomatic and presymptomatic SARS-CoV-2 infections can harbor high viral loads and contribute to widespread transmission within a nursing home (3, 33, 34) . Rapid identification of these people is essential, and frequent facility-wide testing is recommended, particularly in outbreak settings (3) . Although our data suggest that nearly a third of RT-PCR-positive infections were missed overall, the antigen test was able to identify 86% of infections when testing was done during early infection when people are more likely to be infectious. Previous work has shown that people can continue to test positive for SARS-CoV-2 by RT-PCR for weeks after they are no longer infectious (7, 9) . Thus, comparisons of antigen testing with virus culture might provide a more accurate measure of antigen test performance for identifying infectious people. In this evaluation, PPA was very high (95%) among participants who had replication-competent virus in their specimen, suggesting that rapid antigen tests might be more useful for detecting people who are infectious. Pekosz and colleagues (35) found similar agreement (96%) between antigen testing and virus culture using the BD Veritor System for Rapid Detection of SARS-CoV-2, a lateral flow antigen detection test, on a convenience sample of RT-PCR-positive specimens. Of note, we found that 1 participant who had virus culture-positive specimens from 2 consecutive rounds of testing done 6 days apart had corresponding specimens that were also antigen-positive.

The antigen test was effective for identifying SARS-CoV-2 during early infection when viral RNA load might be high but was less effective during late infection. Kissler and colleagues (36) used serial RT-PCR testing to define and characterize infection stage dynamics (proliferation, clearance, and persistence) for symptomatic and asymptomatic infections. In their analysis, the average proliferation stage lasted 2 to 4 days and was similar regardless of symptoms. Thus, by doing point prevalence surveys every 5 to 6 days, we might have identified additional infections but potentially missed the proliferation stage of some new infections. Taken together, these data suggest that frequent antigen testing during an initial outbreak response might be an effective strategy for screening and identifying new SARS-CoV-2 infections that are in the early, proliferative stagethat is, the highly infectious period.

Despite previous concern about false-positive test results (17) , only 8 false positives occurred during this evaluation (PNA, 98%), similar to rates reported in the EUA (99%) (37) . Even with a high PNA because of a large volume of testing in nursing homes with a low percentage of positivity, there are concerns that frequent false positives may be problematic within a facility. Of note, all 8 specimens were collected from people with previously positive results (6 of 7 of whom tested positive by RT-PCR during the evaluation), suggesting that these false positives had some association with true infection and were not solely attributable to user error.

Our findings are subject to several limitations. Antigen test performance for asymptomatic infections might have been overestimated because of challenges in symptom ascertainment that led to misclassification of symptomatic people as asymptomatic. In addition, although virus culture was used to identify replication-competent virus, the inability to culture virus from a given specimen does not mean that replication-competent virus was not present in that specimen or person. Further, antigen testing was done by CDC laboratory staff, and test performance by nonlabo-ratory staff may not be equivalent (38) . Finally, although Kissler and colleagues also showed that repeated quantitative testing with RT-PCR can be used to infer infection stages (36) , the categorizations described for this evaluation might not be generalizable to other populations, test protocols, or testing frequencies and do not account for host factors, including antibody development (36) .

Many antigen tests are inexpensive, fast, and relatively easy to perform and can be used to augment the testing capacity of clinical and public health laboratories. Despite the overall lower PPA compared with the reported EUA data, these findings show that the BinaxNOW antigen test performed well for identifying people who are infectious and will likely perform well when used serially as a screening tool for nascent and emerging COVID-19 outbreaks. Further, the generally high PNA between antigen testing and RT-PCR supports not doing confirmatory testing on antigen-positive individuals when the pretest probability is high, as in a large nursing home outbreak (4) . Taken together, these data suggest that serial antigen testing early and often could be an effective testing strategy to support infection control in nursing homes having a SARS-CoV-2 outbreak. These findings merit further evaluation in other congregate settings, such as university campuses, hospitals, and detention centers. Acknowledgment: The authors thank the residents and staff at the participating nursing home facility for their time and willingness to contribute to this project. They also acknowledge additional CDC contributors who supported this work, in part, through critical review of this manuscript, including Mary Ann Hall, MPH, and Alison Laufer-Halpin, PhD. 

Serial SARS-CoV-2 Rapid Antigen Testing During a Nursing Home Outbreak

Bilateral AN swab collection was done in 2 steps: First, the RT-PCR swab was inserted into 1 naris and the antigen swab into the other; then, each swab was removed and used to sample the opposite naris. When paired specimens included a nasopharyngeal swab for RT-PCR and AN swab for antigen testing, the bilateral AN swab was collected first followed by the nasopharyngeal swab collected from a single naris.

To perform virus culture, 100 ml of clinical specimen was diluted 2-fold across a 96-well plate in serum-free Dulbecco's Modified Eagle Medium supplemented with (Sigma). Vero CCL-81 cells were trypsinized and resuspended in Dulbecco's Modified Eagle Medium plus 10% fetal bovine serum plus 2 Â penicillin-streptomycin plus 2 Â amphotericin B at 2.5 Â 10 5 cells/mL. A 100-ml cell suspension was added directly to the clinical specimen dilutions and mixed gently by pipetting. The inoculated cultures were grown in a humidified 37 C incubator with 5% CO 2 and observed for cytopathic effect daily. When cytopathic effect was observed, presence of SARS-CoV-2 was confirmed by RT-PCR.

Among 69 concordant antigen-positive and RT-PCRpositive specimens that were assessed for virus culture, 20 were virus culture-positive. These concordant specimens had a lower median Ct value (median, 21.1 [range, 15.4 to 26.7] ) than 49 that were virus culture-negative (median, 29.7 [range, 22.5 to 35 .0]) (P < 0.001). The remaining culture-positive specimen was discordant antigen-negative and RT-PCR-positive; this specimen was cultured from a nasopharyngeal swab.

Appendix Appendix Figure. Agreement between antigen testing and RT-PCR and VC among 101 paired specimens with VC result, over time. 

Ag, RT-PCR, and VC results for 101 specimens from 63 individuals. VC was attempted for specimens that were Ag-positive and RT-PCR-negative (circle with X) but not attempted for RT-PCR-positive specimens with Ct values ≥35 (black squares). Ag = antigen; Ct = cycle threshold; RT-PCR = reverse transcription polymerase chain reaction; VC = virus culture. 

COVID-19 nursing home data

Risk factors, presentation, and course of coronavirus disease 2019 in a large, academic long-term care facility

Presymptomatic SARS-CoV-2 infections and transmission in a skilled nursing facility

SARS-CoV-2 antigen testing in long term care facilities

COVID-19 test result turnaround time for residents and staff in US nursing homes

Serial testing for SARS-CoV-2 and virus whole genome sequencing inform infection risk at two skilled nursing facilities with COVID-19 outbreaks -Minnesota

Predicting infectious severe acute respiratory syndrome coronavirus 2 from diagnostic samples

Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards

Virological assessment of hospitalized patients with COVID-2019

Repeat COVID-19 molecular testing: correlation of SARS-CoV-2 culture with molecular assays and cycle thresholds

Pathogenesis and transmission of SARS-CoV-2 in golden hamsters

SARS-CoV-2 virus culture and subgenomic RNA for respiratory specimens from patients with mild coronavirus disease

Interim guidance on duration of isolation and precautions for adults with COVID-19

Interim guidance for antigen testing for SARS-CoV-2. Updated

Testing guidelines for nursing homes

Individual EUAs for antigen diagnostic tests for SARS-CoV-2. 2020. Accessed at www.fda.gov/ medical-devices/coronavirus-disease-2019-covid-19-emergencyuse-authorizations-medical-devices/vitro-diagnostics-euas# individual-antigen on 10

Accessed at www.fda.gov/medical-devices/ letters-health-care-providers/potential-false-positive-results-antigentests-rapid-detection-sars-cov-2-letter-clinical-laboratory on 10

Performance of an antigen-based test for asymptomatic and symptomatic SARS-CoV-2 testing at two university campuses -Wisconsin

Field performance and public health response using the BinaxNOW TM Rapid SARS-CoV-2 antigen detection assay during community-based testing

Rethinking covid-19 test sensitivity -a strategy for containment

Modeling effectiveness of testing strategies to prevent COVID-19 in nursing homes -United States

CDC. Symptoms of coronavirus. 2020. Accessed at

Individual EUAs for molecular diagnostic tests for SARS-CoV-2. 2020. Accessed at www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitrodiagnostics-euas-molecular-diagnostic-tests-sars-cov-2#individual-molecular on 10

Statistical guidance on reporting results from studies evaluating diagnostic tests: guidance for industry and FDA staff

Asymptomatic and presymptomatic SARS-CoV-2 infections in residents of a longterm care skilled nursing facility

Asymptomatic and presymptomatic severe acute respiratory syndrome coronavirus 2 infection rates in a multistate sample of skilled nursing facilities

Antigen-based testing but not real-time PCR correlates with SARS-CoV-2 virus culture. medRxiv. Preprint posted online 5

SARS-CoV-2 viral dynamics in acute infections. medRxiv. Preprint posted online 1

BinaxNOW COVID-19 Ag Card -instructions for use. U.S. Food and Drug Administration

COVID-19: rapid antigen detection for SARS-CoV-2 by lateral flow assay:a national systematic evaluation for mass-testing. medRxiv. Preprint posted online 26

Current Author Addresses

Negley: Georgia Department of Public Health, 2 Peachtree Street NW #14-225

Author Contributions: Conception and design

Analysis and interpretation of the data

Critical revision of the article for important intellectual content

Administrative, technical, or logistic support

Members of the CDC Infection Prevention and Control Team who contributed to this work but did not author it

Members of the CDC COVID-19 Surge Laboratory Group as part of the Laboratory and Testing Task Force who authored this work

Members of the CDC COVID-19 Surge Laboratory Group as part of the Laboratory and Testing Task Force who contributed to this work but did not author it