key: cord-0726838-zqh8t1ym
authors: Moore, N. M.; Li, H.; Schejbal, D.; Lindsley, J.; Hayden, M.
title: Comparison of two commercial molecular tests and a laboratory-developed modification of the CDC 2019-nCOV RT-PCR assay for the qualitative detection of SARS-CoV-2 from upper respiratory tract specimens
date: 2020-05-06
journal: nan
DOI: 10.1101/2020.05.02.20088740
sha: 67653b786a8e50593b43635728646a94bdce61e4
doc_id: 726838
cord_uid: zqh8t1ym

We compared the ability of 2 commercial molecular amplification assays [RealTime SARS-CoV-2 on the m2000 (Abbott) and ID NOW COVID-19 (Abbott)] and a laboratory developed test [modified CDC 2019-nCoV RT-PCR assay with RNA extraction by eMag(R) (bioMeriux) and amplification on QuantStudio 6 or ABI 7500 Real-Time PCR System (Life Technologies)] to detect SARS-CoV-2 RNA in upper respiratory tract specimens. Discrepant results were adjudicated by medical record review. 200 nasopharyngeal swab specimens in viral transport medium were collected from symptomatic patients between March 27 and April 9, 2020. Results were concordant for 167 specimens (84.3% overall agreement), including 94 positive and 73 negative specimens. The RealTime SARS-CoV-2 assay on the m2000 yielded 33 additional positive results, 25 of which were also positive by the modified CDC assay but not by the ID NOW COVID-19 assay. In a follow-up evaluation, 97 patients for whom a dry nasal swab specimen yielded negative results by the ID NOW COVID-19 assay had a paired nasopharyngeal swab specimen collected in viral transport medium and tested by the RealTime SARS-CoV-2 assay; SARS-CoV-2 RNA was detected in 13 (13.4%) of these specimens. Medical record review deemed all discrepant results to be true positives. The ID NOW COVID-19 test was fastest (as soon as 5 minute for positive and 13 minute for negative result.) The RealTime SARS-CoV-2 assay on the m2000 detected more cases of COVID-19 infection than the modified CDC assay or the ID NOW COVID-19 test.

In December 2019, a cluster of patients with pneumonia of unknown origin was linked to exposure to a 37 wet market in Wuhan, Hubei Province, China (1). Very quickly, a novel betacoronavirus was isolated 38 from a lower respiratory tract sample from one patient and the full genome of the virus was sequenced 39

(2). This novel coronavirus, which was named SARS-CoV-2 for its genetic homology to SARS-CoV, spread 40 rapidly across the globe (3-10). As of April 29, 2020, more than 3 million cases of SARS-CoV-2 infection 41 had been identified worldwide, with over 200,000 deaths; approximately one-third of cases have been 42 identified in the United States. 43

Laboratory testing plays a critical role in defining the disease characteristics and epidemiology of 44 an emerging infectious pathogen such as SARS-CoV-2, and in controlling its spread. Early on, laboratory 45 testing for SARS-CoV-2 in the U.S. was performed only at the Centers for Disease Control and Prevention 46 (CDC) laboratories in Atlanta, GA using a reverse transcriptase polymerase chain reaction (RT-PCR) assay 47 that was developed there (11). Subsequently, the CDC test was to be implemented in all state public 48 health laboratories, but roll out was slow due to technical problems. Following the declaration of a 49 public health emergency, the US Food and Drug Administration (FDA) moved to allow in vitro diagnostic 50 assays under an Emergency Use Authorization (EUA) in an attempt to expedite test development by 51 commercial and clinical laboratories. The majority of assays approved through EUA are nucleic acid 52 amplification tests that target conserved regions of the SARS-CoV-2 genome. Abbott Molecular received 53 authorization for the RealTime SARS-CoV-2 assay to be performed on the m2000 real-time platform on 54 March 18, 2020 (12) . The ID NOW™ COVID-19 assay was granted approval under the EUA on March 27, 55 2020. In vitro diagnostic device (IVD) assays with EUA status from commercial manufacturers do not 56 undergo usual FDA review under the De Novo request or the 510(k) premarket notification; as such, 57 limited data comparing these assays are available. 58 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 6, 2020 . . https://doi.org/10.1101 In this study, we compared the performance of two commercial EUA IVD assays and a laboratory 59 developed test that is a modification of the CDC RT-PCR assay for the qualitative detection of SARS-CoV-60 2 RNA directly from upper respiratory tract specimens. 61

Clinical samples. For the initial evaluation of the three test systems, we collected nasopharyngeal swab 63 specimens in 3mL M4-RT viral transport medium (VTM) (Remel, Lenexa, KS) from symptomatic (fever or 64 cough or shortness of breath) adult and pediatric outpatients, emergency department (ED) patients, and 65 inpatients at Rush University Medical Center (RUMC) or Rush Oak Park Hospital (ROPH); both hospitals 66 are in metropolitan Chicago, IL. Specimens were collected between March 27 and April 9, 2020, and 67 tested within 72 hours of collection; specimens were held refrigerated at 4°C if all testing could not be 68 completed on the same day. 69

In a separate follow up evaluation, symptomatic patients who had a negative result on a dry 70 nasal swab that was tested at the point of care by the ID NOW™ COVID-19 system also had a paired 71 nasopharyngeal swab sample collected and transported to the on-site clinical microbiology laboratory 72 for testing by RealTime SARS-CoV-2 on the m2000. 73

Age, sex, and location of swab collection were extracted from the electronic medical record 74 (EMR) for all patients. The study was reviewed and given expedited approval by the RUMC institutional 75 review board, with a waiver of written informed consent. 76

Modified CDC assay. We validated and implemented a modification of the CDC 2019-nCoV assay (11) for 77 clinical use in our laboratory; this was the first SARS-CoV-2 RT-PCR assay we adopted during the COVID-78 19 pandemic. This assay targets two regions of the nucleocapsid (N) gene of the SARS-CoV-2 genome. 79

The human RNase P (RP) gene target is included and used as specimen extraction and amplification 80 control. 81 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101 Nucleic acids were purified and extracted using the eMAG® automated nucleic acid sample 82 extraction system (bioMérieux, Marcy l'etoile, France). Briefly, total nucleic acids were extracted from 83 VTM using a programmed input sample volume of 200µL into 2000µL of easyMAG® lysis buffer with the 84 Specific B protocol to which a final eluted volume of purified nucleic acids was 50µL. We utilized the 85 total sample volume per reaction was 15µL of master mix, combined primer/probe mix, and nuclease 90 free water and 5µL of eluted sample. Assay run parameters were as described in the CDC protocol (11). 91

Samples that gave a cycle threshold (C t ) value <40 for both N1 and N2 targets were considered positive. 92

Samples negative for both N1 and N2 targets had to have a positive amplification curve for the RP gene 93 to be considered a valid negative result. Samples that gave a C t value <40 for either N1 or N2 targets 94 were considered inconclusive and repeat testing was performed per CDC protocol. If results were still 95 inconclusive after repeat testing, a result of inconclusive was reported. Validation results for the 96 laboratory-modified CDC assay are not shown. 97

Abbott Molecular SARS-CoV-2 assay. Next, we verified the RealTime SARS-CoV-2 assay (Abbott 98 Molecular, Des Plaines, IL), which is a qualitative real-time assay performed on the Abbott m2000 99 platform (12). The system includes the m2000sp instrument with automated extraction of nucleic acids 100 using the DNA (total nucleic acid) sample preparation kit in batches of up to 96 samples. The RealTime 101 SARS-CoV-2 assay utilizes two real-time detection probes: one probe combined for the N and RNA-102 dependent RNA polymerase (RdRP) genes, and a second probe for the internal control to assess overall 103 performance, including nucleic acid extraction and possible PCR inhibition. Nasopharyngeal swab 104 samples were heat inactivated at 56°C for 35±5 minutes prior to testing. Automated extraction was 105 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.02.20088740 doi: medRxiv preprint performed using a sample input volume of 500µL VTM, followed by automated addition of amplification 106 pack reagents and extracts (40µL volume used for PCR amplification and detection). Two controls (one 107 positive and one negative) provided by the manufacturer were included with each run of patient 108 samples. Amplification curves were interpreted by the m2000rt system and reported as detected or not 109 detected. 110

In our initial verification of the RealTime SARS-CoV-2 assay, we tested 25 nasopharyngeal swab 111 samples in which SARS-CoV-2 RNA gene sequences had been detected by the laboratory-modified CDC 112 assay, and 30 samples in which SARS-CoV-2 RNA samples were not detected. There was 100% positive 113 and negative agreement between results of the 2 assays [median C t values on the modified CDC assay 114 for positive samples, 25.93 (IQR, 20.3 -28.87) for N1 and 24.6 (IQR, 19.4 -28.35) for N2]. 115 ID NOW™ COVID-19 assay. The third SARS-CoV-2 molecular assay introduced to our laboratory was the 116 ID NOW™ COVID-19 (formerly Alere i), an isothermal nucleic acid amplification test for SARS-CoV-2 RNA 117 that targets the RdRp gene (13). Following an initial 3 minute warm-up of the test system, 200µL of VTM 118 was added to elution buffer in the sample base using the provided disposable transfer pipette, then 119 mixed for 10 seconds with the pipette. Using the sample transfer device, the sample was transferred 120 into the test cartridge, the lid was closed and the instrument automatically initialized the assay. The ID 121 NOW does not report C t values to the user. The instrument software interprets amplification data, and 122 final results are reported on screen as positive, negative, or invalid. 123

Estimation of RNA concentration in respiratory samples. We tested purified genomic RNA from a 124 reference strain of SARS-CoV-2, isolate USA-WA1/2020 (catalog# NR-52285, lot# 70033320) (BEI 125 Resources, Manassas, VA) to generate standard curves for the laboratory-modified CDC assay and the 126

RealTime SARS-CoV-2 assay in order to estimate the concentration of SARS-CoV-2 genome equivalents 127 in nasopharyngeal swab samples. We serially diluted the standard and tested the dilution series in 128 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. including all samples in which SARS-CoV-2 RNA was detected by the RealTime SARS-CoV-2 assay on the 140 m2000 -our standard assay during the study period -and also the next negative sample after the 141 positive sample. Mean age was 50 ± 17 years and 54% were women. Seventy-nine (40%) patients were 142 hospitalized, 29 (36%) of whom were in an intensive care unit; 76 (38%) were cared for in an ambulatory 143 location, including 55 (72%) who were seen in a designated COVID-19 screening clinic; and 45 (23%) 144 were seen in an ED. 145

Assay performance using nasopharyngeal swab samples in viral transport medium. There were 94 146 (47%) samples in which SARS-CoV-2 gene sequences were detected by all assays and 73 (36.5%) samples 147 in which SARS-CoV-2 RNA was detected by none of the assays (Table 1 ). The median cycle number (C n ) 148 for positive samples by the RealTime SARS-CoV-2 assay was 15.34 (IQR, , or approximately 149 447 genome equivalents/µL (Figure 1 ). Overall agreement among the three assays was 83.5% (95% CI, 150 77.7% -88.0%). Two-way positive and negative agreement between results is shown in Table 2 . Positive 151 agreement ranged from 75.2% to 100%, with the lowest agreement between the RealTime SARS-CoV-2 152 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101 and the ID NOW™ COVID-19 assays. Negative agreement ranged from 92.4% (laboratory-modified CDC 153 assay versus RealTime SARS-CoV-2 assay) and 100% (laboratory-modified CDC assay versus ID NOW™ 154 COVID-19 assay, and ID NOW™ COVID-19 assay versus RealTime SARS-CoV-2 assay). 155

For the laboratory-modified CDC assay, SARS-CoV-2 target RNA sequences were detected in 119 156 (60%) samples. Six (3%) samples gave an initial inconclusive result. Upon repeat testing, 4 yielded valid 157 results: 3 were detected and 1 was not detected. The remaining 2 (0.01%) samples repeated as 158 inconclusive (only one of the two targets amplified in the specimen) ( Table 1) The RealTime SARS-CoV-2 on the m2000 assay yielded 127 (63.5%) positive results and no 163 invalid results (Table 1) . The median C n value for positive samples was 17.27 (IQR, 13.27 -21.40), which 164 correlates to an RNA concentration of approximately 147 genome equivalents/µL of sample (Figure 1) . 165

The ID NOW™ COVID-19 assay yielded 94 (47%) positive results (Table 1) . Five (0.03%) samples first gave 166 invalid results; 3 resolved after repeat testing and the remaining 2 repeated as invalid. 167

Analysis of discordant results. There were 33 (17%) samples that yielded discordant results across the 168 three assays (Table 1) . Eight discordant samples were not detected or gave inconclusive results by the 169 laboratory-modified CDC assay but were detected by the RealTime SARS-CoV-2 assay. The median C n 170 value for these samples on the RealTime SARS-CoV-2 was 27.73 (IQR, 27.37 -28.40), or approximately 171 0.34 genome equivalents/µL (Figure 2 ). Thirty-three samples (including 2 invalid samples) were not 172 detected by the ID NOW™ COVID-19 but were detected by the RealTime SARS-CoV-2 assay; 25 of these 173 were also detected by the laboratory-modified CDC assay. The median C n values for these samples on 174

the RealTime SARS-CoV-2 were 21.42 (IQR, 20.80 -23.88), or approximately 13.3 genome 175 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. Table 3 . We estimate that we can batch test and report results for 58 192 samples by the laboratory-modified CDC assay, and 94 samples by the RealTime SARS-CoV-2 assay, in an 193 8-hour shift. The ID NOW™ COVID-19 assay, which was developed for point of care, requires the least 194 hands-on time and provides the fastest results. However, throughput is limited (1 sample/instrument/5-195 15 minutes). 196

Rapid, accurate detection of COVID-19 is essential to ensure speedy and appropriate patient 198 management, outbreak containment, and to better understand the global epidemiology of the virus. 199 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101 Laboratory testing to date has relied primarily on the amplification and detection of viral gene 200 sequences in upper respiratory tract specimens. As new test kits are made available through the EUA 201 pathway, laboratories are confronted with the dilemma of deciding which test or platform to adopt for 202 SARS-CoV-2 detection. Additionally, laboratory directors are faced with numerous questions from 203 clinicians regarding performance characteristics of the tests. Responding to these questions is difficult, 204 since EUA requires only limited test validation (14); assays approved under EUA have not been 205 evaluated in clinical trials, and robust performance data from real world assessments are lacking. 206

Results of the current study help to fill this knowledge gap. We found significant differences in 207 detection of SARS-CoV-2 viral sequences among the RealTime SARS-CoV-2 assay on the m2000, the 208 laboratory-modified CDC assay, and the ID NOW™ COVID-19 assay. The RealTime SARS-CoV-2 assay on 209 the m2000 detected the most cases, followed by the laboratory-modified CDC assay, and then the ID 210 NOW™ COVID-19 assay. Discrepant results were observed almost exclusively in samples with higher C t 211 values, i.e., lower viral titer. These findings suggest differences in lower limit of detection of the assays. 212

For the laboratory-modified CDC RT-PCR assay, this might be explained in part by smaller input sample 213 volumes for extraction (200µL) and amplification (5µL), compared to 500µL extraction and 40µL 214 amplification volumes in the RealTime SARS-CoV-2 assay on the m2000, i.e., there is more available 215 target for amplification and detection in the RealTime SARS-CoV-2 assay on the m2000. Our results 216 comparing the RealTime SARS-CoV-2 assay on the m2000 and the ID NOW™ COVID-19 assay are 217 concordant with those of Harrington et al, who reported increased detection of SARS-CoV-2 RNA gene 218 sequences by the RealTime SARS-CoV-2 assay compared to the ID NOW™ COVID-19 assay (15). 219

In order to eliminate confounding that could have been introduced by testing different sample 220 types on different systems, we evaluated aliquots of the same nasopharyngeal swab/viral transport 221 medium in all three assays. At the time of this study, nasopharyngeal swab specimens in viral transport 222 medium were deemed acceptable sample types for the 3 assays that we assessed. Following the 223 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020 . . https://doi.org/10.1101 completion of our study, the manufacturer amended the package insert of the ID NOW™ COVID-19 224 assay to state that testing viral transport medium could lead to false-negative results. However, in our 225 subsequent analysis of dry nasal swab samples tested by the ID NOW™ COVID-19 assay and paired 226 nasopharyngeal swabs tested by the RealTime SARS-CoV-2 assay on the m2000, we continued to see 227 more true positive results by the he RealTime SARS-CoV-2 assay on the m2000 assay, suggesting that 228 false-negative results were not due entirely to dilution. 229

We observed differences in turnaround time, workflow, and throughput among the three tests. 230

The RealTime SARS-CoV-2 assay on the m2000 had the longest runtime of the three assays: 231 approximately 8 hours for one full run of 94 samples. Runtime of the laboratory-developed CDC assay 232 was similar, but the throughput was less (58 samples in an 8-hour shift). The ID NOW™ COVID-19 assay 233 was the easiest to perform and yielded the fastest results; positive results are generated in as few as 5 234 minutes, which is faster than any other test system available currently in the United States. Ease of use 235 and speed are advantages in settings without laboratory expertise, or when rapid results are needed. 236

The assay platform is small and can be utilized at near-patient settings, thereby increasing the overall 237 testing capacity for SARS-CoV-2 within healthcare facilities. Availability of different platforms provides 238 beneficial flexibility to meet testing needs of different populations and different healthcare settings. 239

Our study has limitations. Because there is not a reference standard for SARS-CoV-2 infection, 240

we were unable to calculate sensitivity or specificity of the assays. Instead, we calculated percent 241 agreement, which is appropriate when a non-standard reference method is utilized to compare assay 242 performance (16). We resolved discrepant results through review of patient medical records, which may 243 have introduced bias, since concordant test results were not confirmed in the same way (17). We 244 enriched for samples that were positive by the RealTime SARS-CoV-2 assay on the m2000, which may 245 have biased in favor of this test. However, our inability to detect any samples that yielded a positive 246 result by either of the other two assays under evaluation and a negative result by the RealTime SARS-247 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101 CoV-2 assay in this study, or in our initial verification of the assay, suggests that the effect of bias is 248 small. Not all testing was performed on the same day due to workflow and personnel limitations, 249 although all testing was completed within 72 hours of sample collection. Storage of specimens at 250 ambient room (22°C) or refrigerated (4°C) temperature has been shown to have little impact on 251 detection of other RNA viruses by . 252

In conclusion, we found that The RealTime SARS-CoV-2 assay on the m2000 detected more 253 cases of COVID-19 infection than the modified CDC assay or the ID NOW™ COVID-19 test. The ID NOW™ 254 COVID-19 test provided fastest results, and the small footprint of the instrument and ease of use are 255 advantages in settings without technical expertise. Both tests are welcome additions to the COVID-19 256 testing armamentarium, and increase nationwide testing capacity for 258 Acknowledgements 259

The following reagent was obtained through BEI Resources, NIAID, NIH: Genomic RNA from SARS-260 Related Coronavirus 2, Isolate USA-WA1/2020, NR-52285. 261 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101 

China Novel Coronavirus I

Coronavirus from Patients with Pneumonia in China

A new 267 coronavirus associated with human respiratory disease in China

313 14. FDA. Policy for Diagnostics Tests for Coronavirus Disease-2019 during the Public Health

Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 317 from nasopharyngeal and nasal swabs from symptomatic patients

User Protocol for Evaluation of Qualitative Test Performance

CLSI document EP12-A2

Discrepant analysis: how can we test a test?

Evaluation of swabs, transport media, and 323 specimen transport conditions for optimal detection of viruses by PCR

Performance agreement for detection of SARS-CoV-2 RNA by laboratory-modified CDC 2019-nCOV RT-PCR assay, RealTime SARS-CoV-2 332 on the m2000

Assay comparison (A/B) Positive Percent Agreement (95% CI) Assay comparison (A/B) Negative Percent Agreement

LDT+/m2000+ 100 (96

Abbreviations: LDT, laboratory-modified CDC 2019-nCOV RT-PCR assay; m2000, RealTime SARS-CoV-2 on the m2000

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. Inconclusive defined as a sample that gave a C t value <40 for either N1 or N2 targets. 331replicates. Trend line equations: Laboratory-modified CDC assay (N1), y=-2.054ln(x) + 40.585, R2=1.0; 344 laboratory modified CDC assay (N2), y=-1.966ln(x) + 40.022, R2 = 0.99; RealTime SARS-CoV-2 on the 345 m2000 assay, y=-1.729ln(x) + 25.899, R 2 =0.99. *Only 2 of 3 replicates amplified and are included in 346 estimate. 347. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020 . . https://doi.org/10.1101 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.02.20088740 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.02.20088740 doi: medRxiv preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10. 1101