key: cord-0914726-laeocs3j
authors: Lima, Amorce; Healer, Vicki; Vendrone, Elaine; Silbert, Suzane
title: Validation and Comparison of a Modified CDC Assay with two Commercially Available Assays for the Detection of SARS-CoV-2 in Respiratory Specimen
date: 2020-06-30
journal: bioRxiv
DOI: 10.1101/2020.06.29.179192
sha: 81de5515c9571b335455eaa7fdae7a456c482408
doc_id: 914726
cord_uid: laeocs3j

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes coronavirus disease 2019 (COVID-19), has spread rapidly around the globe since it was first identified in December of 2019 in Wuhan, China. In a race to contain the infection, researchers and healthcare officials have developed several assays to help diagnose individuals with COVID-19. To help laboratories in deciding what assay to bring into testing lines, factors such as assay availability, cost, throughput, and TAT should be considered. Here we validated a modified version of the CDC assay and used it as a reference to evaluate the performance of the NeuMoDx™ SARS-CoV-2 and DiaSorin Simplexa™ Covid-19 Direct assays. In silico analysis and clinical sample testing showed that the primesr/probes designed by the CDC were specific to the SARS-CoV-2 as they accurately detected all reactive samples with an assay LoD of 200 copies/ml. The performance of the three assays were analyzed using 161 nasopharyngeal swabs specimen tested within 24 hours or 5 days from routine testing. A 100% agreement was observed between the commercial assays and the modified CDC SARS-CoV-2 assay. A deeper look at the Ct values showed no significant difference between NeuMoDx and the modified CDC SARS-CoV-2 assay, whereas DiaSorin had lower overall Ct values than the modified CDC SARS-CoV-2 assay. NeuMoDx and DiaSorin workflows were much easier to perform. NeuMoDx has the highest throughput and shortest TAT, whereas although the modified CDC SARS-CoV-2 assay has comparable throughput to DiaSorin, it has the longest hands-on time, and highest TAT.

There has been a race against time to develop tests for SARS-CoV-2 detection so 70 individuals with COVID-19 could be identified and isolated to slow the spread of the disease. In 71 January 2020, the CDC developed a TaqMan probe based molecular test, and at the end of 72 to help respond to the testing demand in our hospital. All three assays are based on real-time 85 and RP). A no-template water control (NTC) and 2019-nCoV_N_ Positive Control (nCoVPC) 123

were used as template for each of the primer/probe set; Hs_RPP30_Positive Control plasmid 124 was used as template for RP primer/probe where necessary. The real-time reverse 125 transcriptase PCR (rRT-PCR) cycling conditions were set up on the Rotor-Gene 3000 126 thermocycler (Corbett Research, Australia) as followed: 25°C for 2 min, 50°C for 15 min, 95°C 127 for 2 min, followed by 45 cycles of 95°C for 3 s and 55°C for 30 s with fluorescence (FAM) 128 detection during the 55°C incubation step. A sample was positive for SARS-CoV-2 if at least 129 one of the two targets (N1, N2) was detected regardless whether RP was amplified, negative if 130 none of the targets was detected and RP was detected, and invalid if RP and the two targets 131

were not detected. 132

Samples were processed for the NeuMoDx TM SARS-CoV-2 assay and the DiaSorin 133 Analytical sensitivity and specificity of the modified CDC SARS-CoV-2 assay. A 149 series of two-fold dilutions of SARS-CoV-2 strain USA_WA1/2020 RNA were spiked in pooled 150 sputum at concentrations of 800 copies/ml to 0.05 copy/ml in order to determine the limit of 151 detection (LoD) of the assay. All samples were processed and tested in triplicate as described 152

above. The LoD was confirmed by further testing in 20 replicates. The analytical specificity was 153 determined by testing 22 samples which include 15 patient samples, 4 ATCC strains, and 3 154 commercially available nucleic acid controls. 155

Clinical evaluation of the modified CDC SARS-CoV-2 assay. The performance of the 156 modified CDC SARS-CoV-2 assay was established by testing 30 contrived NP swabs and 157 sputum specimens and 30 non-reactive specimens. Of the 30 contrived specimens, 20 were 158 spiked with SARS-CoV-2 strain USA_WA1/2020 RNA at 1x-2x LoD concentrations and 10 were 159 spiked at concentrations spanning the assay's testing range (60,000 copies/ml to 234 160 copies/ml). 161

Covid-19 Direct assay and the modified CDC SARS-CoV-2 assay. A total of 161 NP swabs 163 were used to compare the clinical performance of two commercially available assays against 164 the modified CDC SARS-CoV-2 assay. Of those, 118 samples were tested to compare 165

NeuMoDx SARS-CoV-2 assay with the modified CDC SARS-CoV-2 assay, and 43 were tested 166 to compare Simplexa Covid-19 Direct assay with the modified CDC SARS-CoV-2 assay. 167 Statistical analysis. The results obtained from each assay were compared with those 168 obtained using the modified CDC SARS-CoV-2 assay. EP Evaluator was used to calculate 169 positive percent agreement (PPA), negative percent agreement (NPA), and Cohen's kappa (k) 170 with 95% confidence intervals. modified CDC SARS-CoV-2 assay were designed by the CDC (16). Since the CDC 175 primer/probe panel became available, many SARS-CoV-2 isolates have been sequenced. 176 Therefore, we conducted an analysis based on the currently available sequences in the National 177

Center for Biotechnology Information (NCBI) database as of March 11, 2020. Primer Blast 178 analysis showed that the primer/probe sets were specific to all available sequences for SARS- primer/probe set for human RP gene internal control produced a 57 bp amplicon (Fig. 1B) . 184

The PCR efficiency was determined for each of the primer/probe set by testing a series 185 of 10-fold dilutions of the 200,000 copies/µl concentration of the nCoVPC plasmid. The data 186

showed that the PCR was linearity over 6 orders of magnitude with great PCR efficiency 187 (N1=111% and N2=100) and R 2 of 0.99 for each of the primer/probe set (Fig. 2) . The standard 188 curve generated using the known concentrations was used to accurately determine the 189 concentration of the USA_WA1/2020 RNA. It was found that the USA_WA1/2020 RNA obtained 190 from UTMB was at a concentration of 1.5 x 10 8 copies/ml. analytical specificity of the assay was determined by testing 22 samples, which included 15 199 patient samples, 4 ATCC strains, and 3 commercially available nucleic acid controls. The result 200

showed that none of the targets was detected (Table 3) . were spiked with 234 copies/ml to 60,000 copies/ml. The results showed that the assay 208 detected SARS-CoV-2 in all 30 reactive samples, whereas no amplification was seen in the 30 209 non-reactive samples (Table 4) . CoV-2 assay and NeuMoDx SARS-CoV-2 assay, 104 samples were run within a day and 14 220

were run within 5 days of first testing. All 89 positive and 72 negative samples tested by the 221 modified CDC SARS-CoV-2 assay matched the results using the two commercial assays 222 evaluated yielding a 100% PPV and NPV for each assay (Table 5A) . Two samples that were 223 negative on the modified CDC SARS-CoV-2 assay resulted in indeterminate on the NeuMoDx for the Nsp2. NeuMoDx did not yield a Ct value for one of the two targets in 5 samples: 4 were 225 negative for Nsp2 gene target and 1 was negative for the N gene target. However, they still 226 considered positive as only one detected target was needed for a positive result. 227

Although 100% agreement was observed among the assays, further analysis showed NeuMoDx SARS-CoV-2 and the modified CDC SARS-CoV-2 assay was -0.14, and -2.13 233 between samples run within 5 days. The overall Ct value difference for all samples run between 234 the two assays was -0.346. On the other hand, the average Ct values difference between 235 samples run within 2 days between DiaSorin Simplexa Covid 19 Direct assay and the modified 236 CDC SARS-CoV-2 assay was -2.42, and -6.0 between samples run within 5 days. The overall 237

Ct value difference for all samples between the two assays was -3.47. Although the overall infection and death rates of COVID-19 has been declining in some 254 countries, it has increased in others. Therefore, the ongoing pandemic still poses great risks for 255 many around the world, and with the easing of certain restrictions, the need for health care 256 facilities to be equipped and accurately test for the virus to limit its spread is as crucial as it will 257 ever be. To that extent, laboratories have brought in SARS-CoV-2 assays and molecular 258 platforms to respond to the need of their communities. There have been a few publications on 259 head-to-head comparisons of those assays, including a couple very recently as we preparing 260 this article, in order to shed light on their performance characteristics and help laboratories 261 make informed decisions on acquiring those assays (9, 17-19) . 262

In this study, we validated a modified CDC SARS-CoV-2 assay and compared its 263 performance to two commercial automated sample-to-answer assays for the detection of SARS-264

CoV-2 RNA. We confirmed that the primer/probe sets were specific to all SARS-CoV-2 clades 265 based on the available genome sequences including those that were not available at the time 266 when those primer sets were originally designed. Not only were those primers/probe sets 267 specific to SARS-CoV-2 based on in silico study, there were no false positive results in cross-268 reactivity experiments using a panel of bacterial and closely-related virus targets. We found that 269 the primer/probe sets have high PCR efficiency and a LoD of 200 copies/ml. The clinical 270 sensitivity and specificity of the assay was also evaluated in samples with different 271

concentrations of viral RNA. The results showed that the assay is specific to SARS-CoV-2 as it 272 was only detected in the reactive samples. 273

The other objective of this study was to compare the clinical performance of two 274 commercially available assays in out laboratory, NeuMoDx SARS-CoV-2 assay and Simplexa agreement of 100% between the results obtained on the commercial assays and those on the 277 modified CDC SARS-CoV-2 assay. It is worth noting that while we found that the modified CDC 278

inserts For NeuMoDx SARS-CoV-2 assay and DiaSorin Simplexa Covid 19 Direct is 250 280 copies/ml and 500 copies/ml for assay, respectively. 281

A closer look at the Ct value differences between the modified CDC SARS-CoV-2 assay 282 and the commercial assays suggests that there is not a significant difference between the 283 modified CDC SARS-CoV-2 assay and NeuMoDx SARS-CoV-2; however, there seems to be a 284 greater difference in Ct values between DiaSorin Simplexa CoV-2 Direct assay and the modified 285 CDC SARS-CoV-2 assay, with DiaSorin having lower Ct values. The difference is even greater 286 in samples that were run 5 days after the routine testing on the modified CDC SARS-CoV-2 287 assay. This is in line with previously published data that showed Ct values on DiaSorin was 288 much lower than those on an the modified CDC SARS-CoV-2 assay by an average Ct 289 difference of -2.1 (18). The overall data also suggest that depending on the viral burden in the 290 samples NP samples can be refrigerated for at least 5 days and still maintain the RNA integrity 291 for viral detection by the assays in this study. 292 A limitation of this study was that the same samples were not tested by the three 293 different assays, so a head-to-head comparison of the three assays was not performed. This 294 was due to the limit of available kits for routine testing in patient care. However, we did a head-295

to-head comparison of the assay's workflow. The NeuMoDx 96 molecular system is a sample-296 to-answer and random-access platform that automatically performs nucleic acid extraction, 297 amplification, and signal detection and analysis requiring only very little human interaction for 298 loading and scanning the samples (20). It has the shortest TAT and highest throughput of the 299 three assays. DiaSorin Simplexa CoV-2 Direct assay involves a simple operation procedure that 300

does not include an extraction step; it has, however, much lower throughput than NeuMoDx. 301

The modified CDC SARS-CoV-2 assay is singleplex; each target must be run in different tubes, as opposed to the two commercial assays, which are multiplex. It requires separate extraction 303 steps on the EasyMag which can extract up to 24 samples at a time, longer hands-on time, and 304 has much lower throughput compared to NeuMoDx. It has comparable throughput to DiaSorin if 305 nucleic acid extraction and PCR reaction tubes for the next sample batch are prepared before 306 the PCR cycles of the previous batch ends. Therefore, although the three assays have the 307 same accuracy, the overall workflow favors the commercial platforms. 308

In conclusion, diagnostic laboratories around the world have faced with unprecedented 309 challenges due to the SARS-CoV-2 pandemic. Thousands of SARS-CoV-2 tests are being 310 executed in any given laboratory each day. This testing requirement has not only forced 311 laboratory to bring in new technologists to help with testing, but it has also led to the shortage of 312 testing reagents. Consequently, laboratories had to acquire different assays and platforms to 313 meet testing demand. As much as LDT assays, such as the modified CDC SARS-CoV-2 assay, 314

were instrumental at the onset of the pandemic for COVID-19 testing, their overall testing 315 capacity are limited. Therefore, it is necessary for laboratories to acquire multiple high 316 throughput automated instruments that can test high number of samples quickly with almost the 317 same number of qualified laboratory professionals. Real time PCR determining amplification efficiency of the primer/probe sets. Ten-fold serial dilution of 200,000 copies/µl of nCoVPC plasmid was tested. PCR linearity over 6 orders of magnitude with a limit of detection of 2 copies/µl; N1 slope of -3.05 with a correlation coefficient R2 = 0.99; N2 slope =-3.33 and R2 = 0.99. 

Genotyping coronavirus SARS-CoV-2: methods and implications

Epidemiology and cause of severe 337 acute respiratory syndrome (SARS) in Guangdong, People's Republic of China

Identification of a novel 343 coronavirus in patients with severe acute respiratory syndrome

Isolation 19 direct assay

Clinical Evaluation of the cobas SARS-CoV-2 Test and a 382

Diagnostic Platform Switch during 48 Hours in the Midst of the COVID-19 Pandemic

A familial cluster of pneumonia associated with the 2019 novel 387 coronavirus indicating person-to-person transmission: a study of a family cluster

detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals 406 diagnosed with COVID-19

Comparison of Commercially Available and Laboratory Developed Assays for in vitro 409 Detection of SARS-CoV-2 in Clinical Laboratories

Clinical Evaluation of Three

Sample-To-Answer Platforms for the Detection of SARS-CoV-2

Molecular System

Control