key: cord-319864-t6ql9hz2
authors: Lima, Amorce; Healer, Vicki; Vendrone, Elaine; Silbert, Suzane
title: Validation of a Modified CDC Assay and Performance Comparison with the NeuMoDx™ and DiaSorin® automated assays for Rapid Detection of SARS-CoV-2 in Respiratory Specimens
date: 2020-11-11
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104688
sha: 
doc_id: 319864
cord_uid: t6ql9hz2

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes coronavirus disease 2019 (COVID-19), has spread rapidly around the globe since it was first identified in December of 2019 in Wuhan, China. In a race to contain the infection, researchers and healthcare officials have developed several assays to help diagnose individuals with COVID-19. To help laboratories decide what assay to bring into testing lines, factors such as assay availability, cost, throughput, and TAT should be considered. Here we validated a modified version of the CDC assay and used it as a reference to evaluate the performance of the NeuMoDxTM SARS-CoV-2 and DiaSorin SimplexaTM Covid-19 Direct assays. In silico analysis and clinical sample testing showed that the primers/probes designed by the CDC were specific to the SARS-CoV-2 as they accurately detected all reactive samples with an assay LoD of 200 copies/ml. The performance of the three assays were analyzed using 159 nasopharyngeal swabs specimen tested within 1 to 5 days after routine testing. A 100% agreement was observed between the commercial assays and the modified CDC SARS-CoV-2 assay. A deeper look at the Ct values showed no significant difference between NeuMoDx and the modified CDC SARS-CoV-2 assay, whereas DiaSorin had lower overall Ct values than the modified CDC SARS-CoV-2 assay. NeuMoDx and DiaSorin workflows were much easier to perform. NeuMoDx has the highest throughput and shortest TAT, whereas although the modified CDC SARS-CoV-2 assay has comparable throughput to DiaSorin, it has the longest hands-on time and highest TAT.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which started in Wuhan City in China at the end of December 2019, has spread to over 200 countries and was declared a global pandemic by the World Health Organization (WHO) on March 11, 2020 (1). As of July 12, there were more than 12,740,000 confirmed cases of COVID-19 and 565,000 deaths (https://coronavirus.jhu.edu/map.html). The United States (US) led the world with over 3,280,000 cases of COVID-19 and over 135,000 deaths as of July 12, 2020. The rate of new cases continues to rise in the US, Brazil and India with no apparent end in sight.

To meet diagnostic needs as the pandemic grows, the U.S. Food and Drug Administration (FDA) granted several commercial SARS-CoV-2 tests Emergency Use Authorization (EUA). Since that, there has been a race against time to develop tests for SARS-CoV-2 detection so individuals with COVID-19 could be identified and isolated to slow the spread of the disease. In January 2020, the CDC developed a TaqMan probe-based molecular test (2) . In the following months, several commercial assays became available and have been used in the laboratory under the FDA's EUA (3) (4) (5) (6) . With the rapid development of those tests, came the challenge of assay sensitivity and specificity. Our laboratory at Tampa General Hospital validated a modified version of the CDC assay following the FDA EUA guidelines and brought in commercial assays to help respond to the testing demand. The modified CDC SARS-CoV-2 assay involves off-instrument cell lysis and nucleic acid extraction steps, and an amplification step on a different instrument. The assay includes a panel of primer/probe sets targeting the viral N gene and the human RNase P gene.

The NeuMoDx™ SARS-CoV-2 and DiaSorin Simplexa® Covid-19 Direct assays are fully automated sample-to-answer multiplex assays targeting two different regions of the viral genome.

NeuMoDx assay targets the Nsp2 and the N genes while DiaSorin test targets the ORF1ab and J o u r n a l P r e -p r o o f S gene. The NeuMoDx™ SARS-CoV-2 assay is performed on a random-access, high throughput instrument allowing for with high priority and high volumes testing which is crucial in an outbreak.

In this study, we sought to describe a modified CDC SARS-CoV-2 assay validation and compare its performance and workflow to that of the NeuMoDx SARS-CoV-2 and DiaSorin Simplexa Covid-19 Direct assays using respiratory specimens.

The primer/probe sets used in this validation were selected from regions of the SARS-CoV-2 virus nucleocapsid (N) gene and were described in the CDC EUA protocol for COVID-19 diagnostic testing (7) . The CDC original panel was designed for both universal detection of SARSlike coronaviruses (N3 primer/probe set) and specific detection of the SARS-CoV-2 (N1 and N2 primer/probe sets). Moreover, the original CDC protocol included two sets of controls: HSC (human specimen extraction control) and RP (human RNase P gene to assess specimen quality).

Our modified protocol included the N1 and N2 primer/probe sets and the RP control. Primer and probe sets used in this validation were purchased from Integrated DNA Technologies (Coralville, IA).

Total nucleic acid extraction was carried out on the bioMérieux NucliSens® easyMAG® automated system (bioMerieux, France) from a 200 µl of the sample and eluted in 50 μl of EasyMag elution buffer. A separate reaction mix containing 5 µl of the eluate, 5 µl of TaqPath TM 1-Step RT-qPCR Master Mix (4x), 1.5 µl of combined primer/probe Mix (500nM and 125nM final concentration of primers and probes, respectively) and 13.5 µl of Nuclease-free Water was made for each of the assay target (N1, N2, and RP). A no-template water control (NTC) and nCoVPC plasmid were used as template for each of the primer/probe set; Hs_RPP30_Positive Control plasmid was used as template for RP primer/probe. The RT-PCR cycling conditions were set up on the Rotor-Gene 3000 thermocycler (Corbett Research, Australia) as followed: 25°C for 2 min, J o u r n a l P r e -p r o o f 50°C for 15 min, 95°C for 2 min, followed by 45 cycles of 95°C for 3 s and 55°C for 30 s with fluorescence (FAM) detection during the 55°C incubation step. A sample was positive for SARS-CoV-2 if at least one of the two targets (N1, N2) was detected regardless whether RP was amplified, negative if none of the targets was detected and RP was detected, and invalid if RP and the two targets were not detected.

For the analytical evaluation, we used SARS-CoV-2 RNA (strain USA_WA1/2020) kindly provided by the University of Texas Medical Branch (UTMB) in Galveston. A series of two-fold dilutions of the RNA were spiked in pooled sputum at concentrations of 800 copies/ml to 0.05 copies/ml to determine the limit of detection (LoD) of the assay. All samples were processed and tested in triplicate as described above. The LoD was confirmed by further testing in 20 replicates. The analytical specificity was determined by testing 22 samples previously positive for different respiratory species, including 15 patient samples, 4 ATCC strains, and 3 commercially available nucleic acid controls. The clinical performance was established by testing 30 contrived NP swabs and sputum specimens and 30 non-reactive specimens. Of the 30 contrived specimens, 20 were spiked with SARS-CoV-2 strain USA_WA1/2020 RNA at 1x-2x LoD concentrations and 10 were spiked at concentrations spanning the assay's testing range (60,000 copies/ml to 234 copies/ml).

A total of 159 NP swabs were used to compare clinical performance of the three SARS-CoV-2 assays. Samples were processed as described above for the modified CDC SARS-CoV-2 assay;

for the NeuMoDx SARS-CoV-2 assay and the DiaSorin Simplexa Covid-19 Direct assay samples 

Since the CDC primer/probe panel became available, many SARS-CoV-2 isolates have been sequenced. Therefore, we ascertained that the primer/probe sets were specific to all available sequences for SARS-CoV-2 in the NCBI's GenBank. Multiple sequence alignment of the nucleocapsid gene of SARS-CoV-2 and the two closely related coronaviruses, SARS-CoV and MERS-CoV, showed the positions of the primer/probe sets (Fig. 1A) . The N1 set amplified a 72 bp fragment of the nucleocapsid gene, N2 amplified a 67 bp of the same gene, and the primer/probe set for human RP gene internal control produced a 57 bp amplicon (Fig. 1B) . The PCR efficiency was determined by testing a series of 10-fold dilutions of the 200,000 copies/µl concentration of the nCoVPC plasmid. The data showed that the PCR was linear over 6 orders of magnitude with great PCR efficiency (N1=111% and N2=100) and R 2 of 0.99 for each of the primer/probe set (Fig. 2) .

The analytical sensitivity of the modified CDC SARS-CoV-2 assay was determined by testing pooled sputum samples spiked with a series of two-fold dilutions at concentrations of 800 copies/ml to 50 copies/ml of SARS-CoV-2 strain USA_WA1/2020. The LoD was determined to be 200 copies/ml (Table 1A) . It was confirmed by further testing 20 replicates that were inoculated with 200 copies/ml; all 20 replicates were tested positive (Table 1B) . The analytical specificity of the assay was determined by testing 22 previously positive samples, which included 15 patient samples, 4 ATCC strains, and 3 commercially available nucleic acid controls. The result showed that none of the targets was detected ( Table 2 ). The performance of the assay was evaluated in 60 sputum samples to mimic the extent of viral colonization in respiratory specimens. The assay detected SARS-CoV-2 in all 30 contrived samples; no amplification was seen in the 30 nonreactive samples (Table 3) .

One hundred and fifty-nine NP samples with a wide range of Ct values were used to compare the performance of the three assays. Of the 43 samples used for comparison between modified CDC SARS-CoV-2 assay and Simplexa Covid 19 Direct assay, 37 samples were run within 2 days and 6 were run within 5 days of first testing. Of the 116 samples used for comparison between the modified CDC SARS-CoV-2 assay and NeuMoDx SARS-CoV-2 assay, 102 samples were run within a day and 14 were run within 5 days of first testing. All the samples tested by the modified CDC SARS-CoV-2 assay matched the results using the two commercial assays evaluated yielding a 100% PPV and NPV for each assay (Table 4A ). NeuMoDx did not yield a Ct value for one of the two targets in 5 samples: 4 were negative for Nsp2 gene target and 1 was negative for the N gene target. However, they were still considered positive as only one detected target was needed for a positive result.

Although 100% agreement was observed among the assays, further analysis showed that there were some differences in the Ct values. The average Ct value difference in samples ran within 24 hours between NeuMoDx SARS-CoV-2 and the modified CDC SARS-CoV-2 assay was -0.14, and -2.13 between samples ran within 5 days. The overall Ct value difference for all samples ran between the two assays was -0.346. On the other hand, the average Ct values difference between samples ran within 2 days between DiaSorin Simplexa Covid 19 Direct assay and the modified CDC SARS-CoV-2 assay was -2.42, and -6.0 between samples ran within 5 days. The overall Ct value difference for all samples between the two assays was -3.47.

We assessed the workflow and compared the throughput, time of sample-to-result, and cost per sample for each assay. Based on 8 samples/run, it was estimated to be 2 h and 50 min for sample processing and PCR runs were staggered. The most cost-effective test was the NeuMoDx. Reagents for this test is a little more expensive than the modified CDC SARS-CoV-2 assay, but less expensive than the DiaSorin. The labor involved on the modified CDC SARS-CoV-2 assay, however, is considerably greater than on the NeuMoDx and DiaSorin assays, which ends up increasing the cost of the test.

The ongoing pandemic still poses great risks for many around the world, and with the easing of certain restrictions, the need for health care facilities to be equipped and accurately test for the J o u r n a l P r e -p r o o f virus to limit its spread is as crucial as it will ever be. To that extent, laboratories have brought in SARS-CoV-2 assays and molecular platforms to respond to the need of their communities. There have been a few publications on head-to-head comparisons of those assays, including a couple very recently as we were preparing this article, in order to shed light on their performance characteristics and help laboratories make informed decisions on acquiring those assays (4, (8) (9) (10) .

In this study, we validated a modified CDC SARS-CoV-2 assay and found that the assay is very sensitive and specific to SARS-CoV-2. The clinical performance comparison between NeuMoDx SARS-CoV-2 assay, Simplexa Covid-19 Direct assay, and the modified CDC SARS-CoV-2 assay showed an overall agreement of 100%. The Ct value difference between the modified CDC assay and the NeuMoDx assay suggests that there is not a significant difference between the two assays; however, there seems to be a greater difference in Ct values between Simplexa SARS-CoV-2 Direct assay and the modified CDC SARS-CoV-2 assay, with the former having lower Ct values. The difference is greater in samples that were run 5 days after the routine testing on the modified CDC SARS-CoV-2 assay. This is in line with previously published data that showed Ct values on Simplexa SARS-CoV-2 Direct assay was much lower than those on an a modified CDC SARS-CoV-2 assay by an average Ct difference of -2.1 (9) . The overall data also suggest that depending on the viral burden NP samples can be refrigerated for at least 5 days and still maintain the RNA integrity for viral detection by the assays in this study.

A limitation of this study was that the same samples were not tested by the three different assays, so a head-to-head comparison of the three assays was not performed due to the limited kits available for routine patient care. However, a head-to-head comparison of the assay's workflow was performed. NeuMoDx has the shortest time-to-result, the highest throughput, and is the most cost-effective of the three assays evaluated. Therefore, it allows the clinical laboratory to significantly increase throughput and reduce TAT in a time when more testing is needed, and faster result is crucial to speed up isolation measures. The TAT comparison between the three J o u r n a l P r e -p r o o f assays was based on 8 samples/run since the LIASON® MDX instrument can only accommodate 8 samples/run. The overall throughput of the modified SARS-CoV-2 CDC assay can be improved to 24 samples if the 72-PCR tube adaptor option for the Rotor-Gene 3000 is used. Laboratories that are equipped with 96-well nucleic acid extraction platforms and PCR instruments, as well as the personnel can further scale up testing and significantly increase the throughput of the CDC assay. Nevertheless, the increased demand for testing highlighted a well-known short resource of skilled laboratory personnel and having to train new staff usually takes some time. Acquiring two NeuMoDx96 instruments helped our laboratory to increase the testing capacity from 200 tests/day to about 700 tests/day while maintaining the same number of laboratory personnel.

Therefore, although the three assays have the same accuracy, due to its random-access and high throughput capacity, NeuMoDx SARS-CoV-2 assay outperforms the other two assays.

Diagnostic laboratories around the world have faced with unprecedented challenges due to the SARS-CoV-2 pandemic. The testing requirement has not only forced laboratories to bring in new technologists to help with testing, but it has also led to shortage of reagents. Consequently, multiple assays and platforms are used to meet testing demand. As much as LDT assays, were instrumental at the onset of the pandemic, their overall testing capacity are very limited. Therefore, it is necessary for laboratories to acquire multiple high throughput automated instruments that can test high volume of samples quickly with almost the same number of qualified laboratory professionals.

There is no conflict of interest to disclose. J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f 

J o u r n a l P r e -p r o o f

A Novel Coronavirus from Patients with Pneumonia in China

First Case of 2019 Novel Coronavirus in the United States

Evaluation of the QIAstat-Dx Respiratory SARS-CoV-2 Panel, J o u r n a l P r e -p r o o f the First Rapid Multiplex PCR Commercial Assay for SARS-CoV-2 Detection

Clinical evaluation of a SARS-CoV-2 RT-PCR assay on a fully automated system for rapid on-demand testing in the hospital setting

Rapid and sensitive detection of SARS-CoV-2 RNA using the Simplexa™ COVID-19 direct assay

Clinical Evaluation of the cobas SARS-CoV-2 Test and a Diagnostic Platform Switch during 48 Hours in the Midst of the COVID-19 Pandemic

US CDC Real-Time Reverse Transcription PCR Panel for Detection of Severe Acute Respiratory Syndrome Coronavirus 2

Comparison of Abbott ID Now, Diasorin Simplexa, and CDC FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19

Comparison of Commercially Available and Laboratory Developed Assays for in vitro Detection of SARS-CoV-2 in Clinical Laboratories

Clinical Evaluation of Three Sample-To-Answer Platforms for the Detection of SARS-CoV-2

We are grateful to University of Texas Medical Branch for providing SARS CoV-2 RNA Template for the validation of the modified CDC SARS-CoV-2 assay.

J o u r n a l P r e -p r o o f