key: cord-263279-afdmegq0
authors: Uhteg, Katharine; Jarrett, Junko; Richards, Mahmia; Howard, Craig; Morehead, Elizabeth; Geahr, Melissa; Gluck, Linda; Hanlon, Ann; Ellis, Brandon; Kaur, Harsimar; Simner, Patricia; Carroll, Karen C.; Mostafa, Heba H.
title: Comparing the analytical performance of three SARS-CoV-2 molecular diagnostic assays
date: 2020-04-26
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104384
sha: 
doc_id: 263279
cord_uid: afdmegq0

In December 2019, a novel coronavirus (SARS-CoV-2) was first isolated from Wuhan city, China and within three months, the global community was challenged with a devastating pandemic. The rapid spread of the virus challenged diagnostic laboratories to rapidly develop molecular diagnostic methods. As SARS CoV-2 assays became available for testing on existing molecular platforms, laboratories devoted unprecedented energy and resources into evaluating the analytical performance of the new tests and in some cases developed their own diagnostic assays under FDA-EUA guidance. This study compares the validation of three different molecular assays at the Johns Hopkins Molecular Virology laboratory: the RealStar® SARS-CoV-2 RT-PCR, ePlex® SARS-CoV-2, and the CDC COVID-19 RT-PCR tests. Overall, our studies indicate a comparable analytical performance of the three assays for the detection of SARS-CoV-2.

The highly pathogenic betacoronavirus, SARS-CoV-2, first isolated in Wuhan, China in December 2019 has caused a quickly evolving pandemic [1] [2] [3] . The outbreak was followed by characterization of the SARS-CoV-2 whole viral genome within weeks of its discovery, which allowed the development of various molecular diagnostic assays. The implementation of in-house molecular diagnostics nationwide was slower than the emergence of the pandemic. It was not until February 4th, 2020 that Centers for Disease Control and Prevention (CDC)'s COVID-19 real-time PCR assay received an emergency use authorization (EUA) (https://www.fda.gov/medical-devices/emergencysituations-medical-devices/emergency-use-authorizations# covid19ivd). Clinical microbiology laboratories were not permitted to develop COVID-19 testing in house and apply for their own EUA approvals until February 29th, 2020. Soon after that date, only a few commercial assays were available with insufficient reagents to meet national demands. Of the first assays that were available for validations were the CDC COVID-19 RT-PCR panel assay (IDT, Coralville, IA) as well as the RealStar® SARS-CoV-2 RT-PCR (Altona Diagnostics, Hamburg, Germany), and both were initially validated for clinical use at the Johns Hopkins Hospital Medical Microbiology laboratory.

Assays that offer the required analytical sensitivity and specificity are essential for early diagnosis and consequently early intervention especially for infection prevention and control purposes. Molecular diagnosis using reverese-transcription RT-PCR is the current most conclusive approach for COVID-19 diagnosis. An understanding of the analytical performance of different molecular asssays is essential for proper interpretation of the results and for defining the clinical sensitivity of RNA detection.

We validated three different assays for the molecular detection of SARS-CoV-2: the RealStar® SARS-CoV-2 RT-PCR, ePlex® SARS-CoV-2, and the CDC COVID-19 RT-PCR tests. The analytical sensitivity of the three assays was compared using the same quantified genomic materials, which offered a side by side comparison of their lower limits of detection. The overall accuracy of the three assays was compared using patient' clinical specimens and the reproducibility was studied using contrived specimens.

Johns Hopkins Hospital. The study was approved by the Johns Hopkins University School of Medicine IRB (IRB00246024).

Genomic viral RNA, kindly provided by the University of Texas Medical Branch (UTMB) was used for the analytical sensitivity and reproducibility studies. The genomic RNA was derived from the Strain USA_WA1/2020 originating from Washington, USA from a traveler from Wuhan, China. This isolate demonstrates 100 % consensus match to GenBank MN985325.1. Per UTMB product insert, RNA was purified using TRIzol and the RNA purity was 27 % viral and 73 % host as determined by next-generation sequencing. The viral RNA concentration was determined to be equivalent to 6 × 10 4 pfu/μL(and the genome copies were noted to exceed the pfu counts in the range of 1000: 1)

Clinical specimens used for studies were remnant specimens available at the completion of standard of care testing from patients suspected of COVID-19. Specimens included nasopharyngeal swabs (NP) and bronchoalveolar lavage (BAL). Archived frozen specimens (i.e., NP and BAL) were used as matrix to create contrived samples (matrix negative for SARS-CoV-2).

The RealStar® SARS-CoV-2 RT-PCR Kit 1.0 is from Altona Diagnostics (Hamburg, Germany). This kit detects both B-βCoV specific RNA (E gene) and SARS-CoV-2 specific RNA (S gene). (https://altonadiagnostics.com/en/products/reagents-140/reagents/realstar-realtime-pcr-reagents/realstar-sars-cov-2-rt-pcr-kit-ruo.html)

The GenMark (Carlsbad, CA) ePlex® SARS-CoV-2 Test, for Research Use Only (RUO) assay was performed on the ePlex instrument. The company received FDA-EUA on March 19th. A single use ePlex cartridge automates nucleic acid extraction, amplification, and detection. GenMark has developed an innovative eSensor technology that combines DNA hybridization and electrochemical detection [4] . A volume of 200μLper specimen is added to the sample delivery device. The ePlex® SARS-CoV-2 targets the nucleocapsid (N) protein (https://www. fda.gov/media/136282/download).

The CDC COVID-19 RT-PCR panel assay was developed by the CDC and was granted an EUA on February 4th. The oligonucleotide primers and probes (two primer/probe sets) target regions of the nucleocapsid (N) gene. The panel includes a primer/probe set to detect the human RNase P gene (RP) for extraction and specimen quality evaluations (https://www.fda.gov/media/134922/download).

Automated nucleic acid extraction for both the The RealStar® SARS-CoV-2 and the CDC assays was performed using either the NucliSENS easyMag or eMAG instruments (bioMérieux, Marcy-l'Étoile, France) using software version 2.1.0.1. The input volume for all sources was 500μLand the elution volume was 50 u L. All the specimens were initially processed in either BSL-3 or BSL-2 using BSL-3 biosafety measures and 2 mL of the easyMag/ eMAG lysis buffer was added to each 500μLof the aliquoted specimens in a biosafety cabinet. Specimens were incubated for 10 min and nucleic acid extraction protocol was followed for performing automated off board lysis extraction following bioMérieux protocol.

The RealStar® SARS-CoV-2 RT-PCR Kit 1.0 total reaction volume was 30 μL(10μLextracted sample and 20 μLMasterMix). The kit contains two premade master mixes, A and B, which contain PCR buffer, magnesium salt, primers and probes, reverse transcriptase, and DNA polymerase. The detectors used are FAM (B-βCoV), Cy5 (SARS-CoV-2), and JOE (Internal Control). Taqman RT-PCR was performed using the Prism 7500 Sequence Detection System (Applied Biosystems, Foster City, CA) at the following cycling conditions: 1 cycle at 55.0°C for 20 min, 1 cycle at 95.0°C for 2 min and 45 cycles at 95.0°C for 15 s, 55.0°C for 45 s then 72.0°C for 15 s.

The CDC COVID-19 RT-PCR panel assay was performed using the IDT primers and probes lot # 0000500383 and TaqPathTM 1-Step RT-qPCR Master Mix (4x) (ThermoFisher catalog No A15300). Controls included the IDT 2019-nCoV_N_Positive Control, and The Hs_RPP30 Positive Control plasmids. The assay was performed in three separate reactions per specimen for each target (N1, N2, and the internal control RP). The reaction volume is a total of 20 μL(8.5μLnucleasefree water, 1.5μLprimer and probe mix, 5μLTaqPathTM 1-Step RT-qPCR Master Mix, and 5μLextracted specimen (or controls). The detector used for all the targets is FAM. Taqman RT-PCR was performed using the Prism 7500 Sequence Detection System (Applied Biosystems) at the following cycling conditions: 1 cycle at 25.0°C for 2 min, 1 cycle at 50.0°C for 15 min, 1 cycle at 95.0°C for 2 min and 45 cycles at 95.0°C for 3 s, 55.0°C for 30 s.

To evaluate and compare the analytical sensitivity of the three methods, SARS-CoV-2 negative NP or BAL specimens were extracted using easyMag or EMAG and eluates were spiked with serially diluted SARS-CoV-2 whole viral genomic materials. Dilutions were tested with the RealStar® SARS-CoV-2 RT-PCR, ePlex® SARS-CoV-2, and the CDC COVID-19 RT-PCR tests. The lower limit of detection (LOD) was defined as the lowest concentration at which 95 % of the tested replicates were detected.

For the RealStar® SARS-CoV-2 RT-PCR, extracts from remnant clinical NP or BAL swabs were spiked with serially diluted genomic RNA in 7 concentrations ranging from 120,000 pfu/ mL (1200,000 copies (cp)/ reaction (rxn)) to 0.12 pfu/ mL (1.2 cp/ rxn). Three replicates were tested at all concentrations except for 120 cp/rxn (n = 30) and 12 cp/rxn (n = 24). The LOD for the the RealStar® SARS-CoV-2 RT-PCR for NP specimens was at 1200 cp/ mL (12 cp/ rxn) ( Table 1) .

For the BAL, extractions from archived clinical BAL specimens were spiked with serially diluted genomic RNA into 3 concentrations (cp/ rxn): 1200 (n = 20), 120 (n = 20), 12 (n = 3). The LOD for the the RealStar® SARS-CoV-2 RT-PCR for BAL specimens was at 12,000 copies/ mL (120 cp/ rxn) ( Table 1 ). It is notable that of the three BAL specimens run at 12 cp/ rxn, 2 were positive for the B-βCoV target but negative for the SARS-CoV-2 target.

For the CDC COVID-19 RT-PCR test, similarly, extracts from clinical NP or BAL were spiked with serially diluted genomic RNA. The LOD was identified to be 1200 cp / mL for both NP and BAL specimens (6 cp/ rxn) ( Table 1) .

For the ePlex® SARS-CoV-2, negative NP eluates were spiked with different concentrations of the viral genomic materials and tested following the GenMark instructions for use. Our data indicate that the LOD of the assay is 600 cp/ mL (120 cp/ rxn). No LOD studies were performed for BAL specimens on this assay (Table 1) .

To compare the analytical performance of the three assays, positive and negative SARS-CoV-2 clinical specimens (using the RealStar® SARS-CoV-2 as the reference method as this assay was the first to be offered in house for clinical diagnosis) were tested by the CDC COVID-19 RT-PCR and/ or the ePlex® SARS-CoV-2 assays. Comparing the performance of the CDC COVID-19 RT-PCR to the RealStar® SARS-CoV-2 included testing 20 positive and 48 negative clinical NP specimens. Comparison showed 100 % agreement as well as similar trends in Ct values for the positive specimens (Table 2 ). For the negative specimens 100 % agreement was noted as well (data not shown).

The agreement between the RealStar® SARS-CoV-2 and the ePlex® SARS-CoV-2 was tested by comparing 34 negative and 13 positive clinical specimens (initially diagnosed by RealStar® SARS-CoV-2). Our data showed 100 % agreement for both negatives (data not shown) and positives (Table 3) .

To assess the reproducibility of the three assays, replicate testing within the same run and in different runs was performed. For both the RealStar® SARS-CoV-2 and the CDC COVID-19 RT-PCR, NP and BAL specimens were spiked with dilutions of the SARS-CoV-2 genomic materials and replicates were tested over three separate runs after separate extractions by 3 different operators, using 3 different thermocyclers. For the RealStar® SARS-CoV-2 NP specimens, concentrations of spiked specimens were 12,000 cp/mL (n = 30) and 1200 cp/mL (n = 20). Nineteen negative NP samples were also included (Table 4) . BAL was run at concentrations of 120,000 cp/ mL (n = 21), 24,000 cp/ mL (n = 11), 12,000 cp/ mL (n = 20). Thirty negative BAL sepecimens were included (Table 4) . Our data showed a qualitative reproducibility of 100 %.

For the the CDC COVID-19 RT-PCR assay, concentrations of spiked specimens were 1200 cp/mL (n = 26). Forty negative NP samples were also included (Table 5 ). BAL was run at conecntrations of 1200 cp/mL (n = 27). Thirty negative BAL sepecimens were included ( Table 5 ).

The reproducibility of ePlex® SARS-CoV-2 was assessed by running replicates of negative and contrived NP specimens, spiked with different dilutions within the same day and over three separate runs, by different operators. Overall, the runs showed 100 % reproducibility (Table 6) .

Ct values' range of diagnosed patients after implementing the RealStar® SARS-CoV-2 clinical assay

The RealStar® SARS-CoV-2 assay was the first assay validated and implemented at Johns Hopkins Hospital. To assess the range of viral loads detected by this assay since its implementation, we examined the Ct values of a subset of the initial positive specimens over time (Fig. 1) . Initially, diagnosed cases after the implementation of the assay had Ct 

SARS-CoV-2 is the seventh and most novel human coronavirus. It emerged as a highly pathogenic species in December 2019 in Wuhan City, China and has since quickly spread to all continents except Antartica [5] . As of the time of this writing, the number of confirmed cases is more than one milion with thousands of deaths (https://www. who.int/emergencies/diseases/novel-coronavirus-2019/situationreports). The clinical manifestations of the disease (COVID-19) range widely from mild upper respiratory tract infection to a more severe or critical disease [6, 7] . The rapid spread of the outbreak might be attributed to the successful transmissibility [8, 9] in addition to some evidence that asymptomatic individuals may transmit the infection [10] .

SARS-CoV-2 was first identified in the BAL fluid of a patient from China by metagenomics whole genome sequencing [11] . The quick characterization of the full genome of the virus has enabled the development of multiple molecular diagnostic methods, however, nationwide implementation of molecular diagnostics in the US was delayed. In the US, the CDC defined criteria for defining patients under investigation and priorities for testing, however, extensive testing in other countries was instrumental in controlling the spread of the disease and identifying a more accurate case-fatality rates [12, 13] . Consequently, the implementation of laboratory developed testing in the US was essential to escalate the required testing capacity and to ensure rapid diagnosis that facilitated implementation of containment measures, utilization of high demand personal protective equipment and patient management. As so far, as molecular detection remains the gold standard for diagnosis, it is critical to understand the analytical performance of the available molecular assays.

In this study, we compared the analytical performance of three different molecular assays for the detection of SARS-CoV-2; the RealStar® SARS-CoV-2 RT-PCR, ePlex® SARS-CoV-2, and the CDC COVID-19 RT-PCR tests. The CDC assay and the GenMark ePlex target the N gene, and the Altona RealStar assay detects both the E and S genes. The three assays showed comparable analytical sensitivity that was between 600−1200 viral genome copies/ mL. There was 100 % agreement between the three assays for both negative and positive clinical specimens. Generally, the RealStar® SARS-CoV-2 RT-PCR assay has a higher throughput than the CDC assay as the CDC assay requires three separate wells per specimen. The GenMark ePlex, although it has a relatively short turn-around-time, offers open access and easier workflow, its full implementation is limited by inadequate supplies and inventory. It is in general difficult to understand the differences in the analytical sensitivity of different molecular assays due to inherent variabilities in specimen processing and reference materials used for validations in different laboratories. This study offered a side by side comparison using the same extraction methodology (the RealStar® SARS-CoV-2 and the CDC assays) and viral genomic materials which offers a better assessment of the analytical performance. Many questions remains to be answered about the clinical sensitivity of PCR assays for the diagnosis of COVID-19 and the minimum acceptable analytical sensitivity. Different studies have shown different detection patterns of the viral RNA based on the specimen type and the specimen collection time in relation to the onset of symptoms. A recent study by a Chinese group showed that viral RNA is readily detectable in the nasopharyngeal, sputum, and stool specimens, with sputum specimens showing a more extended detection time frame that extends beyond 4 weeks (a mean of 22 days versus 16.2 days) and for both respiratory sources the peak viral load was during the first week after the onset of symptoms [14] . A different group consistently showed that the sputum as well as broncho-alveolar lavage specimens showed the highest positivity rates [15] . Additional studies from other groups showed lower sensitivity of PCR for early detection in highly suspected patients in comparison to CT scan [16] or serology [17] . Our data shows that many of our diagnosed specimens had viral loads that were below the assay's lower limit of detection (Ct range of 32.74 for the SARS-CoV-2 channel, NP sources (Table 4) , and Fig. 1) , which warrants an analytical sensitivity below 1200 copies/ mL to reduce the number of false negative results. In general, multiple factors other than the analytical sensitivity of the molecular assay could contribute to the clinical performance of various RNA detection methods. This includes the specimen collection time and the specimen quality when collected. Overall, it is appropriate to conclude that the RNA detection remains the assay of choice for COVID-19 confirmed diagnosis and until a better understanding of the dynamics of viral shedding and its correlation to the disease progression is achieved, assays with acceptable analytical performance are essential for enhancing the clinical diagnosis.

In summary, our results show that the analytical performance of the three SARS-CoV-2 assays; the RealStar® SARS-CoV-2 RT-PCR, ePlex® SARS-CoV-2, and the CDC COVID-19 RT-PCR tests is comparable. The clinical sensitivity of PCR in COVID-19 diagnosis is still an area of investigation.

CRediT authorship contribution statement 

A novel coronavirus from patients with pneumonia in China

A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster

Coronaviridae Study Group of the International Committee on Taxonomy of V, The species severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2

The GenMark ePlex((R)): another weapon in the syndromic arsenal for infection diagnosis

Diagnosis and treatment of an acute severe pneumonia patient with COVID-19: case report

Evidence of SARS-CoV-2 infection in returning travelers from Wuhan

Features, Evaluation and Treatment Coronavirus (COVID-19)

Early epidemiological assessment of the transmission potential and virulence of coronavirus disease 2019 (COVID-19) in Wuhan City: China

Presumed asymptomatic carrier transmission of COVID-19

A new coronavirus associated with human respiratory disease in China

Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV2)

Diagnostic testing for severe acute respiratory syndromerelated coronavirus-2: a narrative review

Viral kinetics and antibody responses in patients with COVID-19, medRxiv

Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease

We thank the entire Medical Microbiology division for their assistance with this study. We also thank Altona Diagnostics for their support with the initial implementation of the COVID-19 laboratory developed test. We also thank the University of Texas Medical Branch (UTMB) for providing quantified genomic SARS-CoV-2 RNA material.