key: cord-0771086-jjhi7ud3
authors: Mostafa, Heba H.; Hardick, Justin; Morehead, Elizabeth; Miller, Jo-Anne; Gaydos, Charlotte A.; Manabe, Yukari C.
title: Comparison of the Analytical Sensitivity of Seven Commonly Used Commercial SARS-CoV-2 Automated Molecular Assays
date: 2020-08-05
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104578
sha: d606a05f23e882361966998de3a40b6cb536da30
doc_id: 771086
cord_uid: jjhi7ud3

The SARS-CoV-2 pandemic has challenged molecular microbiology laboratories to quickly implement and validate diagnostic assays and to expand testing capacity in a short timeframe. Multiple molecular diagnostic methods received FDA emergency use authorization (EUA) and were promptly validated for use nationwide. Several studies reported the analytical and/ or clinical evaluation of these molecular assays, however differences in the viral materials used for these evaluations complicated direct comparison of their analytical performance. In this study, we compared the analytical sensitivity (lower limit of detection, LOD) of seven commonly used qualitative SARS-CoV-2 molecular assays: the Abbott Molecular RealTime SARS-CoV-2 assay, the NeuMoDx™ SARS-CoV-2 assay, the Roche Cobas®SARS-CoV-2 assay, the BD SARS-CoV-2 reagents for BD MAX™ system, the Hologic Aptima® SARS-CoV-2 assay, the Xpert Xpress SARS-CoV-2 test, and the GenMark ePlex SARS-CoV-2 test. The comparison was performed utilizing a single positive clinical specimen that was serially diluted in viral transport media and quantified by the EUA approved SARS-CoV-2 droplet digital PCR (ddPCR) assay. Replicate samples were prepared and evaluated for reproducibility across different molecular assays with multiple replicates per assay. Our data demonstrated that the seven assays could detect 100% of replicates at a nucleocapsid gene concentration of (N1 = 1,267 and N2 = 1,392) copies/mL. At a one log less concentration, the Abbott, the Roche, and the Xpert Xpress assays detected 100% of the tested replicates.

The first case of coronavirus disease-19 On February 29 th , the FDA updated their guidance to allow clinical microbiology laboratories and commercial parties to develop in vitro SARS-CoV-2 assays and seek independent EUA (4) .

Beginning in March, the number of SARS-CoV-2 molecular diagnostic assays with EUA increased rapidly, many of them being developed for existing, automated and integrated (nucleic acid extraction and RT-PCR) diagnostic systems (4) .

Molecular detection of SARS-CoV-2 viral RNA has been used as the gold standard for diagnosis of COVID-19 (5) . Hence, an understanding of the analytical and clinical performance of various SARS-CoV-2 molecular methods is essential. In particular, the analytical sensitivity limitations and the relative sensitivity of different assays is important due to the large number of individuals that may carry transmissible virus asymptomatically in low copy number (6) . In addition, the recognition of initially false negative molecular testing in some patients who ultimately tested positive further highlights the importance of understanding the limit of J o u r n a l P r e -p r o o f detection (7) . The initial development of SARS-CoV-2 molecular assays was challenging due to the paucity of clinical specimens, viral controls, and/or material reagents for the validations. Different companies and laboratories used different materials for assessing the analytical and clinical sensitivities of their assays which were quantified differently. For example, the Abbott Molecular RealTime SARS-CoV-2 and Xpert Xpress SARS-CoV-2 assays both utilized recombinant virus for analytical sensitivity determination, while the NeuMoDx™ SARS-CoV-2

Assay utilized genomic RNA, and the GenMark ePlex SARS-CoV-2 test utilized in vitro transcripts for analytical sensitivity determination. These differences, coupled with differences in sample volume, extraction volume and amplification technology has made direct comparison across platforms challenging. Utilizing SARS-CoV-2 droplet digital PCR (ddPCR)(8) quantified single positive clinical specimen, we compared the analytical sensitivity of seven SARS-CoV-2 EUA-granted assays, the Abbott Molecular RealTime, the NeuMoDx™ , the Roche Cobas®, the BD SARS-CoV-2 reagents for BD MAX™ system, the Hologic Aptima®, the Xpert Xpress, and the GenMark ePlex SARS-CoV-2 tests. Comparisons were performed using the same highly quantified specimen and were facilitated by the in-house availability of these assays for SARS-CoV-2 diagnosis at Johns Hopkins Hospital Clinical Laboratory and the Johns Hopkins Center for Innovative Diagnostics for Infectious Diseases. Marcy-l'Étoile, France) as described before (4, 10) The average concentration of both the N1 target gene and N2 target gene copies at each dilution was determined to be 10,956 and 10,759 copies/mL (1:10 5 dilution) and 1,267 and 1,392 copies/mL (1:10 6 dilution), respectively. An additional dilution was prepared and the concentration was estimated to be 126.7 and 139.2 copies/ mL for both the N1 and N2 gene targets. Replicate samples were prepared for each of these concentrations and ranged from a maximum of ten replicates for the Abbott Molecular Although 10 replicates were tested for the majority of platforms, the number of assays performed varied based on availability of reagents and replicates.

Results for each assay, the Abbott Molecular RealTime SARS-CoV-2, the NeuMoDx™ SARS-CoV-2, the Roche Cobas®SARS-CoV-2, the BD SARS-CoV-2 reagents for BD MAX™ system, the Hologic Aptima® SARS-CoV-2, the Xpert Xpress SARS-CoV-2, and the GenMark ePlex SARS-CoV-2 tests are summarized in Table 1 . Replicates at the most diluted set (N1=126.7 copies/ mL and N2=139.2 copies/ mL) were detected 100% of the time with the Abbott RealTime SARS-CoV-2 assay and the Roche Cobas®SARS-CoV-2 assay. The Xpert Xpress SARS-CoV-2 test did not detect the N2 target in one replicate at this concentration, however because it did detect the E target at that concentration, the result was reported positive. All the tested assays were able to detect all the tested replicates at a one log higher concentration of 1,267 and 1,392 copies/mL for the N1 and N2 gene targets.

In this study, we compared the analytical sensitivities of seven common, commercially available assays for the molecular detection of SARS-CoV-2. Our comparison was performed using the same quantified clinical specimen allowing parallel evaluation. Our results show higher analytical sensitivities of the Roche (Cobas), the Abbott, and the GeneXpert (Xpert Xpress) assays when compared to the NeuMoDx (NeuMoDX), the BD MAX, the GenMark (ePlex), and the Hologic (Aptima) assays. The analytical sensitivities of all the assays assessed were in a range less than 1,000 copies/ mL indicating excellent analytical performance.

In general, the analytical performance of molecular assays is usually determined by the selection of the assay, the assay design and the selected target. Most of the current molecular diagnostics for SARS-CoV-2 are real-time PCR (RT-PCR) assays, however technologies that include CRISPR (clustered regularly interspaced short palindromic repeats)-based (18) and

Loop Mediated Amplification (LAMP) assays have also been developed. (19, 20) Most of the assays compared in our study are RT-PCR methods with the exception of the GenMark (ePlex) that involves an initial amplification followed by hybridization and electrochemical detection and J o u r n a l P r e -p r o o f the Hologic (Aptima) that uses a transcription-mediated amplification (TMA) technology. Our study identified only slight differences in the analytical sensitivity of the RT-PCR methods and comparable sensitivities using the GenMark (ePlex) and Hologic (Aptima) assays, indicating comparable performance. The assays we evaluated also target different genes in the SARS-CoV-2 genome, mainly the nucleocapsid (N), envelope (E), non-structural protein (Nsp)2, and open reading frame ORF1/2 ( Table 1 ). The N1 and N2 targets within the N gene were recommended by the CDC (21, 22) and were selected for their assay, while the WHO recommends an initial screening with the E gene followed by confirmation with the RNA dependent RNA polymerase (RdRp) (23) . Our data, as well as other published data illustrate that the assay performance for SARS-CoV-2 detection is not dictated by the selected gene.

However, a recent report concluded higher sensitivity of primers that target the N2 or the E genes (24) . The approach of targeting two genes appears to enhance the assays' sensitivities (Table 1 ) and could also reduce the risk of sensitivity reduction associated with genomic polymorphisms (25) .

Understanding the analytical performance of SARS-CoV-2 assays is essential for better understanding their clinical performance. Standardized materials for determining the analytical sensitivity or lower limit of detection (LOD) for EUA applications and validations were not available and different companies and laboratories used different approaches that included using in vitro synthesized transcripts (IVT) (e.g. GenMark), cultured virus (e.g. Roche), or SARS-CoV-2 genomic RNA (e.g. NeuMoDx) ( Table 2) . As a step forward for optimizing the validations and collecting comparable assays' performance data, the FDA currently offers reference panels for EUA applications which should also assist in different assays' calibrations and on-going monitoring of the performance(26). Although several papers have been published comparing the characteristics and sensitivities between a few SARS-CoV-2 molecular tests J o u r n a l P r e -p r o o f 8 ( Table 2) and (10, (27) (28) (29) (30) (31) (32) (33) (34) (35) (36) (37) (38) , our study provides a direct comparison between 7 of the most commonly used commercially available assays using the same quantified clinical specimen.

There are limitations to our study, including differences in the sample input volume for each assay, differences in the amount of sample processed and analyzed by the assay and differences in the extraction efficiency between assays, which cannot accurately be accounted for, and thus complicate the comparison. Additionally, we did not evaluate the same number of replicates for each assay due to the availability of supplies and replicates as we used dilutions of one specimen).

We demonstrated similar analytical performance of seven FDA EUA commercial SARS-CoV-2 molecular assays; the most sensitive platforms were the Roche and Abbott. With the critical current and future need for the use of molecular SARS-CoV-2 diagnostics not only for diagnosis, but also for asymptomatic large scale screening, it is essential to identify the most sensitive assays to assure early detection and diagnosis of COVID-19, and inform decisions related to contact isolation, measures that are essential to mitigate the current pandemic.

J o u r n a l P r e -p r o o f N/A N/A a-copies/mL determined by ddPCR for the N1 target gene; b-percentage of replicates detected; c-standard deviation; N/Anot applicable, NCnot calculated, * the input volume is based on the volume loaded per specimen and not the volume used per reaction. 

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster

Discontinuation of Transmission-Based Precautions and Disposition of Patients with COVID-19 in Healthcare Settings

Presymptomatic SARS-CoV-2 Infections and Transmission in a Skilled Nursing Facility

Clinical Performance of SARS-CoV-2 Molecular Testing

Comparing the analytical performance of three SARS-CoV-2 molecular diagnostic assays

SARS-CoV-2 Reagents for BD MAX™ System

FDA. ePlex®SARS-CoV-2 Test

Rapid Detection of 2019 Novel Coronavirus SARS-CoV-2 Using a CRISPR-based DETECTR Lateral Flow Assay. medRxiv

Rapid and visual detection of 2019 novel coronavirus (SARS-CoV-2) by a reverse transcription loop-mediated isothermal amplification assay

RT-LAMP for rapid diagnosis of coronavirus SARS-CoV-2

A Novel Coronavirus from Patients with Pneumonia in China

First Case of 2019 Novel Coronavirus in the United States

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Comparative Performance of SARS-CoV-2 Detection Assays Using Seven Different Primer-Probe Sets and One Assay Kit

2020. 26. FDA. Coronavirus (COVID-19) Update: FDA Provides New Tool to Aid Development and Evaluation of Diagnostic Tests That Detect SARS-CoV-2 Infection

Comparison of Four Molecular In Vitro Diagnostic Assays for the Detection of SARS-CoV-2 in Nasopharyngeal Specimens

Comparison of Abbott ID Now, Diasorin Simplexa, and CDC FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19

Rapid and sensitive detection of SARS-CoV-2 RNA using the Simplexa COVID-19 direct assay

Comparison of Commercially Available and Laboratory Developed Assays for in vitro Detection of SARS-CoV-2 in Clinical Laboratories

Clinical Evaluation of Three Sample-To-Answer Platforms for the Detection of SARS-CoV-2

Comparison of Two High-Throughput Reverse Transcription-Polymerase Chain Reaction Systems for the Detection of Severe Acute Respiratory Syndrome Coronavirus 2

Comparison of the Panther Fusion and a laboratory-developed test targeting the envelope gene for detection of SARS-CoV-2

Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from symptomatic patients

Performance of the rapid Nucleic Acid Amplification by Abbott ID NOW COVID-19 in nasopharyngeal swabs transported in viral media and dry nasal swabs

Comparison of two commercial molecular tests and a laboratory-developed modification of the CDC 2019-nCoV RT-PCR assay for the detection of SARS-CoV-2

Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche cobas for the Rapid Detection of SARS-CoV-2

Comparison of the Accula SARS-CoV-2 Test with a Laboratory-Developed Assay for Detection of SARS-CoV-2 RNA in Clinical Nasopharyngeal Specimens

The Detection of SARS-CoV-2 using the Cepheid Xpert Xpress SARS-CoV-2 and Roche cobas SARS-CoV-2 Assays

Comparison of SARS-CoV-2 detection from nasopharyngeal swab samples by the Roche cobas 6800 SARS-CoV-2 test and a laboratory-developed real-time RT-PCR test

Clinical Evaluation of the cobas SARS-CoV-2 Test and a Diagnostic Platform Switch during 48 Hours in the Midst of the COVID-19 Pandemic

Validation of the Hologic's Aptima Unisex and Multitest Specimen collection kits used for Endocervical and Male Urethral Swab Specimen (Aptima Swab) for sample collection of SARS-CoV-2

We thank the entire Medical Microbiology division for their assistance with this study. We also thank BioRad Laboratories for their collaboration. Funding: UM1AI068613 (HPTN), U54EB007958 (Center POCTR STD), CEIRS