key: cord-0909542-ibxo4n9q
authors: Dust, Kerry; Hedley, Adam; Nichol, Kim; Stein, Derek; Adam, Heather; Karlowsky, James A.; Bullard, Jared; Van Caeseele, Paul; Alexander, David C.
title: Comparison of Commercial Assays and Laboratory Developed Tests for Detection of SARS-CoV-2
date: 2020-09-10
journal: J Virol Methods
DOI: 10.1016/j.jviromet.2020.113970
sha: eb729fee1f38a2ae6a83b95aa5970d9e34ccc310
doc_id: 909542
cord_uid: ibxo4n9q

The global COVID-19 pandemic has led to the rapid development of tests for detection of SARS-CoV-2. Studies are required to assess the relative performance of different assays. Here, we compared the performance of two commercial assays, the cobas® SARS-CoV-2 (Roche Diagnostics) and Xpert® Xpress SARS-CoV-2 (Cepheid®) tests, and a laboratory developed RT-PCR test adapted for use on the Hologic® Panther Fusion® (Hologic®) instrument as well as Bio-Rad and QIAGEN real-time PCR detection systems. Performance characteristics for each test were determined by testing clinical specimens and reference material. All assays detect the pan-Sarbecovirus E (envelope structural protein) gene plus a SARS-CoV-2-specific target. The limit of detection for the E gene target varied from ∼2 copies/reaction to >30 copies/reaction. Due to assay-specific differences in sample processing and nucleic acid extraction, the overall analytical sensitivity ranged from 24 copies/mL specimen to 574 copies/mL specimen. Despite these differences, there was 100% agreement between the commercial and laboratory developed tests. No false-negative or false-positive SARS-CoV-2 results were observed and there was no cross-reactivity with common respiratory viruses, including endemic coronaviruses.

 Commercial assays and lab developed tests for detection of SARS-CoV-2 exhibited 100% positive and negative agreement  Nucleic acid extraction method influenced the analytical sensitivity (limit of detection) of the lab developed RT-PCR test

The global COVID-19 pandemic has led to the rapid development of tests for detection of SARS-CoV-2. Studies are required to assess the relative performance of different assays. Here, we compared the performance of two commercial assays, the cobas® SARS-CoV-2 (Roche Diagnostics) and Xpert® Xpress SARS-CoV-2 (Cepheid®) tests, and a laboratory developed RT-PCR test adapted for use on the Hologic® Panther Fusion® (Hologic®) instrument as well as Bio-Rad and QIAGEN real-time PCR detection systems. Performance characteristics for each test were determined by testing clinical specimens and reference material. All assays detect the pan-Sarbecovirus E (envelope structural protein) gene plus a SARS-CoV-2-specific target.

The limit of detection for the E gene target varied from ~2 copies/reaction to >30 copies/reaction. Due to assay-specific differences in sample processing and nucleic acid extraction, the overall analytical sensitivity ranged from 24 copies/mL specimen to 574 copies/mL specimen. Despite these differences, there was 100% agreement between the commercial and laboratory developed tests. No false-negative or falsepositive SARS-CoV-2 results were observed and there was no cross-reactivity with common respiratory viruses, including endemic coronaviruses. Shared Health Diagnostic Services clinical microbiology laboratory have also implemented a laboratory developed RT-PCR test (LDT) that has been modified for compatibility with three different platforms. The use of multiple assays has enabled our laboratories to increase testing capacity, despite the limited supply of commercial test kits and LDT reagents. At the same time, platform-specific differences in assay performance have the potential to compromise SARS-CoV-2 testing (2) . In this study, we compared the performance characteristics of five molecular assays for SARS-CoV-2 detection. The LDT is a two-step RT-PCR assay. The first (screening) step targets the E gene and, as an endogenous control, the human RNase P gene (5) . The second (confirmatory) step targets the SARS-CoV-2-specific N1 region of the N gene (6) . The 

Analytical test characteristics were evaluated using three types of samples:1) A reference panel of simulated specimens derived from cultured SARS-CoV-2 virus that was inactivated by gamma irradiation and then added to viral transport medium (VTM) containing simulated respiratory secretion medium (7); 2) a convenience set of clinical specimens submitted to CPL for routine viral diagnostic testing (i.e., nasopharyngeal swabs in VTM), and 3) AccuPlex™ SARS-CoV-2 Reference Material (SeraCare), which is recombinant viral RNA encapsulated in a replication-deficient mammalian virus.

Agreement between assays was assessed using clinical specimens and a set of 

Limit of detection (LOD) studies used serial dilutions (1000 to 10 genome copies/ml) of 

All methods demonstrated 100% agreement with LDT-1 results ( Table 1) . There were no false-negative or false-positive results, and no cross-reactivity with circulating respiratory viruses, including endemic coronaviruses. Ct values for the E gene target differed between assays. With the simulated and clinical specimens, values for LDT-FUS were consistently higher than the LDT-1, whereas LDT-2 values trended lower.

LOD was determined for the E gene (all assays), as well as N1 (LDT-FUS), N2 (Xpert® Xpress) and orf1a (cobas®). LOD was initially calculated as genome copies/reaction ( Table 1) . The E gene LOD varied from 2 copies/reaction for the LDT-2 to ~10 copies/reaction for the LDT-1, LDT-FUS and cobas®. For the LDT FUS, the secondary, N gene, target performed better, with an LOD of 5.1 copies/reaction, whereas the orf1a target used in the cobas® system, was less sensitive at only 31.4 copies/reaction. The 'black box' nature of the Cephied cartridge precluded accurate calculations for the Xpert® Xpress, but LOD was estimated to be ≤30 copies/reaction for the E gene and ≤7.5 copies/reaction for the N2 gene target.

Due to assay-specific differences in sample processing and nucleic acid extraction, the amount of clinical material used per reaction varied between tests. As such, E-gene LODs were also calculated as genome copies/mL specimen. By this measure, the cobas® (24 copies/mL) was the most sensitive assay, the LDT-2 (100-172 copies/mL) and Xpert® Xpress (~100 copies/mL) were intermediate, whereas the LDT-1 (455 copies/mL) and LDT-FUS (574 copies/mL) were the least sensitive. 'Emergency Use Authorization' process. In Manitoba, Canada, the availability of multiple assays has allowed testing capacity to increase despite shortages in LDT reagents and limited supplies of commercial test kits. At the same time, the rapid implementation of these different tests has raised concerns that results may be compromised by differences among tests. In this study, we compared two commercial assays and three variations of a LDT. Despite some test-specific differences in Ct values and the LOD for the common E gene target, 100% agreement was observed. There were no falsenegative or false-positive results, and no cross-reactivity with common respiratory viruses. Analytical sensitivity studies indicated that the LOD for the E gene ranged from 2 copies/reaction (LDT-2) to ≤30 copies/reaction (Xpert® Xpress). Values for LDT-1, cobas® and LDT-FUS, were 9.1, 9.7, and 10.4 copies/reaction, respectively. However, the overall analytical sensitivity was influenced by assay-specific differences in sample processing and nucleic acid extraction. When input specimens volumes were considered, the LOD for the E gene ranged from 24 copies/mL for the cobas® system to 574 copies/mL specimen for the LDT-FUS. With the Xpert® Xpress and LDT-FUS, sensitivity was improved by the use of the secondary, N gene target.

There are several limitations to this study. Due to the clinical material available, it was not possible to test all clinical specimens with all assays. However, a common panel of simulated SARS-CoV-2 specimens was tested with all platforms. The assay-specific trends (e.g., difference in Ct values relative to the LDT-1 reference standard) were equivalent for the simulated specimens and clinical specimens. The apparent difference in performance with the LDT-2R assay was due a change in the extraction procedure.

The LDT-2R values for the clinical specimens were associated with manual extraction, whereas the lower LDT-2R values for the simulated specimens, and all LDT-2D results, were associated with automated extraction on a QIAcube®. We assessed potential cross-reactivity of the Cepheid® assay against human rhinoviruses, but not other viruses. However, a multicenter evaluation of that assay showed no cross-reactivity with other respiratory pathogens, including human coronaviruses (8) . Recent reports describe performance differences between the cobas® and Xpert® systems for samples that contain low levels of target (9, 10) . The majority (75%) of our clinical specimens were strong positives (LDT-1 Ct ≤ 30) and only two samples had Ct values >35. As such, the excellent agreement we observed may not be generalizable to testing of weakly positive clinical material.

In summary, despite measurable differences in analytical sensitivity, the cobas® SARS-CoV-2 (Roche Diagnostics), Xpert® Xpress SARS-CoV-2 (Cepheid®) and three variations of a LDT performed equivalently and showed 100% agreement when testing simulated and clinical specimens. This suggests that the concurrent use of these platforms does not compromise SARS-CoV-2 detection. In contrast, the availability of 

A Novel Coronavirus from Patients with Pneumonia in China

Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche cobas for the Rapid Detection of SARS-CoV-2

Roche Molecular Systems Inc. cobas® SARS-CoV-2 (package insert). Document

Xpert® Xpress-SARS-CoV-2 EUA Package Insert (GeneXpert System). Document 302-3562

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR EuroSurveillence

US Centers for Disease Control and Prevention. CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel. CDC-006-00019

Simulated Respiratory Secretion for Use in the Development of Influenza Diagnostic Assays

Multicenter Evaluation of the Cepheid Xpert Xpress SARS-CoV-2 Test

Detection of low levels of SARS-CoV-2 RNA from nasopharyngeal swabs using three commercial molecular assays

The Detection of SARS-CoV-2 using the Cepheid Xpert Xpress SARS-CoV-2 and Roche cobas SARS-CoV-2 Assays

We thank our colleagues at the Cadham Provincial Laboratory, Shared Health, and Manitoba Health for their expertise, professionalism and extraordinary work during the pandemic.