key: cord-0996506-x1sfdj6d
authors: TANIDA, KONSTANTIN; KOSTE, LARS; KOENIG, CHRISTIAN; WENZEL, WERNER; FRITSCH, ANDREAS; FRICKMANN, HAGEN
title: Evaluation of the automated cartridge-based ARIES SARS-CoV-2 Assay (RUO) against automated Cepheid Xpert Xpress SARS-CoV-2 PCR as gold standard
date: 2020-08-17
journal: Eur J Microbiol Immunol (Bp)
DOI: 10.1556/1886.2020.00017
sha: 2bbc174e0e07f6dbeae1ddafd166a0d1f7c65212
doc_id: 996506
cord_uid: x1sfdj6d

INTRODUCTION: To evaluate the automated cartridge-based PCR approach ARIES SARS-CoV-2 Assay targeting the ORF-sequence and the N-gene of SARS-CoV-2. METHODS: In line with the suggestions by Rabenau and colleagues, the automated ARIES SARS-CoV-2 Assay was challenged with strongly positive samples, weakly positive samples and negative samples. Further, intra-assay and inter-assay precision as well as the limit-of-detection (lod) were defined with quantified target RNA and DNA. The Cepheid Xpert Xpress SARS-Cov-2 Assay was used as gold standard. RESULTS: Concordance between the ARIES assay and the Cepheid assay was 100% for strongly positive samples and for negative samples, respectively. For weakly positive samples as confirmed applying the Cepheid assay, a relevant minority of 4 out of 15 samples (26.7%) went undetected by the ARIES assay. Intra- and inter-assay precision were satisfactory, while the lod was in the 10(3) DNA copies/reaction-range, in the 10(3) virus copies/reaction-range, or in the 10(3)-10(4) free RNA copies/reaction-range in our hands. CONCLUSIONS: The automated ARIES assay shows comparable test characteristics as the Cepheid assay focusing on strongly positive and negative samples but a slightly reduced sensitivity with weakly positive samples. Decisions on diagnostic use should include considerations on the lod.

Laboratories, Abbott Park, Illinois, USA) approach is presently the most rapid well evaluated molecular point-of-caretesting assay for the diagnosis of COVID-19 but it is outperformed regarding sensitivity by PCR-based assays [7] [8] [9] [10] . One well-evaluated, highly sensitive automated PCR-based testing approach is the cartridge-based Xpert Xpress SARS-CoV-2 Assay (Cepheid, Sunnyvale, California, USA) with a limit of detection of less than 100 copies [11] [12] [13] [14] [15] [16] .

Recently, another automated cartridge-based system ARIES SARS-CoV-2 Assay (Luminex Corporation, Austin, Texas, United States) was cleared by the US Food and Drug Administration (FDA) for in-vitro-diagnostics under emergency use authorization (EUA-IVD), while the system was distributed under a research-use-only (RUO) label in Europa. A validation of the ARIES SARS-CoV-2 Assay against the well described above-mentioned Cepheid assay in line with the recommendations according to Rabenau and colleagues [17, 18] was performed to assess its suitability for in-vitro diagnostic use.

The study was conducted as a test validation using the batch number AB0629 of the automated cartridge-based ARIES SARS-CoV-2 Assay (Luminex, referred to as "ARIES assay" in the following) targeting the ORF-sequence and the Ngene with the well-evaluated Xpert Xpress SARS-CoV-2 Assay (Cepheid, referred to as "Cepheid assay" in the following) targeting the E-gene and the N2-region of the Ngene as the gold standard according to the validation protocol as suggested for laboratory-developed real-time PCR assays by Rabenau and colleagues [17, 18] . This implies comparative testing of strongly positive samples (defined as Cepheid cycle threshold (Ct) value ≤34 for all target genes), weakly positive samples (defined as Cepheid Ct value >34 for at least one target gene), negative samples according to the gold standard, inter-assay and intra-assay precision testing as well as the definition of the limit of detection (lod). Here, the lod was determined using patient samples with defined SARS-CoV-2 copy numbers, positive control plasmids and synthesized positive control RNA. Both assays were applied as described by the manufacturer with 200 mL sample used for the ARIES system and 300 mL sample used for the Cepheid approach.

Positive control DNA and RNA was designed based on sequence fragments of the N-gene (N1, N2, N3, China CDC), the RdRp-gene, the E-gene and the ORF-region derived from the genome of the Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1 (GenBank accession number NC_045512.2) as identified applying the previously described primers [2] [3] [4] [5] , prolonged by ≥10 bases at the 5 0 -end and the 3 0 -end, and linked by EcoR1 restriction sites (sequence details in Table 1 ). A pEX-A128 vector backbone was used to construct the positive control DNA plasmid (eurofins Genomics, Ebersberg, Germany). Positive control RNA of 1,005 bases length was transcribed with a T7 RNA polymerase from a linear template generated by PCR (AmpTec GmbH, Hamburg, Germany). Serial dilution was performed with eSwab collection & preservation of Aerobic, Anaerobic & Fastidious Bacteria (COPAN, Brescia, Italia) transport and storage medium spiked with respiratory sample material from patients; for the positive control RNA, it was also attempted with PCR grade water to prevent preterm RNA degradation due to RNAses included in the human patient samples.

Patient samples were partly derived from the diagnostic routine of the Bundeswehr Hospital Hamburg. Further, patient samples with quantified viral copy numbers were provided by the University Hospital Regensburg, the institution from where external laboratory control assessment schemes for molecular diagnostic tests ("Ringversuche") are provided in Germany. Finally, well characterized material from the external laboratory control assessment scheme "INSTAND EQA scheme (340) virus genome detection of Ct 5 cycle threshold. n.a. 5 not applicable.

European Journal of Microbiology and Immunology 10 (2020) 3, 156-164 SARS-CoV-2 -April 2020" by INSTAND e.V. (D€ usseldorf, Germany) was applied.

As clarified previously (WF-011/19) by the Ethics Committee of the Medical Association of Hamburg, Germany, in line with German National Laws, use of anonymized residual sample materials for test comparison purposes in diagnostic laboratories is neither a "research project involving human beings" in line with § 9 chapter 2 of the "Hamburgisches Kammergesetz f€ ur Heilberufe" (Hamburg's Association of Health Care Professions Act) nor a research project requiring ethical advice according to §15 chapter 1 of the "Berufsordnung f€ ur Hamburger € Arzte und € Arztinnen" (Professional Regulations for Physicians in Hamburg). Therefore, ethical clearance was not required for this assessment.

Assessment with strongly positive, weakly positive and negative samples

With the Cepheid SARS-CoV-2 assay applied as gold standard, sensitivity of the automated ARIES assay with the strongly positive samples was 100% (15/15), specificity was 100% (30/30), resulting in excellent agreement according to Landis & Koch [19] . For the weakly positive samples, however, agreement of the ARIES assay with the Cepheid assay was reduced to 26.7% (4/15) . Overall agreement about all tested samples between the two assays was 81.7% and Cohen's Kappa was 0.63 indicating substantial agreement according to Landis & Koch [19] . Thereby, negative reactions of the ARIES-PCR targeting the ORF-sequence were more frequently observed, even in case of the strongly positive samples, compared with the ARIES N-gene-PCR. Details are provided in Tables 2-4 .

For the assessment of inter-and intra-assay variation, excellent concordance was observed for the strongly positive samples, while slightly more variation was seen for the weakly positive samples in the high ct-value range (Tables 5  and 6 ).

Limit of detection assessment with well characterized samples, DNA plasmids and positive control RNA With plasmid DNA, the lod of the ARIES system with a positive result indicated by at least one positive PCR target as defined by the manufacturer was in the 10 3 copies/reaction-range. Within the 10 2 copies/reaction-range, target DNA in 50% of the repeated runs would have gone undetected. Focusing on the ORF-sequence target, 25% of the runs were positive in the 10 3 copies/reaction-range, 75% in the 10 4 copies/reaction-range, and 100% in the 10 5 copies/ reaction-range. Focusing on the N-gene target, 50% of the runs were positive in the 10 2 copies/reaction range and 100% in the 10 3 copies/reaction range (Table 7) .

With free single-stranded RNA, the limit of detection of the ARIES system was in the 10 4 copies/reaction-range. Within the 10 3 copies/reaction-range, only 50% of the runs would have led to positive results. Focusing on the ORFsequence target, 25% of the runs were positive in the 10 3 copies/reaction-range, and 100% in the 10 4 copies/reactionrange. The reason for the single failed amplification of the European Journal of Microbiology and Immunology 10 (2020) 3, 156-164 ORF-sequence in the 10 5 copies/reaction range remains unclear. Focusing on the N-gene target, 50% of the runs were positive in the 10 3 copies/reaction range and 100% in the 10 4 copies/reaction range (Table 8 ). Due to poor stability of free RNA molecules, their integration in the evaluation has been a particular challenge to the diagnostic assay assessed. With quantified virus particles in patient samples, the lod of the ARIES assay was in the 10 3 copies/reaction-range. While all samples containing 10 3 and more virus copies were detected positive by the ARIES assay, virus RNA in the 10 2 copies/reaction-range and less went undetected.

In summary, the ARIES SARS-CoV-2 Assay did not show any non-specific reactions in our hands in the course of the performed validation procedure. Focusing on diagnostic sensitivity, however, target sequences in a considerable proportion of weakly positive samples which were still detected positive with the automated assay by Cepheid went undetected. With a limit of detection >10 3 virus copies, stages of the infection with lower viral copy number may be missed. Based on previous estimates [9, 20] , respective high copy numbers are only detectable for few days of the infection. In contrast, highly sensitive PCR facilitates the detection of prolonged shedding of viral RNA sequences irrespective of their infectious potential [21] [22] [23] [24] .

Anyway, clinical or hygiene-related decisions based on positive results of the ARIES SARS-CoV-2 Assay should consider its limits of detection which are considerably higher than the ones of the automated Cepheid approach [11] [12] [13] [14] [15] [16] . This is of particular importance in situations when maximum sensitivity is desired, both for clinical decision making or surveillance purposes.

The assessment has a number of limitations. First of all, there was only a limited number of positive samples available for evaluation purposes, so samples had to be taken from different sources. Secondly, the definition of the Cepheid assay as the gold standard is a methodical limitation by itself, because this decision does not consider potential failure of the gold standard itself. For example, it cannot be excluded that there might have been individual SARS-CoV-2-containing samples being positively detected by the ARIES system but missed by the Cepheid approach in broader evaluations including higher sample numbers. However, in spite of such theoretical assumptions, the data provide a preliminary estimation of the diagnostic performance of the validated diagnostic test in line with validation standards as suggested previously [17, 18] .

The automated cartridge-based ARIES SARS-CoV-2 Assay specifically detected SARS-CoV-2 sequences in our validation approach. Compared with the automated Cepheid approach, however, the assay was considerably less sensitive for cases with low target RNA loads, a fact that should be considered in case of decisions for its diagnostic use.

Funding sources: No financial support was received for this study.

A pneumonia outbreak associated with a new coronavirus of probable bat origin

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel

Cited

MiQ 01: Nukleins€ aure-Amplifikationstechniken. Qualit€ atsstandard in der mikrobiologischen Diagnostik. 3. Auflag. Urban & Fischer Verlag

Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from symptomatic patients

Comparison of Abbott ID Now, Diasorin Simplexa, and CDC FDA Emergency Use Authorization methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19

Clinical evaluation of three sample-to-answer platforms for the detection of SARS-CoV-2

Performance of Abbott ID NOW COVID-19 rapid nucleic acid amplification test using nasopharyngeal swabs transported in viral media and dry nasal swabs, in a New York City academic institution

Detection of SARS-CoV-2 by use of the Cepheid Xpert Xpress SARS-CoV-2 and Roche cobas SARS-CoV-2 Assays

Test agreement between Roche Cobas 6800 and Cepheid Gen-eXpert Xpress SARS-CoV-2 Assays at high cycle threshold ranges

Multicenter evaluation of the Cepheid Xpert Xpress SARS-CoV-2 test

Multi-center evaluation of cepheid xpert ® xpress SARS-CoV-2 point-of-care test during the SARS-CoV-2 pandemic

Detection of low levels of SARS-CoV-2 RNA from nasopharyngeal swabs using three commercial molecular assays

Comparison of Cepheid Xpert Xpress and Abbott ID now to Roche cobas for the rapid detection of SARS-CoV-2

Verification and validation of diagnostic laboratory tests in clinical virology

Validierung von Untersuchungsverfahren im Bereich der Virusdiagnostik

The measurement of observer agreement for categorical data

Type I and Type III interferons -induction, signaling, evasion, and application to Combat COVID-19

Prolonged SARS-CoV-2 RNA shedding: Not a rare phenomenon

Factors associated with prolonged viral RNA shedding in patients with coronavirus disease 2019 (COVID-19)

Differences of SARS-CoV-2 shedding duration in sputum and nasopharyngeal swab specimens among adult inpatients with COVID-19

Factors associated with duration of viral shedding in adults with COVID-19 outside of Wuhan, China: A retrospective cohort study

/), which permits unrestricted use, distribution, and reproduction in any medium for non-commercial purposes, provided the original author and source are credited

Authors' contribution: KT, LK, and HF assessed the data and prepared the manuscript. All authors have jointly planned the study, optimized and reviewed the manuscript.Conflict of interest: Nothing to declare.

Simone Priesnitz is gratefully acknowledged for excellent technical assistance. The colleagues from the University of Regensburg are acknowledged for providing the precharacterized patient samples for the evaluation.