key: cord-0735123-0n9szl4l
authors: Kogoj, Rok; Kmetič, Petra; Valenčak, Anja Oštrbenk; Komloš, Kristina Fujs; Seme, Katja; Sagadin, Martin; Korva, Miša; Poljak, Mario
title: Real-life head-to-head comparison of performance of two high-throughput automated assays for detection of SARS-CoV-2 RNA in nasopharyngeal swabs: the Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 assays
date: 2021-05-19
journal: J Mol Diagn
DOI: 10.1016/j.jmoldx.2021.05.003
sha: f288728b31d9005b9ceb2a627c3547ec79ed2b55
doc_id: 735123
cord_uid: 0n9szl4l

Abbott’s Alinity m is a recently launched fully integrated, automated molecular analyzer allowing continuous loading of samples and sample-to-result molecular detection of several microorganisms. Manufacturer-independent clinical evaluation of Alinity m SARS-CoV-2 (Alinity) against cobas 6800 SARS-CoV-2 (cobas) as a standard comparator was performed on 2,157 consecutive nasopharyngeal swabs. Valid initial results of Alinity and cobas were obtained from 2,129/2,157 (98.7%) and 2,157/2,157 (100%) samples, respectively. Overall percent agreement of 98.3% (2,092/2,129; 95%CI:97.6–98.7%), positive percent agreement of 100.0% (961/961; 95%CI:99.6–100.0%), negative percent agreement of 96.8% (1,131/1,168; 95%CI:95.7–97.7%), and a high kappa value of 0.965 (95%CI:0.954–0.976) were observed on 2,129 samples with valid results for both assays. There were 37 discordant results and based on discordant analyses, including previous and/or follow-up PCR results, 22 could be considered Alinity analytically true positive with high probability. Due to the lack of additional information and inability of repeated/further testing, the status of the remaining 15 discordant samples remains unresolved. Comparative real-life throughput of both analyzers assessed while testing 564 samples in parallel across two 8-hour shifts and comparative turnaround time assessment measured while processing in parallel the first 94 routine samples received in the laboratory each working day for 5 consecutive days showed similar real-life performance of both analyzers with certain differences, which has potential importance in some laboratory settings.

Coronavirus disease 2019 (COVID- 19) pandemic has affected over 135 million people, with over 2.9 million COVID-19-related deaths as of 10 April 2021. Highly reliable laboratory diagnostics for COVID-19 are essential for case identification, patient management, and contact tracing. Detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA in nasopharyngeal swabs is still considered the COVID-19 reference laboratory diagnostic standard. 1, 2 Although several commercial SARS-CoV-2 RNA assays have received U.S. Food and Drug Administration (FDA) emergency-use authorization (EUA), only a few have been designed for analyzers with high sample throughput to cope with the unprecedented demand for SARS-CoV-2 RNA testing and allow significant scaling up due to the fully automated sampleto-result solution. 3 The cobas 6800 System and cobas 8800 System (Roche Molecular Systems, Branchburg, NJ, USA) are fully integrated and automated analyzers allowing sample-to-result qualitative and quantitative molecular detection of several microorganisms. The U.S. FDA recently approved a range of cobas 6800/8800 System molecular assays (https://www.fda.gov/medical-devices/vitro-diagnostics/nucleic-acid-based-tests, last accessed April 10, 2021), including assays for SARS-CoV-2, which received FDA EUA on 12 March 2020. Several studies have evaluated the performance of this highly reliable and robust SARS-CoV-2 assay, [4] [5] [6] [7] [8] [9] [10] [11] and it has become a primary comparator in many performance evaluations of novel SARS-CoV-2 RNA assays.

Alinity m (Abbott Molecular, Des Plaines, IL, USA) is another recently launched fully integrated, automated sample-to-result molecular analyzer allowing continuous loading of samples and random-access testing. Seven molecular assays have been developed for use with Alinity m, [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] including an assay for SARS-CoV-2, which received U.S. FDA EUA on 11 May J o u r n a l P r e -p r o o f 5 2020. Unlike the cobas SARS-CoV-2 assay with much-published performance data, peerreviewed literature contains only limited verification and validation data for the Alinity m SARS-CoV-2 assay. 20 Here the results of the large manufacturer-independent clinical evaluation of the Alinity m SARS-CoV-2 assay against the cobas 6800 SARS-CoV-2 assay as a standard comparator are presented. The head-to-head study evaluated the clinical performance of two high-throughput automated assays on 2,157 consecutive nasopharyngeal swabs in routine diagnostic settings. In addition, comparative real-life throughput assessment of two analyzers on 564 samples across two 8-hour shifts as well as comparative turnaround time assessment on the first 94 samples received in the laboratory each working day for 5 consecutive days was performed.

Samples used for head-to-head clinical comparison. Clinical performances of the Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 assays were compared on a total of 2,157 unselected nasopharyngeal swabs routinely collected during a 10-day period in mid-January 2021. All samples were obtained from patients treated in the University Clinical Center Ljubljana, the largest tertiary hospital in the country with 2,138 beds which is also the principal Slovenian COVID-19 hospital with the largest COVID-19 intensive care unit. Samples were collected in a 3 ml commercial transport medium VTM (Liofilchem, Roseto degli Abruzzi, Italy) from 2,157 individuals referred for COVID-19 testing during the study period. The median transport time of samples from the collection site to the laboratory was 1 hour 51 minutes. Upon arrival at the laboratory, swabs were vortexed for 1 minute at maximum speed and two VTM J o u r n a l P r e -p r o o f aliquots were prepared: 800 µl for Alinity m SARS-CoV-2 testing and 700 µl for cobas 6800 SARS-CoV-2 testing.

Alinity m SARS-CoV-2 testing. The Alinity m SARS-CoV-2 assay is a real-time reverse transcriptase (RT) PCR-based assay for the qualitative detection of SARS-CoV-2 RNA in nasal, nasopharyngeal, and oropharyngeal swabs. 20 By using target-specific fluorescent-labeled oligonucleotide probes, the assay allows simultaneous detection and amplification of two SARS-CoV-2-specific sequences targeting the RdRp gene and N gene reported as a combined signal in one channel and individually an internal control (IC) target sequence for evaluation of sample extraction and amplification efficiency reported separately in another channel. The assay is performed on Alinity m-a fully integrated, automated molecular analyzer that allows continuous loading of samples and performs sample preparation, RT-PCR assembly, amplification, detection, calculation, and reporting of the results. Alinity m provides two different reports: "Result" (each signal is reported either as "Not Detected" if a specimen cycle number (CN) is not generated or "CN value" if CN is less than 42) and "Interpretation" (either Negative or Positive). Prior to transferring the results into the laboratory information system (LIS), the results can be reviewed directly in the system or as a printed report.

For this study, 800 µl sample aliquots were transferred into Alinity m Aliquot Tubes, loaded on Alinity m, and tested following the manufacturer's instructions.

Cobas 6800 SARS-CoV-2 testing. The cobas SARS-CoV-2 assay is a two-target RT-PCR for qualitative detection of SARS-CoV-2 RNA in nasopharyngeal and oropharyngeal swab samples. N-glycosylase system for prevention of PCR contamination. The assay is performed on either the cobas 6800 System or cobas 8800 System instrument, which consists of the sample supply module, transfer module, processing module, and analytic module. Automated data management is performed by the manufacturer's software, which assigns test results. The results can be reviewed directly on the system screen, printed as a report, or transferred to an LIS. According to the manufacturer's instructions, a tested sample was considered SARS-CoV-2 positive if cobas showed positive results either for both the ORF1 (target 1) and E (target 2) genes or for the 

Head-to-head clinical comparison. Out of 2,157 nasopharyngeal swabs tested, valid initial results of Alinity m SARS-CoV-2 and cobas 6800 SARS-CoV-2 were obtained from 2,129/2,157 (98.7%) and 2,157/2,157 (100%) samples, respectively. Total of 28 samples were excluded from further analysis due to initial invalid Alinity m SARS-CoV-2 results; all invalid results were due to no amplification of the target and internal control failure. Table 1 Thirty-seven discordant results were identified. Discordance between the assays was completely unidirectional (Table 1) Alinity m SARS-CoV-2 internal control is a noninfectious Armored RNA sequence unrelated to the SARS-CoV-2 sequence and is introduced into each specimen at the beginning of the sample preparation. Internal control is simultaneously amplified by PCR to demonstrate that the entire testing procedure has proceeded correctly for each sample. In this study, 28/2,157 (1.3%) samples showed initial invalid Alinity m SARS-CoV-2 results, all due to no amplification of target and internal control failure. Insufficient leftover volume allowed retesting of only 10/28 study samples with initial invalid Alinity m SARS-CoV-2 results, and after retesting a valid Alinity m SARS-CoV-2 result was obtained in 9/10 (90%) of initially failed samples. Currently, the cobas 6800 System has been installed in our laboratory for 2 years and Alinity m was J o u r n a l P r e -p r o o f installed a week before the study. Although SARS-CoV-2 RNA invalid results were also initially an issue with the cobas 6800 System 4 and were mainly caused by clots, mucus, or physical contamination detected by the instrument during sample aspiration or insufficient sample volume identified in sample tubes or processing plates, the instrument soon "stabilized," and no significant problems with invalid results on the cobas 6800 System have been recorded in 10 months while routinely testing 120,000+ samples using the cobas 6800 SARS-CoV-2. After closing the head-to-head study, we continued monitoring the rate of invalid results on Alinity m 

When selecting a SARS-CoV-2 RNA assay, virologists must consider not only sensitivity and specificity, but also sample throughput, time-to-result, test complexity, reagent and instrument availability, and cost per reportable result. Assay throughput is especially a crucial parameter for large-scale testing. Alinity m and the cobas 6800 System are both fully integrated and automated analyzers allowing sample-to-result detection of SARS-CoV-2 RNA, but this comparative throughput assessment showed potentially important differences (Fig. 1) . Alinity m produced the first reportable results much sooner than the cobas 6800 System (2:35h vs. 3:40h), but the cobas 6800 System finished testing 564 samples in parallel almost 2 hours earlier than Alinity m (11:35h vs. 15:30h). Similarly, although the total hands-on time for testing 564 samples was almost equal for both instruments, the cobas 6800 System required slightly less total instrument handling time than Alinity m, but slightly more total sample handling time. In addition, during testing of 564 samples in parallel across two 8-hour shifts, the cobas 6800 System required the presence of laboratory staff during seven similar time slots whereas Alinity m required staff presence during 12 varying time slots (Fig. 1) . Similar potentially important differences between the analyzers were also recorded during parallel routine processing of the first 94 samples received at the laboratory each working day for 5 consecutive days. This test on 470 samples showed that TATs for most samples were somewhat shorter for Alinity m in comparison to the cobas 6800 System. This discrepancy was mainly due to differences observed in total preanalytical time between the analyzers for most samples and because our laboratory routinely (and most economically) processed 12-sample batches for Alinity m and 94-sample batches for the cobas 6800 System. Thus in different laboratory settings the differences between the analyzers observed in this study could be less evident and less important.

When asked to qualitatively compare their experience with both analyzers for routine detection of SARS-CoV-2 RNA, laboratory staff agreed that the main comparative advantages of Alinity m are rapid TAT for smaller batches and flexible STAT prioritization, and the main disadvantages are the need to pre-thaw and centrifuge reagents, more frequent instrument interactions and sample loading/unloading, and limited (48-hour) onboard stability of positive and negative controls. The main comparative advantages of the cobas 6800 System were higher 24-hour throughput, ready-to-use reagents not requiring thawing or mixing, and less frequent instrument interactions and sample loading/unloading, and the main disadvantage (in our laboratory setting) was needing sample centrifugation to avoid problems during sample aspiration due to clots, mucus, or physical contamination.

In conclusion, the results of the manufacturer-independent evaluation of Alinity m SARS-CoV-2 on 2,157 samples in routine diagnostic settings against cobas 6800 SARS-CoV-2 showed that Alinity m SARS-CoV-2 is a reliable assay for the qualitative detection of SARS-CoV-2 in nasopharyngeal swab samples. Assays showed excellent overall, positive, and negative percent agreements with a high kappa value. Slightly higher analytical sensitivity of Alinity m SARS-CoV-2 was clinically beneficial in a limited number of samples. Comparative real-life throughput and TAT assessments showed similar performance of both assays with performance differences, which could be potentially important in some laboratory settings. 

Diagnostics for SARS-CoV-2 detection: a comprehensive review of the FDA-EUA COVID-19 testing landscape

Laboratory diagnosis of emerging human coronavirus infectionsthe state of the art

Summary of the available molecular methods for detection of SARS-CoV-2 during the ongoing pandemic

Clinical evaluation of the cobas SARS-CoV-2 test and a diagnostic platform switch during 48 hours in the midst of the COVID-19 pandemic

Comparison of commercially available and laboratory-developed assays for in vitro detection of SARS-CoV-2 in clinical laboratories

Clinical evaluation and utilization of multiple molecular in vitro diagnostic assays for the detection of SARS-CoV-2

Comparison of SARS-CoV-2 detection from nasopharyngeal swab samples by the Roche cobas 6800 SARS-CoV-2 test and a laboratory-developed realtime RT-PCR test

Comparison of two high-throughput reverse transcription-PCR systems for the detection of severe acute respiratory syndrome coronavirus 2

Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche cobas for the rapid detection of SARS-CoV-2

Detection of SARS-CoV-2 by use of the Cepheid Xpert Xpress SARS-CoV-2 and Roche cobas SARS-CoV-2 assays

Multicenter comparison of the cobas 6800 System with the RealStar RT-PCR kit for the detection of SARS-CoV-2

Multicenter clinical evaluation of Alinity m HBV assay performance

Alinity m HR HPV assay fulfills criteria for human papillomavirus test requirements in cervical cancer screening settings

Multicenter clinical comparative evaluation of Alinity m HIV-1 assay performance

Evaluation of the Alinity m HIV-1 assay for the quantification of HIV-1 RNA plasma viral load in a high-throughput molecular laboratory in South Africa

AM: Comparison of the Abbott Alinity m and m2000 assays for the quantification of HIV-1, HCV and HBV in clinical samples

Investigating utilising the Alinity m platform to detect hepatitis C virus RNA in dried blood spot samples

Real-world application of the Xpert HBV viral load assay on serum and dried blood spots

Verification and validation of SARS-CoV-2 assay performance on the Abbott m2000 and Alinity m systems

Performance evaluation of the fully automated molecular system Alinity m in a highthroughput central laboratory

Clinical and analytical evaluation of the Alinity m HR HPV assay within the VALGENT-3 framework

Comparison between Abbott m2000 RealTime and Alinity m STI systems for detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and Mycoplasma genitalium

Comparison of four molecular in vitro diagnostic assays for the detection of SARS-CoV-2 in nasopharyngeal specimens

Evaluation of the Aptima™ transcription-mediated amplification assay (Hologic®) for detecting SARS-CoV-2 in clinical specimens

Analytical and clinical comparison of three nucleic acid amplification tests for SARS-CoV-2 detection

Comparison of the Panther Fusion and a laboratory-developed test targeting the envelope gene for detection of SARS-CoV-2

Multicenter evaluation of the NeuMoDx SARS-CoV-2 test

Validation of a modified CDC assay and performance comparison with the NeuMoDx™ and DiaSorin® automated assays for rapid detection of SARS-CoV-2 in respiratory specimens

A recurrent mutation at position 26340 of SARS-CoV-2 is associated with failure of the E gene quantitative reverse transcription-PCR utilized in a commercial dual-target diagnostic assay

Paniz-Mondolfi A: Will the emergent SARS-CoV2 B.1.1.7 lineage affect molecular diagnosis of COVID-19

We thank Robert Krošelj and Maja Accetto Kos for excellent laboratory assistance with cobas 6800 System testing and all members of the COVID-19 diagnostic team, who have been working diligently for the past 12 months to ensure on-time SARS-CoV-2 results.

Alinity m (red curve) and the cobas 6800 System (blue curve) while testing 564 nasopharyngeal swab samples in parallel across two 8-hour shifts. Sample handling times, instrument handling times, and total hands-on times measured by two independent observers for each analyzer are presented with red (Alinity m) and blue (the cobas 6800 System) time boxes; the dark part of each box represents sample handling time, and the lighter part of each box instrument handling time.