key: cord-1007312-918wd3ez
authors: Basu, Atreyee; Zinger, Tatyana; Inglima, Kenneth; Woo, Kar-mun; Atie, Onomie; Yurasits, Lauren; See, Benjamin; Aguero-Rosenfeld, Maria E.
title: Performance of the rapid Nucleic Acid Amplification by Abbott ID NOW COVID-19 in nasopharyngeal swabs transported in viral media and dry nasal swabs, in a New York City academic institution
date: 2020-05-12
journal: bioRxiv
DOI: 10.1101/2020.05.11.089896
sha: 44b048b91a952e5473af79c6f7c2edd7a0a08980
doc_id: 1007312
cord_uid: 918wd3ez

The recent emergence of the SARS-CoV-2 pandemic has posed formidable challenges for clinical laboratories seeking reliable laboratory diagnostic confirmation. The swift advance of the crisis in the United States has led to Emergency Use Authorization (EUA) facilitating the availability of molecular diagnostic assays without the more rigorous scrutiny to which tests are normally subjected to prior to FDA approval. The need to identify the COVID-19 positive cases quickly and accurately has propelled the release of a variety of assays intended to meet the urgent demand. Several Nucleic Acid Amplification Tests (NAAT) platforms are currently available. Our laboratory currently uses two real time RT-PCR platforms, the Roche Cobas SARS-CoV2 and the Cepheid Xpert Xpress SARS-CoV-2. Both platforms demonstrate comparable performance; however the run times for each assay are 3.5 hours and 45 minutes, respectively. In search for a platform with shorter turnaround time, we sought to evaluate the recently released Abbott ID NOW COVID-19 assay which is capable of producing positive results in as little as 5 minutes. We present here the result comparisons between Abbot ID NOW COVID-19 and Cepheid Xpert Xpress SARS-CoV-2 using nasopharyngeal swabs transported in VTM as well as dry nasal swabs for the Abbott assay. Regardless of method of collection and sample type, Abbot ID NOW COVID-19 missed a third of the samples detected positive by Cepheid Xpert Xpress when using NP swabs in VTM and over 48% when using dry nasal swabs.

The novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causing coronavirus disease-2019 (COVID-19) was first identified in Wuhan Jinyintan Hospital, China in December 2019. Since then, it has spread vastly, reaching the status of global pandemic as declared by the World Health Organization on March 11 th , 2020 (1) . As of May 2 nd, 2020, there are 3,356,205 reported cases, and 238,730 confirmed deaths worldwide, in 215 countries (2) . In NY State and New York City, there were 316,415 and 168,845 reported cases and 19,189 and 13,319 deaths respectively as of May 2 nd , 2020 (3, 4) .

Common signs and symptoms with the associated respiratory illness are fairly non-specific and include fever, dry cough, headache, myalgias, arthralgias, and sore throat, with indicators of more severe disease such as shortness of breath and respiratory failure (1) . Nearly 80% of the infected individuals are either asymptomatic or have mild symptoms leading to difficulties to diagnose this disease. COVID-19 has high human-to-human transmission rates (R0 = 2.0 -2.5), and is stable in aerosols and on surfaces, presenting a challenge to mitigation and emphasizing the urgent need for a more rapid, accessible and accurate diagnostic assays (5) .

SARS-CoV-2 virus is detected in various human sampling sources including respiratory and fecal, but testing has been validated for PCR primarily of nasal, nasopharyngeal, and oropharyngeal specimens. Positive samples for SARS-COV-2 were found in Chinese patients 1-2 days prior to symptom onset and persisted up to 2 weeks in severe cases (1) .

While long-term mitigation and prevention goals include prognostic markers, therapeutics, and vaccines (1), the urgency of this rapidly developing crisis in the United States has prioritized the immediate goal of diagnostic testing, advanced by the FDA's Emergency Use Authorization 5 (EUA) (5, 6) . The initiative to expand and accelerate testing has eased the usual scrutiny that new assays would normally undergo prior to release. This obligates clinical laboratories to more carefully assess the overall performance of the various rapidly emerging testing platforms prior to implementation. Several Nucleic Acid Amplification tests (NAAT) platforms are currently available. Our laboratory uses two real time RT-PCR platforms, the Roche Cobas and the Cepheid GeneXpert. Both platforms perform comparably with similar limits of detection (LOD) of SARS-CoV-2 viral RNA, 100-150 and 250 copies/mL, respectively. However, the assay run times of 3.5 hours and 45 minutes, respectively, are still too long for timely decision support in a variety or important clinical situations. (e.g. Emergency Department discharge). In search for a platform with shorter turnaround time, we sought to evaluate the recently released Abbott ID NOW COVID-19 assay. Abbott ID NOW uses isothermal nucleic acid amplification of the RdRp viral target with a claimed LOD of 125 copies/mL. The objective of our study was to evaluate the performance of the rapid Abbott-ID-NOW platform test by using the Cepheid-Xpert-Xpress SARS-CoV-2 test on the GeneXpert Dx as the comparator reference method.

All samples were obtained from patients seen at the New York University Langone Tisch Hospital with diagnosis of suspected COVID-19. Nasopharyngeal (NP) samples were obtained using flocked NP swabs and transported in universal viral transport media (VTM) (Cepheid Xpert Viral Transport Medium; SWAB/B-100) or BD Universal VTM / 2250527) within 1-2 hours of collection.

For the assessment of performance of dry swabs, we obtained 101 dry nasal swabs of both nares as well as 101 NP swabs in VTM from the same patient on the same ED encounter to be tested using both, the Abbott ID NOW COVID-19 and Cepheid Xpert Xpress SARS-CoV-2, 6 respectively. The dry nasal swabs, which are supplied with the Abbott assay (Puritan Medical Products 25-1506 IPF 100) were transported in a sterile tube to the laboratory along with the corresponding NP swab in VTM.

ID NOW COVID-19 is a rapid test that qualitatively detects SARS-CoV-2 viral nucleic acids from nasal, nasopharyngeal and throat swabs. It is an automated assay that utilizes isothermal nucleic acid amplification technology that eliminates the need for denaturation by thermocycling.

The test includes a sample receiver containing elution/lysis buffer, a test base comprising two Xpert Xpress SARS-CoV-2 test is a rapid, real-time RT-PCR test detecting ribonucleic acid (RNA) from the SARS-CoV-2 in nasopharyngeal swab and/or nasal wash/aspirate specimens collected from individuals with suspected COVID-19 diagnosis. The specimens were collected, placed in VTM and run on the GeneXpert Dx. This platform automates and integrates sample preparation, nucleic acid extraction and amplification, and detection of the target sequences. The assay detects 2 nucleic acid targets, namely N2 (nucleocapsid) and E (envelope) wherein N2 is more specific for SARS-CoV-2. The lowest limit of detection (LOD) for this assay is claimed by the manufacturer to be 250 copies/mL. SARS-CoV-2 genomic load is assessed using the number of amplification cycles needed for a positive PCR test (i.e. the cycle threshold [Ct] value). Ct values have an inverse relationship to viral loads and thereby provide a surrogate measurement of the viral genomic load (7) . Consequently, the higher the viral load, the lower the Ct value of a particular specimen and vice versa.

Both assays are used only under Food and Drug Administration's Emergency Use Authorization (EUA). (8) The performance evaluation of the Abbott ID NOW assay was conducted in several steps.

samples using the Xpert Xpress assay, one with high a Ct value 30.8 for N2 and another with low Ct value of 19.3 (i.e. high viral load). Both samples were diluted 1:2, 1:5, 1:10, 1:20, 1:50 and 1:100 in VTM. All the dilutions were tested separately on both platforms.

This included 15 sequential NP specimens submitted in VTM by our Emergency Department from patients with suspected COVID-19 disease. Each specimen was tested on both, the Cepheid GeneXpert and Abbott ID NOW platforms.

Cepheid Xpert Xpress assay. For this analysis, rather than using specimen dilution to simulate a series of decreasing viral loads, a set of 8 previously tested specimens in the Xpert Xpress assay were selected with increasing N2 Ct values ranging from 32.6 to 41.8. All samples were tested on the ID NOW platform and the results compared to those previously obtained on the Cepheid GeneXpert. 8 For all the comparisons mentioned above, all samples were nasopharyngeal swabs transported to the lab in VTM. In response to our findings of lower sensitivity of the ID NOW compared to the GeneXpert, Abbott communicated to us that despite the indications in the package insert, their assay was not optimized for testing nasopharyngeal swabs in VTM; the best performance would be from dry nasal swabs obtained at the point of care (POC). They indicated that they would be modifying their package insert to reflect this change (ID NOW COVID-19 Product Insert, IN190000 Rev.3 2020/04:6-8.). Since the Abbott ID NOW rapid results feature was regarded as a worthwhile advantage over our other assays in use, we compared dry nasal swab (both nares) tested on the Abbott platform versus NP in VTM tested on the Cepheid platform. Both samples were obtained in parallel from patients with suspected COVID-19, seen at the NYU Langone Health Tisch Hospital Emergency Department. Although Abbott indicated that the dry swab could be transported in the same sleeve in which the swabs were packaged, we considered the safety of this modality and chose a sterile tube instead. A total of 101 paired samples collected during 12 hour shifts over 3 consecutive days were included. Ninety-three percent of the samples were transported to the laboratory within 1 hour for testing, the remainder within 2 hours. Testing at the point of care was not practical.

Qualitative Method Comparison was performed using EP Evaluator (Data Innovations, VT 05403) and correlation coefficient using Excel 2010.

Results of the initial verification of the Abbott ID NOW revealed that it performed well with respect to the reference method when a specimen (Sample 1) with low N2 CT value (9.3) was progressively diluted up to 1:100., but this was not the case for Sample 2 with had an initial higher N2 Ct value (30.8) (Table 1) . Decreased sensitivity was noted at 1:10 dilution level (approximated Ct=33.8) and beyond with only a 33% agreement at 1:100 (approximated Ct value of 36.8).

Comparison of results obtained on 15 sequential NP specimens submitted in VTM by our Emergency Department produced 5 false-negative results by Abbott ID NOW (33.3%) ( Table 2 ).

The Cepheid N2 Ct values of the discordant specimens ranged from 36.3 to 44.3.

In order to confirm that the lower sensitivity of the Abbott ID NOW platform was related to the Ct value obtained on the GeneXpert Dx, we used a series of previously tested NP specimens with increasing N2 Ct values (Table 3 ). Similar to the dilution studies, the performance of the Abbott ID NOW started to decline as the N2 Ct values increased throughout the series, although at a higher Ct value (38.0) than that seen in our previous comparisons (~ 33.8, 36.3, respectively). That is, beyond a Ct value of 38, ID NOW performance became increasingly less reliable. For the dry nasal swabs versus NP samples in VTM we included samples from 101 Emergency Department patients (age 28-90) with suspected COVID (onset of symptoms 1 day to 1 month).

Patients were sampled twice during the same encounter using a nasal dry swab of both nares and a NP specimen submitted in VTM. As seen in Table 4 , the Abbott ID NOW using dry nasal swabs There was no correlation between N2 Ct and days of onset of symptoms (R2 = 0.097). Sixteen of the 31 patients whose samples tested positive for SARS-CoV2 RNA by Xpert Xpress were admitted; 6 of these patients had negative results when tested by Abbott ID NOW.

The urgency of the COVID-19 pandemic and the United States government's subsequent use of the Federal Authorization Act has placed an unusual onus of responsibility on the clinical laboratories to substantiate the clinical value and performance of the COVID-related tests they introduce. Accurate diagnostic testing and subsequent isolation of sick individuals is crucial to mitigating continued infectious transmission. Therefore, assay sensitivity must be evaluated on the many newly available COVID-19 testing platforms to minimize the public health risks that come with false negative results.

Our study used several approaches in an attempt to validate the rapid testing methodology by Abbott ID NOW COVID-19 in comparison to Cepheid Xpert Xpress SARS-CoV-2. The appeal of the Abbott technology stemmed from our intention to provide rapid and accurate results in our institution, especially for our Emergency Department.

Our results showed progressively more false negative results with lower viral load (higher N2 Ct values) as determined by Cepheid, but this was not consistent; there were samples testing positive by Abbott platform even when N2 Ct values were above 38 when we did the parallel comparison of dry nasal swabs versus NP samples in VTM. A study from Northwell Health has shown similar discordant results comparing these two assays, with ID NOW COVID-19 producing 7 false negatives out of 107 nasopharyngeal samples, resulting in a sensitivity of 87.7% compared to 98.3% on Xpert Xpress SARS-CoV-2. Their study however, used NP swabs transported in VTM, which the manufacturer claims decreases the sensitivity of their assay. These authors also determined that the GeneXpert had the lowest limit of detection at 100 viral copies/mL, while the ID NOW had a limit of detection of 20,000 copies/mL (9) which is well above the limit of detection stated in the Abbott ID NOW COVID-19 package insert (125 viral copies/mL).

Using a larger sample size, preliminary studies at the Cleveland Clinic also revealed a lower sensitivity for the Abbott ID NOW, with a false negative rate of 14.8%. However, the samples were all stored in viral transport media (10) . Although the initial Abbott package insert had stated that 0.5 to 3.0 mL of viral transport media was acceptable for use in their assay, it also recommended minimizing dilution of the sample is as it may result in decreased test sensitivity.

The use of dry swabs is now recommended by the manufacturer for optimal test performance. To investigate potential improvement in test performance resulting from this change, we evaluated the use of dry nasal swab and NP in VTM obtained in parallel on ED patients. Contrary to manufacturer expectations, our parallel study showed that the sensitivity of Abbott ID NOW using dry swabs (51.6%), was actually lower than when using viral transport media (66.7%) as seen from our earlier study of ED samples.

The literature provides no strong consensus regarding the influence of transport medium and storage conditions on viral load stability. One study looking at differing viral transport media and storage conditions using the Quest and Roche EUA assays showed consistent detection of SARS-CoV-2 RNA after 7 days at room temperature, and up to 14 days refrigerated or frozen. (11) The SARS-COV-2 ID-NOW package insert also directs clinicians to test swabs as soon as possible after collection, and if not possible, to hold it in its original package no more than two hours prior 12 to testing. Outside of inpatient hospital laboratories, such as in outpatient point-of-care testing, independent validation should be performed to assess performance in different settings, including pre-analytic variables.

Extrapolating from our results from 101 patient samples, if the Abbott-ID-NOW COVID-19 assay were to be used as a first step screening, confirmation of over 80% of the tested samples would be required to be confident that the negative results are truly negative. While there was one apparent false-positive result in our parallel study, the overall high PPV (94.1%) and specificity (98.6 %) do support the added value that in 5 -13 minutes a positive result can be interpreted as a true positive. We could not confirm that the apparent false positive result obtained with the Abbott ID NOW was due to sampling variability.

Based on our findings we could argue that the Abbott ID NOW detects samples with high viral load or possibly viable virus that could be of importance for transmission. But, the fact that it misses positive samples on patients being admitted to the hospital with clinical picture of COVID-19 makes this technology unacceptable in our clinical setting. Recent reports indicate that mutations in the RdRp domain have been emerging (12) . It is unknown at this point if the genomic region targeted by the Abbott assay has changed significantly as the virus is spreading throughout the world.

Limitations of our study include relatively small sample size, inability to control for sampling variabilities, time of transport of dry swabs and testing being conducted in the clinical laboratory, rather than point-of-care setting as Abbott ID NOW platform is intended.

Overall, our study revealed low sensitivity with high false negative results by Abbott ID NOW platform irrespective of use of viral transport media, which raises concern regarding the performance of the assay and its suitability as a diagnostic tool for symptomatic patients. The resolution could be to reflex all negative results for confirmation by a method with higher sensitivity. However, such requirement would, except for positives, severely diminish the value of the rapid results of the assay. 

World Health Organization

World Health Organization. 2020. Coronavirus disease (COVID-19) Pandemic, on WHO

New York State Department of Health. 2020. NYSDOH COVID-19 Tracker

COVID-19: Data

Comparison of Four Molecular In Vitro Diagnostic Assays for the Detection of SARS-CoV-2 in Nasopharyngeal Specimens

US Food and Drug Administration (FDA). 2020. FAQs on Diagnostic Testing for SARS-CoV-2, on US FDA

Respiratory syncytial virus genomic load and disease severity among children hospitalized with bronchiolitis: multicenter cohort studies in the United States and Finland

Diagnostic Testing for Severe Acute Respiratory Syndrome-Related Coronavirus-2: A Narrative Review

Clinical Evaluation of Three Sample-To-Answer Platforms for the Detection of SARS-CoV-2

Study Raises Questions About False Negatives From Quick

National Public Radio (NPR)

Evaluation of Transport Media and Specimen Transport Conditions for the Detection of SARS-CoV-2 Using Real Time Reverse Transcription PCR

Emerging SARS-CoV-2 mutation hot spots include a novel RNA-dependent-RNA polymerase variant