key: cord-1053117-q9ij2skz
authors: Chan, Maurice; Koo, Seok Hwee; Jiang, Boran; Lim, Pei Qi; Tan, Thean Yen
title: Comparison of the Biofire FilmArray Respiratory Panel, Seegene AnyplexII RV16, and Argene for the detection of respiratory viruses
date: 2018-09-30
journal: Journal of Clinical Virology
DOI: 10.1016/j.jcv.2018.07.002
sha: 37f71b88b0091fb0d6232dd9e8ee4cfd7c3b6f64
doc_id: 1053117
cord_uid: q9ij2skz

Abstract Background Respiratory infections are common reasons for hospital admission, and are associated with enormous economic burden due to significant morbidity and mortality. The wide spectrum of microbial agents underlying the pathology renders the diagnosis of respiratory infections challenging. Molecular diagnostics offer an advantage to the current serological and culture-based methods in terms of sensitivity, coverage, hands-on time, and time to results. Objectives This study aimed to compare the clinical performance of three commercial kits for respiratory viral detection. Study design The performance of FilmArray Respiratory Panel, AnyplexII RV16, and Argene was compared using clinical respiratory samples (n = 224, comprising 189 nasopharyngeal swabs in Universal Transport Medium (UTM) and 35 endotracheal aspirates), based on common overlapping targets across the platforms. Influenza A “equivocal” and “no-subtype” samples by FilmArray were further compared to a laboratory-developed Influenza A/B test. Results and Conclusions The overall performance of all three platforms appeared to be comparable with regards to sensitivities (95.8–97.9%) and specificities (96.1–98.0%), detection of coinfections, and distinguishment of influenza from non-influenza cases. “Equivocal” and “no-subtype” samples by FilmArray mostly represented weak Influenza A by laboratory-developed test. Lower respiratory tract samples had comparable final-run success-rates and discordant-rates as compared to UTM. Coronavirus HKU1, which was not targeted by AnyplexII RV16, were detected as OC43. The expected test volume would be the main determinant for the selection of platform. Among the platforms, the FilmArray is the most automated but is of the lowest-throughput and has the highest reagent cost.

Respiratory infections are one of the most common reasons for hospital admission [1, 2] . Identification of the causative agent is often omitted or limited to a few pathogens easily detectable by direct antigen tests [2] . However, the reliable diagnosis of respiratory infection is important for therapy [3] , implementation of isolation measures [4] , and better use of antimicrobial therapy [5] . Epidemiological data from the laboratory can be used for prescribing prophylactic treatments for at risk patients [6, 7] , or for alerting to new or seasonal spread of pathogens [2] .

Direct fluorescent antibody testing and viral culture are time consuming and demand significant expertise in the performance of the test and interpretation of test results. On the other hand, rapid antigen testing appeared to lack sensitivity in the detection of influenza and respiratory syncytial virus (RSV), with reported sensitivities as low as 21% and 59% respectively [8] [9] [10] . Nucleic acid detection tests for respiratory virus assays have gained popularity due to their quick turnaround-time and propensity for multiplexing and automation.

The FDA cleared four molecular multiplex respiratory panels, namely the FilmArray RP (respiratory panel) (BioFire Diagnostics, Salt Lake City, UT), the eSensor RVP (respiratory viral panel) (GenMark Dx, Carlsbad, CA), the Luminex xTAG RVPv1, and the Luminex xTAG RVP fast (Luminex Molecular Diagnostics, Austin, TX). These platforms are able to detect 12-20 targets simultaneously, at reported sensitivities and specificities of 78.8%-95.4%, and > 99.0% respectively [3] .

The FilmArray RP (FA-RP) is the most automated, requiring no preextraction and additionally tests for three bacterial agents not detected https://doi.org/10.1016/j.jcv.2018.07.002 Received 12 January 2018; Received in revised form 18 May 2018; Accepted 5 July 2018 by the other three platforms, namely Bordetella pertussis, Chlamydophila pneumoniae, and Mycoplasma pneumoniae. The FA-RP is a melt curvebased real-time PCR (RT-PCR) method, whereby the amplification occurs in two steps: the 1 st step is to enrich for the target nucleic acids in the sample in a multiplex reaction and the 2nd step is to amplify each target in multiple miniaturized individual wells containing single specific primer-pairs. The nucleic acid extraction, distribution of nucleic acid into wells and PCR amplification are fully automated and integrated and occur in a single-use pouch.

The AnyplexII RV16 (Seegene, South Korea) is an alternative to FDA-approved integrated platforms. It requires pre-extraction and prereverse transcription, but runs on common RT-PCR platforms such as CFX-96 (Bio-Rad, Hercules, CA) and AB7500 (Thermo Fisher Scientific, Waltham, MA). It utilizes proprietary TOCE ™ technologies to achieve simultaneous detection of melt curves from 16 viral targets in two-tube reactions.

The Argene real-time PCR assay-respiratory range (Biomerieux, France) is compatible with major extraction platforms and real-time PCR systems as well as on various sample types. The Argene PCR kits have been validated to exhibit good analytical sensitivity and specificity, thus represent a rapid and reliable test for the detection of viruses and bacteria causing respiratory infections. The turn-around time for the assays, not including extraction, is within 2 h.

The objective of this study was to compare the clinical performance (using actual clinical respiratory samples) of the two multiplexing strategies represented by the FA-RP (Version 1.7) and AnyplexII RV16 (Version 1.1), with that from a more conventional, two-plex RT-PCR kit represented by respiratory virus assays from the Argene (Biomerieux, France) range.

The research study protocol was submitted to the SingHealth Centralised Institutional Review Board (CIRB Ref No.: 2015/2930) and was exempted from review based on the nature of the study.

A total of 224 specimens, which comprised 189 nasopharyngeal swab specimens in Universal Transport Medium (UTM, Copan, Italy), and 35 endotracheal aspirate (ETT) specimens were collected from symptomatic patients from August 2015 to March 2016. The first 107 study specimens were collected prospectively in sequential order, and the remaining 117 specimens comprised FA-RP positive Universal Transport Medium (UTM) specimens and respiratory aspirates (positive and negative specimens). The distribution of microorganisms detected in the samples used for this study is shown in Supplemental Table 1 .

The FA-RP, AnyplexII RV16, and Argene assays (comprising Adenovirus/Bocavirus, Coronavirus/PIV, Influenza A/InfluenzaB, RSV/ MPV, and Rhinovirus/Enterovirus) were highly similar in their targets. Bocavirus and the FA-RP bacterial targets (B. pertussis, C. pneumoniae, and M. pneumoniae) were not compared, as Bocavirus was not covered in FA-RP, and the bacterial agents were not covered in AnyplexII RV16 and Argene. Coronavirus (NL63, OC43, KHU1, 229E) and Parainfluenza (PIV1, PIV2, PIV3, PIV4) can be detected by Argene kit but cannot be subtyped and were reported as groups. FA-RP can detect and subtype four Coronavirus subtypes (229E, HKU1, OC43, and NL63) while AnyplexII RV16 covered and subtype three (229E, OC43, and NL63).

Influenza A was not subtyped by AnyplexII RV16 and Argene. Rhinovirus/Enterovirus and RSV were reported as groups by both FA-RP and Argene. In contrast, AnyplexII RV16 distinguished between Rhinovirus and Enterovirus, and could subtype RSV into RSV A and RSV B.

Quality control, assay validation and result interpretation were performed according to individual manufacturer's instructions. For AnyplexII RV16 and Argene, microbial nucleic acids were extracted using the EZ1 Virus Mini kit (Qiagen, Singapore) on EZ1 or EZ1XL (Qiagen, Singapore) automated sample preparation platforms (sample extraction volume: 400 μL, elution volume: 150 μL). The FA performs extraction and nucleic acid amplification on the same platform. UTM samples were used directly for FA-RP or for EZ1/EZ1XL extraction. Other respiratory aspirates underwent pre-processing using a "dunk and swirl" method in UTM [11] prior to nucleic acid extraction. Briefly, the sputum samples were diluted and mixed with an equal volume of sterile water, a sterile swab was then dunked into each sample, withdrawn and then swirled in 700 μL of sterile water. The swab was removed and solution used for testing. AnyplexII RV16 and Argene RT-PCR runs were carried out using CFX-96 (Bio-Rad) and Rotorgene (Qiagen), respectively.

The FA-RP automatically interpreted and showed the results for each target (detected/ not detected/ invalid). For FA-RP, Influenza A results which were "equivocal" or "no subtype", the specimens were further tested by an in-house Influenza A/B multiplex qPCR assay which can detect prevalent Influenza A virus (all subtypes), Influenza A virus H1N1pdm09 subtype and Influenza B virus. Samples which were "equivocal" or "no subtype" by FA-RP were confirmed by Argene or AnyplexII as true positives but with a low viral load. The in-house PCR assay was used as an extended reference test for FA-RP for "equivocal" or "no subtype" samples. For Argene assays, a cycle threshold of < 40 denotes the presence of the target organism in the sample according to manufacturer's instructions. For AnyplexII RV16 assay, melting curve was called for positive or negative results according to kit instruction. For all three platforms, only results from valid runs were used for calculations in this comparison study.

An approach of extended (composite) reference was used for the interpretation of results: that is, a true positive was defined as being positive by two or more of the three platforms compared.

To facilitate comparison, for cases where targets were reported as a group, identification to correct group would be considered as a concordant case when compared with another assay that may provide more a specific identification. For example, detection of Influenza A by AnyplexII RV16 and Argene, and the detection of Influenza A/H1-2009 by FilmArray for the same specimen would be considered as a concordant result. Results from multiple-positive samples were grouped into respective targets for sensitivity and specificity calculations.

The performance of FA-RP method on different sample types (UTM vs ETT) in terms of sensitivity and specificity was evaluated. This was accomplished by comparing the results obtained using FA-RP against those from the composite gold standard between the two sample types. A concordance in results between FA-RP and composite gold standard (i.e. both positive and both negative) was considered a match. Twotailed Fisher's Exact Test was used to assess for any significant difference in the correct diagnosis using FA-RP method (with composite gold standard as reference) between the two sample types. A p-value of less than 0.05 was considered statistically significant.

The first 107 specimens were prospectively collected in sequential order, reflecting the actual clinical distribution of samples sent to the laboratory. 38 specimens (36.5%) were positive by FA-RP. The most common viruses detected were Human Rhinovirus/Enterovirus (12.5%), Influenza A & B (8.7%), and RSV (7.7%). There were 2 samples detected with dual-infection (1.87%).

Subsequent to the first 107 specimens, the samples were collected selectively, with the purpose of selection of additional positive targets for comparative testing. As such, the final collection comprised 60.7% (136) positive samples, representing all targets covered by FA-RP except Influenza (H1). Only one positive specimen was collected for Coronavirus 229E, Parainfluenza virus 4 (PIV4). All other targets were represented by 3-20 samples (Supplemental Table 1 ). There were 11 specimens which were positive for 2 targets, and a single specimen positive for 3 targets. There were 28 Influenza specimens in this study covering all categories of Influenza A results except the "H1 subtype" category.

The overall performance of all three platforms was comparable, with sensitivities and specificities ranging from 95.8% to 97.9%, and from 96.1% to 98.0%, respectively. The discordant rates for FA-RP, Argene, and AnyplexII RV16 were also similar (5.1%, 5.8%, and 5.1% respectively). Discordant cases were randomized across the targets, with no prominent shortcoming specifically associated with any of the platforms tested (Table 1) .

Among the three platforms, Argene had the most false-negatives and was associated with the lowest sensitivity but the highest specificity, while the FA-RP was associated with the highest sensitivity but lowest specificity ( Table 1 ). The differences were not statistically different at p < 0.05. The specificities for the individual assays were ≥99.0% for all the three platforms (Table 1) . For FA-RP, the lowest sensitivity was for detection of adenovirus (92.3%). For Argene, the lowest sensitivity was for detection of Rhino/Enterovirus and RSV (both 88.9%). Any-plexII RV16 had the lowest sensitivity (92.0%) for Influenza A.

There were two dual-positive samples in the sequentially collected samples (1.9%), and 11 (10.3%) dual-positive samples in the entire test panel. One discordant case was observed (CoV and RSV by FA-RP, CoV by Argene and AnyplexII RV16), suggesting that dual positive samples can be diagnosed accurately using FA-RP, Argene and AnyplexII RV16.

There were 7 specimens that represented Influenza A "no sub-type" or "equivocal" results, as reported by FA-RP. These samples were tested using the comparator platforms, and further compared with an in-house qPCR test which tested for the presence of Influenza A/H1-2009/ Influenza B. All three "no sub-type" samples were Influenza A positive by Argene (Ct-values of 26-31), AnyplexII RV16, and in-house tests (Ctvalues of 28-33). The Ct-values for Argene and in-house test were comparable. All four "equivocal" samples were Influenza A positive by Argene, but in-house test and AnyplexII RV16 were negative for two of the samples. These two Influenza A equivocal samples were considered to be positive by FA-RP. All the "equivocal" samples had Ct-values of > 30. This suggests that "equivocal" samples may be weak positive samples and that "no-subtype" samples may also be moderate-to-weak Table 1 Summary of discordant cases in this study. FN, false-negative; FP, false-positive. For the sake of simplicity, entries which represent "0″ or "100%" are replaced by "." instead. [13] [14] [15] [16] [17] positive samples.

Among the UTM samples, 124 out of 126 samples which were tested positive by FA-RP turned out to be positive on composite gold standard. 62 out of 63 samples which were tested negative by FA-RP turned out to be negative on composite gold standard. Hence according to the definition, there were three discordant results-two false positives and one false negative for UTM samples. This translated to 99.2% sensitivity and 96.9% specificity of the FA-RP method using UTM samples.

Among the ETT samples, 9 out 10 samples which were tested positive by FA-RP turned out to be positive on composite gold standard. 24 out of 25 samples which were tested negative by FA-RP turned out to be negative on composite gold standard. Hence, there were two discordant results-one false positive and one false negative for ETT samples. This translated to 90% sensitivity and 95.2% specificity of the FA-RP method using ETT samples.

The statistical analysis showed that 98.4% of UTM versus 94.3% of ETT samples were correctly classified. The percentage difference of 4.1% was not statistically significant (p = 0.175, Fisher's Exact Test).

A wide range of diagnostic kits for respiratory pathogens have emerged, addressing both American (FDA-approved) and European (CE-IVD) markets. These kits vary widely in terms of extensiveness of target coverage, throughput, and degree of automation. Among non-FDA approved platforms, the RespiFinder (Pathofinder), Seeplex RV15 (Seegene), Magicplex RV Panel (Seegene), Clart Pneumovir (Genomica), and ResPlex II Panel (Qiagen) have been compared extensively, with reported sensitivities ranging from as low as 56.2% to 91.6%, albeit with excellent specificities (> 94%) for all pathogens [2] . Among the FDA-approved platforms, the performance of the FA-RP, Luminex xTAG RVP, Luminex xTAG RVP fast, and eSensor RVP have been investigated [3] . Sensitivities ranged from 84.4% (Luminex xTAG RVP fast) to 98.3% (eSensor RVP), while specificities were high for all assays (> 99%) [3] . FA-RP had an overall sensitivity of 84.5%, with substantially lower sensitivity noted for detection of adenovirus [3] .There are comparatively fewer evaluations of AnyplexII RV16, but in a comparative study in 2013, the assay was reported to have an overall sensitivity of 95.2% [12] .

In this comparative study, the FA-RP, AnyplexII RV16 and Argene assays were noted to have comparable sensitivities and specificities of 97.9% and 96.1%; 97.2% and 97.1%, and 95.8% and 98.0%, respectively. In a comparison study between AnyplexII RV16, Seeplex RV15 (Seegene), and xTAG RVP, the overall sensitivity of RV16 was better than that of xTAG RVP [12] . However, improvement of the sensitivity for rhinovirus was required [12] . In this study, FA-RP and AnyplexII RV16 had the same sensitivity (96.2%) for rhinovirus/enterovirus, better than that for Argene (92.6%). Hence, the lack of sensitivity towards rhinovirus previously reported was not observed in this study. The sensitivity of FA-RP to adenovirus had been a concern because the original version (v1.6) of this kit was not able to detect several ADV serotypes [13] . In the modified version (v1.7), the sensitivity improved from 66.6% to 90.5% in retrospective studies, and 42.7% to 83.3% in prospective studies [13] . However, there was still one serotype (serotype 31) that remained undetected in v1.7 and the performance for adenovirus testing was still lower when compared to an in-house assay [14] . In this study, the sensitivity of FA-RP (v1.7) for ADV was 92.3%, comparable to that reported earlier [13] .

Respiratory aspirates are of specific interest for the diagnosis of community-acquired pneumonia. However, the FA-RP is FDA-cleared only for testing on nasopharyngeal swab specimens transported in UTM. In contrast, respiratory aspirates were considered acceptable samples for Argene and AnyplexII RV16, both of which are CE-IVD.

Two previous studies evaluating the FA-RP on a large number of bronchoalveolar lavage (BAL) specimens supported the utility of respiratory aspirates for FA-RP [15, 16] . In this study, there was no final run-QC failure associated with the use of respiratory aspirates on the FA-RP. Furthermore, there was no significant difference in the percentage of matches (comparing FA-RP with composite gold standard) between UTM and ETT sample types. Hence, this and earlier studies suggest that respiratory aspirates are acceptable sample types for FA-RP.

There were 7-8 samples for ADV, Influenza B, and RSV. All other targets were represented by sample sizes ranging from 10-21. Although more samples for ADV, Influenza B and RSV are desired, collection period will be prolonged. Similarly, more Influenza A "equivocal" and "no-subtype" samples will be desirable. Similarly, most of the respiratory aspirates were ETT, while sputum and BAL were poorly represented or not represented at all. There were only one each of PIV4, Coronavirus 229E, and 3-targets positive samples. Hence, these categories were not well represented. Another shortcoming was the use of "positive by at least two platforms" as the reference result for calculating sensitivity and specificity. Hence, an assay with greater sensitivity than all other assays tested would result in reduced specificity. Finally, all the platforms compared herein were multiplexed assays. The Argene duplex-assays were the closest to conventional qPCR assays. Ideally, each target should be compared with individual qPCR assays, but this was not practical due to the lack of laboratory developed tests (LDTs) for every target, as well as financial constraints.

One shortcoming of the FilmArray is that despite its short turnaround time (1 h), only one sample could be processed per run, while most platforms have higher sample throughput (up to 96 samples) despite a longer turnaround time (5.5 h for xTAG, 2.75 h for xTAG fast, 0.92 h for eSensor, and 7 h for AnyplexII RV16) [3, 12] . AnyplexII and Argene are thus of higher throughput and geared towards batched testing, while the low complexity Filmarray is more suited for on-demand analysis and requires minimum hands-on time. Since the Fil-mArray performed comparably to AnyplexII, the laboratory's required turnover and workflow considerations would be major determinants in the selection of a platform for screening respiratory agents. It should also be noted that the FilmArray also had the highest reagent costs among these automated platforms [3] .

It is worth noting that the three chosen technologies employed in this study were designed for very different laboratory purposes. The Biofire system is, regarding its workflow, clearly suitable for emergency or point of care testing with an excellent time to result. The seegene AnyplexII (or the alternative Allplex in one-step RT-PCR) fits well with routine activity involving batched samples for testing on a daily basis. The Argene solution offers quite similar features with the added advantage that the test panel may be adapted with a focus on particular organisms such as Influenzae or RSV during the epidemic period depending on the ongoing epidemiology of respiratory infections.

We have demonstrated the comparable clinical performance of three commercial kits for respiratory viral detection, namely: FilmArray Respiratory Panel (RP), AnyplexII RV16, and Argene using archived clinical respiratory samples, based on common overlapping targets across the platforms. A more extensive study with larger sample size representative of the different viral categories is required to evaluate the platforms' performance at higher resolution.

The authors declare no conflict of interest.

(NMRC/CG/002/2013 -CG12Aug02-15).

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.jcv.2018.07.002.

Why diagnose respiratory viral infection?

Comparative evaluation of six commercialized multiplex PCR kits for the diagnosis of respiratory infections

Comparison of the ofire FilmArray RP, Genmark eSensor RVP, Luminex xTAG RVPv1, and Luminex xTAG RVP fast multiplex assays for detection of respiratory viruses

Respiratory virus surveillance and outbreak investigation

Clinical evaluation of the new high-throughput Luminex NxTAG respiratory pathogen panel assay for multiplex respiratory pathogen detection

Revised recommendations concerning palivizumab prophylaxis for respiratory syncytial virus (RSV)

Oseltamivir prophylaxis in controlling influenza outbreak in nursing homes: a comparison between three different approaches

Safety assessment of Lactobacillus plantarum JDM1 based on the complete genome

Low sensitivity of rapid diagnostic test for influenza

Comparison of Cepheid's analyte-specific reagents with BD directigen for detection of respiratory syncytial virus

Detection of respiratory viruses in sputum from adults by use of automated multiplex PCR

Comparison of anyplex II RV16 with the xTAG respiratory viral panel and seeplex RV15 for detection of respiratory viruses

Evaluation and implementation of FilmArray version 1.7 for improved detection of adenovirus respiratory tract infection

Comparison of the FilmArray assay and inhouse real-time PCR for detection of respiratory infection, Scand

Respiratory virus detection in immunocompromised patients with FilmArray Respiratory Panel compared to conventional methods

Comparison of two multiplex methods for detection of respiratory viruses: FilmArray RP and xTAG RVP

The authors would like to thank biostatistician, Ms Carmen Jia Wen Kam for performing the statistical analysis. This study was supported by a grant from the National Medical Research Council, Singapore M. Chan et al.Journal of Clinical Virology 106 (2018) [13] [14] [15] [16] [17]