key: cord-299664-nexq5ntj
authors: Butt, S.A.; Maceira, V.P.; McCallen, M.E.; Stellrecht, K.A.
title: Comparison of three commercial RT-PCR systems for the detection of respiratory viruses
date: 2014-08-18
journal: J Clin Virol
DOI: 10.1016/j.jcv.2014.08.010
sha: 
doc_id: 299664
cord_uid: nexq5ntj

BACKGROUND: Due to the insensitivity of rapid tests for respiratory viruses, nucleic acid amplification tests are quickly becoming the standard of care. OBJECTIVES AND STUDY DESIGN: The performance of the FilmArray Respiratory Panel (RP) and Verigene RV+ (RV+) were compared in a retrospective analysis of 89 clinical specimens previously determined to be positive for the following viruses by our test of record, Prodesse (Pro): influenza A (29, FluA), influenza B (13, FluB), respiratory syncytial virus (12, RSV), human metapneumovirus (10, hMPV), parainfluenza (14, PIV), and adenovirus (10, AdV). Samples positive for influenza A, B or RSV were tested by both methods, while the remainder were tested by RP only. True positives were defined as positive by two or more assays. RESULTS: Limit of detection (LOD) analyses demonstrated Pro had the lowest LOD for all FluA strains tested, PIV1, PIV2 and AdV; RV+ had the lowest LOD for FluB; and RP had the lowest LOD for RSV, PIV3 and hMPV. Of the 55 samples tested by RV+, all 54 true positive samples were positive by RV+. Of the 89 samples tested by RP, 85 of the 88 true positive samples were positive by RP. From these results, the overall sensitivities for influenza A, B and RSV were 100% and 98% for RV+ and RP, respectively. The overall sensitivity of RP for all viruses was 97%. CONCLUSIONS: In summary, these systems demonstrated excellent performance. Furthermore, each system has benefits which will ensure they will all have a niche in a clinical laboratory.

Respiratory viruses are major contributors to morbidity and mortality in all age groups [1] [2] [3] [4] . Rapid identification is important for both therapeutic and infection control purposes. Traditional rapid viral diagnoses such as immunoassays produce quick results and are simple to perform; however, have sub-optimal sensitivity [5] [6] [7] [8] . Nucleic acid amplification assays (NAAT), which are relatively rapid and have greatly enhanced sensitivity, are becoming the method of choice [9] . More recently, less complex sample-toresult molecular platforms such as Xpert Flu Assay (Cepheid), RP (BioFire Diagnostics), RV+ (Nanosphere), eSensor Respiratory Viral Panel (GenMark Diagnostic) and Liat Influenza A/B (Iquum) have been introduced into the marketplace which have the potential to Specimens, collected between January 2008 and February 2012 and stored at −80 • C, were selected based on results from original testing. Samples were paired for differing viruses and tested as two-sample pools. Discrepant samples were retested individually, while samples discrepant for adenovirus were refereed by viral culture as previously described [22] . Specimens were defined as true positive if they were positive by two or more assays.

The NATtrol TM RVP Verification Panel (ZeptoMetrix, Buffalo, NY) was used for limit of detection analysis by serial dilution (1:10) in normal human plasma for endpoint detection. Viral nucleic acid concentration determinations were based on quantified control viral RNA (Hologic Gen-Probe Inc., San Diego, CA) using Pro.

The Verigene ® RV+ test (Nanosphere, Northbrook, IL) combines conventional PCR with a gold nanoparticle probes detection system. Initially, Test Cartridges, Extraction Trays, and Amplification Trays were loaded into Processor SP after thawing the Amplification Trays for 10-30 min. Specimen in viral transport (0.2 ml) was added to the extraction tray. Automated sample lysing and target amplification of approximately 10% of the lysate was performed on the Verigene Processor SP as a multiplex assay for the detection of FluA, 09H1, sH3, sH1, FluB, RSV-A and RSV-B. Within 5 min of completion of amplification, the RV+ test cartridge was removed from the processor, opened to release the hybridization slide, which was then inserted into the Verigene reader. Results were reported "detected" or "not detected" for each target, or a message of "No Call" (NC) was reported in the event of assay failure, such as an internal control failure. NC samples were retested to obtain a valid result.

The FilmArray Respiratory Panel v1.6 (BioFire Diagnostics Inc., Salt Lake City, UT), which at the time of the clinical evaluation, was a comprehensive panel for the detection of 15 respiratory viral targets (AdV, coronaviruses [CoV] HKU1 and NL63, FluA, 09H1, sH3, sH1, FluB, hMPV, human rhinovirus/enterovirus (HRV), PIV1-4, and RSV) by real-time PCR. The closed system integrates extraction, amplification, and detection into a simple, hands free test. Briefly, 0.3 ml of specimen in viral transport was added to 0.5 ml sample buffer. The FilmArray pouch was injected with 0.5 ml of hydration solution and 0.3 ml of the diluted sample. The pouch was inserted into the FilmArray and within 70 min results were reported as detected, not detected or invalid for each target. Invalid samples were retested to obtain a valid result. In addition, a result of equivocal for FluA occurred when only one of the two FluA specific targets was amplified.

The Prodesse assays for respiratory viruses (Hologic|Gen-Probe, San Diego, CA) are a series of either multiplex or uniplex, realtime PCR assays, including ProFlu+ (detects FluA, FluB and RSV), ProFAST+ (differentiation of 09H1, sH3, and sH1), ProAdeno+, ProHMPV+ and ProParaflu+ (differentiates PIV1-3). Briefly, viral RNA was extracted from 0.2 ml of specimen in viral transport, along with a universal internal control, using NucliSens Magnetic Extraction Reagents and the easyMAG extractor (bioMerieux, Durham, NC) and eluted to a volume of 50 l. RNA extract, 5 l, was amplified on SmartCyclers (Cepheid, Carlsbad, CA) with the appropriate assay as requested for clinical purposes.

Probit analyses for the limit of detection with a 95% probability of detection were performed using SPSS version 8.0 (IBM, Armonk, NY). The chi-square test was performed using Microsoft Excel 2013 (Redmond, WA). P values of 0.05 were considered statistically significant. Confidence intervals (CI) were determined using Graph-Pad (La Jolla, CA).

Pro had the lowest LOD for all FluA viruses, while the LOD for RV+ was within 0.5 log copies/ml of Pro for the 09H1 and sH1, and more than 1 log greater for sH3 (Table 1) . RV+ had the lowest LOD for FluB, but RP had the lowest LOD for RSV. The analytical sensitivity was mixed as to whether Pro or RP was more sensitive for PIV, hMPV and AdV.

The determination of clinical performance involved 89 specimens previously positive for respiratory viruses by Pro including one sample originally positive for sH3 that became negative upon archiving ( Table 2 ). The 54 samples positive for FluA, FluB or RSV and the negative sample were tested by both RP and RV+ as two sample pools. Initially, 24 sample pools were positive for the expected viruses by RV+, 1 was positive with 3 viruses, and 3 resulted as No Call. The NC pooled samples (6) were retested individually; five additional samples were positive for the expected virus while the sixth was positive for two viruses. The discrepant samples, which involved a pool of a sH1 sample and a sH3 sample additionally positive for 09H1 and a sH3 sample additionally positive for FluB, were retested individually. Upon repeat each sample was positive only for the expected virus.

When testing the same samples with RP, 51 of the 54 positive specimens were positive for the expected virus. One sample from a patient with dual RSV and AdV infection was only positive for RSV and one 09H1 sample was equivocal for FluA. In addition, one pool was positive for three viruses (an AdV sample and a sH3 sample additionally positive for 09H1). Again, the samples from this discrepant pool were tested individually. Upon repeat each sample was positive only for the expected virus. The additional 33 specimens positive for PIV, hMPV or AdV were tested with RP, which TP  TN  FP  FN  Invalid  TP  TN  FP  FN  Invalid   09H1N1  15  14  1 a  1 b  15  1 a  sH3N2  12  12  12  1  sH1N1  2  2  2  Flu B  1 3  13  13  1 a  3  RSV  11  11  11  2  hMPV  10 detected the appropriate virus in all but one sample positive for AdV, for a total AdV detection rate of 9/11 AdV positive samples. In addition, AdV was originally detected in all positive specimens by viral culture. From these findings the sensitivities for FluA, B and RSV were 100% and 98% for RV+ and RP, respectively (95% CI = 92.1-100% and 89.3-100%, respectively). This difference in sensitivity was not statistical significance (two-tailed P value = 0.3129). The overall sensitivity of RP for all viruses was 97% (95% CI = 90.1-99.3%). The initial failure rates for these two assays with clinical samples were 10.7% and 0% for RV+ and RP. However, when including the LOD analysis, the failure rates were 9.7% and 1%, respectively. All failed results resolved with repeat testing.

Pro. Both systems are designed for on demand testing. The most cumbersome step for both was the test preparation, including sample addition. RP has an extra step of sample dilution, but the hands-on-time (HOT) was only 5 min (Table 3) . RV+ had the additional steps of thawing for frozen reagents, loading the cartridges and trays, and one return to the instrument post-amplification for cartridge reading, for a total HOT of 21 min. Pro is more manual, including nucleic acid extraction and amplification reagent preparation, but has higher throughput capabilities. Cost analysis demonstrates that RP is the most expensive per test, however there are multiple targets reportable and billable per test, hence the cost/reportable is considerably lower than other methods.

Since their introduction in 2011, there have been numerous studies regarding the performance of RV+ [10, 15, 23] and RP [11, 12, 14, 24] ; however, only one compared the systems side by side [25] Interestingly, in this study both RP and Pro demonstrated significantly superior sensitivity for the detection of FluA as compared to RV+. Hence, it is surprising that in our study the LOD for all FluA strains tested were much lower with RV+ than RP; and both systems demonstrated excellent sensitivity with clinical specimens. Furthermore, Van Wesenbeeck et al. [25] evaluated the LOD of the different assays based on the virus titers in the clinical samples and were able to determine an LOD for RP and Pro for FluA in the range of 3.5-4.5 log RNA copies/ml, which is similar to our findings for Pro but not for RP. One potential reason for the discrepancy between studies is that all specimens in the Belgium study were collected between February and March 2012 and almost all FluA specimens subtyped as sH3. Hence, RV+ may perform suboptimally with the strain of sH3 circulating in Belgium during the winter of 2012. Our clinical samples would have been representative of the typical genetic diversity seen over a period of 4 years. In addition, these authors reported a higher than typical percentage of invalid results with RV+ (15.8%), which may indicate a system problem or improper handling of reagents which is critical for optimal performance.

Other samples in which the expected virus was not detected were two containing AdV. In our study, the LOD with RP for the virus strain in the NATtrol TM RVP Verification Panel was similar to Pro. However, it has been established that RP version 1.6 had low sensitivity for most group C AdV as well as select serotypes with all other groups [11, 15] , whereas the version 1.7 kits demonstrated significant improvements in the range of AdV detected [26] . Interestingly, these two samples were still negative with version 1.7 kits; however, one was now negative with Pro, and the other had a Ct value of more than 37 which is very close to the LOD.

One limitation of these studies is that the clinical performance of Pro cannot be assessed as sample selection was based on Pro test results. In addition, assay specificity cannot be accurately assessed because the population was artificially skewed toward positive specimens rather than the annual respiratory virus positivity rate of 29% that we typically see at our institution. However, false positive results were seen with both RP and RV+. It is possible that the false positive results were an artifact of testing samples in pools. Contamination is another possible source for false positive results. RP is a closed, real-time PCR; hence there is no opportunity for contamination with residual amplicon. Of course, carry over contamination between samples is possible, but unlikely since the samples were all tested individually on one instrument. Although RV+ does involve post-amplification processing, the manufacturer has indicated that amplicons are not released from their cartridges.

In addition to the enhanced workflow, both RP & RV+ have many advantages and disadvantages (Table 4 ). RV+ allows for a relatively simple, highly sensitive option for detecting FluA, B and RSV at a more economical price. But since this system does not detect other respiratory viruses, a second system would be required if a full diagnostic profile is needed. RP has room temperature reagent storage and a total test time of little over an hour, making this system conducive for STAT testing. RP also detects a broad range of organisms, including HRV and CoV. After our studies were performed, the manufacturer added two more strains of CoV, as well as three bacterial targets. Both systems only test one specimen at a time per instrument; hence, multiple instruments are necessary for increased throughput.

Lastly, Pro offers the advantage of high throughput testing and a broad menu with the ability to select different analytes. It is difficult to compare the labor costs for this system with the two on-demand systems as the labor per specimen needed for Pro is highly dependent on the number of specimens tested. None-the-less, all three systems offer excellent sensitivity and each system offers different benefits to fit different laboratory needs. It is apparent that these highly automated assays demonstrate similar performance and greatly reduce the hands-on time.

We acknowledge the kind generosity of Nanosphere and BioFire Diagnostics Inc., for providing the reagents for this study.

Mortality associated with influenza and respiratory syncytial virus in the United States

Respiratory syncytial virus infection in elderly and high-risk adults

Influenza-and respiratory syncytial virus-associated mortality and hospitalisations

Detection of respiratory viruses by molecular methods

Clinical performance of the 3M rapid detection flu A+B test compared to R-Mix culture, DFA and BinaxNOW influenza A&B test

Low sensitivity of rapid diagnostic test for influenza

Diagnosis of 2009 pandemic influenza A (pH1N1) and seasonal influenza using rapid influenza antigen tests

Performance of a rapid antigen test (Binax NOW(R) RSV) for diagnosis of respiratory syncytial virus compared with real-time polymerase chain reaction in a pediatric population

Update on influenza diagnostics: lessons from the novel H1N1 influenza A pandemic

Real-time detection of influenza a, influenza B, and respiratory syncytial virus a and B in respiratory specimens by use of nanoparticle probes

Comparison of the FilmArray Respiratory Panel and Prodesse real-time PCR assays for detection of respiratory pathogens

Comparison of two multiplex methods for detection of respiratory viruses: FilmArray RP and xTAG RVP

Evaluation of the Cepheid Xpert Flu Assay for rapid identification and differentiation of influenza A, influenza A 2009 H1N1, and influenza B viruses

Comparison of the Idaho Technology FilmArray system to real-time PCR for detection of respiratory pathogens in children

Comparative evaluation of the Nanosphere Verigene RV+ assay and the Simplexa Flu A/B & RSV kit for detection of influenza and respiratory syncytial viruses

Laboratory diagnosis of Clostridium difficile infections: there is light at the end of the colon

Clinical accuracy of a PLEX-ID flu device for simultaneous detection and identification of influenza viruses A and B

Comparison of the Biofire FilmArray RP, Genmark eSensor RVP, Luminex xTAG RVPv1, and Luminex xTAG RVP fast multiplex assays for detection of respiratory viruses

Comparison of xTAG respiratory virus panel and Verigene respiratory virus plus for detecting influenza Virus and respiratory syncytial virus

Evaluation of three influenza A and B real-time reverse transcription-PCR assays and a new 2009 H1N1 assay for detection of influenza viruses

Influenza and respiratory syncytial virus detection in clinical specimens without nucleic acid extraction using FOCUS direct disc assay is substantially equivalent to the traditional methods and the FOCUS nucleic acid extraction-dependent RT-PCR assay

A one-step RT-PCR assay using an enzyme-linked detection system for the diagnosis of enterovirus meningitis

The clinical utility of a near patient care rapid microarray-based diagnostic test for influenza and respiratory syncytial virus infections in the pediatric setting

Comparison of the Luminex xTAG RVP Fast assay and the Idaho Technology FilmArray RP assay for detection of respiratory viruses in pediatric patients at a cancer hospital

Comparison of the FilmArray RP, Verigene RV+, and Prodesse ProFLU+/FAST+ multiplex platforms for detection of influenza viruses in clinical samples from the 2011-2012 influenza season in Belgium

Evaluation and implementation of FilmArray version 1.7 for improved detection of adenovirus respiratory tract infection

We thank the Molecular Diagnostics staff for their help with collecting respiratory specimens and performing Pro.

The authors declare no competing interests.

Obtained from Albany Medical Center.