key: cord-0735781-rey2m4nq authors: Steensels, Deborah; Reynders, Marijke; Descheemaeker, Patrick; Curran, Martin D.; Jacobs, Frédérique; Denis, Olivier; Delforge, Marie-Luce; Montesinos, Isabel title: Performance evaluation of direct fluorescent antibody, Focus Diagnostics Simplexa™ Flu A/B & RSV and multi-parameter customized respiratory Taqman® array card in immunocompromised patients date: 2017-03-30 journal: J Virol Methods DOI: 10.1016/j.jviromet.2017.03.013 sha: 4861039a642043488e2794c5141e99915d5366d7 doc_id: 735781 cord_uid: rey2m4nq BACKGROUND: Molecular assays for diagnosis of Flu A, Flu B, and RSV with short turn-around-time (TAT) are of considerable clinical importance. In addition, rapid and accurate diagnosis of a large panel of viral and atypical pathogens can be crucial in immunocompromised patients. OBJECTIVES: First, to evaluate the performance of the Simplexa™ Direct assay system in comparison with direct fluorescent antibody (DFA) and customized Taqman® Array Card (TAC) testing for RSV, Flu A, and Flu B in immunocompromised patients. Second, to evaluate different algorithms for the detection of respiratory pathogens in terms of cost, turn-around-time (TAT) and diagnostic yield. STUDY DESIGN: We collected 125 nasopharyngeal swabs (NTS) and 25 BAL samples from symptomatic immunocompromised patients. Samples for which Simplexa™ and TAC results were discordant underwent verification testing. The TAC assay is based on singleplex RT-PCR, targeting 24 viruses, 8 bacteria and 2 fungi simultaneously. RESULTS: The overall sensitivity was significantly lower for DFA testing than for the two molecular methods (p < 0.05). Performance characteristics of Simplexa™ testing were not significantly different compared to TAC testing (p > 0.1). For BAL samples only, the sensitivity and specificity of the Simplexa™ assay was 100%. In total, 6.7, 16 and 18% of samples were positive for Flu A, Flu B or RSV by DFA, Simplexa™ and TAC testing, respectively. When considering not only these pathogens but also all results for TAC, the method identified 93 samples with one or more respiratory pathogens (62%). A co-infection rate of 15.3% was found by TAC. The estimated costs and TAT were 8.2€ and 2 h for DFA, 31.8€ and 1.5 h for Simplexa™ and 55€ and 3 h for TAC testing. CONCLUSIONS: Performing the Simplexa™ test 24 h a day/7 days a week instead of DFA would considerably improve the overall sensitivity and time-to-result, albeit at a higher cost generated in the laboratory. Performing the TAC would increase the diagnostic yield and detection of co-infections significantly. Viral respiratory infections are a leading cause of morbidity and mortality, especially during winter months (Mahony et al., 2011) . Accurate clinical diagnosis of acute viral respiratory infection is challenging because of overlap of symptoms associated with various viruses and overlap with symptoms associated with other illnesses. Flu A, Flu B and RSV are clinically most important since they are the most frequently encountered, they cause substantial disease burden and a targeted treatment exists. Using a first-line method with a high sensitivity and short TAT to guarantee a result 24 h a day/7 days per week for these pathogens could be of considerable clinical importance. Many commercial as well as in-house developed assays are available, including the Simplexa™ Flu A/B & RSV Direct assay system (FDAcleared), a real-time RT-PCR system that enables the direct amplification, detection and differentiation of Flu A, Flu B and RSV virus RNA from unprocessed nasopharyngeal swabs (no separate sample extraction steps are required). This system could also be used as a point-of- In addition, rapid and accurate diagnosis of a large panel of viral and atypical pathogens could be crucial for an appropriate and sometimes life-saving clinical management in specific patient populations such as immunocompromised patients. At the University hospital Erasme, a recent study showed a relatively low positivity rate of routine conventional tests with direct fluorescent antibody (DFA) assays and viral culture for the detection of respiratory viruses in symptomatic immunocompromised patients (Steensels et al., 2015) . In addition, more than half of all Flu and RSV positive samples were missed. The study compared conventional testing with a customized Taqman® Array Card (TAC) assay, a microfluidic technology based on singleplex, reverse transcription real-time PCR, targeting 24 viruses, 8 bacteria and 2 fungi simultaneously. The goals of this study are, first, to compare the sensitivity and specificity of Simplexa™ for the detection of RSV, Flu A, and Flu B in comparison to DFA, and TAC testing. Second, to compare the cost, turnaround-time (TAT) and diagnostic yield of different algorithms for the detection of respiratory pathogens in this group of immunocompromised patients. And finally, we aim to test the Simplexa™ assay on BAL samples, which has only been validated on nasopharyngeal swabs so far. This study was approved by the ethical committee of the Erasme Hospital. In parallel with previously published study performed during winter season 2014-2015 (Steensels et al., 2015) , we included adult immunocompromised patients with symptoms of an upper or lower respiratory tract infection from November 2015 to April 2016. Based on review of the electronic medical records, subjects were enrolled in five different patient groups according to underlying conditions. For detailed composition of disease groups, see Table 1 . After conventional and Simplexa™ testing, the samples were aliquoted and stored at −80°C until TAC testing. DFA respiratory virus tests and viral culture were performed as described previously (Steensels et al., 2015) . The Simplexa™ Flu A/B & RSV (Focus Diagnostics, Cypress, CA) testing was performed according to manufacturer guidelines, on unprocessed NTS and BAL samples, on the same day as the routine conventional testing. In the case of an invalid result, the sample was diluted 1:2 and rerun. The Simplexa™ assay does not include a sample quality control. Patient samples were retrospectively tested with a customized TAC respiratory panel (v10.0 premarket version Cambridge-Brugge) which included testing for the pathogens shown in Figure 1 (Supplementary file). Two human DNA genes are included as cellular (sample quality) controls. Nucleic acids were extracted on the QiaSymphony (Qiagen, Valencia, CA) using the DSP viral pathogen midi kit and the TACs were run on QuantStudio 7 (Thermo Fisher Scientific, Life Technologies, Carlsbad, CA). The detailed procedure was described previously (Steensels et al., 2015) . A consensus of the two molecular tests was considered as the golden standard. Discordant Simplexa™ and TAC assay results were verified using the multiplex real-time PCR assay FTD Flu/HRSV (Fast-track Diagnostics) according to manufacturing guidelines, on the nucleic acid extract that was used for TAC testing. This confirmatory test was chosen as an independent molecular test for Flu A, Flu B and RSV which targets different genes from those targeted by Simplexa™ and TAC. DFA, Simplexa™ and TAC testing were compared using the exact two-sided McNemar's test. All statistical analyses were performed using the MedCalc software (Mariakerke, Belgium). Total cost per sample and cost per pathogen were calculated, including all necessary consumables and personnel costs. The cost of consumables was based on negotiated prices for our laboratory. In contrast to official list prices, these prices reflect the reality and will be taken into account by laboratories in their ultimate choice between different methods. For each technique, three different lab technicians were timed during sample manipulation on a subset of samples included in the study. The mean hands-on time was used to calculate personnel costs, and the mean time to result was used to estimate turnaround-time. One hundred-twenty five nose-throat samples (83.3%) and 25 (16.7%) BAL samples were obtained from 150 immunocompromised patients who underwent respiratory pathogen testing for symptoms of upper or lower respiratory tract infection. Patient characteristics are shown in Table 1 . Table 2 shows the performance characteristics of DFA, Simplexa™ and TAC testing for RSV, Flu A and Flu B. As expected, the overall sensitivity was significantly lower for DFA testing than for the two molecular methods, even when the 42 inadequate samples were excluded from the analysis (p = 0.0044). These 42 samples were uninterpretable by DFA due to insufficient cells derived from the respiratory sample. However, when considering results for each patho- * lung + kidney (n = 2), lung + heart (n = 2), kidney + heart (n = 1), lung + kidney + heart (n = 1). gen separately, the difference in sensitivity between DFA and molecular methods was only statistically significant for Flu A. The Simplexa™ direct test missed one RSV, one Flu A and two Flu B positive samples in comparison to the TAC assay and verification PCR. The amplification curves of these Simplexa™ results were completely flat. One sample was found strongly positive for Flu A by Simplexa™ (Ct 15), but was negative by DFA, TAC and verification testing. Unfortunately, there was no sample left to repeat Simplexa™ testing. Nevertheless, the difference in individual and overall sensitivity and specificity was not significant compared to TAC testing (p > 0.1). Two samples tested as invalid using Simplexa™ due to internal control failure. Since this could be due to presence of an inhibitor, the samples were diluted and rerun, giving valid results. For BAL samples, the sensitivity and specificity of the Simplexa™ assay was 100%. In total, DFA identified 10 samples (6.7%) with one pathogen each among 150 samples. The Simplexa™ testing found 24 (16%) and TAC 27 (18%) samples positive with one pathogen each. For nose-throat swabs only, DFA, Simplexa™ and TAC testing were positive in 5.6, 15.2 and 17.6%, respectively. For BAL samples, this was 12, 20 and 20%, respectively. An additional objective of the study was to evaluate these different methods for the detection of respiratory pathogens in terms of diagnostic yield. When considering not only Flu A, B and RSV but all results for the TAC, the method identified 93 samples with one or more respiratory pathogens (62%). TAC testing was positive for other pathogens than RSV, Flu A and B in more than half (54%) of Simplexa™ negative samples. Detailed results are shown in Figure 2 (Supplementary file). A co-infection rate of 15.3% was found by TAC (21 samples positive for 2 pathogens, 2 samples positive for 4 different pathogens). Those most frequently involved in co-infections were Coronaviruses (n = 12), HSV (n = 11), Flu A (n = 8) and CMV (n = 5). Out of 24 Simplexa™ positive samples, TAC found 7 samples positive for 2 pathogens (all positive for Flu A plus rhinovirus (n = 1), coronavirus (n = 2), HSV (n = 1), CMV (n = 1), Aspergillus species (n = 1), P. jirovecii (n = 1)). Furthermore, the estimated costs and turn-around-time of DFA, Simplexa™ and TAC testing were evaluated and are shown in Table 3 . Important to note is that these costs are valid for our laboratory, but not necessarily for another lab since this depends greatly on activity and can vary from one country to another. Remarkably, the TAC technology is the most expensive method in terms of cost per sample, but it is the least expensive in terms of cost per parameter. Even though the technical TAT of the different methods is comparable, the true timeto-result can vary greatly depending on organizational aspects. Based on our results, performing a molecular method such as the Simplexa™ or TAC method instead of DFA would be necessary to obtain an acceptable overall sensitivity, albeit at a higher cost generated in the * 42 of 150 samples tested by DFA were reported as inadequate but were counted here as DFA negative because no results were generated; seven DFA-inadequate samples were positive by Simplexa™, 9 were positive by TAC for RSV, Flu A or Flu B. Reference method was consensus of two molecular tests, and multiplex real-time PCR assay FTD Flu/HRSV (Fast-track Diagnostics) for discordant results. laboratory. Performing the TAC would increase the diagnostic yield and detection of co-infections significantly. In line with our previous findings (Steensels et al., 2015) , this study confirms the limited sensitivity of direct fluorescent antibody testing in comparison to molecular methods. Although sampling was performed according to consistent procedure, an important number of samples were uninterpretable by DFA which indicates poor sample quality. Sample quality is very important for traditional methods such as DFA because of their lower sensitivity in comparison to molecular methods and since they require the presence of live infected cells and/or live pathogens. Several papers were published on Simplexa™ Flu A/B & RSV, using an extracted nucleic acid and/or direct sample testing and comparing to conventional methods or molecular tests (Alby et al., 2013; Hindiyeh et al., 2013; Selvaraju et al., 2014; Ko et al., 2013; Woodberry MW1 et al., 2013; Landry and Ferguson, 2014; Svensson et al., 2014) . In this study, the Simplexa™ Flu A/B & RSV Direct assay system showed comparable performance characteristics for Flu A, Flu B and RSV in comparison with TAC testing. The four samples missed by Simplexa™ in this study had a low viral load (median cycle threshold of 34), suggesting a slightly lower sensitivity of the assay in comparison with that of the TAC assay. A possible explanation could be the simple lysis of the sample with insufficient viral RNA return compared to a real extraction of the nucleic acids. The TAC technology targets a large panel of viral, bacterial and fungal respiratory pathogens. Logically, the diagnostic yield of this multi-pathogen molecular method was much higher compared to that of DFA and Simplexa™ testing which target only a limited number of pathogens. Also, in almost one-third of the Simplexa™ positive samples, TAC detected Flu A and the presence of an additional pathogen. Respiratory co-infections, in particular, the association of Flu A with (an)other viral, bacterial or fungal pathogen(s), could be associated with longer hospital stay, a longer ICU stay and even higher mortality (Shah et al., 2016; Echenique et al., 2013; Crotty et al., 2015) . Detecting these co-infections could have an important clinical impact. Performing a method that allows a syndromic approach on all samplesor at least those coming from specific patient populations such as those included in this studywould be diagnostically valuable. When comparing TAC results from this study with those from previous winter season (Steensels et al., 2015) , the overall positivity rate was comparable (62 versus 66.4%). Although the estimated technical TAT of DFA, Simplexa™ and TAC testing are comparable, the true time-to-result can vary greatly depending on organizational aspects. DFA requires experienced lab technicians and is labor intensive. In our lab, DFA is only performed during regular laboratory operating hours and not in weekends, so it can take up to 2-3 days to have a result if the sample is taken on a Friday evening. On the other hand, the Simplexa™ direct assay system is an all-in-one system (no separate sample extraction steps needed) which requires minimal hands-on-time and results are available in about 1 h 15 min from the start of a test run. The assay is easy to perform, without the need of specialized equipment or experienced molecular lab technicians, so it can easily be performed 24 h a day/7 days per week, and even as point-of-care test. Performing the Simplexa™ assay as first-line test would guarantee a fast result for Flu A, Flu B and RSV, regardless of the sampling timing. Having a fast negative result for these pathogens could also result in cost savings thanks to reduced time spent in isolation, shorter length of inpatient stay and a faster discontinuation of empiric antiviral therapy such as oseltamivir (Pettit et al., 2015) . For TAC testing, a separate nucleic acid extraction step is required, as are specialized equipment and lab technicians, and samples cannot be treated individually in the TAC format, so a same-day-result cannot always be guaranteed. Nevertheless, laboratories with a high respiratory sample turn-over could ensure an acceptable TAT, providing their clinicians with a much more complete result. Even though molecular methods still have a significant higher cost than conventional methods such as DFA, it is important to point out the reasonable cost of the TAC technology considering the large number of agents detectable. The TAC method is actually the least expensive in terms of cost per pathogen. In addition, the gain in sensitivity and diagnostic yield could improve patient outcomes and could reduce overall costs for the hospital, making it cost-effective (Olofsson et al., 2011; Mahony et al., 2009) . The added cost of testing with a multipathogen panel such as the TAC assay should be weighed against the potential benefits of increased identification of the causative pathogen (s). Even though targeted antiviral treatment not always exists, other treatment options such as antifungals or corticosteroids could be important. In addition, the diagnosis of a viral respiratory infection can potentially reduce the use of antibiotics (Gelfer et al., 2015; Falsey et al., 2013; Rogers et al., 2015) , it can assist infection control practitioners in providing appropriate infection control measures (e.g. droplet and/or contact precautions and considerations in creating cohorts) and it can stop the more thorough search for a diagnosis avoiding unnecessary medical procedures. Moreover, co-infections are routinely reported using multi-pathogen molecular assays (Chung et al., 2007; Mahony et al, 2007; Pierangeli et al., 2007) . Testing for multiple pathogens in a single assay may also provide a savings of resources including technologist time and expendables and cost less than the aggregate cost of performing multiple uniplex PCR tests. And finally, these tests will contribute to our understanding of the epidemiology and clinical importance of respiratory tract infections. Many commercial as well as in-house developed assays exist today which offer a syndromic approach for the detection of respiratory pathogens. Each system has its advantages and disadvantages, and each user should determine which system is appropriate for their specific laboratory and/or patient population. However, more studies will be needed to determine populations or situations in which these methods would be the most useful in order to optimize their clinical and financial impact (Vallières and Renaud, 2013) . Several testing algorithms could be proposed. Ideally, the TAC technology or another comparable multi-pathogen method would be performed on all patient samples with a guaranteed result within 24 h. Another option would be to perform the Simplexa™ assay 24 h a day/7 days a week as a first-line test in winter period for all patients, and to reserve the more complete TAC technology for specific patient populations such as immunocompromised patients, patients in ICU, neonates, etc. There are some limitations to our study. In contrast to DFA and Simplexa™ testing, TAC and verification testing was performed retrospectively. Therefore, the quality of the samples could have deteriorated due to the freeze-thaw cycle before TAC and verification testing. This study was also limited by the lack of verification testing on all samples. Finally, the cost estimation of the different methods is valid for our laboratory, but would not necessarily be the same in another laboratory. In summary, the Simplexa™ direct assay system is a sensitive and specific method for the detection of Flu A, B and RSV. Performing a molecular method such as the Simplexa™ test 24 h a day/7 days a week instead of DFA would considerably improve the overall sensitivity and time-to-result, albeit at a higher cost generated in the laboratory. Performing the TAC would increase the diagnostic yield and detection of co-infections significantly, and this in turn could be crucial for appropriate clinical management in special populations such as immunocompromised patients. Comparative evaluation of the anosphere Verigene RV+ assay and the Simplexa Flu A/B & RSV kit for detection of influenza and respiratory syncytial viruses Respiratory picornavirus infections in Korean children with lower respiratory tract infections. Scand Epidemiology, co-infections, and outcomes of viral pneumonia in adults: an observational cohort study Clinical characteristics and outcomes in hospitalized patients with respiratory viral co-infection during the 2009 H1N1 influenza pandemic Bacterial complications of respiratory tract viral illness: a comprehensive evaluation The clinical impact of the detection of potential etiologic pathogens of community-acquired pneumonia Evaluation of Simplexa Flu A/B & RSV for direct detection of influenza viruses (A and B) and respiratory syncytial virus in patient clinical samples Evaluation of the Simplexa Flu A/B and RSV test for the rapid detection of influenza viruses Comparison of Simplexa Flu A/B & RSV PCR with cytospin-immunofluorescence and laboratory-developed TaqMan PCR in predominantly adult hospitalized patients Establishing the epidemiology of respiratory virus infections using molecular technology Cost analysis of multiplex PCR testing for diagnosing respiratory virus infections Molecular diagnosis of respiratory virus infections PCR for detection of respiratory viruses: seasonal variations of virus infections Comparison of turnaround time and time to oseltamivir discontinuation between two respiratory viral panel testing methodologies Detection and typing by molecular techniques of respiratory viruses in children hospitalized for acute respiratory infection in Rome, Italy Impact of a rapid respiratory panel test on patient outcomes Influenza and respiratory syncytial virus detection in clinical specimens without nucleic acid extraction using FOCUS direct disc assay is substantially equivalent to the traditional methods and the FOCUS nucleic acid extractiondependent RT-PCR assay Bacterial and viral co-infections complicating severe influenza: incidence and impact among 507 U.S. patients, 2013-14 Clinical evaluation of a multi-parameter customized respiratory TaqMan® array card compared to conventional methods in immunocompromised patients Performance of the Simplexa™ Flu A/B & RSV Direct Kit on respiratory samples collected in saline solution Clinical and economical impact of multiplex respiratory virus assays Comparison of the Simplexa FluA/B & RSV direct assay and laboratory-developed real-time PCR assays for detection of respiratory virus We thank the technicians of our virology and molecular microbiology laboratory for their technical assistance during this study and Launch Diagnostics for providing the 3M™ integrated cycler, Simplexa™ Flu A/B & RSV Direct assay kits and a FTD Flu/HRSV kit (Fast-track Diagnostics) for verification testing. Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jviromet.2017.03.013.