key: cord-318021-4qrf5m8s
authors: Wolfensberger, A.; Jakob, W.; Faes Hesse, M.; Kuster, S.P.; Meier, A.H.; Schreiber, P.W.; Clack, L.; Sax, H.
title: Development and validation of a semi-automated surveillance system—lowering the fruit for non-ventilator-associated hospital-acquired pneumonia (nvHAP) prevention()
date: 2019-03-25
journal: Clin Microbiol Infect
DOI: 10.1016/j.cmi.2019.03.019
sha: 
doc_id: 318021
cord_uid: 4qrf5m8s

OBJECTIVES: Conducting manual surveillance of non-ventilator-associated hospital-acquired pneumonia (nvHAP) using ECDC (European Centre for Disease Prevention and Control) surveillance criteria is very resource intensive. We developed and validated a semi-automated surveillance system for nvHAP, and describe nvHAP incidence and aetiology at our hospital. METHODS: We applied an automated classification algorithm mirroring ECDC definition criteria to distinguish patients ‘not at risk’ from patients ‘at risk’ for suffering from nvHAP. ‘At risk’-patients were manually screened for nvHAP. For validation, we applied the reference standard of full manual evaluation to three validation samples comprising 2091 patients. RESULTS: Among the 39 519 University Hospital Zurich inpatient discharges in 2017, the algorithm identified 2454 ‘at-risk’ patients, reducing the number of medical records to be manually screened by 93.8%. From this subset, nvHAP was identified in 251 patients (0.64%, 95%CI: 0.57–0.73). Sensitivity, negative predictive value, and accuracy of semi-automated surveillance versus full manual surveillance were lowest in the validation sample consisting of patients with HAP according to the International Classification of Diseases (ICD-10) discharge diagnostic codes, with 97.5% (CI: 93.7–99.3%), 99.2% (CI: 97.9–99.8%), and 99.4% (CI: 98.4–99.8%), respectively. The overall incidence rate of nvHAP was 0.83/1000 patient days (95%CI: 0.73–0.94), with highest rates in haematology/oncology, cardiac and thoracic surgery, and internal medicine including subspecialties. CONCLUSIONS: The semi-automated surveillance demonstrated a very high sensitivity, negative predictive value, and accuracy. This approach significantly reduces manual surveillance workload, thus making continuous nvHAP surveillance feasible as a pivotal element for successful prevention efforts.

Hospital-acquired pneumonia (HAP) is divided into two groups: ventilator-associated pneumonia (VAP) and non-ventilator-associated hospital-acquired pneumonia (nvHAP). HAP and lower respiratory tract infections were shown to be the most common hospital-acquired infections (HAIs), constituting a proportion of 26% [1, 2] . Almost two thirds of HAPs are nvHAPs [1, 2] . Although nvHAP is more frequent, and has been shown to be comparable in mortality and costs to VAP [3] , current research, prevention guidelines, and prevention efforts focus almost exclusively on VAP. In 2017, Ewan et al. called nvHAP a 'neglected disease' among HAIs [4] . As a fundamental first step to preventing nvHAP, they argued for obtaining accurate estimates of prevalence and incidence through prospective surveillance [4] .

Apart from point prevalence studies in which 0.5e1.1% of all included patients are reported to be affected by nvHAP on a given day [2, 5, 6] , literature on nvHAP prevalence or incidence is scarce. Weber et al. reported an HAP incidence of 0.37%, of which 44% were nvHAPs, by performing a time-consuming full manual surveillance from using the Centers for Disease Control and Prevention (CDC) surveillance definitions of 1988 [7] . Three authors reported nvHAP incidence rates between 0.12 and 2.28 per 1000 patient days by applying the 2013 CDC surveillance definition to patients with International Classification of Diseases (ICD-9-CM) codes for pneumonia not present on admission [8e10] . These studies may reflect incidence of nvHAP only very roughly, as positive predictive value and sensitivity of discharge diagnostic codes for identifying patients with nvHAP were shown to be as low as 35% and 59%, respectively [11] .

Manual surveillance is resource-intensive, and can be applied to only a limited patient population. Therefore, automated or semiautomated surveillance algorithms based on electronically available patient data represent a promising alternative to manual surveillance.

The IPC-team of the University Hospital Zurich (UHZ) saw the need for continuous surveillance of nvHAP as part of their hospitalwide HAI prevention programme. The primary aim of this study was to develop and validate a semi-automated nvHAP surveillance system, and to assess its sensitivity, negative predictive value and accuracy. Second, we aimed to describe incidence and aetiology of nvHAP to mitigate data scarcity in this field.

The study was conducted at UHZ (Switzerland), a 950-bed tertiary-care teaching hospital covering all medical specialties (n ¼ 21) except paediatrics and orthopaedics. The study period was January 1st to December 31st 2017. We included all patients who were discharged or passed away during this period. We excluded neonatology patients and newborns in the obstetrics department due to differences in the nvHAP surveillance definition for this patient population.

Routinely collected electronic data were used. In our hospital, all patient data are charted electronically via an electronic medical records (EMR) system. Selected data are stored in a clinical data warehouse.

The necessity for a formal ethical evaluation was waived by the Zurich Cantonal Ethics Commission (Req-2016-00623), based on the Swiss law on research on humans.

We used the European Centre for Disease Prevention and Control (ECDC) definition criteria for pneumonia, as also applied in the European point prevalence study [12] . The definition comprises radiological criteria, systemic symptoms (fever >38 C, leucopenia or leucocytosis) and pulmonary symptoms (e.g. cough, sputum production). Pneumonia is defined as hospital-acquired when symptoms start 48 h after admission. If an invasive respiratory device was present in the 48 h preceding symptom onset, the pneumonia is considered a VAP. For this study, we categorized aetiology in three groups according to microbiological sampling site: first, sputum, tracheal aspirate, or upper respiratory tract specimen; second, bronchoalveolar lavage, endobronchial aspirate, or tissue sample; and third, blood culture or antigen detection in blood or urine. We defined 'good-quality sputum and tracheal aspirate' as sputum or tracheal aspirate with <10 squamous epithelial cells (SECs) per low power field (LPF) in microscopic examination [13] . 'Possible fungal pneumonia' was defined when host factors and clinical criteria were met [14] . 'Place of acquisition' was determined as affiliation to department and ward 48 h before the first symptoms of nvHAP, unless a shorter incubation period was evident from patient history.

The automated aspect of the surveillance was developed by the infection prevention and control (IPC) team (AW, SPK, PS, HS) in collaboration with a data warehouse analyst (WJ) based on availability of electronic data and nvHAP definition. We applied an automated classification algorithm ( Fig. 1) [5] to distinguish patients 'not at risk' from those 'at risk' for nvHAP. Based on the ECDC case definition, patients were defined as 'at risk' if they had undergone at least one radiological procedure (x-ray or CT scan) of the chest fulfilling the following criteria: (a) radiological procedure performed 48 h after admission or at any time during hospitalization for patients readmitted within 28 days; (b) systemic symptoms in temporal relationship to radiological procedure; (c) pneumonia not a-priori excluded in the radiologists' report; (d) no invasive respiratory device constantly present during 48 h preceding radiological procedure. As microbiological criteria are not required for the ECDC nvHAP definition, we refrained from including microbiological criteria in the classification algorithm. The 'at-risk' patients were then manually screened for nvHAP, strictly applying the ECDC case definition [12] . Manual surveillance was performed by two skilled nurses of the IPC team (AM, MF) and double-checked by an experienced infectious disease physician (AW). Investigators manually collected data comprising 'possible fungal pneumonia', date of onset, and 'place of acquisition'.

To further reduce the number of patients and radiological procedures to manually evaluate, we adapted the algorithm during an iterative process. After a 3-month period with about 650 'at-risk' patients, the temporal relationship of systemic symptoms to radiological procedure was reduced from e5 and þ3 days to e3 and þ1 day. Additionally, radiological procedures whose reports contained key phrases ruling out pneumonia (see Supplement 1) were excluded. After adaptation, the number of 'at-risk' patients was reduced by 8% and sensitivity analysis showed no difference in the number of patients with nvHAP.

To validate the semi-automated surveillance, we chose full manual surveillance by a skilled IPC nurse, double-checked by an experienced infectious disease physician (both blinded to the results of the algorithm) as the reference standard. As resource considerations prevented the review of all EMRs, full manual surveillance was applied on three validation samples (VS1 to VS3): a random sample of 700 patients of the study population (VS1), all 637 patients of the study population having HAP according to ICD-10 discharge diagnostic codes (VS2), and 754 patients of the year 2016 from four distinct departments, comprising 165 patients with and 589 patients without HAP according to ICD-10 (VS3). Rootcause analysis was conducted on patients with nvHAP not classified as 'at-risk' patients.

To identify hospital areas associated with increased nvHAP rates, we grouped the 21 specialty departments into eight department groups: internal medicine and subspecialties; oncology and haematology; abdominal and urogenital surgery; cardiac and thoracic surgery; traumatology and plastic surgery; eye, ear, head and neck surgery; neurology and neurosurgery; gynaecology and obstetrics.

The calculation of sensitivity, negative predictive value, and accuracy of the semi-automated surveillance system was executed according to standard epidemiological methods. Accuracy was defined as the proportion of true positives and true negatives in all evaluated cases. The c-square test was used to test differences in categorical variables. All calculations were performed with STATA version 15 (Stata Corp., College Station, TX, USA).

A total of 39 519 patients who were discharged or passed away during the 1-year study period were included. The algorithm reduced the number of patients to screen manually by 93.7% (95% CI: 93.5e94.0%) to 2454 'at-risk' patients. Every patient 'at risk' had a mean of 2.3 radiological procedures to be evaluated. A total of 251 patients were identified as having nvHAP according to ECDC criteria. The number of EMRs needing to be screened to detect one nvHAP was 9.8 (95%CI: 8.7e11.4). On average 4.4 min were required to screen one EMR. This would translate to a calculated workload for nvHAP surveillance in our 950-bed tertiary care centre of approximately 40 min per workday when using the semiautomatic surveillance system. Table 1 shows the cross-tabulations of full manual surveillance against semi-automated surveillance for the three validation samples (VS1eVS3), and the respective analyses of sensitivity, negative predictive value, and accuracy. Four additional patients were identified to have nvHAP in VS2 (comprising only patients with ICD-10 HAP) that were not identified as 'at risk' by the algorithm and hence missed. Consequently, compared to full manual surveillance, sensitivity, negative predictive value, and accuracy of semi-automated surveillance were lowest in VS2 with 97.5% (CI: 93.7e99.3%), 99.2% (CI: 97.9e99.8%), and 99.4% (CI: 98.4e99.8%), respectively.

Root-cause analysis showed why the algorithm did not identify the four patients as 'at risk': (a) a patient on the ICU with fever but normal leucocyte count, not identified because temperature measurements for ICU patients are not yet included in EMRs for software reasons; (b) a patient with fever beyond the defined 24 h after a radiological procedure; (c) a patient with pneumonia diagnosed in a positron emission tomography scan only, a radiological procedure not taken into account by our algorithm; and (d) a patient with an x-ray that was not correctly entered in the EMR system.

To describe incidence and incidence density, medical specialty attribution, and microbiology of nvHAP, the four patients with nvHAP identified only during validation were included, totalling 255 patients with nvHAP.

The mean in-hospital incidence of nvHAP was 0.65% (95%CI: 0.57e0.73%) and the overall incidence rate was 0.83/1000 patient days (95%CI: 0.73e0.94) ( Table 2 ). The majority of nvHAPs (72.5%, n ¼ 185) were acquired on general wards, whereas 11.8% (n ¼ 30) and 15.7% (n ¼ 40), were acquired on ICU and IMC, respectively. A few nvHAPs (4.3%, n ¼ 11) were acquired during a previous hospitalization in our hospital and readmitted. Table 3 provides an overview of microbiological aetiology of nvHAP. The vast majority of patients (84%, n ¼ 215) were sampled, and blood culture was the most common sampling technique. Bacterial and viral pneumonias were found in 36% (n ¼ 91) and 5% (n ¼ 13) of patients, respectively. When only including samples of 'good quality' (i.e. sputum or tracheal aspirate with <10 SEC/LPF), the percentage of pneumonias with identification of a bacterial pathogen dropped to 20% (n ¼ 54). Possible fungal pneumonia was found in 5% of patients (n ¼ 13).

We developed and validated a semi-automated surveillance system for nvHAP to allow continuous outcome monitoring as a cornerstone of an infection prevention programme. By applying a classification algorithm, mirroring radiological and systemic criteria of the ECDC definition, the number of patients for manual screening was reduced by more than 90%. The semi-automated surveillance had a very high sensitivity, negative predictive value and accuracy. During 1 year, 255 patients (or one in 154 patients) acquired an nvHAP.

Semi-automated surveillance systems have already been applied and validated to investigate HAIs, most frequently for surgical site infections [15e17] and catheter-related bloodstream infections [18, 19] . Case finding often relies on diagnostic codes, antimicrobial prescription, and/or microbiology. Sensitivity of these algorithms varied from 26% to 95% [5] . For nvHAP, we were not able to identify a study validating a (semi-)automated surveillance system. Instead, two investigations assessed discharge diagnostic codes to substitute conventional nvHAP surveillance with an overall poor sensitivity of 42% and 59% [11, 20] , which declassifies this approach for epidemiological surveillance purposes. By challenging our algorithm by full manual evaluation of the patient population diagnosed and coded to have HAP, we identified four additional patients with nvHAP. Knowing that the sensitivity of ICD-10 coded HAP for HAP according to ECDC definition in our hospital is around 60% [11] , we anticipate having potentially missed another three patients maximumda small number compared to the 251 patients identified by our semi-automated surveillance system. By closely mirroring the ECDC surveillance criteria in our classification algorithm, the sensitivity of our semi-automated surveillance system is close to 100%, higher than the sensitivity of (semi-)automated surveillance systems for various HAIs summarized in the 2013 review by Van Mourik et al. [5] .

The incidence rate of nvHAP in our hospital was similar to that found in previous studies basing their surveillance on coding data [8, 9, 21 ], but incidence was higher than reported by Weber et al., Table 1 Validation of semi-automated surveillance who used the former CDC definitions of 1988 [7] . We found the highest nvHAP incidence rates in departments with a presumably high proportion of immunocompromised, multi-morbid, and elderly patients. Among surgical departments, nvHAP was most common in cardiac and thoracic surgery. These results are consistent with the known risk factors for nvHAP outside the ICU: i.e., malnutrition, chronic renal failure, anaemia, compromised consciousness, and patients undergoing thoracic surgery [21] . For UHZ, University Hospital Zurich. Number of patients in 2017: þ, <100 patients; þþ, 100e500 patients; þþþ, 500e1000 patients; þþþþ, 1000e2000 patients; þþþþþ, >2000 patients. a Internal medicine (patients in 2017: þþþþ), angiology (þþþ), cardiology (þþþþþ), dermatology (þþþþ), emergency medicine (þ), endocrinology (þþ), gastroenterology (þþþþ), geriatrics (þþ), immunology (þ), infectious diseases (þþ), nephrology (þþ), pneumology (þþþþ), and rheumatology (þþþ). b Haematology (þþþþ), nuclear medicine (þþ), oncology (þþþþ), and radio-oncology (þþ). c Visceral surgery (þþþþþ) and urology (þþþþ). d Cardiac surgery (þþþþ) and thoracic surgery (þþþ). e Traumatology (þþþþþ) and plastic surgery (þþþþ). f Ophthalmology (þþþþ), oral and maxillofacial surgery (þþþ), and otorhinolaryngology (þþþþþ). g Neurology (þþþþ), neuroradiology (þþ), neurosurgery (þþþþ), and psychiatry (þþ). h Gynecology (þþþþþ) and obstetrics (þþþþþ). surgical patients, known risk factors are immobilization, aspiration, gastric retention and vomiting, and abdominal surgery [22] . The proportion of patients with an established aetiology was 36%, consistent with the results of other studies [7, 21] . In our hospital, patients only rarely undergo bronchoscopic sampling, partially explaining the low percentage of microbiologically confirmed pneumonias by ECDC definition [12] . On the other hand, viral diagnostics are readily used, leading to higher rates of viral pneumonia as compared with other studies [7] . Oral flora were the only detected pathogens in sputum and tracheal aspirates of 'good quality' in one of three patients. Even though aspiration pneumonia is considered to be often caused by anaerobic and facultatively aerobic bacteria of the mouth [23] , the relevance of this finding is not clear, and contamination may be an alternative explanation.

Some limitations of our study should be considered. First, due to limited resources, we were unable to apply our reference standard of manual surveillance to the whole patient population. We aimed to address this limitation by conducting full manual surveillance on three different validation populations, including a validation sample comprising all patients with HAP according to ICD-10 coding data. Second, even careful manual case evaluation is associated with an inherent risk for interrater variability. We addressed this issue by double-checking all borderline cases and all cases with nvHAP by a second observer. Third, both semi-automated and full manual surveillance rely on radiological procedures demonstrating a pneumonic infiltrate. Patients without x-ray (e.g. due to restrictively performed diagnostic procedures) are missed by both surveillance systems, and so are patients who were transferred to another hospital or discharged home. Fourth, due to our selected algorithm criteria, patients with delayed radiological procedure and patients whose pneumonia was diagnosed in a radiological procedure of the abdomen or in a PET scan can be omitted, as shown by our rootcause analysis. Lastly, our surveillance system may require modification before it is apt for implementation in hospitals with differing characteristics of EMRs and data warehouses. Adaptations would also be needed when CDC instead of ECDC nvHAP definitions are to be used, as CDC definitions include special diagnostic criteria for immunocompromised and elderly patients.

This semi-automated surveillance system opens up various possibilities, such as timely feedback of infection rates, a key element in HAI prevention programmes [24, 25] . Further, we identified medical specialties with high nvHAP rates, allowing targeting of prevention efforts to the departments concerned. The resulting dataset identified the aetiological spectrum of nvHAP, revealing a surprisingly high percentage of viral infections, which should be considered in future prevention strategies. The continuous application of this semiautomated surveillance system will allow monitoring of the effectiveness of our ongoing nvHAP prevention bundle.

In conclusion, this novel classification algorithm proved highly successful in filtering out a large majority of patients 'not at risk' for nvHAP, thus drastically reducing manual surveillance workload. This made the establishment of a state-of-the-art semi-automated continuous surveillance of nvHAP feasible, facilitating timely feedback of infection rates, identifying departments with high nvHAP rates, and monitoring the effectiveness of nvHAP prevention measures. This surveillance system constitutes a breakthrough and central building block of nvHAP prevention, a previously neglected HAI. Implementation and validation of this semi-automated surveillance system (or adaptations of it) in other hospitals would be of special interest to investigate its overall validity and generalizability.

AW, WJ, HS, SPK, and PS designed the study. WJ, AM, MF, and AW acquired the data, and AW and SPK performed statistical analysis. AW, WJ, HS, SPK, and PS analysed and interpreted the data. AW drafted the manuscript, and WJ, MF, AM, SPK, PS, LC and HS provided a critical review of the manuscript for important intellectual content. All authors agree with the content and conclusions of this manuscript.

All authors declare no conflicts of interest. Aline Wolfensberger is supported by the academic career programme 'Filling the gap' of the Medical Faculty of the University of Zurich. Development of the algorithm was supported by 'Innovation pool', a University Hospital Zurich funding programme for developing new approaches in medical diagnostics and treatment. The funding source was not involved in the study design, collection and interpretation of the data, or manuscript submission.

Health care-associated infections: a meta-analysis of costs and financial impact on the US health care system

Multistate point-prevalence survey of health care-associated infections

The breadth of hospital-acquired pneumonia: nonventilated versus ventilated patients in Pennsylvania

Hospital-acquired pneumonia surveillancedan unmet need

Automated surveillance for healthcare-associated infections: opportunities for improvement

Point prevalence survey of healthcare associated infection and antimicrobial prescribing

Microbiology of ventilator-associated pneumonia compared with that of hospital-acquired pneumonia

Basic nursing care to prevent nonventilator hospital-acquired pneumonia

Hospital acquired pneumonia prevention initiative-2: incidence of nonventilator hospital-acquired pneumonia in the United States

Clinical attributes of non-ventilator-associated hospital-acquired pneumonia

Should International Classification of Diseases codes be used to survey hospitalacquired pneumonia?

CDC/NHSN surveillance definitions for specific types of infections

Rejection criteria for endotracheal aspirates from adults

Revised definitions of invasive fungal disease from the European organization for research and treatment of cancer/invasive fungal infections cooperative group and the national institute of allergy and infectious diseases mycoses study group (EORTC/MSG) consensus group

Evaluation study of different strategies for detecting surgical site infections using the hospital information system at Lyon University Hospital

Enhanced surgical site infection surveillance following hysterectomy, vascular, and colorectal surgery

Improved surveillance for surgical site infections after orthopedic implantation procedures: extending applications for automated data

Development of trigger-based semi-automated surveillance of ventilator-associated pneumonia and central line-associated bloodstream infections in a Dutch intensive care

Automated surveillance for central line-associated bloodstream infection in intensive care units

Frequency of hospital-acquired pneumonia in electronic and paper-based patient record

Risk factors for hospital-acquired pneumonia outside the intensive care unit: a case-control study

Incidence and potential risk factors for hospital-acquired pneumonia in an emergency department of surgery

Geriatric oral health and pneumonia risk

The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals

Feeding back surveillance data to prevent hospital-acquired infections

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cmi.2019.03.019.