key: cord-0017519-folw0uhc
authors: Vukoja, Marija; Dong, Yue; Adhikari, Neill K. J.; Schultz, Marcus J.; Arabi, Yaseen M.; Martin-Loeches, Ignacio; Hache, Manuel; Gavrilovic, Srdjan; Kashyap, Rahul; Gajic, Ognjen
title: Checklist for Early Recognition and Treatment of Acute Illness and Injury: An Exploratory Multicenter International Quality-Improvement Study in the ICUs With Variable Resources
date: 2021-02-23
journal: Crit Care Med
DOI: 10.1097/ccm.0000000000004937
sha: d48067b0c17fa4533db82acd1bd43a6b6e039756
doc_id: 17519
cord_uid: folw0uhc

OBJECTIVES: To determine whether the “Checklist for Early Recognition and Treatment of Acute Illness and Injury” decision support tool during ICU admission and rounding is associated with improvements in nonadherence to evidence-based daily care processes and outcomes in variably resourced ICUs. DESIGN, SETTINGS, PATIENTS: This before-after study was performed in 34 ICUs (15 countries) from 2013 to 2017. Data were collected for 3 months before and 6 months after Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation. INTERVENTIONS: Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation using remote simulation training. MEASUREMENTS AND MAIN RESULTS: The coprimary outcomes, modified from the original protocol before data analysis, were nonadherence to 10 basic care processes and ICU and hospital length of stay. There were 1,447 patients in the preimplementation phase and 2,809 patients in the postimplementation phase. After adjusting for center effect, Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation was associated with reduced nonadherence to care processes (adjusted incidence rate ratio [95% CI]): deep vein thrombosis prophylaxis (0.74 [0.68–0.81), peptic ulcer prophylaxis (0.46 [0.38–0.57]), spontaneous breathing trial (0.81 [0.76–0.86]), family conferences (0.86 [0.81–0.92]), and daily assessment for the need of central venous catheters (0.85 [0.81–0.90]), urinary catheters (0.84 [0.80–0.88]), antimicrobials (0.66 [0.62–0.71]), and sedation (0.62 [0.57–0.67]). Analyses adjusted for baseline characteristics showed associations of Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation with decreased ICU length of stay (adjusted ratio of geometric means [95% CI]) 0.86 [0.80–0.92]), hospital length of stay (0.92 [0.85–0.97]), and hospital mortality (adjusted odds ratio [95% CI], 0.81 (0.69–0.95). CONCLUSIONS: A quality-improvement intervention with remote simulation training to implement a decision support tool was associated with decreased nonadherence to daily care processes, shorter length of stay, and decreased mortality.

T he delivery of high-quality evidence-based medicine is expected to minimize complications and improve outcomes in critically ill patients. However, quality-improvement studies in ICU have yielded conflicting results (1) (2) (3) . Most were performed in high-income country (HIC) ICUs, and no large-scale global quality-improvement efforts have been undertaken in low-and middleincome countries (LMICs) (4) despite their appeal for resource-limited settings where the burden of critical illness is high, and multiple barriers preclude evidence-based care (5) . In LMICs, however, a systematic approach to error prevention with the use of checklist has been shown to improve patients' safety in surgical theaters (6) .

To facilitate timely and accurate best practice delivery in critically ill patients in ICUs with variable resources, an international collaboration developed the Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN), a clinical decision support tool that provides evidence-based checklists for structured ICU admission and rounding. Based on user-centered design, we developed CERTAIN that is fast, easy to use and applicable in diverse clinical settings (7, 8) . We hypothesized that care assisted by CERTAIN would improve the processes of care and outcomes of critically ill patients in ICUs with variable resources.

This was a pragmatic, prospective, before-after qualityimprovement study conducted between November 1, 2013, and December 31, 2017. In the preimplementation phase (phase 1, 3 mo or minimum 50 patients per ICU, whatever was reached first), we collected baseline data including ICU characteristics, demographics, baseline severity, comorbidities, limitations on life support interventions, daily care processes, and clinical outcomes of patients admitted to the participating ICUs. Following structured CERTAIN implementation (phase 2), the same variables were collected in the postimplementation phase (phase 3, 6 mo or minimum 150 patients per ICU, whatever was reached first). A detailed description of the methods is published (8) . The study was approved by the research ethics boards of all participating hospitals.

The study included ICUs in 10 LMICs and five HICs, across five continents (eTable 1, Supplemental Digital Content 1, http://links.lww.com/CCM/G189); country income status was defined by the World Bank (9) . Those ICUs was approached to participate based on previous collaboration with the investigators and through contacts established through members of the European Society of Intensive Care Medicine global working group and American Thoracic Society international health committee. The participating ICUs had variable size (median 12 beds, range, 5-52) and resources; a detailed description is given in eTable 1 (Supplemental Digital Content 1, http://links.lww. com/CCM/G189).

CERTAIN is a web-based decision support tool displaying relevant clinical information with a systematic approach. It incorporates evidence-based checklists, algorithms, and educational modules on performing critical procedures and has been reported previously (8) . CERTAIN consists of two modules: an admission module and a rounding module (eFigs. 1-3, Supplemental Digital Content 1, http://links. lww.com/CCM/G189). A mobile version and paper version were also provided in case of problems with internet connection. Items of admission and rounding checklists are presented in Figures 1 and 2 . The PDF format of the admission checklist, rounding checklist, and individual syndrome cards are also available for download on the CERTAIN website: https://www.icertain.org/library. The tool was tested in a simulated environment before the implementation phase (10, 11) .

CERTAIN implementation was done through an education program that engaged with local teams including at least three ICU members. After 2-4 weeks of access to online curriculum (slide presentations, published papers), the team conducted a follow-up online remote training via simulation coaching and debriefing (eFig. 4, Supplemental Digital Content 1, http://links. lww.com/CCM/G189) (8, 11) . Following a train-thetrainer session, the local trainer was certified to train their local staff. Once local physicians and nurses completed the training, the participating center proceeded to clinical implementation. The core of the intervention was a structured approach to admission and daily To enter the postimplementation phase, all centers were required to meet greater than 80% adherence to both the rounding and admission checklists for 4 consecutive weeks during the implementation phase. Adherence was tracked weekly through self-reported online form. CERTAIN was implemented sequentially in each ICU, so that by the end of the study, all ICUs had received the intervention (eFigs. 5 and 6, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

In both pre-and postimplementation phases, data were collected prospectively on consecutive patients (≥ 18 yr) admitted to the study ICUs. Readmissions, patients who were admitted for simple monitoring, patients who had a planned ICU admission for routine postoperative surveillance lasting less than 24 hours after uncomplicated surgery (this and previous criterion as defined by the site investigators), and patients who were transferred from an ICU outside the participating hospital were excluded.

Data were collected using the secure Research Electronic Data Capture system (12) . All data collectors were trained by the coordinating team and via online tutorials. The coordinating center audited data collection and met investigators regularly via monthly videoconference.

The study's coprimary outcomes include the following: 1) nonadherence to daily processes of care and 2) ICU and hospital length of stay (LOS). Secondary outcomes include 1) ICU mortality, 2) hospital mortality, 3) 28-day mortality, 4) blood product utilization, 5) infections, and 6) survival time. We observed the nonadherence to processes on days 0 (the day of ICU admission), 1, 2, 3, 7, 14, and 21, as long as the patient remained in the ICU. Nonadherence was recorded for the following 10 daily care processes: 1) deep vein thrombosis (DVT) prophylaxis, 2) peptic ulcer prophylaxis, 3) oral care, 4) head of bed (HOB) elevation to at least 30° above horizontal, 5) spontaneous breathing trial (SBT), 6) family conference discussion (for patients on mechanical ventilation), 7) assessment of central venous catheter (CVC) removal (for patients with CVC), 8) assessment for urinary catheter removal (for patients with urinary catheter), 9) assessment to continue or discontinue current antimicrobials (for patients receiving antimicrobial therapy), and 10) assessment to continue or discontinue current sedation (for patients receiving sedation). Nonadherence to these daily processes of care was quantified as an incidence rate, defined as the ratio of the number of observations of not receiving basic care procedures among eligible patients (events) to the number of total observations of in which the specific intervention was indicated (exposure) expressed per 1,000 days of specific intervention (eTable 3, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

The published study protocol defined the primary outcome as adherence to care practices; we changed this to nonadherence because it was a "rare" event, and we used Poisson regression for analysis. Modifications from the original protocol were done before data analysis and are presented in eTable 7 (Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Blood product utilization was measured by documenting RBC, fresh frozen plasma (FFP), and platelet transfusions. For infections, we recorded incidence rates of ventilator-associated pneumonia (VAP), catheterrelated bloodstream infections (CRBSIs), and urinary catheter infections. Although we provided standard operating definitions for the infection outcomes (13), the final diagnosis was based on medical records; we did not request microbiologic data to validate this diagnosis. For ICU and hospital LOS, the interval between the ICU admission date and the ICU or hospital discharge date was calculated in days. Patient status at ICU, hospital discharge, and at 28 days was assessed by reviewing the medical records and by postdischarge telephone contact (for discharged patients), respectively.

We did not assess adherence to some of the prespecified outcomes (e.g., shock resuscitation and sepsis treatment) due to limited resources for real-time data collection (eTable 5, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). The incidence rates of VAP, CRBSI, and urinary catheter infections, and RBC, FFP, and platelet transfusions were added after the study protocol publication but before study completion.

Detailed statistical methods are presented in the Supplementary Digital Content (Supplemental Digital Content 1, http://links.lww.com/CCM/G189). Categorical data are reported as counts and proportions and continuous data as mean (sd) or median (interquartile range [IQR]).

The study encompasses six sets of clinical outcomes that were measured before and after CERTAIN implementation. These outcomes were studied in all centers. Assessments of CERTAIN's effect were performed by developing linear, logistic, and Poisson models as the data required. These models were used to estimate ratios that expressed the effect of CERTAIN. The model's variables included CERTAIN implementation, center (to account for the clustering of patients within center), and patient characteristics (to account for imbalances between the pre/post implementation phases). For example, in the model for mortality, the regression coefficient (sd) for the CERTAIN implementation variable was used to estimate the adjusted odds ratio (aOR) and its CI. The p values are based on the likelihood ratio test using nested and full models. All tests are two sided, and a p value of less than 0.05 was considered statistically significant. All statistical analysis was conducted using the R statistical software, Version 3.4.(R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.) Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) were used as a framework for conducting the study (14, 15).

A total 55 centers were approached, of which 46 accepted to participate and 34 centers completed the study. A total of 4,256 patients completed the study: 1,447 in preimplementation and 2,809 in postimplementation phases (Fig. 3) . Twelve centers did not complete the study: eight centers dropped out during preimplementation phase, three during implementation phase, and one center in final postimplementation phase. Reasons for drop-out included lack of resources and change in leadership leading to a shift in priorities. Baseline demographics and severity of illness of patients in pre-and postimplementation phases were similar. More patients in the postimplementation phase were admitted from home, whereas patients in preimplementation phase had more comorbidities and more limitations of life support interventions at ICU admission ( Table 1) .

The median duration of ICU study participation was 96 weeks (IQR, 54-139 wk) (eTable 4, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Following implementation of CERTAIN, there was a significant decrease in nonadherence to eight of 10 targeted daily care processes compared with preimplementation period ( Table 2 and Fig. 4 ). There were 2,840 mechanically ventilated patients resulting in 10,069 observation days (3453 in preimplementation and 6,616 in postimplementation phases).

Compared with the preimplementation period, the incidence of nonadherence to DVT and peptic ulcer prophylaxis was significantly reduced in the postimplementation period (adjusted incidence rate ratio [aIRR], 0.74; 95% CI, 0.68-0.81 and 0.46; 95% CI, 0.38-0.57, respectively). There was a decrease in nonadherence to daily assessment of CVCs (from 647 to 566 per 1,000 CVC days; aIRR, 0.85; 95% CI, 0.81-0.90) and urinary catheters (674 vs 533 per 1,000 urinary catheter days; aIRR, 0.84; 95% CI, 0.80-0.88) ( Table 2 ). There was also an improvement in adherence to daily assessment of medications. The incidence rate for not assessing the need for antimicrobials was reduced from 340 to 219 per 1,000 observation days (aIRR, 0.66; 95% CI, 0.62-0.71). Omissions in daily assessment of sedation need decreased from 359 to 212 per 1,000 observation days (aIRR, 0.62; 95% CI, 0.57-0.67) ( Table 2 ). The intervention was also associated with reduced nonadherence to the best practices of conducting daily SBTs and holding family conferences. There was no change in nonadherence to daily oral care and HOB elevation.

The mean ICU LOS was 6.5 days (sd 3.0 d) in preand 5.8 days (sd 2.8 d) days in postimplementation phases, and the mean hospital LOS was 14.9 days (sd 3.1 d) in preimplementation versus 14.6 days (sd 2.8 d) in postimplementation phases (adjusted ratio of geometric means [aRoGM], 0.86; 95% CI, 0.80-0.92, and 0.92; 95% CI, 0.85-0.97, respectively). Results were similar after adjusting for baseline patient covariates, income level, and center (aRoGM, 0.86; 95% CI, 0.80-0.92 and 0.92; 95% CI, 0.86-0.98, respectively) ( Table 3) .

The overall (combined pre-and postimplementation phases) ICU, hospital, and 28-day mortality were 

In a post hoc subgroup analysis according to World Bank-defined country income status, the improvements were more pronounced in LMICs (reduction in nonadherence in 8/10 processes) compared with HICs (reduction in nonadherence in 6/10 processes) (eTables 5 and 6, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). Omissions in adherence to SBT, assessment for CVC and urinary catheter removal, and antimicrobial and sedation use decreased in both HIC and LMICs, whereas omissions in DVT and peptic ulcer prophylaxis and holding family conferences were reduced only in LMICs.

Omissions in adherence to HOB elevation decreased only in HICs. The implementation of CERTAIN was associated with reduction in incidence rates of CRBSI and FFP transfusions in LMICs but not HICs (eTables 5 and 6, Supplemental Digital Content 1, http://links. lww.com/CCM/G189). In a related post hoc analysis, significant interaction between income status and CERTAIN implementation effects on ICU, hospital, and day 28 mortality was observed (p < 0.001, p = 0.004, p = 0.002 respectively). After adjusting for baseline imbalances, the outcomes. We demonstrated that CERTAIN implementation is associated with significant improvements of evidence-based care delivery. CERTAIN implementation was associated with improvement of eight of 10 practices with no improvement in HOB elevation and daily oral care practice. The baseline adherence to many processes of care was high, with the greatest improvement in processes with lower adherence in preimplementation phase. Previous studies on checklist use in ICUs have yielded conflicting results (1, 2, 6, (16) (17) (18) (19) (20) . Improvements were seen in HOB elevation, daily interruption of sedative infusions, peptic ulcer disease and deep venous thrombosis prophylaxis, oral care for ventilated patients, electrolyte repletion, initiation of physical therapy, and documentation of restraint orders (21, 22) . Overall, the results were not consistent across studies (23) (24) (25) . This variation in effect may be due to differences in checklist selection and outcome definition. The way in which checklists are implemented may also influence the results, as active prompting has been shown to be superior to a checklist alone (20) . Most studies, including this one, used a before-after design and thus are prone to secular trends. Importantly, previous studies were mostly performed in HICs that have more resources for quality-improvement interventions and an established organizational safety culture. Notably, daily care processes in our study improved more in LMICs than HICs, which may relate to higher adherence to baseline processes of care in HICs but also fewer centers enrolled from HICs.

In LMIC ICUs, lack of structured training and lack of physicians are major barriers to implementing evidence-based medicine (26) . A systematic approach and provision of low-cost basic critical care procedures can minimize death and reduce costly complications in all environments (27, 28) . Simple interventions such as checklists improved outcomes in surgical and trauma patients globally without significantly increasing costs (6, 29) . So far, data on efficacy of such quality-improvement processes in ICUs of developing countries are scarce. Recently, a quality-improvement initiative in sepsis was associated with improved compliance to quality indicators and reduced mortality in a middle-income country (30) . Advances in information technology have yielded opportunities for online remote training and quality improvement without costly on-site education. Such use of collaborative network infrastructure has already shown to improve adaptation of care practices in HIC community ICUs (3). We observed a significant decrease in number of CVC infections in the postimplementation phase with no effect on prevention of VAP and urinary catheter infections. Studies on preventing ICU infections with quality-improvement interventions have shown different results (16, 19, 31) . This heterogeneity of effect may be due to nonuniform definitions of ICU infections. Mortality was significantly lower in the post-CERTAIN phase. In a stratified analysis according to income status, the mortality reduction was present only in LMICs and remained significant after adjusting for baseline imbalances between pre-and postimplementation groups. In contrast, a randomized trial to introduce prompting and checklists did not decrease hospital mortality and ICU infections in Brazilian ICUs (2) but did show modest improvements in four of seven processes of care. The main difference is that in our study, the intervention (training and checklist) not only included a rounding checklist (CERTAIN rounds) but also a structured admission and resuscitation checklist (CERTAIN admission). Similar to our findings, a nonrandomized evaluation of the global implementation of a surgical checklist was associated with marked reductions in mortality and postoperative complications in economically diverse group of hospitals (6), but the same effect was not observed in a HIC setting (32) .

Our study has several limitations. Before-after studies are prone to confounding by secular improvements and Hawthorne effect. Several aspects of management may have changed over time; for example, transfusion triggers may have declined. Therefore, cause and effect cannot be reliably determined. Second, in this pragmatic study, data were collected by bedside clinicians rather than dedicated study coordinators. We encouraged adherence to the intervention throughout the study period and measured it via self-reporting rather than direct observation. Third, the baseline characteristics of pre-and postintervention groups differed. The preimplementation cohort had more comorbidities and more limitations on life-sustaining measures, both of which could contribute to the mortality differences seen. Nevertheless, the association of CERTAIN with decreased LOS and mortality remained after adjustment for the baseline imbalances. Although the statistical analysis may be able to control for some of this confounding, residual confounding may persist. Fourth, the diagnosis of CVC and urinary catheter infections and VAP were based on medical records rather than adjudication including microbiological data. Fifth, interventions such as catheter insertion bundles, nutrition, or decubitus ulcer prevention were not examined. The evaluated daily care processes were mostly addressed by the CERTAIN rounding checklist, and the impact of the admission checklist and decision support algorithms on outcomes could not be reliably assessed in the absence of validated measures. Sixth, ICU-specific factors including staffing, number of beds, baseline performance, and prior internal quality-improvement interventions may have influenced the results. The time frame in which the ICUs adopted and implemented the intervention differed significantly due to the duration of the study period. Challenges in CERTAIN implementation are discussed elsewhere (33) . Only those hospitals motivated to improve quality care were committed time and resources to complete the study (eFig. 7, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). However, the observed association persisted after adjustment for center. The patients who were transferred from an ICU outside the participating hospital were excluded from this study; the results may not be applicable on those patients. Last, some of the elements of SQUIRE guidelines including cost and strategic tradeoffs, opportunity cost, and sustainability were not analyzed. Given these limitations, additional carefully designed studies are required to validate CERTAIN association with improved outcomes.

Despite these limitations, the results of this study show that systematic checklist implementation is feasible even in resource-limited settings and is associated with improvements in processes of care. To further facilitate CERTAIN use in daily work flow, we enabled translation into local language and developed a paper version of the tool. In addition, we have conducted educational courses (domestics, international, and online programs) and developed a train-the-trainer course to share our experience globally (www.icertain.org).

The implementation of a structured electronic decision tool with checklists in international ICUs has shown an association with reductions in nonadherence to basic care processes, CRBSI risk, and mortality.

In memoriam of Dr. Bekele Afessa, one of the pioneers of critical care practice improvement whose teachings inspired this work. We would like to thank Dr. Chris Farmer, Dr. Terri Clemmer, Dr. Atul Gawande, and Dr. Ken Hillman for helpful advice during study design and conduct. We thank Michael Malinchoc, MS, for performing statistical analysis. We thank the participating ICUs and their teams for their enthusiastic work and contribution to this project. 

Systematic review of safety checklists for use by medical care teams in acute hospital settings-limited evidence of effectiveness

Writing Group for the C-ICUI, the Brazilian Research in Intensive Care: Effect of a quality improvement intervention with daily round checklists, goal setting, and clinician prompting on mortality of critically ill patients: A randomized clinical trial

A multifaceted intervention for quality improvement in a network of intensive care units: A cluster randomized trial

Critical care outcomes in resource-limited settings

Critical care and the global burden of critical illness in adults

Safe Surgery Saves Lives Study Group: A surgical safety checklist to reduce morbidity and mortality in a global population

Checklist for early recognition and treatment of acute illness (CERTAIN): Evolution of a content management system for point-of-care clinical decision support

Checklist for early recognition and treatment of acute illness: International collaboration to improve critical care practice

World Bank Country and Lending Groups: World Bank Country and Lending Groups

Prompting with electronic checklist improves clinician performance in medical emergencies: A high-fidelity simulation study

Feasibility of an international remote simulation training program in critical care delivery: A pilot study

Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support

National Healthcare Safety Network: CDC/NHSN Surveillance Definitions for Specific Types of Infections

Revised Standards for Quality Improvement Reporting Excellence

Development of the quality improvement minimum quality criteria set (QI-MQCS): A tool for critical appraisal of quality improvement intervention publications

Matching Michigan Collaboration & Writing Committee: 'Matching Michigan': A 2-year stepped interventional programme to minimise central www

venous catheter-blood stream infections in intensive care units in England

Measurable outcomes of quality improvement using a daily quality rounds checklist: One-year analysis in a trauma intensive care unit with sustained ventilator-associated pneumonia reduction

A multicenter, phased, cluster-randomized controlled trial to reduce central line-associated bloodstream infections in intensive care units*

An intervention to decrease catheter-related bloodstream infections in the ICU

Prompting physicians to address a daily checklist and process of care and clinical outcomes: A single-site study

Improving care for the ventilated patient

Implementation of a mandatory checklist of protocols and objectives improves compliance with a wide range of evidence-based intensive care unit practices

Improving nursephysician communication and satisfaction in the intensive care unit with a daily goals worksheet

Improving communication in the ICU using daily goals

Improving communication in a pediatric intensive care unit using daily patient goal sheets

Improving ICU services in resource-limited settings: Perceptions of ICU workers from low-middle-, and high-income countries

The global need for essential emergency and ritical care

Caring for critically ill patients in humanitarian settings

Implementation of the world health organization trauma care checklist program in 11 centers across multiple economic strata: Effect on care process measures

SPDM against sepsis project: Implementation of sepsis bundles in public hospitals in Brazil: A prospective study with heterogeneous results

The application of human engineering interventions reduces ventilator-associated pneumonia in trauma patients

Introduction of surgical safety checklists in Ontario, Canada

Challenges of implementing a stepped wedge knowledge delivery intervention in 20 countries

Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN) Investigators: Aida