key: cord-305575-mdy0fcnn
authors: Zampieri, Fernando Godinho; Soares, Marcio; Salluh, Jorge Ibrain Figueira
title: How to evaluate intensive care unit performance during the COVID-19 pandemic
date: 2020
journal: Rev Bras Ter Intensiva
DOI: 10.5935/0103-507x.20200040
sha: 
doc_id: 305575
cord_uid: mdy0fcnn

nan

reduce healthcare-associated infections. (8) Adherence to evidence-based medicine (EBM) practices is the first obvious marker of a good performing ICU, and is a candidate for early performance assessment. Therefore, measuring and tracking adherence to EBM for measures and processes of care can provide insightful and actionable information. (5, 9) Of course, many pressing issues may hamper the attempts to measure and improve performance during the COVID-19 pandemic, including the abrupt shift in the ICU case-mix (e.g. increased severity and number of ventilated patients), need for changes in the whole ICU operation due to droplet precautions measures, costs increases due to additional personal protection equipment, and even a reduction of the available staff either due to illness or burnout. Finally, although data is starting to be published, we have no current tool to accurately predict either COVID-19 mortality or LOS. This represents a major limitation, not only for SMR/SRU but also this reduces the potential use of other metrics based on cumulative outcomes, such as variable-adjusted life displays. Much caution is needed if one aims at using SMR at this moment. Illness severity scores usually performed poorly when single conditions (including sepsis or acute respiratory distress syndrome) are considered. (2, 4, 6) Additionally, larger periods (usually 2 or 3 months) are required to allow a relevant number of patients with hospital outcomes. Therefore, if SMR and SRU are to be used and benchmarked, they should not be considered alone, neither be solely based on their absolute values, as larger temporal trends will be required. We, therefore, advoke that other variables should be measured to better understand the outcomes and help ICU directors to identify where to invest and/or change practices, aiming to achieve better outcomes. A comprehensive, but pragmatic, understanding of the case-mix and resource use, and its benchmarking, can be both feasible and insightful, ( Table 1 ) and focus on adherence to the process of care may add substantial value to an approach strictly focused on outcomes.

COVID-19 pandemic represents an abrupt change in the ICU outcomes ("producing survivors" process), with a sudden shift in the input, changes in process care, lack of effective and specific treatment protocols, an exceptional speed in changes of ICU routines, among other factors. This situation can be aggravated by a lack of proper equipment to provide life support, especially in strained ICUs or resource-constrained scenarios. For some ICUs, the limiting factor can be lack of equipment, lack of staff, late patient referral, or all the above. An individual assessment of cases with unfavorable outcomes using simple Ishikawa ("fishbone") diagrams may be useful, particularly early in the pandemic. However, as the cases accumulate, the evidence must come from larger series with proper analysis.

Additional ways to measure performance can be borrowed from economics, especially using the production-possibility frontier and data envelopment analysis. (10) Data envelopment analysis is an interesting econometric process where inputs and outputs are considered, and a benchmark performed. This analysis is flexible in the sense it accommodates with different metrics; for example, inputs may include staff levels, available equipment for organ support, number of beds and number of requested admissions (and their respective average illness severity) and outputs can include the number of survivors, mechanical ventilation free-days, ICU-free days, etc. It can also aid the identification of potential restraining issues between units (Figure 1 ). This may be useful, even for ICU comparison of performance over time, and benchmarking with other units.

Measuring the ICU performance was never so important neither so difficult as during the COVID-19 pandemic. While few data on prognostic scores is available, therefore limiting the use of more traditional metrics, ICUs should focus on measuring indirect performance parameters, especially analyzing case-mix, outcomes, and the rate of adherence to best practices. (oxygenation impairment of admitted patients, average severity, staff level) and "outputs" (mechanical ventilation free days and survival). The same unit at 4 different points is shown. There are changes in illness severity, staff level, oxygenation over time, which results in differences in outputs. These trends together with relative efficiency are shown in panel (B) . Note that at moments 1 and 2 the efficiency is maximized when compared with times 0 and 3 (marked with "*"), despite a reduction in staff level from 1 -2 and fluctuations in severity. At point 3, performance seems to worsen (lower survival, less mechanical ventilation free days which are disproportional to increase in admission severity). Data envelopment could point that staff reduction is probably the limiting step in this toy example. Min -minimum; Max -maximum; MV -mechanical ventilation; PF -partial pressure arterial oxygen/fraction inspired oxygen.

Postgraduate Program in Translational Medicine and Department of Critical Care

HCor-Hospital do Coração -São Paulo (SP)

Why try to predict ICU outcomes?

ICU severity of illness scores: APACHE, SAPS and MPM

Prospectively defined indicators to improve the safety and quality of care for critically ill patients: a report from the Task Force on Safety and Quality of the European Society of Intensive Care Medicine (ESICM)

Understanding intensive care unit benchmarking

New perspectives to improve critical care benchmarking

What every intensivist should know about prognostic scoring systems and risk-adjusted mortality. Rev Bras Ter Intensiva

Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19)

Organizational characteristics, outcomes, and resource use in 78 Brazilian intensive care units: the ORCHESTRA study

Development of actionable quality indicators and an action implementation toolbox for appropriate antibiotic use at intensive care units: a modified-RAND Delphi study

Efficiency analysis of federally funded hospitals: comparison of DoD and VA hospitals using data envelopment analysis