key: cord-0707269-oyaast8u
authors: Beane, Abi; Salluh, Jorge I.F.; Haniffa, Rashan
title: What Intensive care registries can teach us about outcomes
date: 2021-10-01
journal: Curr Opin Crit Care
DOI: 10.1097/mcc.0000000000000865
sha: 613ddc749789c872cc3be1a1f28b2626438dae2c
doc_id: 707269
cord_uid: oyaast8u

PURPOSE OF REVIEW: Critical care registries are synonymous with measurement of outcomes following critical illness. Their ability to provide longitudinal data to enable benchmarking of outcomes for comparison within units over time, and between units, both regionally and nationally is a key part of the evaluation of quality of care and ICU performance as well as a better understanding of case-mix. This review aims to summarise literature on outcome measures currently being reported in registries internationally, describe the current strengths and challenges with interpreting existing outcomes and highlight areas where registries may help improve implementation and interpretation of both existing and new outcome measures. RECENT FINDINGS: Outcomes being widely reported through ICU registries include measures of survival, events of interest, patient reported outcomes and measures of resource utilisation (including cost). Despite its increasing adoption, challenges with quality of reporting of outcomes measures remain. Measures of short-term survival are feasible, but those requiring longer follow-ups are increasingly difficult to interpret given the evolving nature of critical care in the context of acute and chronic disease management. Furthermore, heterogeneity in patient populations and in healthcare organisations in different settings makes use of outcome measures for international benchmarking at best complex, requiring substantial advances in their definitions and implementation to support those seeking to improve patient care. SUMMARY: Digital registries could help overcome some of the current challenges with implementing and interpreting ICU outcome data through standardisation of reporting and harmonisation of data. In addition, ICU registries could be instrumental in enabling data for feedback as part of improvement in both patient centred outcomes and in service outcomes; notably resource utilisation and efficiency.

Critical care registries are synonymous with measurement of intensive care unit outcomes regionally and nationally [1] . Critical care registries are increasingly common globally. Recent high profile publications have highlighted the value and impact of registries in enabling continuous and replicable measure of quality and in supporting efforts to improve the effectiveness of healthcare internationally [2] . More recently registries are playing a pivotal role in capturing and reporting outcomes data in response to the global COVID-19 pandemic [3] .

Historically the outcomes captured by registries (and registry based studies) were selected primarily based on the need for a basic evaluation of ICU performance and on research questions of interest, with some consideration given to the feasibility of capturing the desired outcomes within the study scope and available budget. In ICU, measures of mortality and morbidity dominated. The role of ICUs and their patient populations is however evolving. Admission for optimisation of physiology prior to complex surgery and for acute symptom management in the context of chronic disease and palliative care is now commonplace in many ICUs. Also, as governmental agencies, accreditation services and medical societies established quality of care and outcome measures, several ICU registries have incorporated those in their core data set. In addition there is growing recognition of the need to identify and define outcomes that consider perspectives beyond those of ICU clinicians and policy makers to include patients, their families and the wider multidisciplinary stakeholder group. [4, 5] . This review aims to summarise literature on outcome measures currently being reported in registries internationally, describe current challenges with interpreting existing outcomes measured and highlight areas where registries may help improve implementation and interpretation of both existing and new outcome measures. captured, although some registries also include some "fixed-time" (30, 60-day) reports on vital status. Whilst all-cause mortality is broadly relevant for most specialities and is useful for determining that a patient has died (as opposed to for example being lost to follow up), its utility in determining quality of care is limited. [2] . The latter requires the distinction between expected and excess mortality, and by nature of the distinction, identification of deaths which may have been avoided. However, reporting of procedure-related deaths including those related to a complication of the procedure or treatment for a complication remain widely absent from the majority of registries. [7] .

Using risk-adjusted mortality represents a significant improvement to ensure the use of "mortality" as a quality of care indicator. The measurement of standardised mortality rates which are derived using prognostic scores is less consistent. [8, 9, 10] . Although all registries use standardised mortality rates (SMRs), internationally they vary in both the models used and the time points at which they are applied. The most frequently used are those based on APACHE (II,III and IV) and SAPS (2 and 3) which are used by registries in Europe, South America, Japan, whilst well established registries such as ICNARC (UK) and the Australia and New Zealand registry (ANZICS) use their own scoring systems-developed and validated specifically for their populations (ANZROD and ICNARC mortality model, respectively) [7, 11, 12] . More recently in emerging LMIC ICU registries including those in Asia (India, Nepal, Sri Lanka and Pakistan) and Latin America (Brazil) work is underway to develop and validate simpler scoring systems (e.g Asian validation of E-TropICS and Brazilian validation of the SMS-ICU score); that reflect diagnosis and aetiologies more prevalent in such settings, and which attempt to increase score utility, and minimise data missingness. [13, 14, 15, 16] . Despite being of increasing importance in understanding the effectiveness of ICU care on patient recovery, the majority of ICU registries internationally still do not routinely measure long term survival following hospital discharge. [2, 18, 19] . Despite the increasing presence of electronic health records in high income settings, limited interoperability between in-hospital and social care databases has until recently hindered linking long term outcomes with ICU care. [2, 24] . In low and middle income countries (LMICs) practical challenges such as paper records, lack of human resources and communities' potential distrust of follow up after ICU all hamper efforts to establish follow up services [10] where medium to long term outcomes have been reported survival is less than 50% [24] .

Events of interest typically include complications, or events associated with increased resource use and mortality. A recently published review of registry reported indicators of quality-describes 51 different quality indicators, of which 20 were categorized as outcome events of interest. [7] . The most frequently reported events of interest focused on potentially avoidable adverse events such as healthcare associated infections (HCAIs), pressure sores, duration and complications of invasive mechanical ventilation, prolonged ventilation and unplanned readmission. [4, 7, [20] [21] [22] [23] . Wide variation and inconsistency in collection, definition and reporting of such measures was described, hindering utility and interpretability of such outcomes. Several large scale studies analysing incidence of HCAIs failed to draw meaningful conclusions due to underreporting, missingness of data and inconsistencies in extracting and reporting of data through the registries. [4, 10, 23, 27, 28] . These inconsistencies were greatest where extraction was by hand from paper records. Concerns have also been raised as to whether reporting such events actually leads to improvements in care quality. [20] . Events of interest are often used to suggest suboptimal care, however in focusing on the negative, these indicators and their measurement provides little insight for teams seeking to inform actionable improvement or reinforce good practice. Whilst arguably more acceptable to stakeholders, very few of the event based outcomes reported in registries focused on the presence or inclusion of events associated with positive care outcomes. [25] [26] [27] . Notable exceptions were the NICE registry in the Netherlands, and EpiMed (Brazil) both of which are using audit and feedback mechanisms inbuilt within electronic registry platforms to try and increase desirable events as part of quality improvement initiatives.

Resource utilization measures capture the patient's interactions with the healthcare system. Measures including occupancy rates, organ support, medications and ICU and hospital lengths of stay as well as events associated with increased resource utilisation, e.g. readmission to ICU in the same hospital admission are of increasing interest for managers and health systems and have a major impact on patient and family experience of ICU care. [33] . In addition, these measures are used alongside measures of expenditure to determine cost and cost effectiveness of critical care services. [25] . Whilst ICU registries internationally capture the patient centered measures, the potential of organisational structures in evaluating outcomes has only recently begun to be understood. Recent cost effectiveness evaluations from ANZICS have suggested that increased sizes and occupancy of ICU may improve efficiency [29] . The use of standardized metrics may improve the benchmarking on resource use and be considered a proxy of ICU efficiency ( Figure 1 ). For example APACHE IV and SAPS3 scores have been used by registries in the UK, Brazil and Australia to risk-adjusted LOS as a surrogate for resource utilization. [25, 40, 51] . More recently the COVID-19 pandemic has brought into sharp focus the vulnerability and finite nature of critical care resources globally. Resource and cost data from ICU registries in India was used to provide insights into the impacts of COVID-19 on ICU service utilisation and in doing so highlighted how contextual factors; organisational cultures, team structures may influence resource utilisation and service outcomes. [55] Moving forward ICU registries could provide increasingly valuable contributions to surveillance of ICU resource utilisation, efficiency and the impact of ICU on public health outcomes.

Patient-reported outcome measures (PROMs) reflect the patients' perceptions of their health status and their perspective on health and disease. [30] PROMs have become an increasingly important avenue for research in ICU given increasing numbers of survivors. [31] . For the most part, PROMs are confined to registry led studies and are not routinely captured in ICU registries. Selection and operationalisation of specific PROMs for use within registries is not, however, straight forward and substantial heterogeneity of capturing and reporting remains. Furthermore, the number of possible PROMs is potentially overwhelming, their definition, character and interpretability complex and their ability to detect change in quality Europe PMC Funders Author Manuscripts of care over time uncertain. [26] . Burden of data collection on both the patient and for the healthcare team is a major consideration. In LMICs, these concerns are compounded by uncertainty over how patients and healthcare providers currently understand quality of care and how current expectations of care may invalidate such measures. [2] Nonetheless, PROMs are increasingly used, to support quality assurance and improvement initiatives, and may be of particular value in resource constrained settings-to help refine indications for ICU admission, and help close the quality gap. [2] . The use of novel technologies such as mobile application-based questionnaires as well as linkage with electronic medical records may facilitate data collection beyond ICU admission and allow large scale, consistent information on quality of life, return to daily activities and work among other domains (Table 1) .

As previously mentioned, survival remains a central outcome measure in ICUs. Whilst ICU mortality rates and associated measures such as length of stay have steadily decreased across upper and middle income countries over the last 10 years, demand for critical services and ICU admission in these same settings is rising. In contrast ICU mortality in LMICS whilst varying between regions remain disproportionately high (30-80%) and remain inversely correlated to national income levels [4, 14, 25, 27] . In higher income settings, where critical care services are increasingly well established, early referral and admission of patients from wards who require single organ support and admission of patients for optimisation of physiology prior to major surgery has undoubtedly contributed to improved ICU and hospital survival rates. Potentially the increasing availability of transfer from ICU to rehabilitation services for patients following major orthopaedic and neurological injury may influence both ICU survival and ICU lengths of stay. Such investments in acute and chronic care remain largely absent in resource limited settings with significant variation among and within countries. [28, 29] As a consequence, ICU in these settings are often used as a terminal destination for patients at the end of life, often following cardiac arrest, or in the presence of long-standing organ failure. [27, 30] To increase complexity further, dying in hospitals in many parts of the world is neither culturally desirable or socially acceptable for patients, their families or for their treating clinicians. In such instances ICU admission may be negotiated by specialists as a way to provide mechanical ventilation as a bridge to family members being able to prepare to take their loved one home to die. [4] . For patients already in ICU, arrangements may be made for transfer from the ICU directly home, perhaps with an endotracheal tube in situ. Indiscriminate reporting of mortality figures in these settings, or worse, attempts to draw comparison internationally is unadvisable. Given the legal, cultural, and socioeconomic factors likely to influence decision-making, and recording and reporting of mortality, greater international multidisciplinary perspective is required. [31] . Conversely, cultural differences may play a role in increasing the use of ICU by terminal patients resulting in both admission of patients with low survival expectations but also on low degree of implementation of palliative and end-of-life care measures that ultimately lead to prolonged, expensive and futile use of intensive care resources. 

Crucially clinicians and healthcare users in ICU require support and training to interpret outcomes measures. This may improve trust among clinicians and researchers seeking to improve care using registry data, as well as patients and policy makers. National registries and international initiatives such as the LOGIC (Linking of global intensive care) have an important role in providing standardised metrics and improving the ability for its comparison by using reliable and risk-adjusted measures. [35] How intensive care registries can improve implementation and interpretation of ICU outcomes

Standardised outcome measures provide a set of outcome measures that are feasible to capture in registries that are important to clinical providers, patients and their families. [26] . Standardisation of outcome measures in registries, means variables are mapped to terminologies that facilitate consistent and replicable collection [6] . Support for multiple efforts to develop common data elements for use in registries is growing. The use of established standardized outcome measures is essential so that registries can contribute to evolving evidence and quality improvement practice. Critical care registry projects including "Collaboration for Research, Implementation and Training in Asia and Africa (CCAA)" "Global Open Source Severity of Illness Score"and a study looking at organisational characteristics, outcomes and resource utilisation in Brazil have demonstrated the potential in using large scale registry databases to use data as a driver of quality improvement. [40, 41, 47,48,] . Currently the National Institute of Health, Duke Clinical Research Institute and the Pew Charitable Trust are investing in registry data standards for concepts in registries. Standard terminologies not only improves efficiency when establishing registries but also promotes data sharing, and linking of datasets from different sources [42] ; critical for increasing interoperability between registries internationally. In addition, harmonisation of existing critical care data sets using common data models is underway. Pioneering work by groups such as the Observational Health Data Sciences and Informatics (OHDSI) community are working in partnership with ICU registries in the Netherlands, and in Asia to apply a common data models (CDM) to cleanse and standardise existing registry data.

[43]. The CDM allows for the increased interoperability of information, systematic analysis of data from different data sources using common representation (terminologies, vocabularies, coding schemes), and then performing systematic analyses using a library of analytical tools. These tools could help enable standardised outcome measures thus improving its ability to be interpreted across diverse populations.

Central to strengthening the quality and utility of ICU outcome measures is improved communication and collaboration between ICU registries and healthcare databases internationally. Perhaps the most comprehensive and current effort towards achieving this is Linking of Global Intensive Care (LOGIC). [46, 47, 48] A collaboration of registries from 13 countries, it includes aggregate data on over seven million ICU encounters from Asia, Australasia, Europe, North America and South America. Its aim is to promote data sharing Europe PMC Funders Author Manuscripts and international benchmarking through pragmatic reporting of crude aggregated national data from registries. Whilst efforts to harmonise data sets and standardise outcomes are still underway, LOGIC's approach to reporting aggregate data in a single common platform is the first step for many ICU registries in collaboration.

The increasing implementation of digital registries and their potential to communicate with mobile technology and wearable devices could rapidly accelerate the feasibility of routine and sustainable measurement of PROMs. [49] . Data obtained from questionnaires enabled by mobile applications can and are starting to provide patients and clinicians with near real time data on medium to long term quality of life, functional recovery and psychological well being. [50] . Centralising information to cloud-based registries is both feasible and resource light. Leveraging already accessible tools could be especially important in middle and lower middle income settings, where absence of long term outcomes is in part perpetuated by the financial and human resources costs associated with long term follow up.

Actionable indicators are gaining prominence in the healthcare improvement arena. Data pertaining to ICU outcome measures; specifically events of interest (e.g HCAI rates) are increasingly available through registries dashboards and allow ICUs to compare rates over time. Such measures may have a role in promoting improvement in ICU, providing they are selected, implemented and interpreted in partnership with healthcare providers. The Agency for Healthcare Research and Quality have advocated for outcome measures to be used to improve performance in daily practice. Positive reinforcement from changes in measurement reported over time may complement existing tools for audit and feedback as part of quality improvement initiatives. As described above, both the multi national registry Epimed [51] , and Dutch intensive care registry have shown the potential for registries as more than simply reporting ICU outcomes. [52, 53] . Both used registry data as part of a feedback loop to reinforce positive behaviours in clinical care as part of quality improvement initiatives. Such methods are likely to become increasingly central to how ICU registries are used in healthcare systems internationally providing the validity, actionability and reproducibility of data from registries can be assured.

Registries have contributed significantly to the reporting and interpretation of ICU outcome measures internationally. ICU registries have an increasingly central role in benchmarking ICU care, however there are still challenges to implementation and interpretation of current outcome measures. Alongside variation in definitions, implementation and reporting of outcomes in ICU, international comparisons using current measures are making progress. Advances in IT and data science have created opportunities to change the way ICU outcomes are reported and . could additionally support the use of actionable measures to improve patient's outcomes. The metric used is the standardized resource use (SRU) based on SAPS 3. Each yellow dot represents an ICU. Lowest rates represent better resource use and efficiency (lower observed/expected resource use rates). 

Improving the quality of the intensive care follow-up of ventilated patients during a national registration program

High-quality health systems in the Sustainable Development Goals era: time for a revolution. The Lancet global health

Leveraging a Cloud Based Critical Care Registry for COVID-19 Pandemic Surveillance and Research in Low-and Middle-Income Countries

The association between outcome-based quality indicators for intensive care units

Access and Quality Index based on mortality from causes amenable to personal health care in 195 countries and territories, 1990-2015: a novel analysis from the Global Burden of Disease Study

Registries for Evaluating Patient Outcomes: A User's Guide

**) The study provides a systematically review the literature on quality indicators for evaluating critical care and describe their current evidence as well as the variances in measurement

The Danish intensive care database

Quality improvement in critical care: selection and development of quality indicators. Canadian respiratory journal

Applicability of the APACHE II model to a lower middle income country

The ANZROD model: better benchmarking of ICU outcomes and detection of outliers

External validation of the Intensive Care National Audit & Research Centre (ICNARC) risk prediction model in critical care units in Scotland

Indian Registry of IntenSive care (IRIS) collaborators. Validation of a simplified risk prediction model using a cloud based critical care registry in a lower-middle income country

Author manuscript; available in PMC

Intensive Care Outcomes and Mortality Prediction at a National Referral Hospital in Western Kenya

Predicting mortality in low-income country ICUs: the Rwanda Mortality Probability Model (R-MPM). PloS one

Simplified prognostic model for critically ill patients in resource limited settings in South Asia

Implementing an intensive care registry in India: preliminary results of the case-mix program and an opportunity for quality improvement and research

Quality of life in the five years after intensive care: a cohort study

Improving the quality of the intensive care follow-up of ventilated patients during a national registration program. Public Health

National registries: Lessons learnt from quality improvement initiatives in intensive care

Epidemiology, patterns of care, and mortality for patients with acute respiratory distress syndrome in intensive care units in 50 countries

Traumatic brain injury (TBI)outcomes in an LMIC tertiary care centre and performance of trauma scores

Central line-associated bloodstream infections in Australian intensive care units: Time-trends in infection rates, etiology, and antimicrobial resistance using a comprehensive Victorian surveillance program

Device-associated nosocomial infection rates in intensive care units at Cairo University hospitals: first step toward initiating surveillance programs in a resource-limited country

Five-year mortality and hospital costs associated with surviving intensive care. American journal of respiratory and critical care medicine

The Danish intensive care database

Building a continuous multicenter infection surveillance system in the intensive care unit: findings from the initial data set of 9,493 patients from 71 Italian intensive care units. Critical care medicine

Targeted surveillance of nosocomial infection in intensive care units of 176 hospitals in Jiangsu province

The financial cost of intensive care in Australia: a multicentre registry study

Long-term outcomes after critical illness: recent insights

Core outcomes sets for studies evaluating critical illness and patient recovery. Current opinion in critical care

Understanding patient-important outcomes after critical illness: a synthesis of recent qualitative, empirical, and consensus-related studies

Council of the World Federation ofSocieties of Intensive and Critical Care Medicine. Critical care of tropical disease in low income countries: report from the Task Force on TropicalDiseases by the World Federation of Societies of Intensive and Critical CareMedicine

Epidemiology of antibiotic-resistant wound infections from six countries in Africa

MD8on behalf of the Linking of Global Intensive Care Collaboration (LOGIC)

Improving outcomes for severe sepsis in Africa: one step closer

Intensive care unit capacity in low income countries: a systematic review

Variation in critical care services across North America and Western Europe

) The present study describes the current status and challenges for implementing core datasets for ICU patient during and after hospitalization

Family care, visiting policies, ICU performance, and efficiency in resource use: insights from the ORCHESTRA study

33: THE GLOBAL OPEN SOURCE SEVERITY OF ILLNESS SCORE (GOSSIS). Critical Care Medicine

Registry Data Standards

Data Resource Profile: the Dutch National Intensive Care Evaluation (NICE) Registry of Admissions to Adult Intensive Care Units

Identification and assessment of potentially high-mortality intensive care units using the ANZICS centre for outcome and resource evaluation clinical registry

The Epimed monitor ICU database®: a cloud-based national registry for adult intensive care unit patients in Brazil TT -Epimed monitor ICU database®: um Registro nacional baseado na nuvem, Para pacientes adultos internados em unidades de terapia intensiva. Rev Bras Ter intensiva

Establishing a critical care network in Asia to improve care for critically ill patients in low-and middle-income countries

New perspectives to improve critical care benchmarking

Centralized patient-reported outcome data collection in transplantation is feasible and clinically meaningful

The Epimed Monitor ICU Database®: a cloud-based national registry for adult intensive care unit patients in Brazil. Rev Bras Ter Intensiva

Small steps beyond benchmarking. Rev Bras Ter Intensiva

**) The present registry-based multicenter study describes an intervention based on audit-feedback to improve the adherence to evidence-based pain

Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial

Impact of COVID-19 on non-COVID intensive care unit service utilization, case mix and outcomes: A registry-based analysis from India

Not applicable.

Dr Salluh is supported in part by individual research grants from CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) and FAPERJ (Fundação Carlos Chagas Filho de Amparo à Pesquisa do Rio). Drs A Beane and R Haniffa are partially funded by "Wellcome Trust Innovations Project (215522) and UKRI GECO Grant MR/V030884/1".

Registries provide a pivotal role in reporting ICU outcome measures.Whilst ICU survival is increasing globally, registries reveal that there is ongoing disparity in outcomes internationally.Data on long-term mortality and functional outcomes in ICU registries are globally lacking.Main challenges to interpreting ICU outcomes reported by registries include the evolving role of ICU care in acute and chronic management, limited feasibility of disease severity scores and wide variation in health system infrastructure.Implementation of medical informatics offers novel possibilities for overcoming current challenges in utilising and interpreting registry reported ICU outcome data Beane et al. Page 12Curr Opin Crit Care. Author manuscript; available in PMC 2021 October 01.