key: cord-0004643-hrkj2y8o
authors: Villa, Gianluca; Amass, Timothy; Giua, Rosa; Lanini, Iacopo; Chelazzi, Cosimo; Tofani, Lorenzo; McFadden, Rory; De Gaudio, A. Raffaele; OMahony, Sean; Levy, Mitchell M.; Romagnoli, Stefano
title: Validation of END-of-life ScorING-system to identify the dying patient: a prospective analysis
date: 2020-03-09
journal: BMC Anesthesiol
DOI: 10.1186/s12871-020-00979-y
sha: 09f7bbd71050d32582c212c65d8a726ca1660190
doc_id: 4643
cord_uid: hrkj2y8o

BACKGROUND: The “END-of-Life ScorING-System” (ENDING-S) was previously developed to identify patients at high-risk of dying in the ICU and to facilitate a practical integration between palliative and intensive care. The aim of this study is to prospectively validate ENDING-S in a cohort of long-term critical care patients. MATERIALS AND METHODS: Adult long-term ICU patients (with a length-of-stay> 4 days) were considered for this prospective multicenter observational study. ENDING-S and SOFA score were calculated daily and evaluated against the patient’s ICU outcome. The predictive properties were evaluated through a receiver operating characteristic (ROC) analysis. RESULTS: Two hundred twenty patients were enrolled for this study. Among these, 21.46% died during the ICU stay. ENDING-S correctly predicted the ICU outcome in 71.4% of patients. Sensitivity, specificity, positive and negative predictive values associated with the previously identified ENDING-S cut-off of 11.5 were 68.1, 72.3, 60 and 89.3%, respectively. ROC-AUC for outcome prediction was 0.79 for ENDING-S and 0.88 for SOFA in this cohort. CONCLUSIONS: ENDING-S, while not as accurately as in the pilot study, demonstrated acceptable discrimination properties in identifying long-term ICU patients at very high-risk of dying. ENDING-S may be a useful tool aimed at facilitating a practical integration between palliative, end-of-life and intensive care. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT02875912; First registration August 4, 2016.

being aware of the value of palliative care services to help meet the families' and patients' needs and align therapy to the prognosis of the patient balanced with their preferences and values [4, 5] .

Beyond those clearly at the final stages of irreversible diseases (i.e. end-of-life), all patients admitted in the ICU require an early integration between a comprehensive palliative approach and intensive care treatment [4, 6] . The probability of dying of each critical care patient should be evaluated early on at the ICU admission and continuously reassessed during the entire ICU length of stay; the amount of palliative care treatments should thus be integrated accordingly [4] .

Although excellent scoring systems are available in the ICU for prognostic and clinical monitoring purposes (e.g. Acute Physiologic Assessment and Chronic Health Evaluation, APACHE, and Sequential Organ Failure Assessment score, SOFA [7] ), an accurate identification of end-of-life patients in the ICU is still cumbersome [8] [9] [10] . As example, the usefulness of several scores (e.g. the APACHE score) is mainly validated at the ICU admission, when the patients' responsiveness to intensive care treatments is not clear yet. Other scoring systems (e.g. the SOFA score) are well validated to monitor the organ dysfunctions over time in the ICU; nevertheless, the requirement of biochemical data for the scoring calculation may limit their routine use/daily application [7] . Finally, the requirement for, and to what degree, palliative care should be integrated with intensive care for all ICU patients is often not objectively defined and instead determined by individual physician's perspective [11, 12] .

The "END-of-Life ScorING-System" (ENDING-S) was previously developed to: 1) identify patients at very high risk of dying in the ICU and 2) facilitate a practical integration between palliative, end-of-life and intensive care treatments [11] . In a pilot study, ENDING-S presented acceptable calibration and discrimination properties in identifying patients at very high risk of dying in the ICU, with a receiver operating characteristic-area under the curve (ROC-AUC) analysis equal to 0.98 (95%CI, 0.97 to 1) and agreement between the predicted probability and the observed frequency of death in the ICU (p > 0.05 at Hosmer-Lemeshow test) were preliminarily observed [11] .

The aim of this observational study is to prospectively validate ENDING-S in a cohort of critical care patients with an ICU length of stay longer than 4 days.

This observational prospective study was performed in three ICUs: Rhode Island Hospital's medical ICU (Providence RI), Rush Medical Center's medical ICU (Chicago IL) and Azienda Ospedaliera Universitaria Careggi's surgical and medical ICU (Florence Italy). The institutional review boards of each center reviewed and approved the protocol (clinicaltrials.gov Identifier: NCT02875912; first retrospective registration: August 4, 2016). Written consent for analysis and publication of clinical data was obtained from all consentable patients. If the patient was not able to sign consent forms at the study enrollment, permission for analysis and publication of clinical data was obtained from a surrogate or waived in accordance with local ethics committee.

All adult patients admitted in the ICU from September 2015 to March 2017 were considered eligible for the study. Patients admitted to the ICU for end-of-life care were excluded. In order to consider only the long-term ICU patients, those with an ICU length of stay shorter than 4 days were excluded from the analysis. Data abstraction forms were prospectively completed for all eligible patients. In particular, ENDING-S and SOFA scores were calculated daily from the ICU day 4 to the ICU discharge for each enrolled patient. In accordance with the previous paper [1] , ENDING-S score was calculated as: where Days of MV/ICU LoS expresses the ratio between the current days in which the patient requires mechanical ventilation (MV) and the current ICU length of stay (LoS, quantitative variable), Days of Vasoactive drugs/ICU LoS expresses the ratio between the current days in which the patient requires vasoactive drugs and the current ICU length of stay (quantitative variable), Sepsis expresses a septic condition (dichotomous variable: 1 if currently affected, otherwise 0), ICU LoS expresses the current ICU length of stay (quantitative variable).

The ICU patients' management has not changed according to the value of ENDING-S observed.

According to the patients' outcome, the enrolled population was divided into two groups, either "survived" or "died". Death in the ICU being used as a surrogate to identify those ICU patients that had been at the end of their life prior to death. For "died" patients, maximization of palliative care and end-of-life care are thus certainly required. For "survived" patients, a certain form of palliative care is required according to the patients' multidimensional evaluation and expected prognosis, but very unlikely these patients require end-of-life care.

The association between the daily values of ENDING-S and patients' outcome at the ICU discharge was tested through a logistic regression analysis (OR, 95% CI). The ENDING-S predictive properties were evaluated with a ROC analysis. The effectiveness of ENDING-S cut-off 11.5 was previously validated for prediction of death during the ICU stay. The positive predictive value (PPV), negative predictive values (NPV), sensitivity and specificity of ENDING-s were calculated.

Similarly, the association between the daily values of SOFA score and patients' outcome was tested through a logistic regression analysis. The effectiveness of SOFA score was assessed for prediction of death during the ICU stay. In the absence of a specific cut-off point, only a ROC analysis was performed for this scoring system.

Continuous parameters observed in the population are reported as median [interquartile range] or mean ± standard deviation (SD), where appropriate; dichotomous parameters are expressed as crude number and percentage. A pvalue of 0.05 has been considered for statistical significance.

Data was analyzed using STATA 9.1 software (STATA corp, 490, Lakeway Drive College Station, 77,845, Texas, US).

Nine hundred and eleven patients were prospectively screened for this multicenter study. Among these, 220 patients had an ICU length of stay longer than 4 days and thus prospectively enrolled and considered for the analysis. The enrollment procedures are reported in Fig. 1 .

Among the enrolled patients, 21.46% (47/220) died during the ICU stay, while 78.64% (173/220) survived and were discharged from the ICU. The patients' characteristics, for both survivor and death groups, are reported in Table 1 .

Among patients who died, 32 of 47 (68.09%) had ENDING-S values higher than 11.5 (true positive), while the remaining 15 (31.91%) had at least an ENDING-S < 11.5 during the ICU stay (false negative). However, among patients who survived to ICU discharge, 125 of 173 (72.25%) had ENDING-S values smaller than 11.5 (true negative), while the remaining 48 (27.75%) had at least an ENDING-S ≥ 11.5 during the ICU stay (false positive). Given these characteristics, ENDING-S correctly predicted for 157 patients (157/ Sensitivity and specificity associated with ENDING-S cut-off of 11.5 were 68.09 and 72.25%, respectively; positive and negative predictive values were 60 and 89.28%, respectively. ROC-AUC of 0.79 was found for ENDING-S in this validation set (Fig. 2, Panel a) , 95% CI [0.71-0.86]. Fig. 1 The enrollment process. Over the entire population potentially eligible for this prospective study, 194 patients were excluded because admitted in the ICU for comfort measure only (CMO), for lack of family members (required for the qualitative analysis of the study, data not presented), or because pregnant or prisoner. In order to consider only "long term" ICU patients, those with an ICU length of stay (LoS) < 4 days were excluded. Finally, 103 patients refused to be enrolled in this observational study, forms were not completed for 43 patients (1 consent form and 42 clinical data forms) and 23 patients were excluded because not English or Italian native speaking

Considering daily values of SOFA score both for survived and not survived patients, a mean SOFA score of 8.4 ± 4.1 was observed for patients who died, while 4 ± 2.6 was observed for patients who survived at the ICU discharge. A ROC-AUC of 0.88 (95%CI 0.81-0.96) was found for daily values of SOFA score in predicting ICU death (Fig. 2, Panel b) .

In this observational study, the previously defined ENDING-score was prospectively tested in a cohort of critical care patients with an ICU length of stay longer than 4 days in order to validate its discriminative effect in identifying patient at very high risk of dying in the ICU.

The comparison between the ICU outcome observed for every enrolled patient and that expected according to the ENDING-S cut-off of 11.5 reveals a direct association between ENDING-S and patients' ICU outcome p < 0.0001). In particular, every incremental increase in ENDING-S value increased the OR estimate of death in the ICU by 1.18.

Compared with discrimination properties showed by ENDING-S in the pilot study, less efficient characteristics were observed in this prospective validation. Indeed, a ROC-AUC of 0.98 (95%CI, 0.97-1.00) was observed in Similar to the ROC-AUC, the sensitivity and specificity values previously observed in the pilot study are reduced in this external validation study [11] .

The identification of end-of-life patients is quintessential for an adequate integration between qualitative and intensive care treatments during the ICU stay [13, 14] . Notably, palliative care should not be considered as an alternative for the intensive care in these patients; this care should instead be concomitantly made available for the patients and their family early on from the ICU admission [15] . The importance of palliative care with respect to intensive care should be proportional with the probability that the patient is at very high risk of dying in the ICU and in accordance with the specific patient and family needs.

Identification of the patient at very high risk of dying is not the only limitation for palliative care integration in the ICU [16] . Indeed, the "relative amount" of palliative care that should be considered adequate for a specific patient in a specific moment during the ICU stay is difficult to define. Most of data in literature show that the management of palliative and end-of-life care is still determined by the physician's subjective experiences, religion, level of expertise and other nonobjective and unquantifiable variables [1, 15, [17] [18] [19] . Used beside other tools and comprehensive clinical evaluations, this objective tool, able to accurately identify dying patients and to suggest the adequate ratio between qualitative and intensive care for those patients at very high-risk of dying in the ICU, might help the physician appropriately integrate palliative care in the ICU [9] .

Assuming that the progression toward end-of-life should be characterized by an increasing presence of palliative care in the global management of the patient, the probability of being at very high risk of death in the ICU can be used as an indicator of the percentage need of palliative and mainly end-of-life care integration. Treatments based on patients' and families' needs (i.e. targeted on communication, psychological, social personal and/or spiritual well-being), should be progressively prioritized within the efforts of the health care team. In these terms, evaluating the ENDING-S and SOFA score might also be potentially useful for guiding palliative care integration in the ICU (Fig. 3) .

While SOFA does outperform ENDING-S, ENDING-S may still be considered clinically useful when compared to a SOFA score in two ways. First, the negative predictive value of ENDING-S of 89.28% suggests that an ENDING-S score less than 11.5 is clinically valuable in aiding clinicians in identifying patients that are likely to survive. Combining this with the fact that an ENDING-S score does not require any laboratory data as the SOFA score does, ENDING-S can be calculated every day, easily, on each patient. SOFA score requires laboratory data which was often missing in this cohort limiting the usefulness of a SOFA score in providing daily prognostic information.

Another important difference between ENDING-S and SOFA score is in the variation during the ICU length of stay of clinically stable patients who spend their end-oflife period in the ICU before dying. Although in critical condition, several patients have high and unchanged values of SOFA score over time, specifically before dying. Thus, a prognostic tool only based on SOFA score would not change, failing to alert the physician to the increasing amount of palliative and end-of-life care required for that patient. On the other hand, as ENDING-S is influenced by the progression of days spent in the ICU, it increases over time even in absence of clinical changes. This suggests that, by virtue of staying in the ICU, patients could require a progressively increasing amount of qualitative care over time.

There are several limitations in this study; among these, the use of ICU mortality as a surrogate to identify end-of-life is the most important. The aim of ENDING-S is to identify those patients likely being at very high risk of dying in the ICU and suggest the physician to appropriately consider palliative care integration for these patients. Interestingly, an objective definition of end-of- For each patient, the higher the ENDING-s or SOFA score, the higher the probability of ICU death, the higher the amount of palliative care interventions (in green) that should be integrated with intensive care treatment (in blue). Palliative care and intensive care should not be mutually exclusive; they should instead integrate each-other during the entire course of the patient's disease from the diagnosis and the initial organ dysfunction to the occurrence of multiorgan failure and end-of-life condition (within the dashed line). An appropriate scoring system should be characterized by a slope in score/ outcome probability able to promote intensive care and palliative care integration continuously, and across different levels of patient's severity life status is still lacking in clinical practice, even for patients who are not hospitalized [8] . For this reason, as in other papers [20, 21] , the outcome observed at the ICU discharge has been used as a surrogate for end-of-life condition for patients both enrolled in the pilot study and for those enrolled in this prospective validation. Notably, despite death in the ICU certainly being associated with the end-of-life of patients, ICU survival does not necessarily exclude the end-of-life condition or the requirement of palliative care [22] . Unfortunately, a systematic follow-up for patients discharged from the ICU is lacking in this study.

Another important conceptual limitation refers to the exclusive use of the patient's prognosis to guide the palliative care/intensive care integration. Several authors agree that palliative care should be based on needs of patients and their family instead of being exclusively based on patients' prognosis. A comprehensive qualitative evaluation should be integrated on ENDING-S parameters to further improve its role in guiding palliative care. Issues such as patients and family needs, perception of the disease, communication between family members and with health care providers should be considered and analyzed with this aim.

Although values observed in the pilot study resulted in a slightly overestimated prediction model, acceptable discrimination properties have been demonstrated for ENDING-S in identifying patients at very high risk of dying with an ICU stay longer than 4 days. Although SOFA score was more effective and accurate in predicting patients' death in the ICU, the ENDING-S score offers the benefit of not requiring laboratory data, a strong negative predictive value for ICU death, and daily changes by including length of stay. These factors may allow ENDING-S to be meaningfully integrated into daily practice and aid in the integration between palliative, end-of-life and intensive care. Nevertheless, further prospective validations and comparisons between EDNING-S and other standardized score are necessarily to better characterize the clinical application of this tool.

End-of-life decision-making and quality of ICU performance: an observational study in 84 Italian units

Use of intensive care at the end of life in the United States: An epidemiologic study*

The right choice at the right time

End of life care in Italian intensive care units: where are we now? Minerva Anestesiol

Family Care rituals in the ICU to reduce symptoms of post-traumatic stress disorder in family members-a multicenter, multinational, before-and-after intervention trial

Implementing primary palliative care best practices in critical care with the Care and Communication Bundle

Organ dysfunction scores in ICU

End-of-life decision making in the intensive care unit

A practical tool to identify patients who may benefit from a palliative approach: the CARING criteria

Medical futility: predicting outcome of intensive care unit patients by nurses and doctors--a prospective comparative study

Development of END-oflife ScorING-system to identify critically ill patients after initial critical care who are highly likely to die: a pilot study

Instruments for the identification of patients in need of palliative care: a systematic review protocol of measurement properties

Physicians ' perceptions of the value of prognostic models : the benefits and risks of prognostic confidence

Effect of a quality-improvement intervention on end-of-life care in the intensive care unit: a randomized trial

Meeting standards of high-quality intensive care unit palliative care: clinical performance and predictors

Developing guidelines that identify patients who would benefit from palliative care services in the surgical intensive care unit

End-of-Life Practices in European. Units: The Ethicus Study

End-oflife practices in European intensive care units: the Ethicus study

Hospital variation and temporal trends in palliative and end-of-life care in the ICU

Palliative care in the cardiac intensive care unit

Predictors of death in an Intensive Care Unit: contribution to the palliative approach TT -Fatores preditores de óbito em Unidade de Terapia Intensiva: contribuição para a abordagem paliativista

Early palliative Care reduces end-of-life intensive Care unit (ICU) use but not ICU course in patients with advanced cancer

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Not applicable.

We have the following interests: GV received from Baxter honoraria for lectures, from Pall International support for travel expenses, hotel accommodations, and registration to meetings. SR received from Baxter, Orion Pharma, and Vygon honoraria for lectures, from ICU Medical, MSD and Medtronic grants for consultancy, form Baxter, BBraun, Pall International, and Vygon support for travel expenses, hotel accommodations, and registration to meetings. CC received support for meetings (travels, hotel accommodations, and/or registration) by BBraun, Astellas, MSD, Pfizer, Pall International, Baxter, and Orion Pharma, for lectures by Orion Pharma. ARDG received research grants from MSD Italia, Baxter, Pall international. There are no patents, products in development or marketed products to declare. This does not alter our adherence to all the journal policies.

Authors' contributions G. V.: This author substantial helped to conception and design, acquisition of data, analysis and interpretation of data; T. A.: This author was accountable for all aspects of the work thereby ensuring that questions related to the accuracy or integrity of any part of the work were appropriately investigated and resolved; R. G.: This author helped to conception and design, acquisition of data; I. L.: This author helped to conception and design the study; C. 

Ente Cassa di Risparmio di Firenze (grant number 2014.0679), the "Cox foundation" (grant number: NA), the "Philip and Irene Toll Gage Foundation" (grant number: NA) and the National Heart Lung and Blood Institute (NHLBI grant number: T32 HL134625) have economically supported the feasibility and management of this study through research grants aimed at study coordination, statistical analysis and recruitment of investigators for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

The datasets analysed during the current study are available from the corresponding author on reasonable request.

The institutional review boards of each center reviewed and approved the protocol; in particular the "Comitato Etico Area Vasta Toscana Centro" approved the study protocol for the Azienda Ospedaliero Universitaria Careggi in Florence, the Internal Review Board of the Rhode Island Hospital approved the study protocol in Providence and the Internal Review Board of the Rush University Medical Center approved the study protocol in Chicago. (clinicaltrials.gov Identifier: NCT02875912). Written consent for analysis and publication of clinical data was obtained from all consentable patients. If the patient was not able to sign consent forms at the study enrollment, permission for analysis and publication of clinical data was obtained from a surrogate or waived in accordance with local ethics committee. All methods of consent were approved by the appropriate ethics committees.

Not Applicable.