key: cord-0026842-na7lwstn authors: Hu, Yuyuan; Chen, Dongling; Li, Qian; Yin, Guichun; Zhang, Xianjun; Wang, Yachun title: A prediction model for 30-day deaths of cirrhotic patients in intensive care unit hospitalization date: 2022-02-04 journal: Medicine (Baltimore) DOI: 10.1097/md.0000000000028752 sha: 4858613a86eecea0bf2272cdb594fc72f878740b doc_id: 26842 cord_uid: na7lwstn The aim of this study was to establish a prediction model for 30-day deaths of cirrhotic patients in intensive care unit. A case-control study involving 1840 patients was conducted in the Medical Information Mart of the Intensive Care Database III version 1.4. The logistic regression with L1 regularization was used to screen out the variables. The 30-day in-hospital death was used as the dependent variable and the selected variables were used as the independent variable to build a random forest model. The performance of the model was validated by the internal validation. The variables screened by logistic regression analysis were the age, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, Oxygen saturation, white blood cells, platelets, red cell distribution width, glucose, blood urea nitrogen, bicarbonate, total bilirubin, hematocrit, alanine transaminase, aspartate transaminase, bilirubin, Simplified Acute Physiology Score II and Sequential Organ Failure Assessment. The areas under the curve of the random forest model based on these variables was 0.908, and the performance of this model were internally validated with an areas under the curve of 0.801. The random forest model displayed that Simplified Acute Physiology Score, Sequential Organ Failure Assessment, blood urea nitrogen, total bilirubin and bilirubin were more important predictors for the 30-day death of cirrhotic patients in intensive care unit. A prediction model for death of cirrhotic patients was developed based on a random forest analysis, providing a tool to evaluate the patients with a high risk of 30-day in-hospital deaths to help clinician make preventive intervention to decrease the mortality. Hepatic cirrhosis is the terminal stage of liver disease, and can cause continuous damage to liver cells. [1] It is worth noting that the onset and course of this disease are generally slow, which may conceal for 3 to 5 years or more than 10 years. [2] In recent years, the morbidity and mortality of hepatic cirrhosis are both increasing, and has ranked the fourteenth of the most frequent cause of adult deaths worldwide. [3] Studies in the UK and Sweden report that the annual incidence rate of hepatic cirrhosis is 15.3-132.6 per 100,000 people. [4] A screening program in France shows that the prevalence of hepatic cirrhosis is 0.3%. [5] However, due to the insidious onset, this disease is often asymptomatic at early stages; therefore, the actual prevalence may be higher than reported. [6] Cirrhosis can lead to various fatal complications, such as hepatocellular carcinoma, hepatic encephalopathy, gastrointestinal hemorrhage, infections and so on. [7, 8] All of which may increase the mortality of patients and bring financial burdens to patients' families and the society. [9] Critically ill cirrhosis is a type of clinical critical illness with a high death rate and attracts increasing attention. [10] Existing study has focused on patients with critically ill cirrhosis. A survey abroad has indicated the 30-day in-hospital mortality of patients with critically ill cirrhosis and bacterial ascites reaches 33%. [11] The high death rate has highlighted the importance of early identifying the patients with high risk, which is helpful to earlier make treatments. Our study aimed to establish a prediction model to predict the 30-day deaths of cirrhosis patients in intensive care unit (ICU) hospitalization using the data from Medical Information Mart of the Intensive Care Database III version 1.4. The prediction model is helpful to provide clinicians with an early prediction of hospital mortality and subsequent hints for adequate treatments of patients with high risk to decrease the 30-day in-hospital mortality. The patients' data were collected from MIMIC-IIIv1.4, which provided access for the public and was free of charge. The database contained the information of over 50,000 ICU patients who visited Beth Israel Deaconess Medical Centre (BIDMC, Boston, MA, USA) from 2001 to 2012. [12] Considering that our data were accessed from MIMIC-III database, an openly available dataset, there was no need of ethic approval and informed consent. All variables were recorded within 24 hours of ICU admission. For variables measured more than once, the result of the first measurement was included in the analysis. Data of patients were collected, including age, gender, heart rate, respiratory rate, systolic blood pressure (SBP), diastolic blood pressure (DBP), Oxygen saturation (SpO2), white blood cells (WBC), platelets (PLT), red blood cell (RBC), red cell distribution width (RDW), levels of glucose, creatinine, blood urea nitrogen (BUN), bicarbonate, total bilirubin (TBIL), hematocrit, hemoglobin, alanine transaminase, aspartate transaminase (AST), bilirubin, Simplified Acute Physiology Score (SAPS) II, Sequential Organ Failure Assessment (SOFA), chronic obstructive pulmonary disease (COPD), heart failure, diabetes, and septicemia. Missing values were deleted and outliers were replaced with null values. We fitted and imputed the missing values based on random forest. Sensitivity analysis was performed using the baseline data of the training set before and after the imputation, and no significant differences were shown (Supplementary Digital Content Table S1 , http://links. lww.com/MD2/A882). The outcome variables of this study were 30-day deaths of ICU patients with liver cirrhosis, and the start date of the record was the date that patients admitted to the hospital. The prediction model was developed using random forest method. Total samples were split into training set (n = 1288) and testing set (n = 552) with the ratio of 7:3. The logistic regression with L1 regularization was conducted to select variables from the training set for the construction of the prediction model, which was subsequently validated by a 6-fold cross-validation. The selected variables were independent variables, and 30-day in-hospital death was dependent variable. The number of decision trees was 800, and the maximum depth To determine the performance of the model, we calculated the accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the area under the curve (AUC). The cut-off points were ascertained based on the Youden Index. The model performance was validated using the data of testing set. The learning curve was used to evaluate the stability of this model and the calibration curve was used to assess imitative effect. The feature importance computed by Gini importance was used to assess the important variables. The Gini importance was computed from the Random Forest structure, and the average over all trees in the forest was the measure of the feature importance. The model has been uploaded in GitHub (https://github.com/abcgut/data_ model). The statistical analyses were performed using SAS9.4 and Python 3.7 software. Categorical data were presented as the number of cases and the constituent ratio (N (%)), and Chi-Squared test or Fisher exact probability method was adopted for the comparisons. Normally distributed continuous variables were expressed as mean ± standard deviation (Mean ± SD), and t-test was applied for the comparisons between the survival group and the death group. Those quantitative data of skewed distribution were displayed as median and quartiles (M [Q1, Q3]), and the between-group comparisons were tested by Wilcoxon. All statistical tests were two-sided, and P < .05 was considered to be statistically significant. A total of 1851 ICU patients with cirrhosis were selected from MIMIC database, with 11 patients missing the data of blood routine and liver function index. After processing the abnormal values and missing values, 1840 patients were finally included. These patients were randomly divided into the training set (n = 1288) and testing set (n = 552) ( Fig. 1 ). As shown in Table 1 , no Table 1 Baseline characteristics of training set and testing set. Characteristics of death group and survival group in the training set were compared in Table 2 , and results displayed the statistically significant differences between 2 groups in age, heart rate, respiratory rate, SBP, DBP, SpO2, WBC, PLT, RBC, RDW, glucose, creatinine, BUN, bicarbonate, TBIL, AST, bilirubin, SAPS II, SOFA and septicemia (P < .05). The data of training set were used to establish the random forest model. ALT = alanine transaminase, AST = aspartate transaminase, BUN = blood urea nitrogen, COPD = chronic obstructive pulmonary disease, DBP = diastolic blood pressure, PLT = platelets, RBC = red blood cell, RDW = red cell distribution width, SAPS = Simplified Acute Physiology Score, SBP = systolic blood pressure, SOFA = Sequential Organ Failure Assessment, SPO 2 = Oxygen saturation, TBIL = total bilirubin, WBC = white blood cells. Prediction effect of the random forest model. (Fig. 2) . The internal validation was conducted to assess the efficacy of prediction model using the testing set and the results revealed that the AUC was 0.801 (95% CI: 0.799-0.802), the accuracy, sensitivity, specificity, PPV and NPV of prediction model were 0.734, 0.710, 0.744, 0.535 and 0.861, respectively ( Table 3 ). The receiver operating characteristic (ROC) curves of the model and internal verification were shown in Figure 3 . The learning curve of random forest model was presented in Figure 4 , and it showed that the prediction effect of the model on the training set and testing set tended to be steady with the increase of sample size, indicating that the performance of the model was relatively stable. The imitative effect of current model on the training set and testing set was respectively shown in Figure 5a and 5b. In this case-control study, we collected clinical data of 1840 cirrhotic patients in MIMIC-III database and established a model to predict 30-day in-hospital deaths using a random forest analysis. The final results displayed that the random forest model was effective in predicting 30-day death of ICU patients with cirrhosis. SAPS II, SOFA, BUN, TBIL and bilirubin were important factors for the 30-day death of cirrhotic patients. These results showed that our model could help clinicians identify the high-risk patients and earlier make intervention to improve patients' prognosis. Most of the studies tended to establish predictive models using prognostic scores to explore the 30-day outcomes of patients. [28] [29] [30] [31] Jacqueline et al conducted North American Consortium for the Study of End-Stage Liver Disease-Acute-on-Chronic Liver Failure Score to assess mortality risk in hospitalized cirrhotic patients. Multivariable modeling demonstrated that this score was an independently validated tool to predict 30-day survival in cirrhotic patients. The sensitivity and specificity were 84% and 70%, respectively. Huang and Yao [31] established a new predictive model with combination of ascites albumin, neutrophil to lymphocyte ratio, and MELD. Through logistic multivariate regression analysis, ascites albumin, neutrophil to lymphocyte ratio, and MELD were identified as the 3 independent risk factors related to the 30-day death of patients with liver cirrhosis and Logistic regression model has certain requirements for sample size, which theoretically requires a large sample, otherwise the test formula is unreasonable. Furthermore, logistic regression model cannot solve the problem of multicollinearity. As far as we know, there is rarely study using random forest model to predict the death of cirrhotic patients within 30 days of admission up to now. In this study, we established a random forest model to The random forest model could accommodate numerous variables, and had strong predictive power and better tolerance to data outliers. The AUC of our model was as high as 0.908 with the sensitivity of 83.6% and the specificity of 80.6%, indicating the good performance of the random forest model in clinical application. The role of SAPS II and SOFA in predicting hospital mortality of ICU patients has been reported in numerous studies. [13] [14] [15] [16] [17] Dupont et al [16] conducted a retrospective study to assess the predictive abilities of different prognostic scores, and results revealed the superiority of SOFA and Model for End-Stage Liver Disease (MELD) score compared to other prognostic scores for mortality prediction in ICU patients hospitalized with a diagnosis of cirrhosis. SOFA was considered as the best prognostic score to evaluate cirrhotic patients in the ICU according to nearly all of the literature. [13, [18] [19] [20] Our study also identified SOFA as an important predictor for death, and SAPS II presented better discriminative ability for death of cirrhotic patients within 30-day hospitalization. A prior prospective study reached a conclusion that SAPA II and SOFA showed better prediction performance than MELD in ICU mortality for cirrhotic patients. [13] In the future, larger sample sizes are needed to verify the priorities of different prognostic scoring systems in ICU cirrhotic patients. Additionally, elevated BUN and bilirubin were found to be independently correlated with hospital mortality. [16] Ning et al [21] discussed the clinical features and prognosis in Chinese cirrhotic patients with ascites, and found the concentration of BUN was an independent risk factor for 30-day hospital mortality. The serum bilirubin level better reflects the liver's synthetic and excretory functions, thus, the mass of prognostic scoring systems included TBIL and bilirubin as ingredients. [22, 23] Our study demonstrated that BUN, TBIL and bilirubin were significant predictors for 30-day admission death. Previous studies provided a specific explanation. [24] [25] [26] It is reported that intrahepatic cholestasis, portal flow distortion or shunting, and hemolysis caused by splenomegaly may all lead to the increased level of bilirubin. Recent research compared the value of bilirubin and TBIL for predicting prognosis of cirrhotic patients, and results showed bilirubin performed better predictive value. [27] Our study used random forest model to predict the 30-day inhospital deaths of ICU cirrhotic patients, and logistic regression was used to screen out the important variables to establish the prediction model. The internal validation had confirmed that the random forest model could perform well. However, some limitations should be concerned. First, our data were collected from the Medical Information Mart of the Intensive Care Database III version 1.4, which inevitably existed data missing. Data missing may affect the performance of the model, and some potential valuable variables for prediction may exclude due to severe data missing. Second, since our study included the patients with critically ill cirrhosis, the prediction performance in those with mild or moderate cirrhosis was unclear. Moreover, our study was lack of external validation, which needed to perform in the future. In conclusion, our study developed and validated a random forest model to predict the 30-day in-hospital death for cirrhotic patients in ICU. Our model had showed a good predictive performance, and suggested more attentions on SAPS II, SOFA, BUN, TBIL, and bilirubin, indicating that it may be popularized in clinical practice. Applying this model, clinicians may be provided with an early prediction of hospital mortality and take adequate treatments in time for high-risk patients to decrease the 30-day in-hospital mortality of cirrhosis patients in ICU. Survival of endogenous hepatic stem/ progenitor cells in liver tissues during liver cirrhosis A population-based cohort study of mortality of intensive care unit patients with liver cirrhosis Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study Liver cirrhosis Roudot-Thoraval F. The burden of liver disease in Europe: a review of available epidemiological data A/H1N1/09 influenza is associated with high mortality in liver cirrhosis Standard assessments of frailty are validated predictors of mortality in hospitalized patients with cirrhosis The multi-dimensional burden of cirrhosis and hepatic encephalopathy on patients and caregivers Palliative care and health care utilization for patients with end-stage liver disease at the end of life Transfusion with cryoprecipitate for very low fibrinogen levels does not affect bleeding or survival in critically Ill cirrhosis patients High rates of 30-day mortality in patients with cirrhosis and COVID-19 MIMIC-III, a freely accessible critical care database Prospective evaluation of the prognostic scores for cirrhotic patients admitted to an intensive care unit Review article: scoring systems for assessing prognosis in critically ill adult cirrhotics Short-term prognosis in critically ill patients with cirrhosis assessed by prognostic scoring systems Retrospective evaluation of prognostic score performances in cirrhotic patients admitted to an intermediate care unit Medicine (2022) 101:5 www.md-journal Comparison of the predictive value of scoring systems on the prognosis of cirrhotic patients with suspected infection Outcomes of critically ill patients with cirrhosis admitted to intensive care: an important perspective from the non-transplant setting Comparison of the severity of illness scoring systems for critically ill cirrhotic patients with renal failure Multiple organ system failure in critically ill cirrhotic patients. A comparison of two multiple organ dysfunction/failure scoring systems Clinical and bacteriological features and prognosis of ascitic fluid infection in Chinese patients with cirrhosis Surgery and portal hypertension A model to predict survival in patients with end-stage liver disease Jaundice in cirrhosis Bilirubin metabolism in liver cirrhosis Hepatocyte senescence explains conjugated bilirubin aemia in chronic liver failure Direct bilirubin is more valuable than total bilirubin for predicting prognosis in patients with liver cirrhosis NACSELD acute-onchronic liver failure (NACSELD-ACLF) score predicts 30-day survival in hospitalized patients with cirrhosis Validation of the clinical frailty scale for prediction of thirty-day mortality in the emergency department Improved prognosis of septic shock in patients with cirrhosis: a multicenter study * Establishment of a predictive model of death within 30 days for patients with liver cirrhosis and bacterial ascites Editor: Nikolay Efimov.The authors have no funding and conflicts of interests to disclose.Supplemental Digital Content is available for this article.The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.