key: cord-0065965-iztxq5j3
authors: Christou, Chrysanthos Dimitris; Naar, Leon; Kongkaewpaisan, Napaporn; Tsolakidis, Alexandros; Smyrnis, Panagiotis; Tooulias, Andreas; Tsoulfas, Georgios; Papadopoulos, Vasileios Nikolaos; Velmahos, George Constantinos; Kaafarani, Haytham Mohamed Ali
title: Validation of the Emergency Surgery Score (ESS) in a Greek patient population: a prospective bi-institutional cohort study
date: 2021-07-22
journal: Eur J Trauma Emerg Surg
DOI: 10.1007/s00068-021-01734-9
sha: 0a2aceadf61a4a5dc79f015a6f028e705254886b
doc_id: 65965
cord_uid: iztxq5j3

PURPOSE: The Emergency Surgery Score (ESS) is a reliable point-based score that predicts mortality and morbidity in emergency surgery patients. However, it has been validated only in the U.S. patients. We aimed to prospectively validate ESS in a Greek patient population. METHODS: All patients who underwent an emergent laparotomy were prospectively included over a 15-month period. A systematic chart review was performed to collect relevant preoperative, intraoperative, and postoperative variables based on which the ESS was calculated for each patient. The relationship between ESS and 30-day mortality, morbidity (i.e., the occurrence of at least one complication), and the need for intensive care unit (ICU) admission was evaluated and compared between the Greek and U.S. patients using the c-statistics methodology. The study was registered on "Research Registry" with the unique identifying number 5901. RESULTS: A total of 214 patients (102 Greek) were included. The mean age was 64 years, 44% were female, and the median ESS was 7. The most common indication for surgery was hollow viscus perforation (25%). The ESS reliably and incrementally predicted mortality (c-statistics = 0.79 [95% CI 0.67–0.90] and 0.83 [95% CI 0.74–0.92]), morbidity (c-statistics = 0.83 [95% CI 0.76–0.91] and 0.79 [95% CI 0.69–0.88]), and ICU admission (c-statistics = 0.88 [95% CI 0.81–0.96] and 0.84 [95% CI 0.77–0.91]) in both Greek and U.S. patients. CONCLUSION: The correlation between the ESS and the surgical outcomes was statistically significant in both Greek and U.S. patients undergoing emergency laparotomy. ESS could prove globally useful for preoperative patient counseling and quality-of-care benchmarking.

Emergency General Surgery (EGS) is associated with increased mortality and morbidity compared with elective surgery [1] . While EGS accounts for 7.1% of hospitalizations in the US, it disproportionally contributes to a large portion of overall perioperative mortality and morbidity [2] [3] [4] . Existing risk stratification calculators and decision-making tools such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) risk calculator were not explicitly designed for EGS patients, and thus, do not account for the acuity of the disease while erroneously assume that the same risk factors impact EGS and non-EGS patients in a similar fashion [5, 6] .

The Emergency Surgery Score (ESS) was derived in 2016 as a novel preoperative risk assessment tool specifically designed for patients undergoing EGS [7, 8] . ESS was initially validated as a predictor of 30-day mortality. Since then, ESS has been consistently shown to predict not only mortality but also postoperative complications and the need for postoperative intensive care unit (ICU) admission [9] [10] [11] [12] [13] [14] [15] . More recently, an Eastern Association for the Surgery of Trauma (EAST) prospective multicenter study confirmed ESS as a reliable predictor of surgical outcomes, including 30-day postoperative mortality, morbidity, and the need for admission to the ICU [12] .

As such, ESS is now being used in multiple institutions as a tool for bedside counseling of EGS patients and families, and its use as a benchmarking tool across hospitals is being evaluated [16] [17] [18] . However, ESS was originally derived from a United States (U.S.) national database and has been exclusively studied in the U.S. patient population. Its performance in non-U.S. patients remains unknown. We sought to prospectively study ESS's performance in a Greek patient population cohort compared to a similar U.S. patient population.

We conducted a prospective observational cohort study in two tertiary academic centers in Thessaloniki, Greece, and Boston, Massachusetts, USA. The Greek department is a public, academic, general surgery department in a tertiary (level I/II equivalency) hospital covering the emergencies of Thessaloniki and the surrounding areas. The US department is a private, academic, general surgery department in a tertiary, ACS level I trauma hospital covering the emergencies of Boston and the surrounding areas.

This study was conducted in accordance with STROCSS 2019 Guidelines [19] and was registered in "Research Registry," a publicly accessible database for studies involving human subjects with the unique identifying number 5901. Over a 15-month period (from 01/2019 until 03/2020 for the Greek department and from 04/2018 until 06/2019 for the US department), all adult patients who underwent emergency laparotomy in either hospital were included to capture a high-risk, severely ill cohort of patients. An emergency laparotomy was defined as one performed as soon as possible following diagnosis or after the onset of related preoperative symptomatology, where an unnecessary delay could potentially endanger the patient's well-being and outcome [20] . Laparoscopies and inguinal hernia repairs were excluded, since they do not constitute high-risk laparotomies. Trauma-and vascular-related laparotomies were excluded, since they do not reflect the underlying pathophysiology of EGS patients. The Institutional Review Boards of both institutions approved this study granting a waiver of informed consent.

Preoperative, intraoperative, and postoperative information was systematically collected in all patients that underwent an emergency laparotomy during the above study period. Standard American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) definitions were used [20] . The primary outcome of this study was 30-day mortality. Secondary outcomes included in-hospital morbidity and the requirement for postoperative ICU admission. Thirty-day mortality was defined as the death of any patient within 30 days from the index emergency laparotomy. Postoperative morbidity was a composite outcome including the occurrence within 30 days postoperatively of one or more of the following complications: superficial, deep, and organ/space surgical site infections, wound dehiscence, deep venous thrombosis, pulmonary embolism, pneumonia, unplanned intubation, progressive renal insufficiency, acute renal failure, urinary tract infection, cerebrovascular accident (stroke), cardiac arrest requiring cardiopulmonary resuscitation, myocardial infarction, bleeding necessitating postoperative blood transfusions, sepsis, septic shock, ventilator requirement for > 48 h, coma > 24 h, peripheral nerve injury, and graft/prosthesis/flap failure. The requirement for postoperative ICU level of care was defined as postoperative ICU admission at any time during the index hospitalization.

Algorithms were generated to calculate the ESS for each patient using the demographic data, comorbidities, and laboratory values included in the database. The variables included in the ESS have been thoroughly described in our previous studies by Sangji et al. [7, 8] . None of the patients included in the current analysis had missing data required for the calculation of ESS.

We assessed ESS's ability to predict 30-day mortality, inhospital morbidity, and the postoperative need for ICU level of care using the receiver operator characteristic (ROC) analyses. Receiver operative characteristic curves were computed, and the c-statistics (area under the curve-AUROC) were calculated. The probability of a randomly chosen subject with a positive outcome having a higher score compared to a randomly selected subject without the outcome of interest is represented by the c-statistic [21] . Based on a common c-statistic classification, we described the correlation between the ESS score and the positive outcome as acceptable (0.7 ≤ AUROC < 0.8), excellent (0.8 ≤ AUROC < 0.9), and outstanding (AUROC ≥ 0.9) [22] .

Moreover, we compared the distribution of the variables used in the ESS calculation between the patients' two cohorts. Categorical variables were compared with Fisher's exact or Chi-square test, where indicated, and the results are summarized in an absolute number of patients and percentages. The significance level was set at a p value < 0.05. All statistical analyses were performed using the Stata v15.1 (StataCorp 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC).

A total of 214 patients were included, 102 (48%) Greek, and 112 (52%) in the U.S. The mean age of the population was 64 ± 18 years, and 94 (44%) were females. The most common indication for surgery was hollow viscus perforation (25%), and the median ESS was 7 [4] [5] [6] [7] [8] [9] [10] .

The distribution of the ESS in the Greek and U.S. patient populations is shown in Fig. 1 . Table 1 compares the Greek and U.S. patient populations. Notably, Greek patients were more likely to be white (100% vs. 84%, p value < 0.001) and less likely to be transferred from an outside emergency department (1% vs. 18%, p value < 0.001) or an acute care hospital inpatient facility (2% vs. 34%, p value < 0.001). The rest of the patients presented directly to each of the hospital's EDs, rather than being transferred from another hospital's ED or were inpatients already at the time of consultation from the surgical team. Furthermore, Greek patients had lower rates of functional dependence (10% vs. 25%, p value = 0.004), steroid use (1% vs. 7%, p value = 0.037), and preoperative ventilator dependency (2% vs. 16%, p value < 0.001). Regarding the distribution of laboratory values, U.S. patients had higher alkaline phosphatase levels (23% vs. 12%, p value = 0.032) and international normalized ratio (28% vs. 10%, p value < 0.001). Finally, U.S. patients were found to have significantly higher ESS as compared to their Greek counterparts (median: 8 vs. 6, p value = 0.013).

The overall 30-day patient mortality rate was 16%; the unadjusted 30-day mortality was similar in Greek and US patients (15% vs. 18%, p value = 0.582). The median duration from the index surgical procedure to mortality was 4 [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] days. As ESS increased, the 30-day mortality rates gradually increased; for example, ESS of 1, 6, 11, and 19 were associated with 0%, 14%, 44%, and 100% mortality, respectively. The overall c-statistic for mortality was 0.81[95% CI: 0.74-0.88]. Figure 2 shows the receiver operative characteristic curves for the correlation of ESS with 30-day mortality in the Greek vs. U.S. cohorts of patients. Briefly, the ESS correlation was acceptable in the Greek cohort of patients (c-statistic = 0.79 [95% CI: 0.67-0.90]), and excellent with 30-day mortality in the U.S. population (c-statistic = 0.83 [95% CI: 0.74-0.92]). Specifically, for the Greek patients, the mortality rates for patients with ESS of 1, 6, 9, and 15 gradually increased from 0 to 20%, 50%, and 100%, respectively. Comparing the two ROC curves in the Greek and US population, we did not identify any significant differences regarding ESS's performance in the two cohorts of patients (p value = 0.53).

The overall incidence of postoperative complications was 55%. Table 2 illustrates the rates of all isolated postoperative complications included in the composite morbidity outcome. The most commonly observed complication was sepsis/septic shock (27%). As ESS increased, there was a step-wise increase in postoperative complications rates; for example, ESS of 1, 6, 11, and 19 were associated with 5%, 29%, 89%, and 100% morbidity rates, respectively. (4) (5) (6) (7) (8) (9) (10) (11) 85 (76) 64 (63) High score (12) (13) (14) (15) (16) (17) (18) (19) 16 (14) 11 (11) 

A total of 86 patients (40%) were admitted to the ICU postoperatively. The most common indication for ICU admission was hemodynamic instability (26%), followed by the need for mechanical respiratory support (25%). ICU admission rates gradually increased at higher ESS; For example, ESS of 1, 6, 11, and 19 correlated with ICU admission rates of 0%, 24%, 72%, and 100%, respectively. The correlation between the ESS and the requirement for postoperative ICU level of care was excellent in both Greek and U.S. patients with the requirement of postoperative ICU level of care (Fig. 4, c- 

In this study, we demonstrate that ESS performs well in predicting mortality, morbidity, and the postoperative need for ICU admission, not only in U.S. patients but also in non-U.S. patients, specifically a Greek patient population undergoing high-risk emergency laparotomy. Despite some notable differences in the two populations' baseline characteristics, ESS retained its reliability in predicting surgical outcomes. As such, our findings suggest that ESS could be used outside the U.S. for preoperative bedside counseling of patients and their families and for benchmarking the quality of EGS care on a broad scale. A recent multidisciplinary survey highlighted that physicians use risk stratification tools, mostly preoperatively [23] . Many popular risk classification tools such as the ASA, the ACS-NSQIP, the P-POSSUM, and the APACHE calculators require intraoperative variables to be calculated [24] [25] [26] . The NEWS score, similarly to ESS, utilizes only preoperative variables in the ED; however, it was not explicitly designed for EGS patients [27, 28] . Therefore, ESS offers an advantage in EGS patients not only because it was derived from this specific patient population but also because it accounts for the acuity of disease on presentation, and its accuracy is superior. In addition, it exclusively utilizes preoperative variables and is thus amenable to use before the operation. While the ACS-NSQIP Surgical Risk Calculator performs well in various surgical settings, it falls short of ESS performance regarding EGS [26, [29] [30] [31] [32] , most probably because it does not include variables that specifically affect the outcome in EGS [33] .

Unlike patients undergoing elective surgery, EGS patients often have a different pathophysiologic background, and many require ICU admission to recover [34] . ESS can be useful as a triage tool for ICU admission, especially in countries with limited access to critical care beds, such as Greece [35] . The previously proposed ESS cut-off value of ≥ 7 to admit a patient to the ICU [13] could be adjusted by each healthcare facility to reflect their specific existing resources and infrastructure. In addition, smaller facilities with a very limited number of available ICU beds could use the ESS to transfer patients to a healthcare facility of a higher level of care. Some studies have attempted to use the Surgical Apgar Score as a tool to predict the need for ICU admission [36, 37] . However, most of the cases involved elective surgery rather than EGS, and their performance is inferior to that of ESS [36] (c-statistic = 0.76).

Surgical risk is not linear, and artificial intelligence methods such as machine learning may improve our ability to assess surgical patients' risk beyond ESS in the future. For example, machine learning methods were recently used to develop the Predictive OpTimal Trees in Emergency Surgery (POTTER) Calculator [38] . The POTTER Calculator uses a machine learning method called Optimal Classification Trees [39] and demonstrates how the non-linear methods can provide evidence-based, accurate, user-friendly, and potentially actionable predictions of surgical outcomes in EGS patients [38, [40] [41] [42] .

Our study has a few limitations. The main limitation of our study is that it included only EGS patients who underwent surgery. Therefore, it does not apply to EGS patients being managed non-operatively. Second, we limited our study to emergency laparotomy, and therefore cannot generalize our findings to other EGS procedures. Third, additional data points (e.g., other comorbidities, transfer locations, advance directives of the patients, CMO status, etc.) not included in the ESS calculation were not collected in this study and could not be accounted for. Fourth, we had a low number of patients included in both cohorts of patients, which may have affected our study results. Furthermore, differences in infrastructure, capabilities, and eligibility of patients for laparoscopy between the two departments were not accounted for and could potentially affect our results. Finally, despite its prospective nature, our study did not evaluate ESS's potential impact on decision-making in EGS patients.

We have therefore demonstrated that ESS performs well in predicting mortality, morbidity, and the need for postoperative ICU admission in a population outside the U.S. ESS could be used in various surgical settings as a tool for preoperative patient counseling, risk stratification, benchmarking the quality of care, and for postoperative patient triage to higher levels of care.

The excess morbidity and mortality of emergency general surgery

The public health burden of emergency general surgery in the United States: A 10-year analysis of the Nationwide Inpatient Sample-2001 to 2010

30-Day and 1-year mortality in emergency general surgery laparotomies: an area of concern and need for improvement?

Critical differences between elective and emergency surgery: identifying domains for quality improvement in emergency general surgery. Surg

A calculator for mortality following emergency general surgery based on the american college of surgeons national surgical quality improvement program database

Perioperative risk factors impact outcomes in emergency versus nonemergency surgery differently: Time to separate our national risk-adjustment models

Derivation and validation of a novel physiological emergency surgery acuity score (PESAS)

Derivation and validation of a novel emergency surgery acuity score (ESAS)

The Emergency Surgery Score (ESS) accurately predicts the occurrence of postoperative complications in emergency surgery patients

Does the emergency surgery score accurately predict outcomes in emergent laparotomies?

Emergency Surgery Score accurately predicts the risk of post-operative infection in emergency general surgery

Prospective validation of the Emergency Surgery Score (ESS) in emergency general surgery: an EAST Multicenter Study

Can the emergency surgery score (ESS) be used as a triage tool predicting the postoperative need for an ICU admission?

The Emergency Surgery Score (ESS) accurately predicts outcomes in elderly patients undergoing emergency general surgery

Can the Emergency Surgery Score (ESS) predict outcomes in emergency general surgery patients with missing data elements? A nationwide analysis

The surgical risk scale as an improved tool for risk-adjusted analysis in comparative surgical audit

POSSUM: a scoring system for surgical audit

A comparison of the abilities of nine scoring algorithms in predicting mortality

Guideline: strengthening the reporting of cohort studies in surgery

A user guide for the 2014 ACS NSQIP participant use data file

The meaning and use of the area under a receiver operating characteristic (ROC) curve

Applied logistic regression

Multidisciplinary survey of current and future use of emergency laparotomy risk assessment scores in New Zealand

ASA classification and perioperative variables as predictors of postoperative outcome

Prognostic scoring systems for mortality in intensive care units-the APACHE model

Comparison of the risk prediction systems POSSUM and P-POSSUM with the Surgical Risk Scale: a prospective cohort study of 721 patients

The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death

A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study

Comparison of observed to predicted outcomes using the ACS NSQIP risk calculator in patients undergoing pancreaticoduodenectomy

The applicability of POSSUM and P-POSSUM scores as predictors of morbidity and mortality in colorectal surgery

Predictive value of the ACS NSQIP calculator for head and neck reconstruction free tissue transfer

Predicting postoperative complications after acute care surgery: how accurate is the ACS NSQIP surgical risk calculator?

Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons

Increased ICU resource needs for an academic emergency general surgery service*

The surgical Apgar score is strongly associated with intensive care unit admission after high-risk intraabdominal surgery

The surgical apgar score predicts postoperative ICU admission

Surgical risk is not linear: derivation and validation of a novel, userfriendly, and machine-learning-based predictive OpTimal trees in emergency surgery risk (POTTER) calculator

Optimal classification trees

Artificial intelligence in healthcare: past, present and future

Individual surgical decision-making and comparative effectiveness research

Strategies for improving comorbidity measures based on Medicare and Medicaid claims data