key: cord-0792579-tdjajatm
authors: Sun, Louise Y.; Bader Eddeen, Anan; Ruel, Marc; MacPhee, Erika; Mesana, Thierry G.
title: Derivation and Validation of a Clinical Model to Predict Intensive Care Unit Length of Stay After Cardiac Surgery
date: 2020-10-21
journal: J Am Heart Assoc
DOI: 10.1161/jaha.120.017847
sha: c0936b9eafa9e367bc6849df9aed9845adb2b01c
doc_id: 792579
cord_uid: tdjajatm

BACKGROUND: Across the globe, elective surgeries have been postponed to limit infectious exposure and preserve hospital capacity for coronavirus disease 2019 (COVID‐19). However, the ramp down in cardiac surgery volumes may result in unintended harm to patients who are at high risk of mortality if their conditions are left untreated. To help optimize triage decisions, we derived and ambispectively validated a clinical score to predict intensive care unit length of stay after cardiac surgery. METHODS AND RESULTS: Following ethics approval, we derived and performed multicenter valida tion of clinical models to predict the likelihood of short (≤2 days) and prolonged intensive care unit length of stay (≥7 days) in patients aged ≥18 years, who underwent coronary artery bypass grafting and/or aortic, mitral, and tricuspid value surgery in Ontario, Canada. Multivariable logistic regression with backward variable selection was used, along with clinical judgment, in the modeling process. For the model that predicted short intensive care unit stay, the c‐statistic was 0.78 in the derivation cohort and 0.71 in the validation cohort. For the model that predicted prolonged stay, c‐statistic was 0.85 in the derivation and 0.78 in the validation cohort. The models, together termed the CardiOttawa LOS Score, demonstrated a high degree of accuracy during prospective testing. CONCLUSIONS: Clinical judgment alone has been shown to be inaccurate in predicting postoperative intensive care unit length of stay. The CardiOttawa LOS Score performed well in prospective validation and will complement the clinician's gestalt in making more efficient resource allocation during the COVID‐19 period and beyond.

S ince having been declared a Public Health Emergency of International Concern by the World Health Organization on January 30, 2020, the coronavirus disease 2019 (COVID-19) outbreak has rapidly redefined societal norms and challenged healthcare systems across the globe. COVID-19 was declared a pandemic on March 11, 2020 . By then, the availability of intensive care unit (ICU) resources had already begun to fall short of the increasing number of critically ill patients in some regions. Amidst this crisis, surgical patients continue to require lifesaving ICU resources. Although elective surgical procedures have been universally postponed, a significant number of patients with advanced, symptomatic cardiac diseases continue to require cardiac surgery on an urgent basis to prevent disease decompensation and death. This need challenges system capacity, given the complex comorbidities that often coexist with cardiac surgical disease, as well as the demand for ICU monitoring after cardiac surgery.

The current paradigm of triage decision-making is primarily driven by clinicians' judgment and experience, which has been shown to be highly inaccurate in predicting prolonged cardiac surgical ICU (CSICU) length of stay (LOS). 1 Although several objective clinical CSICU LOS models have been proposed, they are all built on small single-center data sets, lack multicenter external validation, and rely on intraoperative and postoperative data to achieve modest discrimination. With a goal to save more lives while maintaining an efficient and adaptable allocation of critical care resources during this crisis, we derived and ambispectively validated a pair of clinical prediction models to provide individualized predictions of CSICU LOS after major cardiac surgery.

The data set from this study is held securely in coded form at the Institute for Clinical Evaluative Sciences (ICES). While data-sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS.

This is an ambispective study, where we began by deriving models to predict low and high ICU resource use after cardiac surgery (defined by CSICU LOS of ≤2 and ≥7 days, respectively), using data available at the University of Ottawa Heart Institute (UOHI). We validated these models using a concurrent cohort of non-UOHI cardiac surgery patients in Ontario. We then tested these models prospectively at the UOHI.

Included were adult patients aged ≥18 years who underwent coronary artery bypass grafting, and/or aortic, mitral, and tricuspid valve surgery. Excluded were patients who underwent procedures requiring circulatory arrest, as well as cardiac transplantation and ventricular assist devices. For patients with multiple cardiac procedures during the study period, only the index procedure was included in the analyses.

The UOHI research ethics board approved the study and waived the need for individual patient consent. All 6625 patients who underwent cardiac surgery at the UOHI between November 1, 2009, and March 31, 2015, and met the selection criteria were included in the derivation cohort. We used prospectively collected clinical data from Cardiocore, a multimodular data repository that captures detailed demographics, comorbidities, procedural details, and outcomes of all patients who underwent cardiac surgical procedures at the UOHI, a university-affiliated tertiary referral center that performs the full scope of cardiac operations. Cardiocore is managed by a multidisciplinary committee and undergoes regularly scheduled quality audits. [2] [3] [4] Validation Cohort

The validation cohort consisted of cardiac surgical patients from 7 other cardiac care centers in Ontario who met the selection criteria between October 1, 2008 and December 31, 2018. Ontario is the most populous province in Canada, with 13 million residents and one of the most ethnically diverse jurisdictions in the world. The use of data was authorized under section 45 of Ontario's Personal Health Information Protection Act, which does not require review by a research ethics board. 5 We used the clinical registry data from CorHealth Ontario, and population-level administrative healthcare databases available at ICES. CorHealth maintains a detailed prospective registry of all patients who undergo invasive cardiac procedures in Ontario, including demographic, comorbidity, and proceduralrelated information. CorHealth data undergo selected chart audits and core laboratory validation. 6 Using unique confidential identifiers, we linked the CorHealth Ontario registry (date and type of cardiac procedures, physiologic, and comorbidity data) with the Canadian Institute for Health Information Discharge Abstract Database (DAD; comorbidities and hospital admissions), the Ontario Health Insurance Plan (OHIP) database (physician service claims), and the Registered Persons Database (vital statistics). These administrative databases have been validated for many outcomes, exposures, and comorbidities, including CLINICAL PERSPECTIVE What Is New?

• We derived and ambispectively validated CardiOttawa, a bimodal score for predicting short (≤2 days) and prolonged (≥7 days) intensive care unit length of stay after cardiac surgery.

What Are the Clinical Implications?

• The CardiOttawa Score will complement the clinician's gestalt in making more efficient resource allocation and may be used as quality benchmark during the coronavirus disease 2019 (COVID-19) period and beyond. [7] [8] [9] [10] Potential covariates considered in the analyses are detailed in Table 1 and included age, sex, body mass index, smoking, hypertension, left ventricular ejection fraction, myocardial infarction within 30 days before surgery, Canadian Cardiovascular Society (CCS) angina class, New York Heart Association class, atrial fibrillation, endocarditis, stroke, peripheral arterial disease, glomerular filtration rate, dialysis, diabetes mellitus treated with oral hypoglycemics and/or insulin, anemia, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and type of surgery. The definitions for these variables are provided in Table S1 .

As with our previous studies, height and weight were identified from the CorHealth registry, and procedural urgency was ascertained from CorHealth and OHIP using an established algorithm. [11] [12] [13] [14] In addition, comorbidities were identified from the CorHealth Ontario registry and supplemented with data from DAD and OHIP using International Classification of Diseases, Tenth Revision (ICD-10) (Canada), codes 15 within 5 years before the index procedure, according to validated algorithms. 7, 9, 16, 17 Outcomes The primary outcome was the total length of CSICU stay during the index surgical admission. Specifically, short CSICU LOS was defined as ≤2 days and prolonged LOS was defined as ≥7 days.

Continuous variables were compared with a 2-sample t test or Wilcoxon rank sum test for non-normally distributed data. Categorical variables were compared with a chi-square test. 

Model discrimination in both the derivation and validation data sets was assessed using the c-statistic. We assessed calibration using the Hosmer-Lemeshow chisquare statistic and by comparing the number of observed versus expected events in each risk quintile. We assessed model performance using the Brier score. 18 For each of the LOS models, we constructed a predictiveness curve in the validation data set by plotting ordered risk percentile on the x-axis, and the probabilities of LOS ≤2 and ≥7 days, respectively, on the y axis. Other measures of model performance, such as sensitivity and specificity and positive and negative predictive values, were determined by examining LOS in higher or lower risk groups at the optimal cutoff value. We prospectively tested these predictive models at our institution between April 6 to 20, 2020, and descriptive statistics for the testing period are presented below. Analyses were performed using SAS version 9.4 (SAS Institute Inc), with statistical significance defined by a 2-sided P value <0.05.

Among the 6625 patients in the derivation cohort, 4201 (63.4%) stayed in the CSICU for ≤2 days and 692 (10.4%) for ≥7 days. Among 65 410 patients in the validation cohort, 50 442 (77.1%) stayed in the CSICU for ≤2 days and 3364 (5.1%) for ≥7 days. The baseline characteristics of both cohorts were similar, with the exception that patients in the derivation cohort were younger and more likely to undergo complex surgery, smoke, and have atrial fibrillation and anemia. Patients in the validation cohort were more likely to have CCS angina class 4 symptoms and undergo isolated coronary artery bypass grafting ( Table 1) .

The multivariable predictors of short and prolonged CSICU LOS are presented in Tables 2 and 3 , respectively. Of the candidate covariates evaluated, younger age; female sex; lower body mass index, CCS angina class, and New York Heart Association class; higher left ventricular ejection fraction; and absence of atrial fibrillation, endocarditis, stroke, peripheral arterial disease, anemia, higher glomerular filtration rate, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and procedure type, were predictors of short CSICU LOS.

Age and sex were forced into the prolonged LOS model on the basis of clinical significance. Other multivariable predictors of prolonged CSICU LOS were body mass index, New York Heart Association class, left ventricular ejection fraction, hypertension, atrial fibrillation, endocarditis, anemia, glomerular filtration rate, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and procedure type.

In the derivation data set, the c-statistic of the multivariable model was 0.78 and the Hosmer-Lemeshow chi-square statistic was 12.71 (P=0.12). In the validation data set, the c-statistic of the multivariable model was 0.71 and the Hosmer-Lemeshow chi-square statistic was 626.9 (P<0.001). The Brier score was 0.16. Figure-Panel A) , 60% of patients had predicted probabilities exceeding the average rate of short stay. The optimal cutoff point on the receiver operating characteristic curve was at a predicted probability of 76.3%, with the following 

In the derivation data set, the c-statistic of the multivariable model was 0.85 and the Hosmer-Lemeshow chisquare statistic was 18.54 (P=0.02). In the validation data set, the c-statistic of the multivariable model was 0.78 and the Hosmer-Lemeshow chi-square statistic was 131.43 (P<0.001). The Brier score was 0.047. Table 5 is a calibration table showing the rates of prolonged CSICU LOS according to each risk quintile. The number of observed patients with an LOS ≥7 days was similar to that predicted among all quintiles. Specifically, the average observed probability of short stay was 0.8% in quintile 1 (predicted probability 0.9%), 1.7% in quintile 2 (predicted 1.6%), 3.0% in quintile 3 (predicted 2.5%), 5.5% in quintile 4 (predicted 4.6%), and 14.8% in quintile 5 (predicted probability 17.2%). On examining the predictiveness curve (Figure-Panel B) , 22% of patients had predicted probabilities that exceeded the average rate of prolonged stay. The optimal cutoff point on the receiver operating characteristic curve was at a predicted risk of 3.9% (sensitivity, 73.2%; specificity, 68.8%; positive predictive value, 11.3%; negative predictive value, 97.9%). At the 25th, 50th, and 75th percentiles of risk, sensitivities were 95.6%, 85.3%, and 64.1%, respectively, whereas negative predictive values were 99.1%, 98.5%, and 97.5%, respectively.

A small number of patients died before postoperative day 7, amounting to 24 (0.56%) of the derivation cohort and 583 (0.89%) of the validation cohort. As perioperative death and prolonged ICU LOS are highly correlated, we tested the ability of the CardiOttawa to predict death before postoperative day 7 or ICU LOS ≥7 days as a composite outcome in the validation data set. Model performance for this composite outcome was mostly unchanged as compared with that of prolonged ICU LOS alone. Specifically, the c-statistic was 0.79, Hosmer-Lemeshow P<0.001, and Brier Score 0.051.

The β coefficients for the logistic models are presented in Tables 2 and 3 (online calculator available at https:// cardi ottawa.ottaw aheart.ca/).

During the beta testing period from April 6 to 20, 2020, a total of 42 patients who were evaluated with the CardiOttawa LOS Score proceeded to have surgery on an urgent basis. Using a predictive threshold of ≥70%, 35 of 38 (92.1%) patients who were predicted to have CSICU LOS of ≤2 days actually did. One patient was predicted to have an LOS of ≥7 days but had intraoperative death. The remaining 3 patients were classified as "indeterminate" (ie, had predicted probabilities of ≤50% for both short and prolonged LOS). Of these patients, 2 had an LOS of between 2 and 7 days and 1 had an LOS of ≥7 days.

Triaging decisions for cardiac surgery may be improved with the aid of objective evidence to more efficiently allocate ICU resources. However, evidence-based decision-support tools are lacking for this patient group. We developed and ambispectively validated clinical models to predict the likelihood of low and high CSICU resource use as defined by short (≤2 days) and prolonged (≥7 days) LOS, using variables that are readily available at the time of surgical referral. Our models predicted well during prospective testing.

Unlike previous models that were developed using small data sets and monotonically focused on predicting prolonged LOS, the CardiOttawa LOS Score demonstrated excellent performance in Ontario, which is the most populous and ethnically diverse province in Canada. An online calculator for these logistic models is available at https://cardi ottawa.ottaw aheart.ca/. The CardiOttawa LOS Score may help to optimize daily operative planning, whereby scheduling of cases with varying postoperative resource requirements could be staggered to maximize the number of urgent cases performed, while minimizing non-COVID ICU bed occupancy at any given time.

To our knowledge, the multicenter CardiOttawa LOS Score is thus far the best performing model, and the only validated model to provide bimodal LOS prediction after cardiac surgery. Previous models have focused on predicting prolonged CSICU LOS, which has been inconsistently defined in the literature (ranging between ≥1 and ≥10 days). 19 In a decade-old study that systematically reviewed published CSICU LOS models and externally validated them using single-center data (n=11 395), the areas under the receiver-operating characteristic curve of 6 general cardiac surgery models ranged between 0.57 and 0.75 for predicting LOS of ≥2 days. Of these, the Parsonnet and the European System for Cardiac Operative Risk Evaluation (EuroSCORE), which were originally intended for the prediction of mortality, had the highest discrimination (area under the receiver operating characteristic curve 0.75 and 0.71, respectively). 20 In the original single-center study that evaluated the performance of the EuroSCORE in LOS prediction (n=1562), the additive EuroSCORE was found to have areas under the receiver operating characteristic curve of 0.76 and 0.67 for predicting CSICU LOS of ≥7 and 2 days, respectively. The logistic EuroSCORE performed similarly in predicting these end points. 21 The CardiOttawa LOS Score is calibrated to modern practices and outcome patterns. It is comparably as parsimonious as the Parsonnet score and is simpler than the EuroSCORE while retaining elements of importance to triage decision-making, such as the presence of endocarditis and disease symptom severity. It predicts CSICU LOS of ≤2 days with an area under the receiver operating characteristic curve of 0.71 and ≥7 days with an area under the receiver operating characteristic curve of 0.78 in a large representative validation cohort of >65 000 patients. The CardiOttawa predictor variables are consistent with those identified in the literature 19 and capture important information on patient demographics, comorbid conditions, and the urgency and complexity of the scheduled procedure. Triaging decisions for cardiac surgery have traditionally been driven by clinical judgment, which may be no better than a coin toss in predicting the exact CSICU LOS after surgery. 1 In an era when the importance of ICU and hospital resource management cannot be overemphasized, it is worth noting that although clinicians are adept at identifying patients who will require a short CSICU LOS, they are only able to correctly identify those requiring a prolonged LOS 39% of the time. 1 Thus, our high-performing prolonged LOS model is well suited to complement the clinician's gestalt in the decision-making process.

The implications of the CardiOttawa LOS Score relate to its ability to support triaging decisions by complementing the physician's assessment of disease acuity and clinical factors with real-world data. The potential impact of the CardiOttawa LOS Score depends on the average CSICU LOS durations specific to each institution. At institutions with lower CSICU LOS durations after cardiac surgery, this score may help to identify the high resource users, whereas at institutions with longer CSICU LOS durations, this score may identify patients who are likely to have a rapid transition through the CSICU. Given its robust performance in prospective validation, the CardiOttawa LOS calculator could be used to benchmark the predicted versus observed CSICU LOS as a quality metric. It could also be used to identify patients who may benefit the most from preoperative optimization (ie, those who are most likely to require a prolonged LOS). Prospective studies are needed to examine whether a risk-stratified approach to optimizing conditions such as anemia and glycemic control could reduce CSICU LOS, while carefully balancing the potential benefits of optimization against the risk of delaying the procedure. The caveat that applies to all decision-support tools is pertinent, because the CardiOttawa LOS Score is intended to assist the clinician, who should ultimately synthesize the predictive score with clinical judgment in making decisions.

Major strengths of the CardiOttawa LOS Score are its generalizability in the broad cardiac surgery population, its suitability for use at the time of surgical referral, and its bimodal LOS prediction. As these models are intended to guide decisions regarding the timing of surgery based on disease acuity and anticipated ICU resource needs at a system level, it is important for model validation to be performed in a patient sample that is representative of the population they are intended to serve.

Our study has some limitations. First, as universal drug coverage is only available to Ontarians ≥65 years, we were unable to include information on prescription medications for all patients in the modeling process. However, medications have not routinely been incorporated in cardiac surgical risk models to date, and decision-support tools require a balance between variable inclusiveness and model simplicity, limiting the incorporation of an exhaustive list of potential factors. Second, we lack certain detailed physiologic measures such as brain natriuretic peptide in the databases used. However, brain natriuretic peptide is not routinely performed in the perioperative setting. Third, we lack certain procedure-related details such as the use of minimally invasive surgical techniques. However, such information is usually unavailable at the time of triage, before assignment of surgical staff and operative consultation by the attending surgeon.

More recent, a number of artificial intelligence algorithms have emerged to assist with CSICU LOS prediction, with some demonstrating promising results. However, these algorithms are still in the development phase and suffer from similar limitations as published statistical models (eg, single center with even smaller sample sizes, lack of external reproducibility, and a practical means of implementation). 22, 23 Further work is needed before they can be launched into clinical practice.

The CardiOttawa LOS Score is a set of simple clinical risk models that predict the likelihood of a short (≤2 days) postoperative CSICU LOS with moderate accuracy, and a prolonged (≥7 days) LOS with a high degree of accuracy. The importance of these predictive models is underscored by the inclusion of a population-based patient sample, its bimodal LOS prediction, and its utility in guiding triaging decisions in the COVID-19 era and beyond. The care and outcomes of all patients requiring ICU resources may be substantially improved if clinical judgment is supported by objective quantification in the planning of care.

Can clinicians predict ICU length of stay following cardiac surgery?

Defining an intraoperative hypotension threshold in association with de novo renal replacement therapy after cardiac surgery

Defining an intraoperative hypotension threshold in association with stroke in cardiac surgery

The association between tracheostomy and sternal wound infection in postoperative cardiac surgery patients

Hepatitis A vaccine versus immune globulin for postexposure prophylaxis

Determinants of variations in coronary revascularization practices

Accuracy of administrative databases in identifying patients with hypertension

Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. Toronto, Canada: Institute for Clinical Evaluative Sciences

Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm

A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario

Prevalence and long-term survival after coronary artery bypass grafting in men and women with heart failure and preserved vs reduced ejection fraction

Disability-free survival after coronary artery bypass grafting in women and men with heart failure

Sex differences in long-term survival after major cardiac surgery: a population-based cohort study

Long-term outcomes in patients with severely reduced left ventricular ejection fraction undergoing percutaneous coronary intervention vs coronary artery bypass grafting

Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data

Identifying individuals with physician diagnosed COPD in health administrative databases

Identifying cases of congestive heart failure from administrative data: a validation study using primary care patient records. Chronic Dis Inj Can

Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors

Prediction of prolonged length of stay in the intensive care unit after cardiac surgery: the need for a multi-institutional risk scoring system

Prediction models for prolonged intensive care unit stay after cardiac surgery: systematic review and validation study

Is EuroSCORE useful in the prediction of extended intensive care unit stay after cardiac surgery?

Predicting length of stay in intensive care units after cardiac surgery: comparison of artificial neural networks and adaptive neuro-fuzzy system

Neural network prediction of ICU length of stay following cardiac surgery based on pre-incision variables

European system for cardiac operative risk evaluation

The Society of Thoracic Surgeons National Database

Prediction of creatinine clearance from serum creatinine

The authors also acknowledge the use of data compiled and provided by the Canadian Institute for Health Information. These data sets were linked using unique encoded identifiers and analyzed at ICES. The analyses, conclusions, opinions and statements expressed in the manuscript are those of the authors, and do not necessarily reflect those of the above agencies.

This study was supported by an operating grant from the Canadian Institutes of Health Research. Sun was named National New Investigator by the Heart and Stroke Foundation of Canada and is supported by the Ottawa Heart Institute Research Corporation and the Tier 2 Clinical Research Chair in Big Data and Cardiovascular Outcomes at the University of Ottawa. Ruel and Mesana are supported by endowed research chairs at the UOHI. This study is supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The authors acknowledge that the clinical registry data used in this analysis are from participating hospitals through CorHealth Ontario, which serves as an advisory body to the MOHLTC, is funded by the MOHLTC, and is dedicated to improving the quality, efficiency, access, and equity in the delivery of the continuum of adult cardiac and stroke care in Ontario, Canada.

None. Requirement for inotropic support with evidence of end organ hypoperfusion or dysfunction or intraaortic balloon pump in situ before surgery These definitions are in keeping with definitions employed by EuroSCORE 25 and/or the STS database. 26