key: cord-0011247-gts0xwfa
authors: Glass, George; Hartka, Thomas R.; Keim-Malpass, Jessica; Enfield, Kyle B.; Clark, Matthew T.
title: Dynamic data in the ED predict requirement for ICU transfer following acute care admission
date: 2020-03-19
journal: J Clin Monit Comput
DOI: 10.1007/s10877-020-00500-3
sha: 238ab35a2dd7d72fc61d52369a196b83cb605876
doc_id: 11247
cord_uid: gts0xwfa

Misidentification of illness severity may lead to patients being admitted to a ward bed then unexpectedly transferring to an ICU as their condition deteriorates. Our objective was to develop a predictive analytic tool to identify emergency department (ED) patients that required upgrade to an intensive or intermediate care unit (ICU or IMU) within 24 h after being admitted to an acute care floor. We conducted a single-center retrospective cohort study to identify ED patients that were admitted to an acute care unit and identified cases where the patient was upgraded to ICU or IMU within 24 h. We used data available at the time of admission to build a logistic regression model that predicts early ICU transfer. We found 42,332 patients admitted between January 2012 and December 2016. There were 496 cases (1.2%) of early ICU transfer. Case patients had 18.0-fold higher mortality (11.1% vs. 0.6%, p < 0.001) and 3.4 days longer hospital stays (5.9 vs. 2.5, p < 0.001) than those without an early transfer. Our predictive analytic model had a cross-validated area under the receiver operating characteristic of 0.70 (95% CI 0.67–0.72) and identified 10% of early ICU transfers with an alert rate of 1.6 per week (162.2 acute care admits per week, 1.9 early ICU transfers). Predictive analytic monitoring based on data available in the emergency department can identify patients that will require upgrade to ICU or IMU if admitted to acute care. Incorporating this tool into ED practice may draw attention to high-risk patients before acute care admit and allow early intervention.

The majority of hospital admissions now come through the Emergency Department (ED) [1] and Emergency Medicine physicians usually provide the initial clinical determination of the appropriate destination for these patients. Misidentification of illness severity may lead to patients being admitted to an acute care unit then unexpectedly transferring to an ICU as their condition deteriorates [2] . While early unexpected ward-to-ICU transfers of ED admissions are infrequent (around 3% at 12 h post-admission) [3] , these events result in an increased need for ventilator support [4] , increased in-hospital and 30-day mortality [5] [6] [7] , and prolonged length-of-stay (LOS) [5, 8] . The rates of ED floor admissions with early ICU transfer has been proposed as a quality metric because of these worsened outcomes [9, 10] . For these reasons, methods for identification of the trajectory of patients prior to admission and determining the appropriate level of care is important.

Guidelines for appropriate ICU admission are in use, but current tools have significant limitations. The Society of Critical Care Medicine (SCCM) provides guidelines which rely primarily on need for specific resources (i.e., mechanical ventilation or vasopressor requirements) [11] . As such, outside of these clear indications for ICU care, the decision to admit to a floor ward versus ICU is based on the patient's perceived stability and consideration of other, often less clear, social and clinical determinants [12] . Early warning scores (EWS) exist for identifying acute and intensive care patients at risk for decompensation: CURB-65 predicts mortality due to pneumonia [13] ; sequential organ failure score (SOFA) and quick SOFA (qSOFA) scores predict mortality due sepsis [14] ; and Acute Physiology And Chronic Health Evaluation (APACHE) scores predict in hospital mortality at ICU admission [15] . It is unclear how well these scores predict unexpected ward-to-ICU transfers. While some previous studies have developed models for predicting unexpected ICU transfers, these studies are limited to patients presenting with specific disease entities [16] , do not take advantage of vital sign and laboratory data [17, 18] , and do not differentiate time of ward-to-ICU transfer after admission [18, 19] .

Accurate identification of ED patients that will decompensate within 24 h if admitted to acute care would allow for targeted interventions: early ICU consultation, ICU or step-down unit admission, or closer follow-up monitoring. This early intervention may avert later clinical deterioration, improving patient outcomes. It has been previously shown that patients with "borderline" ICU admission criteria for pneumonia have better outcomes when admitted to the ICU [12] . Given this, the development of an predictive model to help identify patients at risk for unexpected ward-to-ICU transfer is warranted. Such a tool should utilize available information, provide the flexibility to handle missing information, and provide real-time analysis based on updated data. The objective of this study was to develop a predictive analytic tool to identify ED patients at risk for decompensation within 24 h following acute care admission. The long-term goal of this approach would be to provide real-time risk assessment for clinical decision support to physicians and aid determination of appropriate disposition.

We undertook a retrospective cohort design to identify consecutive ED-to-inpatient admissions to the University of Virginia Medical Center (UVa), a 612-bed academic, tertiary-care center. Annual ED volume is approximately 60,000 with a 37.7% admission rate. Adult critical care units include a medical ICU, surgical-trauma ICU, coronary care ICU, thoracic and vascular surgery ICU, neuro/neurosurgical ICU, surgical-trauma intermediate care unit (IMU), thoracic and vascular surgery IMU, and a neuro/neurosurgical IMU. The primary outcome was acute care admission, followed within 24 h by transfer to IMU, ICU, or death (i.e., early ICU transfer). The time of inpatient admission and that of IMU or ICU transfer were determined using data warehouse admit-discharge-transfer records.

Patients were enrolled between Jan. 1, 2012 and Dec. 31, 2016 . The study included all adult patients (18 years old or greater) admitted through the ED to acute care: two general medicine wards, two cardiovascular medicine wards, and one ward each of cardiovascular surgery, medical/surgical, surgery, orthopedic/trauma, neurology, neurosurgery, hematology/oncology, and geriatric/palliative wards. We excluded patients admitted to pediatric units, labor and delivery, psychiatry, and post-anesthesia and care transfer units. Patients with DNR orders prior to, or within 24 h following, admission were excluded. This study was approved by the UVa Institutional Review Board with a waiver of consent.

The event population was patients with early ICU transfer following inpatient admission to acute care. All data available for patients during their ED stay were included in the analysis.

Patient data were extracted from the electronic data warehouse. Only data elements available during the ED visit were used as feature variables. Dynamic data included flowsheet vital signs and laboratory measurements frequently available at admission. We extracted 7 nurse-documented vital signs: pulse rate (99.8% available), respiratory rate (99.4%), oxygen saturation (99.6%), temperature (96.3%), Glasgow coma scale (GCS, 33.4%), and systolic and diastolic blood pressure (99.7%). We extracted 12 laboratory results from basic metabolic panels and complete blood counts, both 92% available. These intermittent features were combined using sample-and-hold, and censored when the value was older than 24 and 48 h for vital signs and labs, respectively. Vital signs and laboratory measurements were treated as continuous variables in the models, except that GCS (available for 33.4% of admits) was treated as a binary feature of whether or not it was measured. We also included binary features indicating whether or not a less-frequently available laboratory test was available: complete metabolic panel (75% available), CBC with differential (37%), arterial blood gases (3%), lactate (28%), troponin I (16%), and prothrombin time (56%).

We included demographics as well as clinical characteristics available on admission. This included age, length of time boarding in the ED, and length of ED stay as continuous variables in the model. We also included the following as binary variables: male sex, presence of a blood culture order in the ED, administration of intravenous (IV) antibiotics in the ED, triage acuity category, admission at night (11 pm to 7 am), evening admission (3 pm to 11 pm), admitting Charlson comorbidity index (CCI) diagnoses [20] , and use of supplemental oxygen on admission. The CCI diagnoses were determined using ICD9 and ICD10 codes [21, 22] .

Finally, we included several pre-existing risk scores. CURB-65 estimates mortality due to pneumonia and is intended to identify patients requiring inpatient admission. The national early warning score (NEWS) is a scoring system designed for track-and-trigger use across the National Health Service [23] . The systemic inflammatory response syndrome (SIRS) criteria, SOFA and qSOFA are designed to identify patients with suspected infection at high risk for poor outcomes [14, 24, 25] . Frost et al. present a risk model for early ICU transfer [18] . The APACHE II score [26] identifies patients at high risk for in-hospital mortality [15] . The laboratory-based acute physiology score (LAPS) identifies patients at risk for early ICU transfer [27] . We calculated these scores based on data available in the ED prior to admission to acute care, and used the last score measured during the ED stay. For APACHE we were unable to determine chronic organ insufficiency and immunocompromise. For LAPS we were not able to identify high risk patients for whom different imputations should be used because the laboratory score preliminary model was not available [27] .

We developed a predictive model on the entire cohort and used cross-validation to reduce bias in feature selection and performance characteristics. Only the last values available prior to admission were used for modeling. All modeling was performed in R [28] using the rms package [29] . We constructed a binary logistic regression model to identify patients at risk for early transfer. We first removed, blinded to the outcome, the most predictable features correlated more than R 2 of 0.9 with other features. Missing values were imputed with median of all available values. Next, we built a model with all remaining features and removed features using fast backward elimination [30] . Another model was built with restricted cubic spline transformations using three knots of nonlinearity on each feature with enough unique values. The model (linear or non-linear) with the highest Akaike Information Criterion (AIC) was retained. We performed tenfold cross-validation with fast backward elimination to ensure unbiased feature selected, removing features eliminated and retraining the model. Finally, we calculated the predicted risk using leave-one-out cross validation [31] . Briefly, the first hospital admission was identified as index admission, and we built a model on the remaining N-1 admissions. The predicted risk for the index admission was estimated using this model, and the procedure repeated for each of N admissions. In this way risk estimates for each admission were out-of-sample.

Risk marker performance was evaluated on the cross-validated risk predictions for two use cases of acute care admission decision: (1) continuous risk estimation during stay in the ED and (2) threshold-based alerting strategies for use before admission [30] . For continuous risk estimation we calculated the area under the receiver operating characteristic (AUC) and the area under the precision-recall curve (AUCPR). The AUCPR was calculated using the PRROC package for R [32] . The AUC, also known as C-statistic, was calculated using the Mann-Whitney U test. The 95% confidence intervals for AUC and AUCPR were calculated using 200 bootstrap runs. For each bootstrap run, predicted risk and the associated outcome were resampled with replacement and the AUC was calculated. The 95% confidence interval was then defined as the 5th and 195th values of AUC.

For threshold-based alerting, we pre-specified a target rate for early ICU transfer (i.e., we set the sensitivity) and determined the number of alerts required to achieve that target. We selected the last estimated risk prior to admission. The threshold was determined using the pre-specified sensitivity and the resulting alert rate, positive predictive value (PPV), and specificity were recorded.

There were 77,507 admissions through the ED during the study period. The inclusion and exclusion criteria yielded 42,332 admissions for analysis. Table 1 shows a breakdown of the study population inclusion and exclusion criteria. There were 496 events (1.2%), including 425 early ICU transfers, 62 early IMU transfers, and nine early deaths. Table 2 shows characteristics of the study population, i.e., acute care admits with and without early ICU transfer. Acute care admits with early ICU transfer had 18.0-fold higher mortality and stayed in the hospital 3.4 days longer than acute care admits without early ICU transfer. We note that event patients with early IMU transfer did not have significantly different hospital length of stay or mortality than those with early ICU transfer: 5.7 vs. 6.2 days, p = 0.19 and 4.8% vs. 10.1%, (p = 0.18). Table 2 also shows characteristics of adult admits directly to ICU for comparison. A secondary analysis for ICU admits without mechanical ventilation or vasopressors is also shown, as this population was felt to more closely represent a patient population which could have been potentially triaged to acute care or "floor" status. Acute care admits with early ICU transfer are older and have significantly higher mortality and length of stay than any other group. Admits with early ICU transfer have 3.8-fold higher mortality and 1.8 days longer hospital stay than direct ICU admits without vasopressors or mechanical ventilation. Pre-admit APACHE II scores for these two groups are not significantly different. Acute care admits with early ICU transfer had 46% higher mortality (p = 0.004) and stayed in the hospital 1.1 days longer (p < 0.001) than admits directly to ICU without DNR orders, commensurate with findings of Liu et al. [5] . Figure 1 shows the characteristics of the risk marker model using cross-validated risk estimates. The features in the model are shown in Fig. 1a ordered by goodness-of-fit quantified by chi-squared minus degrees of freedom. No features were removed during cross-validation fast backward feature selection. The most important feature is high heart rate, followed by need for supplemental oxygen, longer boarding time, and abnormal sodium. The qSOFA and CURB65 scores were also selected for the model. Time series of predicted risk leading up to the time of admission is shown in Fig. 1b . Risk estimates are shown in risk relative to average risk of early ICU transfer. That is, 1.0 represents the average risk, 2.0 is twice the average risk, and so on. White points indicate that risk estimates are significantly higher than risk estimates for the same patients 3 h prior. Risk estimates for admits with early ICU transfer (solid) are significantly higher than risk estimates for control patients, and rise from about twice the average risk to three times the average during their stay in the ED. The risk is significant beginning about 6 h prior to inpatient admission. Figure 1c shows the density of risk estimates for patients with (solid) and without (dashed) early ICU transfer. Figure 1d shows the predicted versus observed risk of early ICU transfer for deciles of predicted risk. Better calibrated models are closer to the line of identity, and better discriminating models have larger separation between deciles. The risk model is well calibrated in the middle 80% of predicted risk, overestimates risk for high risk patients, and underestimates risk for low risk patients. The model has good discrimination: the observed risk of admits with the highest decile of predicted risk is 7.9-fold (95% CI 5.0-12.9) higher than admits in the lowest decile of predicted risk.

The AUC of the risk marker using cross-validated risk estimates was 0.697 (95% CI 0.671-0.723). The AUCPR of the risk marker using cross-validated risk estimates was 0.54 (0.044-0.073). Figure 2 shows the characteristics of alerts based on the risk marker model. For reference, there were 162.2 acute care admits through the ED per week, of which 1.9 admits had early ICU transfer. Selecting a sensitivity Table 3 . The cross-validated AUC of the risk marker model is 45% higher than the best performing EWS, the NEWS score (the AUC of a random classifier is 0.5). The PRAUC of the risk marker model is 330% higher than the best performing EWS, the NEWS score (the PRAUC of a random classifier is the event rate, 1.2%). The risk marker model requires fewer than one third the number of alerts to achieve a sensitivity of 10%. (solid lines) admits. c Density of log odds predicted by risk marker, shown for event (solid) and control (dashed) admits. d Observed and predicted relative risk for risk marker. The observed relative risk is shown as a function of the relative risk predicted by the risk marker model. Each point represents 10% of measurements

Predicting which patients are at risk for unplanned wardto-ICU transfer at the time of admission has the potential to significantly improve patient safety. In our population, patients admitted to a hospital ward then transferred to an ICU have significantly worse mortality than those admitted to the ICU directly, similar to findings in previous studies [5] [6] [7] [8] 33] . Our model demonstrated the ability to identify patients at risk for ward-to-ICU transfers. If implemented, our model would allow an alert to be activated prior to admission for the subset of patients at high risk for transfer to an ICU. Implementation would result in fewer than 2 such alerts per week, which is important to prevent alarm fatigue. Despite being infrequent, these alerts would have the potential to decrease ward-to-ICU transfers by 10%, and significantly outperform existing models such as NEWS and LAPS.

Our model was built to perform continuous risk assessment. Risk estimates made by the model were significantly higher for cases than controls throughout the ED stay. In addition, the risk estimates for patients with subsequent ward-to-ICU transfer increased during their ED stay. These patients had twice the average risk six hours prior to admission, which rose to three times the average risk immediately prior to admission. This indicates that not only are risk estimates useful for identifying high risk patients, but that risk score trajectories may also be important.

Prior clinical decision support tools used in clinical practice have tended to be simple, point-based systems for ease of recall and use. Better integration profiles and standards for the hospital enterprise now allow for more complex integration of predictive models into clinical practice. More data points and complex weighting of variables can be used by integrating decision tools directly into data acquisition systems and the electronic medical record (EMR). This has been successfully implemented in monitoring for neonatal sepsis and showed a 20% mortality reduction just by making risk estimates available to clinicians at the bedside [34] .

Some have expressed skepticism towards adoption of such clinical scoring systems, stating that clinical decision instruments are seldom favorably compared to physician gestalt [35] . While we recognize that scoring systems and risk scores cannot supplant physician experience, they can certainly provide valuable information to a treating provider. This is evidenced by reductions in rates of septic shock and morality associated with clinical implementation of predictive analytic tools [34, 36] . Our current work also largely sidesteps this issue, as the event population is virtually pathognomonic of a failure in gestalt with resultant inappropriate initial disposition and demonstrable harm. Additionally, in a previous analysis where point of care clinicians were interviewed and observed following implementation of continuous predictive analytics in the ICU, they noted that the analytics were viewed as a key piece of clinical information once adopted into workflow through routine documentation, reported as a risk score during patient rounds and handoffs of care [37] . Future studies will be directed towards implementation of a continuous predictive analytics tool in the ED to determine the impact on early acute care to ICU transfer. To date, there has not been a similar analysis conducted in an ED setting where the patient ratios, clinical presentations, and workflow patterns are very different than other environments. Due to the relative rarity but clinical relevance of the event of interest, it would be critical to incorporate implementation and study of continuous predictive analytics within existing clinical decision support frameworks. Specifically, alert risk thresholds could be established to aid providers in making the initial decision for floor versus ICU admit. Beyond the initial disposition decision, this score could be communicated to the oncoming admission team to guide management decisions early in the post-admit trajectory (e.g., through more frequent vital sign ascertainment or early attending assessment). Additionally, such a tool may assist task-saturated physicians in a supervisory role to appropriately triage attention to patients at higher risk of clinical decompensation. In order to impact distal patient outcomes of unintended ICU/IMU upgrade, technologies must move from passive surveillance to integrated clinical decision support [38] .

While previous studies have examined ICU transfer at any point following admission [18, 19] , we focused on ICU transfer in the first 24 h because these are patients whose outcomes may be most readily impacted by changes in ED care. The relative rarity of these events makes them difficult to predict. The rate of these early ICU transfer in our study population was 1.2%, which is on the lower end of the range seen in similar studies (1.2-4%) [18, 39] . Despite the low rate of these events, they are associated with significantly worsened outcomes. In our study population, hospital length of stay and mortality were both increased in patients with early ward-to-ICU, even relative to patients admitted directly to the ICU.

Our analysis demonstrated that factors specific to a patient's current presentation were the most important to determine risk for early transfer. The most important features were high heart rate, need for supplementary oxygen, longer time boarding in the ED, and abnormal sodium. Existing EWS were incorporated into the model: qSOFA, an aggregate of vital signs (respiratory rate, Glasgow coma scale, and blood pressure) and CURB65 (age, blood pressure, respiratory rate, and higher urea nitrogen levels). Low calcium and the clinical decision to draw a troponin (irrespective of result) were also selected as features. This means that incorporating the latest vital sign and laboratory information into real-time prediction is essential to creating an accurate model of patient risk. Better identification of high-risk patients in the ED might lead to earlier or more targeted treatment, may prompt direction admission to the ICU, and could reduce early ICU transfer and the associated poorer outcomes.

This study is limited in that the model was developed and tested at a single center. Models may need to be tuned for a specific institution with a different population or with different clinical protocols. Our study was also limited by not incorporating continuous data from cardiorespiratory bedside monitors. Our results indicate that current patient state is important. Continuous cardiorespiratory dynamics provide the highest resolution look at patient physiology and would provide additional relevant information [40] . Additionally, the reason for upgrade to ICU-level care was not readily available in our dataset. While such information would help provide a more robust framework in which to interpret of the results, we believe the presence of meaningful markers of deterioration in the event population serves as an effective proxy for appropriateness of ICU transfer. We also recognize that there are other nonclinical features that may influence the decision to admit a patient to either the acute care or ICU following an ED visit, including bed availability, local practice patterns, and nurse staffing considerations [11] . Future study involving a multi-center or prospective design could offset some of this variability and provide more robust, granular data.

In summary, it is likely that predictive analytic monitoring based on ED data can help identify patients that will require upgrade to ICU or IMU if admitted to acute care. Incorporating this tool into ED practice may draw attention to these high-risk patients and allow early intervention. Intervention before admission may ameliorate or prevent deterioration and reduce the associated morbidity and mortality.

The Evolving Role of Emergency Departments in the United States

The unplanned intensive care unit admission

Rate of preventable early unplanned intensive care unit transfer for direct admissions and emergency department admissions

An observational study to determine the effect of delayed admission to the intensive care unit on patient outcome

Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system

Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit*

Outcomes of direct and indirect medical intensive care unit admissions from the emergency department of an acute care hospital: a retrospective cohort study

Direct and delayed admission to an intensive care or high dependency unit following discharge from the emergency department: associated patient characteristics and hospital outcomes

Hospital ward transfer to intensive care unit as a quality marker in emergency medicine

Validity of unplanned admission to an intensive care unit as a measure of patient safety in surgical patients

ICU admission, discharge, and triage guidelines: a framework to enhance clinical operations, development of institutional policies, and further research

Association of intensive care unit admission with mortality among older patients with pneumonia

Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

The third international consensus definitions for sepsis and septic shock (Sepsis-3)

APACHE-acute physiology and chronic health evaluation: a physiologically based classification system

Predictors of patients who present to the emergency department with sepsis and progress to septic shock between 4 and 48 hours of emergency department arrival

Disease progression in hemodynamically stable patients presenting to the emergency department with sepsis

Emergency department tachypnea predicts transfer to a higher level of care in the first 24 hours after ED admission

Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation

Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation

A new method of classifying prognostic comorbidity in longitudinal studies: development and validation

Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases

Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data

National Early Warning Score (NEWS)-standardising the assessment of acute-illness severity in the NHS. London: Royal College of Physicians

Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis

Serial evaluation of the SOFA score to predict outcome in critically ill patients

Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases

R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing

Rms: regression modeling strategies

The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets

PRROC: computing and visualizing precision-recall and receiver operating characteristic curves in R

Impact on patient outcome of emergency department length of stay prior to ICU admission

Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial

Structured clinical decision aids are seldom compared with subjective physician judgment, and are seldom superior

Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit

Advancing continuous predictive analytics monitoring: moving from implementation to clinical action in a learning health system

Monitoring cardiorespiratory instability: current approaches and implications for nursing practice

Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system

Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: a retrospective cohort study

Hospital-level variation in the use of intensive care