key: cord-0833670-n2aqo10s
authors: Covino, Marcello; Sandroni, Claudio; Santoro, Michele; Sabia, Luca; Simeoni, Benedetta; Bocci, Maria Grazia; Ojetti, Veronica; Candelli, Marcello; Antonelli, Massimo; Gasbarrini, Antonio; Franceschi, Francesco
title: Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores
date: 2020-09-09
journal: Resuscitation
DOI: 10.1016/j.resuscitation.2020.08.124
sha: 087ab994411f8da32654985b1ee2a7f3b9f3626f
doc_id: 833670
cord_uid: n2aqo10s

AIMS: To identify the most accurate early warning score (EWS) for predicting an adverse outcome in COVID-19 patients admitted to the emergency department (ED). METHODS: In adult consecutive patients admitted (March 1-April 15, 2020) to the ED of a major referral centre for COVID-19, we retrospectively calculated NEWS, NEWS2, NEWS-C, MEWS, qSOFA, and REMS from physiological variables measured on arrival. Sensitivity, specificity, positive (PPV) and negative predictive value (NPV), and the area under the receiver operating characteristic (AUROC) curve of each EWS for predicting admission to the intensive care unit (ICU) and death at 48 h and 7 days were calculated. RESULTS: We included 334 patients (119 [35.6%] females, median age 66 [54-78] years). At 7 days, the rates of ICU admission and death were 56/334 (17%) and 26/334 (7.8%), respectively. NEWS was the most accurate predictor of ICU admission within 7 days (AUROC 0.783 [95% CI, 0.735-0.826]; sensitivity 71.4 [57.8-82.7]%; NPV 93.1 [89.8-95.3]%), while REMS was the most accurate predictor of death within 7 days (AUROC 0.823 [0.778–0.863]; sensitivity 96.1 [80.4-99.9]%; NPV 99.4[96.2–99.9]%). Similar results were observed for ICU admission and death at 48 h. NEWS and REMS were as accurate as the triage system used in our ED. MEWS and qSOFA had the lowest overall accuracy for both outcomes. CONCLUSION: In our single-centre cohort of COVID-19 patients, NEWS and REMS measured on ED arrival were the most sensitive predictors of 7-day ICU admission or death. EWS could be useful to identify patients with low risk of clinical deterioration.

After the first cases were identified in Wuhan (China) in December 2019, the novel coronavirus SARS-CoV-2 has caused a pandemic of respiratory illness named COVID-19. 1 To date, more than 20 million cases have been reported worldwide, including more than 750,000 deaths. 2 Typical COVID-19 patients present with fever, myalgia, fatigue, and dry cough. 3, 4 Severe cases progress to severe dyspnoea and hypoxemia within one week after the onset of symptoms. [3] [4] [5] In hospitalized COVID-19 patients, the prevalence of hypoxemic respiratory failure is around 20%, and more than 25% of them may require intensive care treatment. 6 The increasing number of COVID-19 cases has challenged healthcare systems worldwide. In this context of overwhelming demand for medical assessment and triage in emergency departments (EDs), early warning scores (EWS) may be useful. EWS are physiological scoring systems based on a rapid and quantitative assessment of changes in vital signs, 7 and were initially developed to identify and track hospitalised patients at risk of deterioration outside critical care areas in order to ensure early stabilisation and transfer to intensive care unit (ICU) when appropriate, and to prevent avoidable cardiac arrests. 8, 9 However, these scores have also subsequently been investigated as a tool to predict critical illness in ED patients, 10 and some, as the Rapid Emergency Medicine Score (REMS), 11 have specifically been developed for this purpose.

In studies performed on the general ED population, the National Early Warning Score (NEWS) 12 was the most accurate in predicting both in-hospital mortality and ICU admission. 10 A recent document from the Swiss Society of Intensive Care Medicine 13 suggested using a score greater than 6 of a modified version of NEWS (hereafter referred as to NEWS-C) for deciding admission to ICU of COVID-19 patients. 14 However, this recommendation is based on previous evidence on non-COVID-19 patients, 15, 16 and it has not been validated so far. At present, it is not clear which EWS is the most accurate for triaging COVID-19 patients in the ED.

The aim of the present study was to compare the ability of EWS assessed on ED arrival to identify adult COVID-19 patients at risk for ICU admission or death within seven days. The EWS assessed in this study J o u r n a l P r e -p r o o f included Modified EWS (MEWS), 17 NEWS, NEWS2, 18 NEWS-C, Quick Sepsis Related Organ Failure Assessment (qSOFA), 19 and REMS. Moreover, in order to measure the added value of EWS to the conventional ED triage process, we also compared the accuracy of EWS with that of the Regional Triage System for Infectious Diseases in use in our hospital for COVID-19 patients.

This is a retrospective observational study (clinicaltrials.gov NCT04371562) conducted in the ED of the Fondazione Policlinico Universitario Agostino Gemelli, in Rome. Policlinico Gemelli is the largest teaching hospital in Rome and a referral centre for COVID-19 in central Italy. Our ED has a catchment area of 1,8 M inhabitants, with an average of 80k access/year. The Regional Triage System for Infectious Diseases in our ED is based on a protocol issued by the Health Service of Lazio Region, where our hospital is located. The protocol, summarised on ESM Tables 1a-b, follows an ABCDE scheme (airway, breathing, circulation, disability, exposure) with additional variables such as age, blood glucose level, plus additional symptoms and signs. The triage system assigns a priority value from 1 to 5, in descending order of severity. Triage is performed by a nurse certified in a full time 7-day course followed by a 6-month supervised training.

The criteria for ICU admission of COVID-19 patients in our hospital during the study period included circulatory shock requiring vasopressors and the need for positive pressure respiratory support, including non-invasive ventilation. High-flow oxygen treatment is provided in general wards. ICU admission criteria were consistent throughout the study period. At present, no EWS is in use in our hospital.

J o u r n a l P r e -p r o o f

From March 1 to April 15, 2020 we included all adult (>18y) patients assessed in our ED for suspected COVID-19 whose diagnosis was confirmed with real-time reverse-transcriptase-polymerase-chain-reaction assay of nasal and pharyngeal swab specimens. Testing for COVID-19 was conducted according to the World Health Organization (WHO) interim guidance. 20 We excluded pregnant women and patients who were on circulatory shock and positive pressure support on arrival to ED. Patients with a do-not-resuscitate/do-not-intubate (DNR/DNI) order in place were also excluded..

Two authors blinded to patients' outcome retrieved clinical information from the electronic medical records (EMR) of the ED database. Extracted data included age, sex, clinical history and presentation, temperature, heart rate (HR), respiratory rate (RR), blood pressure (BP), Glasgow Coma Scale (GCS) score, oxygen therapy, and peripheral oxygen saturation (SpO2). All clinical signs, including SpO2, were measured upon ED arrival. Table 2 . For NEWS2 calculation, patients were considered at risk of type 2 respiratory failure (T2RF) if they had a confirmed history of chronic obstructive pulmonary disease.

The primary study outcomes were death and ICU admission within seven days from ED arrival. Secondary outcomes were death and ICU admission within 48h from ED arrival. Outcomes were retrospectively assessed by reviewing EMRs of the hospital medical database. Receiver operating characteristic (ROC) curve analysis was used to estimate the performance of the evaluated scores in predicting the study outcomes. For each score threshold value, the following was calculated: sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (+LR), and negative likelihood ratio (-LR). The Youden index was used to estimate the optimal thresholds for sensitivity and specificity. The comparison between the areas under the ROC curve (AUROCs) was made according to the DeLong method. 21 A p value ≤ 0.05 was regarded as significant.

To check the robustness of our analysis we imputed data in patients with less than three missing parameters using a multiple imputation approach 22 . The results of imputation were added to the original study cohort with the complete dataset of vital parameters and EWS were recalculated. The resulting AUROCs were compared to the AUROC obtained in the study population by using the z-statistics (2 sided). Further data on imputation are available in Appendix 1. 

This study has been approved by the local Ethics Committee (authorization #001705520) and has been performed in accordance with the ethical standards established in the 1964 Declaration of Helsinki and its later amendments. All patients gave their informed consent for clinical data analysis.

During the study period, 3955 patients were seen in the ED, of whom 1801 were evaluated for suspect COVID-19 and 512 were found positive to the COVID-19 test. Of these, 336 had sufficient data on EMR for calculating EWS. Two patients with a DNR/DNI order in place were excluded, leaving 334 patients for final analysis (Figure 1 ). Patients' median [IQR] age was 66 [54 -78] years (range 20 -98 years) and 215 (64.4%) were male ( Table 1) .

Thirty-eight (11%) COVID-19 patients were discharged from ED and quarantined. These patients were contacted daily to assess their clinical condition. The remaining 296 patients were admitted to hospital, of whom 56 (17% of the total) were admitted to ICU. Fifty patients were admitted to ICU within 48 hours, while the remaining six patients were admitted on days 3-7 after ED arrival. Patients who needed ICU admission were older, had lower SpO2 and higher respiratory rate on ED arrival, and more frequently reported a history of hypertension, obesity, and chronic heart failure or coronary artery disease (Table 1) .

Overall, 26 patients (7.8%) deceased within seven days after ED arrival, among whom 11 (3.3%) deceased by 48 hours. These patients were significantly older, had a higher respiratory rate, and a lower SpO2 on ED arrival. As for ICU admission, in patients who died a history of hypertension and chronic heart failure or coronary artery disease was more commonly reported ( Table 1) . Table 3 ).

For both outcomes, the overall accuracy of the NEWS-C was consistently lower than that of NEWS in our population. Among all EWS, MEWS and qSOFA showed the lowest overall accuracy. Their AUROCs were significantly lower than that of the triage system of our hospital, except for prediction of death at 48h for qSOFA ( Table 3) .

ESM figures 1 and 2 include the ROC curves for prediction of ICU admission and death, respectively, along with a statistical comparison of the AUROCs of the scores included in the study.

Among the 176 patients excluded because of missing parameters, 78 (44.3%) had less than three missing parameters and were eligible for imputation. Their available parameters are shown in Appendix 1, Table A1 .

Among these 78 patients, six deaths (7.7%) and nine ICU admissions (11.5%) occurred within 7 days.

The EWS AUROCs in the extended population including 78 patients with imputed vital parameters (total 412 patients) were not significantly different from those of the 334 patients with complete original data. (Appendix 1, Tables A2 and A3).

Our study showed that in COVID-19 patients NEWS and REMS assessed on arrival at the emergency department were the most accurate scores for predicting the risk of ICU admission and death, respectively, both at 48h and at 7 days. Moreover, the accuracy of these scores was comparable with that of the comprehensive triage score used in our department.

J o u r n a l P r e -p r o o f NEWS is among the most accurate tools for predicting patient deterioration outside ICU. 10 In a comparative study including 198,755 observation sets collected from 35,585 consecutive acute medical admissions in UK, 23 NEWS was more strongly associated with ICU admission or death compared to 33 other scoring systems (not including REMS). Moreover, in the specific setting of acute care patients admitted for suspected infection in ED or hospital wards, NEWS has proven to be more accurate for predicting death or ICU transfer than other tools that were purposely designed to assess infected patients, such as qSOFA and systemic inflammatory response syndrome (SIRS) criteria. 24 In 2017, NEWS has been updated to NEWS2 by adding a new SpO2 scoring scale for patients with type 2 respiratory failure (T2RF), 25 even if the added value of this modification in improving discrimination of adverse outcomes in patients with documented T2RF has not been demonstrated yet. 26 NEWS2 is recommended by the Royal College of Physicians for use in COVID-19 patients. 27 In our study, NEWS2 did not perform better than NEWS in predicting ICU admission or death. However, the prevalence of T2RF patients in our population was very low (19/334 patients; 5.7%) . This is in accordance with data from other studies showing that hypercapnia is uncommon in COVID-19 patients, even in those admitted to ICU, 28 while hypoxia from early pulmonary involvement is the most common presentation feature. 4, 29 NEWS-C, another modified version of NEWS, has recently been suggested for assessment of COVID-19 patients. 14 Age ≥65 years was added to NEWS-C as an independent risk factor based on recent reports in patients with influenza or other non-COVID infectious diseases. 15, 16 Indeed, older age is associated with increased mortality in COVID-19 patients. 30 However, NEWS-C did not perform better than NEWS in predicting ICU admission or mortality in our study. This result can be explained considering that our population was relatively old. In fact, the median age of survivors vs. non-survivors was 64 years vs. 79.5 years in our COVID-19 population, as compared with 51 vs.68 years respectively in a recent report from J o u r n a l P r e -p r o o f China. 30 These data may reflect differences in demographics (median population age is 34.4 years in China vs. 42.5 years in Europe and 47.3 years in Italy 31 ) and in policies for ICU admission and triage. When weighting for age, REMS may be particularly appropriate, since -unlike other EWS -it stratifies age (five groups from <45 to >74 years; see also ESM Table 2 ). In our COVID-19 population, REMS was the most accurate score for predicting mortality. Among the scoring systems we evaluated, REMS was the only one specifically developed to predict mortality in ED patients. 11, 32 The qSOFA has been proposed as a prompt to consider possible sepsis outside the ICU. Criteria to calculate qSOFA include altered mental status (GCS<15), tachypnoea (RR ≥22), and hypotension (systolic BP≤100). In the original validation study, a qSOFA score of 2 or higher was associated with a significantly increased risk of hospital mortality. 19 However, the accuracy of qSOFA was low in our study, confirming results from previous studies conducted in non-COVID potentially septic patients. 24, 33, 34 Notably, most patients in our cohort did not have any of the three qSOFA criteria, and none of them had a qSOFA score of 2 or higher. Among the three qSOFA criteria, the prevalence of hypotension and altered mental status on ED admission in our patient population was particularly low (6.6% and 3.3%; data not shown). This is in line with recent reports showing that extrapulmonary organ dysfunction occurs later in the clinical course of COVID-19. 35 When compared with other scoring systems, including our institutional triage score, NEWS and REMS had the highest sensitivity and NPV for adverse outcomes. In other words, they performed particularly well in identifying COVID-19 patients who did not require ICU admission or were at low risk of death, respectively.

When triaging patients in ED, having a high NPV (i.e., a low rate of false negative predictions) for adverse outcomes is paramount in order to avoid inappropriately admitting a patient at risk of deterioration to a non-critical care area. However, because of the relatively low incidence of adverse outcomes in our population, all the EWS we evaluated had a low PPV (i.e., a low rate of true positive predictions), which implies a risk of overtriage when these scores are used to decide admission to a critical care area. Although J o u r n a l P r e -p r o o f consensus suggests that overtriage is generally preferable to undertriage, 36 this may not be the case when meeting the demands of healthcare resources is difficult. The prognostic indications from physiological scores should be interpreted within a more comprehensive clinical approach including patient's comorbidities, functional status, and preferences.

In our study we assessed EWS only once, on patient arrival to the ED. Since EWS have been designed as track-and-trigger systems to detect deterioration of vital signs over time, their sequential measurement in COVID-19 patients admitted to non-monitored wards may represent the logical continuation of their use in ED. The value of EWS in this context deserves further investigation.

Some limitations of our study should be acknowledged. First, this was a single-centre study conducted on a relatively small population, and its confirmation on a larger, multicentre cohort is warranted. Secondly, the EWS were retrospectively calculated from data recorded on the patients' EMR that were not specifically collected for this purpose. This resulted in missing data necessitating imputation. Thirdly, ICU admission is an institution-dependent outcome, which may limit the generalizability of our results.

In summary, our study showed that in patients with confirmed COVID-19 assessed on ED arrival, REMS and NEWS were the most accurate indices for predicting in-hospital death and ICU admission within seven days, respectively. Due to their high NPV, these indices could be particularly useful for identifying patients who are at lower risk of poor outcome and can be admitted to a non-monitored ward. This supports their use as an adjunct to the process of triage and disposition of COVID-19 patients in the ED.

Prospective multicentre cohort studies are needed to provide an external validation of our results. 

Characteristics of and Important Lessons From the Coronavirus Disease

Outbreak in China: Summary of a Report of 72314 Cases From the Chinese Center for Disease Control and Prevention

European Centre for Disease Prevention and Control COVID-19: Situation update worldwide Accessed

Clinical Characteristics of Coronavirus Disease 2019 in China

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease

Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan

Rapid response systems

Rapid response systems: are they really effective?

Can binary early warning scores perform as well as standard early warning scores for discriminating a patient's risk of cardiac arrest, death or unanticipated intensive care unit admission?

Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): A narrative review

Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients

A national early warning score for acutely ill patients

Swiss Society Of Intensive Care Medicine. Recommendations for the admission of patients with COVID-19 to intensive care and intermediate care units (ICUs and IMCUs)

Novel coronavirus infection during the 2019-2020 epidemic: preparing intensive care units-the experience in Sichuan Province, China

A Risk Classification Model to Predict Mortality Among Laboratory-Confirmed Avian Influenza A H7N9 Patients: A Population-Based Observational Cohort Study

A Comparison of the Quick Sequential (Sepsis-Related) Organ Failure Assessment Score and the National Early Warning Score in Non-ICU Patients With/Without Infection

Modified early warning score predicts the need for hospital admission and inhospital mortality

The National Early Warning Score 2 (NEWS2)

Royal College of Physicians NEWS2 and deterioration in COVID-19 Accessed

Baseline Characteristics and Outcomes of 1591 Patients Infected With SARS-CoV-2 Admitted to ICUs of the Lombardy Region

Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study

Clinical characteristics of 113 deceased patients with coronavirus disease 2019: retrospective study

United Nations Department of Economic and Social Affairs Population Dynamics . World Population Prospects

Prognostic performance of the Rapid Emergency Medicine Score (REMS) and Worthing Physiological Scoring system (WPS) in emergency department

Comparing Mortality Prediction by Quick Sequential Organ Failure Assessment With Emergency Physician Judgment

NEWS2 is Superior to qSOFA in Detecting Sepsis with Organ Dysfunction in the Emergency Department

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

ICU Admission, Discharge, and Triage Guidelines: A Framework to Enhance Clinical Operations, Development of Institutional Policies, and Further Research

Abbreviations: TRIAGE (Triage code assigned in ED)

Warning Score; NEWS National Early Warning Score; NEWS-C: modified NEWS; qSOFA quick Sequential Organ Failure Assessment

+LR Positive Likelihood Ratio; -LR Negative Likelihood Ratio; PPV Positive Predictive Value; NPV Negative Predictive Value

None.J o u r n a l P r e -p r o o f