key: cord-1025675-9yqp030n
authors: Durantez-Fernández, Carlos; Polonio-López, Begoña; Martín-Conty, José L.; Maestre-Miquel, Clara; Viñuela, Antonio; López-Izquierdo, Raúl; Mordillo-Mateos, Laura; Jorge-Soto, Cristina; Otero-Agra, Martín; Dileone, Michele; Rabanales-Sotos, Joseba; Martín-Rodríguez, Francisco
title: Comparison of Nine Early Warning Scores for Identification of Short-Term Mortality in Acute Neurological Disease in Emergency Department
date: 2022-04-14
journal: J Pers Med
DOI: 10.3390/jpm12040630
sha: 1139ffc252c5cf7d55b3526de510d57d8fdb79b2
doc_id: 1025675
cord_uid: 9yqp030n

(1) Background: The aim was screening the performance of nine Early Warning Scores (EWS), to identify patients at high-risk of premature impairment and to detect intensive care unit (ICU) admissions, as well as to track the 2-, 7-, 14-, and 28-day mortality in a cohort of patients diagnosed with an acute neurological condition. (2) Methods: We conducted a prospective, longitudinal, observational study, calculating the EWS [Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), VitalPAC Early Warning Score (ViEWS), Modified Rapid Emergency Medicine Score (MREMS), Early Warning Score (EWS), Hamilton Early Warning Score (HEWS), Standardised Early Warning Score (SEWS), WHO Prognostic Scored System (WPSS), and Rapid Acute Physiology Score (RAPS)] upon the arrival of patients to the emergency department. (3) Results: In all, 1160 patients were included: 808 patients were hospitalized, 199 cases (17%) required ICU care, and 6% of patients died (64 cases) within 2 days, which rose to 16% (183 cases) within 28 days. The highest area under the curve for predicting the need for ICU admissions was obtained by RAPS and MEWS. For predicting mortality, MREMS obtained the best scores for 2- and 28-day mortality. (4) Conclusions: This is the first study to explore whether several EWS accurately identify the risk of ICU admissions and mortality, at different time points, in patients with acute neurological disorders. Every score analyzed obtained good results, but it is suggested that the use of RAPS, MEWS, and MREMS should be preferred in the acute setting, for patients with neurological impairment.

Acute neurological diseases were about 10% to 20% of the medical admissions at Emergency Departments (ED) before the coronavirus disease [1] [2] [3] , accounting for the third cause of admissions after cardiovascular and respiratory diseases [4] . Common neurological diseases include a variety of major disorders such as stroke, headache, and seizures [2] [3] [4] ; traumatic brain injuries exhibit a significant incidence as well [5] .

The reorganization of medical assistance in consequence of the effort focused on the ongoing COVID-19 pandemic, together with the reduction in patients, who are avoiding the hospital, have entailed a decrease in interventions related with neurological problems in many countries [6, 7] . These indirect effects of the COVID-19 pandemic have increased the risk of mortality of these neurological diseases, which have previously shown a high mortality risk worldwide (6.55 million deaths by stroke in 2019) [8, 9] , with a subsequent overload of ED and Intensive Care Units (ICU).

Until now, many studies have focused on possible strategies to decrease the crowding of ED and ICUs. In this context, it was recently demonstrated that the implementation of assistance protocols in ED with the use of Early Warning Scores (EWS) could help to reduce absolute in-hospital mortality rate [10] .

When the focus was moved to neurological diseases, particularly acute cerebrovascular diseases, it was demonstrated that a model including age, stroke severity, Early Warning Score, and Performance Status was found to be valid to predict one-year mortality [11] . Furthermore, the onset severity of neurological diseases could be misleading (many clinical patterns are similar and confounding at onset), and a part of those could present an unexpected rapid clinical deterioration. Keeping in mind this aspect, it appears crucial to develop and fix strategies to better predict not only mortality but also the risk of rapid clinical deterioration in neurological diseases.

Protocol assistance for a coordinated response is necessary to assess the severity of the prognostic and the best initial decision [12, 13] , which certainly involves the use of Early Warning Scores (EWS).

Most of the studies use only one score to evaluate acute neurological disorders (such as stroke [14] [15] [16] [17] [18] , traumatic brain injury [5, 19] , acute hypertensive intracerebral haemorrhage [20] , or patients with medium and long-term deterioration [20, 21] ). A wide variety of EWS is now available, and it is crucial to determine which are the best ones for each pathology, from the clinical context and from the phenomenon that is wanted to be predicted [12, 13] .

To the best of our knowledge, until now no studies performed a direct comparison between the different EWS. The aim of the study was screening the performance of nine Early Warning Scores to identify patients at high-risk of premature impairment and to detect intensive care unit-admissions, as well as to track the 2-, 7-, 14-, and 28-day mortality in a cohort of patients diagnosed with an acute neurological condition.

We conducted a prospective, longitudinal, observational study with adult patients transferred with high priority by ambulance to the EDs of five hospitals of the Health Public System of Castilla y León (Spain) between 1 January 2019 and 30 August 2021. The study conforms to the broader EQUATOR (Improving the Quality and Transparency of Health Research) guidelines [11] The 

The sample was selected from all patients over 18 years of age transferred with high priority by ambulance to the EDs of the selected hospitals with a diagnosis of neurological disorders (ischaemic stroke, seizures, haemorrhage, confusion syndrome, degenerative disease, headache, vertigo, tumour, infection, neuromediated syncope, coma). For this study, all patients who were minors or who met any of the following exclusion criteria were excluded: cardiorespiratory arrest, death before or during transfer, pregnancy, discharge in situ, psychiatric pathology, and/or diagnosis of terminal illness.

During the investigation, an attempt was made to minimize the risk of bias by excluding patients without follow-up through the clinical history, from whom the vital signs necessary for the calculation of the scales could not be obtained, or who did not give their informed consent. A family member or legal guardian was contacted to obtain informed consent for patients with an altered clinical condition or level of consciousness.

The principal outcome was the presence of high-risk of premature impairment (intensive care unit-admissions). Secondly, the performance of the scores for 2-, 7-, 14-, and 28-day mortality was evaluated.

For the study, the clinical variables were analyzed on arrival at the patient's emergency department, which allowed the calculation of the EWS indices, specifically the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), VitalPAC Early Warning Score (ViEWS), Modified Rapid Emergency Medicine Score (MREMS), Early Warning Score (EWS), Hamilton Early Warning Score (HEWS), Standardised Early Warning Score (SEWS), WHO Prognostic Scored System (WPSS), and Rapid Acute Physiology Score (RAPS).

The MEWS scale was first used as a tool to assess the state of the hospitalized patient; however, its usefulness has spread as a predictor of ICU admissions and predictions of hospital mortality [22, 23] . NEWS was categorized by several studies as the most reliable scale to measure a patient's deterioration at the prehospital level [24, 25] . MREMS is a scale derived from REMS (Rapid Emergency Medicine Score), which seeks to predict hospital mortality in real time, adding the variables age and oxygen saturation, of which, for example, RAPS lacks, which comes from the Acute Physiology score and Chronic Health Evaluation II (APACHE-II) and has been shown to be a good rapid prediction tool [26, 27] . HEWS [28] was designed to improve the early detection of all hospitalized patients at risk for deterioration, particularly among those with suspected infection. In SEWS, the inclusion of oxygen saturation is shown to have a significant relationship with short-to medium-term mortality [29] . Finally, WPSS can classify patients into five risk groups with different survivals and probabilities evolution [30] .

For these scores, it was necessary to collect information on respiratory rate, oxygen saturation (SpO 2 ), need for supplemental oxygen, heart rate, temperature, age, systolic blood pressure (SBP), mean arterial pressure (MAP), and level of consciousness using the Glasgow Coma Scale (GCS) or, alternatively, the AVPU scale (alert, voice, pain, no response).

During triage, the emergency personnel oversaw collecting the clinical and evaluation data to calculate the scores of the different EWS. SpO 2 , SBP, MAP, temperature, and heart rate measurements were obtained using the Connex ® Vital Signs Monitor (Welch Allyn, Inc., Skaneateles Falls, NY, USA). Respiratory rate was measured by directly observing the total number of complete respiratory cycles for one minute. The patient's level of consciousness was assessed using the GCS. All data were included anonymously in an electronic database created for this study.

The rest of the hospital variables (hospitalized patients, ICU admissions, mortality rates, diagnosis of the corresponding group according to the International Classification of Diseases 11th Revision) were collected by an associate researcher from each hospital, by reviewing the electronic medical records 30 days after the index event.

Quantitative variables are presented as a median and interquartile range (IQR), while for categorical variables absolute and relative frequencies were used. Quantitative variables were analyzed using a normality test (Kolmogorov Smirnov) and were compared with a non-parametric Mann-Whitney U test. Effect sizes (ES) were calculated with the Rosenthal r test and classified according to these parameters: Trivial (<0. To compare the predictive capacity of the EWS, the area under the curve (AUC) of the receiver operating characteristic (ROC) was calculated, to predict the need for ICU admission and 2-, 7-, 14-, and 28-day mortality. Using the Youden test, we selected the cut-off point with the best results on each scale for sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio, negative likelihood ratio, and diagnostic accuracy (DA). We used the chi-squared test to generate contingency tables that would allow us to establish association relationships through odds ratios. The accuracy of the scores was compared using DeLong's test.

All analyses were performed with XLSTAT ® BioMED software for Microsoft Excel ® version 14.4.0 (Microsoft Inc., Redmond, WA, USA) and IBM SPSS Statistics version 20.0 (IBM Corp., Armonk, NY, USA). In all tests, a confidence level of 95% and a p-value below 0.05 were considered significant.

Between 1 January 2019 and 30 August 2021, we recorded a total of 1160 cases of adult patients with neurological disease who were referred to the ED of the five participating hospitals. Figure 1 show the flowchart. number of complete respiratory cycles for one minute. The patient's level of consciousnes was assessed using the GCS. All data were included anonymously in an electronic databas created for this study.

The rest of the hospital variables (hospitalized patients, ICU admissions, mortalit rates, diagnosis of the corresponding group according to the International Classificatio of Diseases 11th Revision) were collected by an associate researcher from each hospita by reviewing the electronic medical records 30 days after the index event.

Quantitative variables are presented as a median and interquartile range (IQR), while fo categorical variables absolute and relative frequencies were used. Quantitative variables wer analyzed using a normality test (Kolmogorov Smirnov) and were compared with a non-pa ametric Mann-Whitney U test. Effect sizes (ES) were calculated with the Rosenthal r test an classified according to these parameters: Trivial (<0. To compare the predictive capacity of the EWS, the area under the curve (AUC) of th receiver operating characteristic (ROC) was calculated, to predict the need for ICU admissio and 2-, 7-, 14-, and 28-day mortality. Using the Youden test, we selected the cut-off point wit the best results on each scale for sensitivity, specificity, positive predictive value (PPV), neg tive predictive value (NPV), positive likelihood ratio, negative likelihood ratio, and diagnost accuracy (DA). We used the chi-squared test to generate contingency tables that woul allow us to establish association relationships through odds ratios. The accuracy of th scores was compared using DeLong's test.

All analyses were performed with XLSTAT ® BioMED software for Microsoft Exce version 14.4.0 (Microsoft Inc., Redmond, WA, USA) and IBM SPSS Statistics version 20 (IBM Corp, Armonk, NY, USA). In all tests, a confidence level of 95% and a p-value below 0.05 were considered significant.

Between 1 January 2019 and 30 August 2021, we recorded a total of 1160 cases of adu patients with neurological disease who were referred to the ED of the five participating ho pitals. Figure 1 show the flowchart. The median age was 71 years (IQR: 53-82 years, range 18-97), and 46% (537 patients) were females. The main reasons for medical check-up were ischaemic stroke (369 cases, 32%) and seizure (282 cases, 24%), and their priority of care according to hospital triage was mainly level II (50%) or level III emergency care (40%). In total, 808 patients were hospitalized, with ICU care required in 199 cases (17%). The mortality of the patient ranged from 6% (64 cases) within 2 days, to 10% (114 cases) within 7 days, to 13% (145 cases) within 14 days, to 15% (173 cases) within 21 days, to 16% (183 cases) within 28 days.

Tables 1 and 2 compare the clinical variables from initial assessment and the hospital care data of the patients with the variables resulting from this study: the requirement of ICU care and patient mortality at 2-, 7-, 14-, and 28-days. Considering the clinical deterioration of patients with different types of neurological diseases, we found that the scales with the best capacity to predict the need for ICU care were the RAPS with an AUC of 0.790 (95% CI: 0.751-0.829) and the MEWS with an AUC of 0.789 (95% CI: 0.750-0.828) ( Table 3) . Regarding the capacity to predict mortality, the MREMS obtained the best score for 2-day mortality with an AUC of 0.929 (95% CI: 0.885-0.973) and for 28-day mortality with an AUC of 0.856 (95% CI: 0.820-0.891) ( Table 3) . Table 3 . AUROC, cut-off points for combined sensitivity and specificity with the best score (Youden's test) for the different scales analyzed (for intensive care unit, 2-, and 28-day mortality).

Intensive Care Unit 3 and Table S1 shows the best cut-off values for each EWS, according to sensitivity and specificity (Youden test) for each of the studied events (ICU, 2-, 7-, 14-, 28-day mortality), which allowed to discern their predictive value and diagnostic accuracy. Comparisons of the AUC of the different scoring systems (Tables S2 and S3) show that they are significantly different with RAPS and MEWS, having the highest efficacy to predict the ICU admissions and MREMS for 2-and 28-day mortality.

To the best of our knowledge, this is the first study to explore whether several EWS accurately identify the risk of ICU admissions and mortality at different time points in patients with acute neurological disorders. Our findings showed that both RAPS and MEWS performed well. However, regarding mortality prediction, MREMS showed the best capacity to predict it both in the short term and in the long term. Previous studies demonstrated that the use of combined scores, in which EWS was one of the factors included, were able to predict one-year mortality in haemorrhagic and ischaemic stroke patients, with a similar AUC of 85% [11] . Liljehult et al. retrospectively analyzed a cohort of 1113 stroke patients and evaluated the performance of a combined score that included EWS. On the other hand, our total cohort was similar, but we analyzed a not-homogeneous population that included different neurological conditions, so the two studies are not directly comparable. Other studies drew similar conclusions about the usefulness of EWS in mortality and ICU hospitalization prediction in ED, but most are focused on a particular neurological condition as well as traumatic brain injury, brain neoplasm, brain haemorrhage, etc. [19] .

Importantly, when applied to neurological disorders, we found a difference in performance of EWS that is mainly based on similar parameters with different relative weight. In this aspect, we must underline that EWS has been developed and used for areas such as ED and ICU, in which the stratification of patients into risk groups helps rapid evaluation, triage, and early management [31] [32] [33] . Furthermore, EWS can improve the care of critically ill patients, who must be transported from acute settings to hospitals, segregating those with a good prognosis from those with a bad one. Importantly, most of the patients included were diagnosed with ischaemic stroke, epilepsy, and cerebral haemorrhage that included traumatic and not-traumatic etiology, so that our results are mainly affected by these three conditions.

Particularly, in the present cohort, 32% of the patients were diagnosed with stroke, representing the most frequent pathology analyzed. In Spain, stroke care is based on a "Hub and Spoke" system that, together with multimodal imaging, was demonstrated to improve access to revascularization treatment [34] . Revascularization could be achieved mainly by intravenous thrombolysis and/or mechanical thrombectomy: the last one is usually available in Hub Hospitals, but not in Spoke Hospitals, and is indicated for large vessel occlusions [35] . Large vessel occlusion strokes frequently reduce GCS scores, and their general clinical conditions are often worse than those observed in patients with stroke of small vessel occlusions, leading to higher scores in EWS [36] . Keeping in mind this point the use of EWS, particularly of the scales that we demonstrated to have best performances in terms of mortality and ICU risk, could permit to differentiate the transport destiny, bringing "high risk" stroke patients directly to Hub Hospital, reducing treating times and improving the quality of stroke care, permitting a rapid access to advanced imaging and, if necessary, to mechanical thrombectomy. In other words, the use of RAPS, MEWS, and MREMS could help to fasten and ameliorate acute stroke treatment, reducing pre-hospital timings.

It should also be considered that the management of traumatic brain injuries and cerebral haemorrhages depend on the presence of neurosurgery department: interestingly, not all the patients with these conditions are potential candidates for neurosurgery. So, the use of EWS could stratify the patients who are at risk of a rapid clinical deterioration, to anticipate transport to a tertiary hospital (that offers 24 h on-call neurosurgery) or rapid on-site access to neurosurgery, when available.

The major limitations depend on the design and the sample size of the study. Firstly, we conceived and developed an observational study that it is known to increase the risk of a selection bias. However, the fact that several neurological conditions were included could have significantly reduced it. Another important limitation is the sample size: we included more than 1000 patients, but the majority of those were affected by strokes, haemorrhages, and seizures, so that the weight of the final analysis mostly depends on these conditions, hence, de facto reducing our cohort to 50% of the total.

In the sample distribution, according to the need for intensive care of the patients, we found that age groups were significantly different; this situation may be due to confounding bias with older patients, who often have limited ICU care [37, 38] , despite of the impact of hospitalization in older adults is bigger, especially for polimedication [39] . In total, 90% of the patients admitted to the ICU presented as a base pathology ischaemic stroke, seizures, or haemorrhages. We consider it interesting to carry out future studies to address the predictive capacity of EWS, discriminating between "unplanned ICU admission" and "direct ICU admission from the EDs" in patients with these pathologies. Another limitation consists of the lack of a longer follow-up, which could add interesting information about the presence of residual functional deterioration and long-term mortality. All these limitations did not allow more meaningful conclusions about the role of EWS in stroke patients, and more studies with specific analysis are warranted.

In conclusion, this study suggests that the use of RAPS, MEWS, and MREMS should be preferred in the acute setting in patients with neurological impairment. High scores indicate worse prognosis and the potential necessity of more intensive and specific treatments.

Supplementary Materials: The following supporting information can be downloaded at: https://www. mdpi.com/article/10.3390/jpm12040630/s1. Table S1 : AUROC, cut-off points for combined sensitivity and specificity with best score (Youden's test) for the different scales analyzed (for 7-, 14-and 21-day mortality); Table S2 : Comparison of the AUROC of the different scores (ICU admissions); Table S3 : Comparison of the AUROC of the different scores (2-day mortality). Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Acknowledgments: The authors thank the nursing and medical staff from the ambulances and the emergency departments of all hospitals that belong to the public health system of Castilla and León (SACYL), for their assistance in facilitating data collection.

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Acute neurology: A suggested approach

Neurological patient care in emergency departments. A review of the current situation in Spain

Prevalence of acute neurology: A 2-week snapshot in a district general hospital

Identification of Serious Adverse Events in Patients with Traumatic Brain Injuries, from Prehospital Care to Intensive-Care Unit, Using Early Warning Scores

Stroke care during the COVID-19 pandemic: Experience from three large European countries

Stroke Care in the United Kingdom During the COVID-19 Pandemic

Worldwide stroke incidence and early case fatality reported in 56 population-based studies: A systematic review

A systematic analysis for the Global Burden of Disease Study

Clinical impact of implementing a rapid-response team based on the Modified Early Warning Score in wards that offer emergency department support

Early Prediction of One-Year Mortality in Ischemic and Haemorrhagic Stroke

Acute stroke care during the COVID-19 pandemic. Ictus Madrid Program recommendations. Atención al ictus agudo durante la pandemia por COVID-19. Recomendaciones Plan Ictus Madrid

Stroke Care during the COVID-19 Pandemic: International Expert Panel Review

Predictive power of early-warning scores used in hospital emergency departments: A systematic review and meta-analysis

Performance of universal early warning scores in different patient subgroups and clinical settings: A systematic review

Early warning score predicts acute mortality in stroke patients

IScore: A risk score to predict death early after hospitalization for an acute ischemic stroke

Modified early warning score and risk of mortality after acute stroke

Performance of Modified Early Warning Score (MEWS) for Predicting In-Hospital Mortality in Traumatic Brain Injury Patients

Early prediction of death in acute hypertensive intracerebral hemorrhage

Variables associated with 7-day, 30-day, and 1-year fatality after ischemic stroke

A catalogue of reporting guidelines for health research

Performance of Modified Early Warning Score (MEWS) and Circulation, Respiration, Abdomen, Motor, and Speech (CRAMS) score in trauma severity and in-hospital mortality prediction in multiple trauma patients: A comparison study

The comparison of modified early warning score with rapid emergency medicine score: A prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department

Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department

Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department

Comparing the effectiveness of three scoring systems in predicting adult patient outcomes in the emergency department

Prognostic accuracy of the Hamilton Early Warning Score (HEWS) and the National Early Warning Score 2 (NEWS2) among hospitalized patients assessed by a rapid response team

Prediction of in-hospital mortality and length of stay using an early warning scoring system: Clinical audit

Validation of WHO classification-based Prognostic Scoring System (WPSS) for myelodysplastic syndromes and comparison with the revised International Prognostic Scoring System (IPSS-R). A study of the International Working Group for Prognosis in Myelodysplasia (IWG-PM)

The impact of the use of the Early Warning Score (EWS) on patient outcomes: A systematic review

Monitoring vital signs using early warning scoring systems: A review of the literature

Outreach and Early Warning Systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards

Advanced imaging in acute ischemic stroke: An updated guide to the hub-and-spoke hospitals

Mechanical Thrombectomy in Patients Presenting with NIHSS Score <6: A Safety and Efficacy Analysis

Glasgow Coma Scale on Presentation Predicts Outcome in Endovascular Treatment for Acute Posterior Large-Vessel Occlusion

Admitting Elderly Patients to the Intensive Care Unit-Is it the Right Decision?

Should we deny ICU admission to the elderly? Ethical considerations in times of COVID-19

Interactions between drugs and geriatric syndromes in nursing home and home care: Results from Shelter and IBenC projects