key: cord-0764144-yqygc5e8
authors: Guan, Gigi; Lee, Crystal Man Ying; Begg, Stephen; Crombie, Angela; Mnatzaganian, George
title: The use of early warning system scores in prehospital and emergency department settings to predict clinical deterioration: A systematic review and meta-analysis
date: 2022-03-17
journal: PLoS One
DOI: 10.1371/journal.pone.0265559
sha: e1e51168769cf04706405fdd38225fe34fce1378
doc_id: 764144
cord_uid: yqygc5e8

BACKGROUND: It is unclear which Early Warning System (EWS) score best predicts in-hospital deterioration of patients when applied in the Emergency Department (ED) or prehospital setting. METHODS: This systematic review (SR) and meta-analysis assessed the predictive abilities of five commonly used EWS scores (National Early Warning Score (NEWS) and its updated version NEWS2, Modified Early Warning Score (MEWS), Rapid Acute Physiological Score (RAPS), and Cardiac Arrest Risk Triage (CART)). Outcomes of interest included admission to intensive care unit (ICU), and 3-to-30-day mortality following hospital admission. Using DerSimonian and Laird random-effects models, pooled estimates were calculated according to the EWS score cut-off points, outcomes, and study setting. Risk of bias was evaluated using the Newcastle-Ottawa scale. Meta-regressions investigated between-study heterogeneity. Funnel plots tested for publication bias. The SR is registered in PROSPERO (CRD42020191254). RESULTS: Overall, 11,565 articles were identified, of which 20 were included. In the ED setting, MEWS, and NEWS at cut-off points of 3, 4, or 6 had similar pooled diagnostic odds ratios (DOR) to predict 30-day mortality, ranging from 4.05 (95% Confidence Interval (CI) 2.35–6.99) to 6.48 (95% CI 1.83–22.89), p = 0.757. MEWS at a cut-off point ≥3 had a similar DOR when predicting ICU admission (5.54 (95% CI 2.02–15.21)). MEWS ≥5 and NEWS ≥7 had DORs of 3.05 (95% CI 2.00–4.65) and 4.74 (95% CI 4.08–5.50), respectively, when predicting 30-day mortality in patients presenting with sepsis in the ED. In the prehospital setting, the EWS scores significantly predicted 3-day mortality but failed to predict 30-day mortality. CONCLUSION: EWS scores’ predictability of clinical deterioration is improved when the score is applied to patients treated in the hospital setting. However, the high thresholds used and the failure of the scores to predict 30-day mortality make them less suited for use in the prehospital setting.

Initially used in the intensive care unit (ICU), the Early Warning System (EWS) scores have been employed in multiple healthcare facilities, including hospital wards, the emergency department (ED), and prehospital community settings [1, 2] . These scores primarily aim to detect the clinical deterioration in patients by tracking their vital signs, with high EWS scores triggering a response to prevent any potential clinical decline. It has been observed that patients' vital signs usually change before any clinical deterioration [3, 4] . Consequently, if early and timely interventions are adequately performed, adverse outcomes of patients may be prevented. The earliest EWS score was validated in 1981 for only the ICU patients. Modifications over time were developed to suit various hospital inward settings, with some being more specific to certain conditions such as blunt trauma or sepsis [5] [6] [7] [8] . However, the fundamentals of the scores have generally been consistent and involve measurements of variations in vital signs e.g., systolic blood pressure, oxygen saturation, temperature, heart rate, and Glasgow Coma Scale, which are used to calculate a cumulative score. Other versions of the EWS employ advanced therapies, laboratory testing, and demographic information of patients [2, 9] . The EWS scores constitute a standardised practice across prehospital and hospital settings in the UK [6] and in hospital settings in some parts of Australia.

Numerous systematic reviews and meta-analyses have identified optimal EWS scores in hospital settings (e.g., wards or the ED) [3, [10] [11] [12] [13] [14] . However, only a limited number of reviews focused on prehospital settings [15, 16] . Available systematic reviews demonstrate that EWS scores can be utilised to potentially improve patient outcomes [11] [12] [13] [14] [15] but, since several EWS scores are used across different settings, it is unknown which score should be used in the ED or prehospital setting to best predict clinical deterioration [17, 18] . Furthermore, the selection of EWS scores of best cut-off points to accurately predict outcomes such as short-term and long-term mortality or ICU admission has not been performed [6] [7] [8] .

This systematic review and meta-analysis aimed to estimate the pooled odds of predicting clinical deterioration in hospitalised patients, including short (�3-day) and long-term (�30-day) mortality and ICU admission, by stratifying the EWS score cut-off points as used in the ED and prehospital settings. Length of stay in hospital and cardiac or respiratory arrests were also investigated. Score (RAPS) and the Cardiac Arrest Risk Triage (CART) [10, [12] [13] [14] [15] 19] . These scores were selected since they rely solely on observations readily available to health care professionals in the prehospital setting and are easy to calculate and apply. Thus, we avoided the costly and time-consuming pathological and other complex testings, which are less frequently performed in the prehospital environment. A PICO framework was used to inform the literature search strategy (S1 File).

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was used to design and report this systematic review. The protocol was registered with the PROSPERO-International register of systematic reviews (registration number of CRD42020191254).

Original research following experimental, quasi-experimental, or observational designs using EWS scores in either the ED or prehospital setting were eligible for inclusion. Additionally, studies were included if they reported on patients aged �14 years presenting with medical conditions including sepsis or injury associated conditions, with study outcomes including in-hospital mortality within 3-30 days of hospitalisation, cardiac arrest/respiratory arrest, and length of stay. Only studies published in English were included with no restriction to the year of publication.

Studies reporting on obstetric, maternal, or palliative care patients were excluded. Furthermore, articles focusing solely on Severe Acute Respiratory Syndrome Coronavirus 2 infections and rare conditions (e.g., portal hypertension) were excluded.

Four databases (CINAHL, Embase, PubMed/MEDLINE, and Web of Science) were systematically searched in February 2021. Key search terms included prehospital/ambulance/paramedic, ED/emergency room, and the selected EWS scores. All synonyms and MeSH terms were included in the searches (S2 File). In addition, the references of the identified articles were manually searched for additional articles that might have been missed in the electronic searches.

Three co-authors (GG, GM, CL) independently screened all articles based on title and abstract. In addition, author GG screened all potential articles cross-checked by CL (80%) and GM (20%). Any disagreements or conflicts were discussed to make the final decision for inclusion.

Data extracted included author name, year and country of publication, study setting (ED or prehospital), sample size, study outcomes, EWS scores, their cut-off points and sensitivity and specificity in predicting the desired study outcomes, mean or median age of study population, sex proportion, study design, and study inclusion and exclusion criteria. The Newcastle-Ottawa Scale was used to assess the methodological quality of the included articles [20] . GG and SB independently conducted the assessments, with conflicts resolved after discussion with co-authors.

An odds ratio (OR) was computed as a summary measure of the predictive accuracy of each EWS from (TP x TN) / (FP x FN), in which TP, TN, FP, and FN respectively express true positive, true negative, false positive and false negative. The confidence interval (CI) of the OR was estimated as:

where Zα/2 denotes the critical value of the normal distribution at α/2 (e.g., for a CI 95%, α is 0.05, and the critical value reaches 1.96).

To express the diagnostic accuracy of the EWS scores, the log ORs, together with their corresponding log standard errors, were meta-analysed using DerSimonian and Laird randomeffects models [21] . The analyses were conducted by the EWS scores cut-off points, patient outcomes (short-and long-term mortality from hospital admission and ICU admission) and study setting (ED or prehospital). We defined short-term mortality as death within three days of admission and long-term mortality as death within 30 days of admission. The heterogeneity between studies was estimated using I 2 statistic. Meta-regressions were constructed to quantify the proportion of variances between studies, as explained by sample size, sex proportion, and age. Deeks' funnel plot asymmetry test was used to test publication bias. Pooled diagnostic ORs were compared after converting them to z scores. The p values were estimated using the normal distribution table. Sensitivity analyses were conducted by risk of bias.

All analyses were conducted using Stata/SE 15.1 (Stata Corp LP., College Station, Texas, USA).

The electronic databases searches identified 11,565 potential references. After removing duplicates, applying inclusion and exclusion criteria, and excluding irrelevant articles based on the title and abstract, 260 articles were included in the full-text review. Of these, 20 articles with a total sample of 89,928 patients were included for meta-analysis (Fig 1) .

The characteristics of the 20 articles included in the analysis are presented in Tables 1-3 . Among these studies, 13 were conducted in the ED [22] [23] [24] [25] [26] [27] [28] [29] , including five articles on patients presenting with sepsis alone [30] [31] [32] [33] [34] , and seven in the prehospital setting [9, [35] [36] [37] [38] [39] [40] . The study designs included prospective cohort (n = 11) [23, 25-30, 33, 36-38] , retrospective cohort (n = 8) [9, 22, 24, 31, 32, 34, 35, 39] , and pragmatic clinical trial design (n = 1) [40] . In the ED setting, ten studies used MEWS [22] [23] [24] [25] [26] [27] [28] [30] [31] [32] , and four studies used NEWS [24, 29, 33, 34] .

Five studies evaluated NEWS2 [36] [37] [38] [39] [40] in the prehospital setting, and three studies used NEWS [9, 35, 40] . Overall, the meta-analysis included 18,270 and 71,658 patients from the ED and prehospital settings, respectively.

Quality and risk of bias assessments of each included study are found in S3 File. A sensitivity analysis was conducted with and without studies with high risk of bias in the ED setting, as shown in Figs 2 and 3. Of all studies, two were rated poor (10%), and the remaining 18 (90%) were rated good.

The pooled diagnostic ORs (DOR) of MEWS (cut-off points of �3 and �4) and NEWS (cutoff point �6) to predict 30-day mortality and of MEWS (cut-off point �3) to predict ICU admission were estimated (Fig 2) . 

As illustrated in Fig 4, 

Studies reporting 30-day mortality had moderate to high heterogeneity (I 2 >70%) in the ED and prehospital settings. Meta-regressions, including studies from the ED setting, were constructed to detect the variables, potentially contributing to the between-studies differences.

Only patients' age contributed to the heterogeneity of the study variables, explaining 92% of the between-study variance. We could not perform meta-regression in the prehospital setting due to the limited number of studies investigating 30-day mortality.

Since studies often reported multiple results for different EWS scores. We included the highest and lowest reported ORs in each study in two separated Deeks' funnel asymmetry tests to 

While the application of EWS scores in the ED is well documented, only limited studies assessed their function and applicability in the prehospital setting. This systematic review and meta-analysis investigated the predictability of various EWS scores, as utilised in the ED or prehospital setting, to predict up-to-3-and 30-day mortality and admission to ICU. We found that different cut-off points of various EWS scores applied in the ED had comparable abilities to predict clinical decline, including death among hospitalised patients. Conversely, in the prehospital setting with relatively high cut-off points, the EWS only predicted short-term clinical deterioration failing to predict 30-day mortality. Similar to other studies, our systematic review demonstrated the ability of EWS scores to predict 30-day mortality when applied in the ED setting [41, 42] . However, unlike other studies that suggested optimal cut-off points of different EWS scores to predict clinical deterioration [43] [44] [45] [46] [47] , this systematic review did not detect any significant variation in the predictability of different scores by different cut-off points when applied in the ED. The choice of cut-off point may depend on the severity of illness and the acuteness of the investigated condition. Patient presentation varies across different settings; in the prehospital setting, the general population attended by paramedics are less severely ill (with a considerable majority having non-urgent low acuity presentations) than patients who were sick enough to have been brought to the ED [48] . In critically ill patients, lower cut-off points are able to predict clinical deterioration [16, 49, 50] , which may indicate that the predictability of the score is outcome and patient population specific, as evidenced by this systematic review with both settings requiring different optimal thresholds. The cut-off points applied in ED are typically lower than those used in the prehospital setting. The high cut-off points in the prehospital setting may imply that EWS scores are adequate to predict short-term clinical deterioration among the critically ill, as the higher EWS scores target more severely ill patients who are at high risk of clinical deterioration in the short term [16, 49, 50] . Thus, the high thresholds used in the prehospital setting may be targeting the critically ill patients who will constitute a relatively small proportion of the overall prehospital patient population that paramedics treat. The review conducted by Williams et al. on the use of EWS scores in the prehospital setting also suggests that critically ill patients are the best candidates for the use of prehospital EWS scores [16] . The authors also argue that achieving an optimal EWS score is difficult due to the short duration of the interaction between paramedics and patients. The reporting of high cut-off points in the prehospital setting is also due to a trade-off in sensitivity and specificity. Lower cut-off points often result in poor sensitivity and specificity in the prehospital setting. Conversely, the cut-off points in the ED are often similar to the cut-off points used in in-hospital settings suggesting that EWS scores can be compared between the ED and in-hospital wards, whereas this comparison becomes less valid when it is conducted against the prehospital setting. In our review, NEWS2 with a cut-off point of 5, 7 or 9 and NEWS with a cut-off point of 7 had similar predictability. This is supported by a study that compared NEWS and NEWS2 at the same threshold of 7 without detecting significant difference between the two scores when predicting short-term mortality [3] . The Royal College of Physicians in London argue that NEWS2 is superior to NEWS in predicting clinical deterioration [51, 52] ; however, this was not supported by our findings. Similarly, Hodgson and colleagues demonstrated that NEWS2 did not outperform NEWS in predicting clinical deterioration in patients admitted to hospital with acute exacerbation of chronic obstructive pulmonary disease, [52] , which was one of the main reasons why chronic hypoxia patient presentations were as an additional parameter in NEWS2. Based on the available studies and our systematic review, NEWS and NEWS2 had similar predictabilities. In this review, the analysis on patients experiencing sepsis was conducted separately partly because the diagnosis of sepsis requires pre-defined cut-off values for the vital signs. Patients experiencing sepsis may differ from other patients in such that they will have higher baseline EWS scores. For example, common criteria for sepsis include presence of two or more of the following: temperature > 38˚C or < 36˚C, heart rate > 90/min, respiratory rate > 20/min or PaCO2 < 32 mm Hg (4.3 kPa), white blood cell count > 12 000/mm 3 or <4000/mm 3 or > 10% cells with immature bands [53] . These criteria indicate systemic clinical decline and may result in elevated EWS scores. Our systematic review shows that the predictive ability of EWS scores of long-term mortality for patients experiencing sepsis is not different from the mainstream ED patient population.

The results of the systematic review apply only to the EWS scores assessed in this study. The analysis lacked the power to evaluate medical versus trauma conditions separately. Patients' main complaints and diagnoses were unknown and could not be accounted for. The number of studies reporting cardiac and/or respiratory arrest and length of stay were limited and could not be included for meta-analysis. We also acknowledge the lack of reporting of short-term mortality in ED. The articles that reported short-term mortality failed to include cut-off points, making the authors unable to draw conclusions based on available data. Confounding factors that may have affected long-term mortality following hospital admission were unknown and were not accounted for.

The accuracy and predictability of the EWS scores depend on several factors, such as the outcome measure, population, and the types of clinical settings. In the ED setting, the patient population is by default more morbid than that managed by paramedics in the community. This, in turn, may explain the ability of low EWS cut-off points to predict clinical deterioration in the ED. We report that different EWS cut-off points in the ED have similar predictability. Studies using EWS scores in the prehospital setting utilised relatively high cut-off points. This may indicate that early warning scoring systems may be less applicable for the general population treated by medical care personnel in the prehospital setting. This scoring system may only be suited for critically ill patients treated in the prehospital setting. Our findings suggest that EWS scores applied in the prehospital setting cannot accurately predict long-term events, including 30-day mortality. However, this SR demonstrates that EWS scores used in the prehospital setting can predict short term clinical decline. 

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

Predictive value of the prehospital NEWS2-L -National Early Warning Score

Lactate-for detecting early death after an emergency

Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward

Accurate respiratory rates count: So should you. Australasian emergency nursing journal

Assessing prognosis with modified early warning score, rapid emergency medicine score and worthing physiological scoring system in patients admitted to intensive care unit from emergency department

NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party

Prognostic Accuracy of Quick SOFA is different according to the severity of illness in infectious patients

New scoring system for intra-abdominal injury diagnosis after blunt trauma

Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: An observational cohort study

The impact of the use of the Early Warning Score (EWS) on patient outcomes: a systematic review

Can early warning scores identify deteriorating patients in pre-hospital settings? A systematic review

The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: A systematic review

Pre-hospital National Early Warning Score (NEWS) is associated with in-hospital mortality and critical care unit admission: A cohort study

Modified early warning score in assessing disease conditions and prognosis of 10,517 pre-hospital emergency cases

The effectiveness of physiologically based early warning or track and trigger systems after triage in adult patients presenting to emergency departments: A systematic review

The Newcastle-Ottawa Scale (NOS) for assessing the quality if nonrandomized studies in meta-analyses

Meta-analysis in clinical trials

Performance of Modified Early Warning Score (MEWS) and Circulation, Respiration, Abdomen, Motor, and Speech (CRAMS) score in trauma severity and in-hospital mortality prediction in multiple trauma patients: a comparison study

Combination of Lactate with Modified Early Warning Score and Rapid Emergency Medicine Score in Geriatric Patients Admitted to Emergency Department to Predict 28-Day Mortality

Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department

The comparison of modified early warning score and Glasgow coma scale-age-systolic blood pressure scores in the assessment of nontraumatic critical patients in Emergency Department

Evaluating the Use of a Modified Early Warning Score in Predicting Serious Adverse Events in Iranian Hospitalized Patients: A Prognostic Study

The significance of National Early Warning Score for predicting prognosis and evaluating conditions of patients in resuscitation room

Modified Early Warning Score and VitalPac Early Warning Score in geriatric patients admitted to emergency department

NEWS and qSIRS superior to qSOFA in the prediction of 30-day mortality in emergency department patients in Hong Kong

Severity illness scoring systems for early identification and prediction of in-hospital mortality in patients with suspected sepsis presenting to the emergency department

Classifying sepsis patients in the emergency department using SIRS, qSOFA or MEWS

Prediction of mortality in adult emergency department patients with sepsis

The early identification of disease progression in patients with suspected infection presenting to the emergency department: a multi-centre derivation and validation study

Predicting mortality in patients with suspected sepsis at the Emergency Department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score

Prehospital National Early Warning Score predicts early mortality

Analysis of the early warning score to detect critical or high-risk patients in the prehospital setting

Can the prehospital National Early Warning Score 2 identify patients at risk of inhospital early mortality? A prospective, multicenter cohort study

Accuracy of National Early Warning Score 2 (NEWS2

Pre-hospital triage performance and emergency medical services nurse's field assessment in an unselected patient population attended to by the emergency medical services: a prospective observational study

The Value of Prehospital Early Warning Scores to Predict in-Hospital Clinical Deterioration: A Multicenter, Observational Base-Ambulance Study

Validation of a modified Early Warning Score in medical admissions

Prognostic value of Modified Early Warning Score generated in a Chinese emergency department: a prospective cohort study

The prognostic value of the Modified Early Warning Score in critically ill patients: a prospective, observational study

The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department

Modified Early Warning Score and VitalPac Early Warning Score in geriatric patients admitted to emergency department

Modified Early Warning Score (MEWS) Identifies Critical Illness among Ward Patients in a Resource Restricted Setting in Kampala, Uganda: A Prospective Observational Study

Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. American Journal of Respiratory and Critical Care Medicine

Seeking ambulance treatment for 'primary care' problems: a qualitative systematic review of patient, carer and professional perspectives

Can the prehospital National Early Warning Score identify patients most at risk from subsequent deterioration?

Validation of the National Early Warning Score in the prehospital setting

NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party

A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation

The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)