key: cord-0772997-6i7lju2p
authors: Su, Ying; Ju, Min-jie; Xie, Rong-cheng; Yu, Shen-ji; Zheng, Ji-li; Ma, Guo-guang; Liu, Kai; Ma, Jie-fei; Yu, Kai-huan; Tu, Guo-wei; Luo, Zhe
title: Prognostic Accuracy of Early Warning Scores for Clinical Deterioration in Patients With COVID-19
date: 2021-02-01
journal: Front Med (Lausanne)
DOI: 10.3389/fmed.2020.624255
sha: 2a7b42f85efc31fcda9d8a59615cbd017fbb5fd4
doc_id: 772997
cord_uid: 6i7lju2p

Background: Early Warning Scores (EWS), including the National Early Warning Score 2 (NEWS2) and Modified NEWS (NEWS-C), have been recommended for triage decision in patients with COVID-19. However, the effectiveness of these EWS in COVID-19 has not been fully validated. The study aimed to investigate the predictive value of EWS to detect clinical deterioration in patients with COVID-19. Methods: Between February 7, 2020 and February 17, 2020, patients confirmed with COVID-19 were screened for this study. The outcomes were early deterioration of respiratory function (EDRF) and need for intensive respiratory support (IRS) during the treatment process. The EDRF was defined as changes in the respiratory component of the sequential organ failure assessment (SOFA) score at day 3 (ΔSOFA(resp) = SOFA (resp) at day 3–SOFA(resp) on admission), in which the positive value reflects clinical deterioration. The IRS was defined as the use of high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation. The performances of EWS including NEWS, NEWS 2, NEWS-C, Modified Early Warning Scores (MEWS), Hamilton Early Warning Scores (HEWS), and quick sepsis-related organ failure assessment (qSOFA) for predicting EDRF and IRS were compared using the area under the receiver operating characteristic curve (AUROC). Results: A total of 116 patients were included in this study. Of them, 27 patients (23.3%) developed EDRF and 24 patients (20.7%) required IRS. Among these EWS, NEWS-C was the most accurate scoring system for predicting EDRF [AUROC 0.79 (95% CI, 0.69–0.89)] and IRS [AUROC 0.89 (95% CI, 0.82–0.96)], while NEWS 2 had the lowest accuracy in predicting EDRF [AUROC 0.59 (95% CI, 0.46–0.720)] and IRS [AUROC 0.69 (95% CI, 0.57–0.81)]. A NEWS-C ≥ 9 had a sensitivity of 59.3% and a specificity of 85.4% for predicting EDRF. For predicting IRS, a NEWS-C ≥ 9 had a sensitivity of 75% and a specificity of 88%. Conclusions: The NEWS-C was the most accurate scoring system among common EWS to identify patients with COVID-19 at risk for EDRF and need for IRS. The NEWS-C could be recommended as an early triage tool for patients with COVID-19.

The outbreak of coronavirus disease 2019 (COVID-19) has recently become a public health emergency of international concern (1). A novel coronavirus, termed as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was isolated as the pathogen of COVID-19 (2) . As of October 18, 2020, there have been more than 40 million confirmed COVID-19 cases and 1.1 million deaths globally from World Health Organization reports.

With a sharp increase in the number of cases and limited medical resources, healthcare systems worldwide are facing unprecedented challenges (3) . Although the majority of patients with COVID-19 have mild symptoms, patients with advanced age and chronic comorbidities such as hypertension tend to have poor outcomes (4, 5) . Patients infected with SARS-CoV-2 tend to get worse from illness onset with a median duration of 7 days, in which severe type may deteriorate to acute respiratory distress syndrome (ARDS) or multiple organ failure (6, 7) . During the outbreak of the COVID-19 crisis, early and quick recognition of patients who are at high risk of clinical deterioration would therefore be significantly important (8) . A severity-based approach is urgently needed for triaging high risk patients with COVID-19 (9) .

The Early Warning Scores (EWS) are a variety of physiologic scoring systems widely used in the world. These systems are based on bedside indices that can be obtained easily and rapidly such as heart rate, respiratory rate, systolic blood pressure, and peripheral oxygen saturation (SpO 2 ), allowing quick and accurate identify patients at high risk of clinical deterioration. Now, various EWS were developed for early recognition of clinical deterioration. The National Early Warning Score (NEWS), the most common EWS, was initially recommended by the Royal College of Physicians (RCP) (10) . It has been proved that NEWS was associated with ICU admission and death outside ICU (11, 12) . Its updated version, the National Early Warning Score 2 (NEWS2), with a new SpO 2 scoring scale, was published by the RCP in 2017 to improve prediction for clinical deterioration in patients with a hypercapnic respiratory failure (13) . Other versions of EWS such as the Modified Early Warning Score (MEWS) (14, 15) and Hamilton Early Warning Score (HEWS) (16) have also been recently developed to improve the early recognition of hospitalized patients at risk for deterioration, with a significant degree of variation in the clinical variables and the weightings assigned.

Currently, guidelines from the RCP recommends the use of the NEWS2 for initial assessment in patients with COVID-19 (17) . Moreover, NEWS-C, a new version of modified NEWS, has also been recommended for triage decisions in patients with COVID-19 (18, 19) . However, the recommendations were only based on expert opinions and have not been fully validated in COVID-19 patients.

In this study, we aimed to compare the performance of EWS including NEWS, NEWS2, NEWS-C, HEWS, MEWS, and quick sepsis-related organ failure assessment (qSOFA) to predict early deterioration of respiratory function (EDRF) and the need for intensive respiratory support (IRS) in patients with COVID-19.

The study was approved by the Ethics Commission of Renmin Hospital of Wuhan University (WDRY2020-K048) and was conducted in accordance with the amended Declaration of Helsinki. Written informed consent was waived by the Ethics Commission in the setting of COVID-19 crisis in Wuhan.

Patients with age ≥18 years and confirmed COVID-19 admitted between February 7, 2020 and February 17, 2020 were screened in our study. We excluded patients for pregnancy, death within 48 h of admission, or having a Do Not Resuscitate order. COVID-19 was diagnosed by the real-time RT-PCR method on nasal or pharyngeal swab specimens.

Baseline demographics, clinical characteristics on hospital arrival including symptoms, vital signs, and oxygen therapy, laboratory findings, treatments, and outcomes were prospectively collected by two trained reviewers. The NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA were calculated based on the demographic and clinical characteristics of each patient (Supplementary Table 1 ).

Respiratory function was assessed according to the respiratory component of the sequential organ failure assessment score (SOFA resp ). The EDRF was defined as a positive change in respiratory function at day 3 ( SOFA resp = SOFA resp at day 3-SOFA resp on admission). The positive value of SOFA resp reflects clinical deterioration. The IRS was defined as the use of high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation. IRS was considered if the patients met the following criteria: a respiratory rate of ≥30 breaths per minute and arterial oxygen saturation (SaO 2 ) ≤ 93% or a ratio of the partial pressure of arterial oxygen (PaO 2 ) to inspired oxygen (FiO 2 ) of 300 mmHg or less while the patient was receiving oxygen therapy of ≥10 L/min for at least 15 min. The choices of respiratory support method (high flow nasal cannula oxygen therapy, noninvasive or invasive mechanical ventilation) were at the discretion of the attending clinicians.

Data distribution was evaluated using the Kolmogorov-Smirnov test. Continuous variables were expressed as mean ± standard deviation or median interquartile range as appropriate.

Categorical variables were expressed as frequencies and percentages. Baseline data were compared using the Student's t-test or the Mann-Whitney U test for continuous variables and the chi-square test or Fisher's exact test for categorical variables. Receiver operating characteristic (ROC) curves were constructed to assess the performance of EWS, and the optimal cut-off values were calculated by the Youden index.

The area under the receiver operating characteristic curve (AUROC) were compared by the method described by Hanley and McNeil (20) . All statistical analyses were performed using the SPSS software package, version 13.0 (SPSS, Inc., Chicago, IL, USA) and MedCalc software 15.0 (MedCalc Software Ltd, Ostend, Belgium). A two-tailed P value of <0.05 was considered statistically significant.

Between February 7, 2020 and February 17, 2020, a total of 123 patients with COVID-19 were screened for inclusion. Of these patients, seven patients were excluded, including one patient with pregnancy, three patients who died within 48 h after admission, and three patients with DNR order. Finally, 116 patients were included for this study. The baseline characteristics were shown in Table 1 . Of 116 patients, the median age was 63 [IQR 51, 72] years and 47.4% were men. Fever was the most common symptom (86.2%), followed by fatigue (85.3%), cough (69.0%), and dyspnea (56.9%). The baseline NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA at admission were 5 [3, 7] , 6 [5, 9] , 6 [5, 8] , 3 [2, 5] , 2 [2, 3] , and 1 [0, 1], respectively. The distributions of all patients by NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA at admission were presented in Figure 1 .

The baseline PaO 2 /FiO 2 on admission was 292 [245, 326] mmHg. At day 3, the median PaO 2 /FiO 2 was 314 [241, 330] mmHg. A total of 27 (23.3%) patients developed EDRF according to the SOFA resp (SOFA resp at day 3-baseline SOFA resp ), in which a positive value reflected clinical deterioration. Patients with EDRF tended to be older and had a higher rate of hypertension than those without EDRF (all P < 0.01). Compared with the patients without EDRF, the patients with EDRF have higher proportions of dyspnea (88.9 vs. 47.2%; P < 0.001) and higher respiratory rate [28 (21, 22) vs. 22 (20, 23) breaths/minute; P < 0.01] but lower baseline PaO 2 /FiO 2 value [245 (167, 303) vs. 305 (272, 328) mmHg; P < 0.001]. On admission, patients with EDRF had higher NEWS, NEWS-C, HEWS, MEWS, and qSOFA than non-EDRF patients (all P < 0.05; Table 1 ). However, the NEWS2 between patients with EDRF and non-EDRF was comparable (P = 0.16; Table 1 

A total of 24 patients (20.7%) needed IRS during the period of hospital stay. Patients with IRS also tended to be older and had a higher rate of hypertension than those without IRS (all P < 0.05). Compared with the patients without IRS, the patients requiring IRS have higher proportions of dyspnea (91.7 vs. 47.8%; P < 0.001) and a higher respiratory rate [32 (24, 25) vs. 22 (20, 26) ; P < 0.001] but a lower baseline PaO 2 /FiO 2 value [196 (150, 260) vs. 305 (274, 329); P < 0.001]. Patients with IRS had a higher NEWS, NEWS-C, NEWS2, HEWS, MEWS, and qSOFA than non-IRS patients (all P < 0.001; Table 1 ). A total of 20 patients (17.2%) developed both IRS and EDRF in this cohort. The hospital mortality rate was 7.8%. The mortality in patients with EDRF or IRS was higher than those without EDRF or IRS (all P < 0.001).

To assess the utility of EWS to predict EDRF and need for IRS, the ROC curves were constructed and the AUROCs were calculated (Figure 2) . Table 2 listed AUROC, optimal cutoff value, sensitivity, specificity, and positive and negative predictive values of EWS.

Among these EWS, NEWS-C was the most accurate scoring system for predicting EDRF {AUROC 0.79 [95% confidence interval (CI), 0.69-0.89]}. The AUROC of the NEWS-C for predicting EDRF was much higher than that for NEWS2 (0.59, 95% CI 0.46-0.72; P < 0.001) and qSOFA (0.62, 95% CI 0.51-0.74; P < 0.001). The AUROC of NEWS-C for predicting EDRF was also larger than NEWS (0.73, 95% CI 0.62-0.84), HEWS (0.75, 95% CI 0.63-0.86), and MEWS (0.71, 95% CI 0.59-0.83), although the difference is not statistically significant ( Table 3) . A NEWS-C ≥ 9 had a sensitivity of 59.3% and a specificity of 85.4% for predicting EDRF.

Among these EWS, NEWS-C was the most accurate scoring system for predicting IRS [AUROC 0.89 (95% CI, 0.82-0.96)]. The AUROC of the NEWS-C for predicting IRS was much higher than that for NEWS2 (0.69, 95% CI 0.57-0.81; P < 0.001), MEWS (0.80, 95% CI 0.70-0.90; P = 0.03), and qSOFA (0.72, 95% CI 0.61-0.82; P < 0.001). The AUROC of NEWS-C for predicting IRS was also higher than NEWS (0.86, 95% CI 0.78-0.95) and HEWS (0.87, 95% CI 0.77-0.97), although the difference is not statistically significant ( Table 4) . For predicting IRS, a NEWS-C ≥ 9 had a sensitivity of 75% and a specificity of 88%.

Until now, there are limited studies to evaluate the predictive value of EWS in patients with COVID-19. In the current study, we found that the NEWS-C was the most accurate scoring system among common EWS for predicting EDRF and IRS in patients with COVID-19. On the contrary, NEWS 2 had the lowest accuracy for predicting both outcomes. EWS have been developed and widely used around the world for early recognition of clinical deterioration (21) . The NEWS, endorsed by RCP, is already used for predicting deterioration in many hospitals across the United Kingdom (10) . It is reported that the NEWS has better performance than other EWS to identify patients at risk of ICU admission and mortality (12) . Moreover, the NEWS was more accurate in predicting clinical deterioration than qSOFA in infected patients outside the ICU (24) .

The NEWS 2, updated version of NEWS, was recommended by RCP in 2017. The new SpO 2 scoring scale in NEWS 2, with a lower SpO 2 threshold than NEWS, was implemented to avoid over-use of supplemental oxygen and facilitate management in hypercapnic patients (27) . Recently, the NEWS 2 has been recommended for predicting clinical deterioration in patients with COVID-19 (17) . In our study, NEWS 2 had a lower performance than NEWS in predicting EDRF and IRS. This is in accordance with previous study, which demonstrated that NEWS 2 did not predict clinical outcome in elderly patients with COVID-19 (28) . The possible reasons were as follows: (1) the incidence of type II respiratory failure in patients with COVID-19 was low in this study; and (2) the NEWS2 modifications to NEWS may not improve discrimination of poor outcome in hospital patients including those with type II respiratory failure (26) . Therefore, NEWS 2 may be inappropriate for triage decision in patients with COVID-19. A modified NEWS, termed NEWS-C, has also been recommended for triage decision in patients with COVID-19 (18, 19) . The MEWS (14) and HEWS (16) have been developed to early identify clinical deterioration in generally hospitalized patients, with a significant degree of variation in the clinical variables and the weightings assigned. In this study, the NEWS-C had largest AUROC for predicting EDRF and IRS in these EWS. NEWS-C modifications to EWS added an age ≥65 years as an independent component. Several studies have showed that old aging was independently associated with mortality in patients with COVID-19 (4, 23) . Therefore, it may offer better predictive performance than other EWS. qSOFA, consisting of three clinical variables (mental status, respiratory rate, and blood pressure), has been proposed as a rapid screening tool for infected patients (29) . The effectiveness of the qSOFA has been validated in various heterogeneous sepsis patients (30, 31) . Recently, several reports have demonstrated that the qSOFA can accurately assess the severity of community-acquired pneumonia (32) (33) (34) . However, qSOFA had a lower performance in predicting clinical deterioration compared with other EWS in our study. This may be partially explained by the low percentage of hypotension and alter mental status in this cohort. The finding was also consistent with previous studies, in which qSOFA may not be appropriate to identify critically ill patients with COVID-19 (22, 35) .

This study had several limitations. First, this was a retrospective study with relatively small sample size. A larger cohort validation is still required. Second, the changes in EWS during the treatment process was not recorded in this study. Third, as the intubation rate and mortality in this population were lower than that in critically ill patients (25, 35, 36) , caution must be taken in extrapolating the results of the study for critically ill patients. Additional assessments of organ dysfunction should be required in critically ill patients with COVID-19.

The NEWS-C was the most accurate scoring system among common EWS to identify patients with COVID-19 at risk for EDRF and need for IRS. The NEWS-C could be recommended as an early triage tool for COVID-19.

The original contributions presented in the study are included in the article/Supplementary Materials, further inquiries can be directed to the corresponding author/s.

The studies involving human participants were reviewed and approved by Ethics Commission of Renmin Hospital of Wuhan University. The ethics committee waived the requirement of written informed consent for participation.

Trajectory of the COVID-19 pandemic: chasing a moving target

A novel coronavirus from patients with pneumonia in China

Use of critical care resources during the first 2 weeks

Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China

Early estimation of the case fatality rate of COVID-19 in mainland China: a data-driven analysis

Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Lower mortality of COVID-19 by early recognition and intervention: experience from Jiangsu Province. Ann Intensive Care

Management of critically ill patients with COVID-19 in ICU: statement from frontline intensive care experts in Wuhan

A national early warning score for acutely ill patients

Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): a narrative review

The ability of the national early warning score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death

Can the prehospital national early warning score 2 identify patients at risk of in-hospital early mortality? A prospective, multicenter cohort study

Prognostic value of modified early warning score generated in a Chinese emergency department: a prospective cohort study

Detection of deteriorating patients after Whipple surgery by a modified early warning score (MEWS)

Examining the utility of the Hamilton early warning scores (HEWS) at triage: retrospective pilot study in a Canadian emergency department

Royal College of Physicians. NEWS2 and Deterioration in COVID-19

Novel coronavirus infection during the 2019-2020 epidemic: preparing intensive care units-the experience in Sichuan Province, China

Recommendations for the admission of patients with COVID-19 to intensive care and intermediate care units (ICUs and IMCUs)

A method of comparing the areas under receiver operating characteristic curves derived from the same cases

Risk stratification of hospitalized patients on the wards

Prognostic accuracy of the SIRS, qSOFA, and NEWS for early detection of clinical deterioration in SARS-CoV-2 infected patients

COVID-19 with different severity: a multi-center study of clinical features

Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit

Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City Area

A comparison of the ability of the national early warning score and the national early warning score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study

The national early warning score 2 (NEWS2)

Letter to the editor: variability but not admission or trends in NEWS2 score predicts clinical outcome in elderly hospitalised patients with COVID-19

The third international consensus definitions for sepsis and septic shock (sepsis-3)

Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3)

Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit

Efficacy and accuracy of qSOFA and SOFA scores as prognostic tools for community-acquired and healthcare-associated pneumonia

Comparison of the qSOFA and CRB-65 for risk prediction in patients with community-acquired pneumonia

Use of CRB-65 and quick sepsis-related organ failure assessment to predict site of care and mortality in pneumonia patients in the emergency department: a retrospective study

Critically ill SARS-CoV-2-infected patients are not stratified as sepsis by the qSOFA

Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study

YS, G-wT, and ZL: conception and design. M-jJ, R-cX, and ZL: administrative support. YS, M-jJ, S-jY, J-lZ, and G-gM: provision of study materials or patients. YS, S-jY, J-lZ, KL, R-cX, and G-gM: collection and assembly of data. YS, G-wT, R-cX, M-jJ, and ZL: data analysis and interpretation. All authors: manuscript writing and final approval of manuscript. 

The authors appreciate the great sacrifice and contribution of the people from Wuhan, Hubei province, China.

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed. 2020.624255/full#supplementary-material