key: cord-0909134-6hsdlogg
authors: Holten, Aleksander Rygh; Grotle Nore, Kristin; Tveiten, Caroline Emilie Van Woensel Kooy; Olasveengen, Theresa Mariero; Tonby, Kristian
title: Predicting severe COVID-19 in the Emergency Department
date: 2020-10-21
journal: Resuscitation plus
DOI: 10.1016/j.resplu.2020.100042
sha: eff1744800712cca435155b4e0fbe200e8c402b6
doc_id: 909134
cord_uid: 6hsdlogg

Background COVID-19 may lead to severe disease, requiring intensive care treatment and challenging the capacity of health care systems. The aim of this study was to compare the ability of commonly used scoring systems for sepsis and pneumonia to predict severe COVID-19 in the emergency department. Methods Prospective, observational, single centre study in a secondary/tertiary care hospital in Oslo, Norway. Patients were assessed upon hospital admission using the following scoring systems; quick Sequential Failure Assessment (qSOFA), Systemic Inflammatory Response Syndrome criteria (SIRS), National Early Warning Score 2 (NEWS2), CURB-65 and Pneumonia Severity index (PSI). The ratio of arterial oxygen tension to inspiratory oxygen fraction (P/F-ratio) was also calculated. The area under the receiver operating characteristics curve (AUROC) for each scoring system was calculated, along with sensitivity and specificity for the most commonly used cut-offs. Severe disease was defined as death or treatment in ICU within 14 days. Results 38 of 175 study participants developed severe disease, 13 (7 %) died and 29 (17 %) had a stay at an intensive care unit (ICU). NEWS2 displayed an AUROC of 0.80 (95 % confidence interval 0.72-0.88), CURB-65 0.75 (0.65-0.84), PSI 0.75 (0.65-0.84), SIRS 0.70 (0.61 – 0.80) and qSOFA 0.70 (0.61-0.79). NEWS2 was significantly better than SIRS and qSOFA in predicating severe disease, and with a cut-off of5 points, had a sensitivity and specificity of 82 % and 60 %, respectively. Conclusion NEWS2 predicted severe COVID-19 disease more accurately than SIRS and qSOFA, but not significantly better than CURB65 and PSI. NEWS2 may be a useful screening tool in evaluating COVID-19 patients during hospital admission. Trial registration : ClinicalTrials.gov Identifier: NCT04345536. (https://clinicaltrials.gov/ct2/show/NCT04345536).

quick Sequential Failure Assessment (qSOFA), Systemic Inflammatory Response Syndrome criteria (SIRS), National Early Warning Score 2 (NEWS2), CURB-65 and Pneumonia Severity index (PSI). The ratio of arterial oxygen tension to inspiratory oxygen fraction (P/F-ratio) was also calculated. The area under the receiver operating characteristics curve (AUROC) for each scoring system was calculated, along with sensitivity and specificity for the most commonly used cut-offs. Severe disease was defined as death or treatment in ICU within 14 days. Results: 38 of 175 study participants developed severe disease, 13 (7 %) died and 29 (17 %) had a stay at an intensive care unit (ICU). NEWS2 displayed an AUROC of 0.80 (95 % confidence interval 0.72-0.88), CURB-65 0.75 (0.65-0.84), PSI 0.75 (0.65-0.84), SIRS 0.70 (0.61 -0.80) and qSOFA 0.70 (0.61-0.79). NEWS2 was significantly better than SIRS and qSOFA in predicating severe disease, and with a cut-off of 5 points, had a sensitivity and specificity of 82 % and 60 %, respectively. Conclusion NEWS2 predicted severe COVID-19 disease more accurately than SIRS and qSOFA, but not significantly better than CURB65 and PSI. NEWS2 may be a useful screening tool in evaluating COVID-19 patients during hospital admission.

Introduction COVID-19 was first discovered in Wuhan, China, in December 2019, 1 and has since spread to every continent. The pandemic caused by SARS-CoV-2 has had a devastating impact on healthcare worldwide, 2 exceeding local health-care capacity in many regions of the world. 3, 4 It is estimated that about 5 % of COVID-19 patients develop critical disease. 5 Viral pneumonia is the most common organ manifestation, and many patients need respiratory support. 6, 7 Although the majority of patients with COVID-19 admitted to hospital can be managed on general wards with supplemental oxygen, there is limited knowledge on how to identify patients that will ultimately need invasive ventilatory support. A reliable screening tool to identify patients at risk would help allocate limited monitoring resources.

Several tools have been developed for risk stratification in patients with sepsis and pneumonia. Sepsis [8] [9] [10] [11] and pneumonia 12-14 scoring systems have been evaluated separately in COVID-19, but few comprehensive, prospective comparison of these tools in COVID-19 have been published 15 .The accuracy of sepsis and pneumonia scoring systems in COVID-19 is therefore still uncertain. 16 Commonly applied scoring systems for sepsis in the emergency department (ED) include; a) the Quick Sequential Failure Assessment (qSOFA) score, 17 b) Systemic Inflammatory Response Syndrome criteria (SIRS), 18 and c) National Early Warning Score 2 (NEWS2). 19 For pneumonia, the two most commonly used scoring systems are CURB-65 20,21 and Pneumonia Severity Index (PSI). 22 Both have been prospectively validated to predict mortality. PSI is far more comprehensive and time consuming than CURB-65, though currently recommended by the American Thoracic Society and Infectious Diseases Society of America. 23 Lastly, the ratio between arterial oxygen tension and inspiratory oxygen fraction (P/F-ratio) is used in the assessment of Acute Respiratory Distress Syndrome (ARDS), defined as P/F-ratio ≤300 mmHg (40kPa). 24 Although not strictly a scoring system, the P/F-ratio is considered to be important in assessing respiratory failure in critically ill patients, 25 and may be another tool for the initial assessment.

In this prospective observational study, we have evaluated and compared the predictive characteristics of commonly used scoring systems for sepsis and pneumonia applied to a cohort of consecutive COVID-19 patients admitted to our hospital. Our aim was to evaluate their accuracy in the ED in predicting the development of severe COVID-19 infection within 14 days after hospital admittance, in order to assist initial triage and allocation of limited monitoring resources.

Quick Sequential Organ Failure Assessment (qSOFA) (1 point each; score range, 0-3 points) 17 Altered mental state Systolic blood pressure ≤100 mmHg Respiratory rate ≥22 breaths/min Systemic Inflammatory Response Syndrome (SIRS) (1 point each; score range, 0-4 points) 18 Temperature >38°C or <36°C Heart rate > 90 beats/min Respiratory rate >20 breaths/min or PaCO2 <4.3 kPa White blood cell count >12 000 cells/mm 3 or < 4000 cells/mm 3 CURB-65 (1 point each; score range, 0-5 points) 20 Confusion Urea > 7 mmol/l Respiratory rate ≥30 breaths/min Systolic blood pressure <90 mmHg or diastolic blood pressure ≤ 60 mmHg Age ≥ 65 years Pneumonia severity index (risk class I-V) 22 Step 1. If any, assign patient to risk class II-V according to step 2. If non, assign patient to risk class I Age > 50 years History of neoplastic disease, congestive heart failure, cerebrovascular disease, renal disease or liver disease Any of the following abnormalities on physical examination: Altered mental status, pulse ≥ 125 beats/min, respiratory rate ≥ 30 breaths/min, systolic blood pressure <90 mmHg or temperature <35°C or ≥40°C.

Step 2. Point scoring system (point assigned for each characteristic). If the score is ≤70 the patient is assigned to risk class II, 71-90 to risk class III, 91-130 to risk class IV and >130 risk to risk class V. Age (1 point/year) Female sex (10 points) Nursing home resident (10 points) Neoplastic illnesses (30 points) Liver disease (20 points) Congestive heart failure (10 points) Cerebrovascular disease (10 points) Renal failure (10 points) Altered mental state (20 points) Respiratory rate ≥30 breaths/min (20 points) Systolic blood pressure <90 mmHg (20 points) Temperature <35°C or ≥40°C (15 points) Pulse ≥ 125 beats/min (10 points) Arterial pH <7.35 (30 points) Blood urea nitrogen ≥ 11 mmol/l (20 points) Sodium <130 mmol/l (20 points) J o u r n a l P r e -p r o o f Glucose >14 mmol/l (10 points) Haematocrit < 30 % or Haemoglobin <9 g/dl (10 points) PaO2 <60 mmHg (>8 kPa) (10 points) Pleural effusion (10 points) National Early Warning Score 2 (NEWS2) (score range 0-20) 19 The following six physiological parameters are scored from 0 to 3. Respiratory Rate Oxygen Saturations (with lower oxygen saturation targets for patients with confirmed hypercapnia) Temperature Systolic blood pressure Heart Rate Level of consciousness Additional 2 point if any Supplemental Oxygen is administered.

All SARS-CoV2-positive patients >18 years old, admitted to the OUH during the period from 01.03.20 -30.06.20 were evaluated for study participation. Patients transferred from other hospitals, and patients electively hospitalized for medical conditions unrelated to SARS-CoV-2, were excluded. Since the aim of this study was to evaluate the value of the scoring systems in the ED, only the calculation of qSOFA, SIRS-criteria, CURB-65, PSI, NEWS2 and P/F-ratio from the patient's first assessment at the hospital were included. Severe disease was defined as death or treatment in ICU within 14 days after admittance. There were no established criteria for ICU admission. The decision to transfer a patient to ICU was taken by experienced intensivist on duty.

The included patients were included in the quality registry "Covid19 OUS", at Oslo University Hospital (OUH). The quality registry was approved by the data protection officer at OUH on March 13 th 2020 (Reference number 20/08822). Informed consent was waived in accordance with the data protection officer. Patients were included prospectively, but before March 13 th , due to the acute onset of the pandemic in Norway, the first eight patients were included retrospectively. The register contains all patients hospitalized with confirmed SARS-CoV-2 (positive real-time reverse transcriptase chain reaction (RT-PCR)) regardless of symptoms and clinical findings. Identification of COVID-19 positive patients admitted to OUH has been done by cross-checking results of positive SARS-CoV-2 RT-PCR provided by the Department of Microbiology at OUH, and by use of the diagnose coding system "International Classification of Diseases" (ICD10) by identifying the diagnosis codes J12.8 (other viral pneumonia) in combination with U07.1 (COVID-19 identified). The clinicians who assessed the patients were otherwise unrelated to the study. Data were extracted from patient journals and recorded electronically by register staff in the Medinsight registration tool. The records were controlled by two of the authors (ARH, TMO). This paper has been developed according to the STARD-guidelines for reporting diagnostic accuracy studies. 26 

The area under the receiver operating characteristics curve (AUROC) for severe disease was calculated for each scoring system. Sensitivity, specificity, positive and negative predicative values of the most commonly used cut-off values for SIRS-criteria (2/4), qSOFA (2/3), CURB-65 (2/5), PSI (3/5) and NEWS (5), together with a P/F-ratio less than 300 mmHg (40 KPa), were determined. Patients with missing data were excluded from the individual calculation.

Continuous variables are given in median and interquartile ranges, and compared using the independent Student's t test. Categorical variables are given in numbers and percentages, and were compared with the chi-square test. Statistical analyses were calculated using the IBM SPSS Statistics (version 26) and GraphPad Prism (version 8). 95 % confidence intervals for sensitivity, specificity and positive and negative predicative values were calculated by Medcalc (https://www.medcalc.org/). Pvalues less than 0.05 were considered significant. (table 3) . qSOFA had a sensitivity of 26 % and a specificity of 95 %. SIRS displayed a sensitivity of 79 % and a specificity of 52 %. The pneumonia prediction scores CURB-65 and PSI had sensitivities of 58 % and 71 % and specificities of 80 % and 70 %, respectively. The positive predicative values (PPV) of the scoring systems were generally low. NEWS2 ≥ 5 showed the highest negative predicative value (NPV) of 92 %, only significantly higher than qSOFA ≥ 2.

The AUROC for P/F-ratio for predicting severe disease was 0.81, which is comparable with NEWS2, CURB-65 and PSI. The sensitivity and specificity for the P/F-ratio with cut-off ≤300 mmHg (≤40kPa) were 70 % and 73 %, respectively.

This prospective observational study of five different scoring systems applied on COVID-19 patients at admittance to hospital found that NEWS2 predicts severe disease in COVID-19 patients significantly better than the sepsis scoring systems qSOFA and SIRS. The AUROC was not significantly higher for NEWS2 than for CURB-65 and PSI.

With a cut-off of NEWS2 set to five, the sensitivity of 82 % and the NPV of 92 % are moderately good. However, the specificity of 60 % and PPV of only 37 % makes it challenging to use in clinical practice. When the NEWS2 cut-off value is increased, both the specificity and the PPV increase, but at the expense of sensitivity. NEWS2 is currently used in assessing patients with suspected severe infections in several EDs around the world 27, 28 .Even though a single assessment during hospital admission obviously has limited predictive capability, NEWS2 might still be useful as a clinical decision tool also J o u r n a l P r e -p r o o f in COVID-19. Reliable identification of patients at high risk of developing severe COVID-19 could have important implications. A predictive tool would be helpful in selecting the patients most likely to benefit from additional monitoring, inclusion in interventional trials or receiving intensive care.

There have been several attempts at developing new tools to predict development of severe COVID-19. 29 Liang and colleagues developed a prognostic scoring system consisting of ten factors including X-ray abnormalities, age, symptoms, comorbid conditions and biomarkers. 30 The score is validated in four additional cohorts and obtained an AUROC of 0.88 both in the development cohort and the validation cohorts. It is even launched as a freely available online risk calculator. While promising, the score is demanding in use and requires several diagnostic tests before the score can be calculated.

In contrast to sepsis, which commonly presents with multiorgan failure, COVID-19 is often characterized by solitary respiratory failure. 7, 31 In contrast to the other sepsis scoring tools, three of seven parameters in the NEWS2 score relate to degree of respiratory failure, and this could explain its relatively high performance compared to the other scoring systems we evaluated. However, maximal scores in NEWS2 are reached at respiratory rate ≥ 25, oxygen saturation ≤ 91% and the need for any supplemental oxygen -parameters often far exceeded in COVID-19 patients at time of admission. It is possible that the prognostic accuracy of the NEWS2 score could be improved by modifying the score. For instance, respiratory rate ≥ 40 breaths per minute, oxygen saturation ≤ 80% and need for ≥ 10 litres of oxygen might further add to the total score. Novel predictive strategies, whether based on new artificial Intelligence technology 32 or modifications of existing scores, need to be prospectively tested in adequately sized training and validation cohorts.

Of the 175 hospitalized COVID-19 patients included in our study, 38 were defined as severe disease; non-survivors or by need of intensive care treatment. Thus, adequate clinical observations are of outmost importance. This is hindered by a multitude of infection control measures such as timeconsuming donning and doffing of personal protective equipment. It is therefore important that the routines for observations are simple and well documented. Serial assessments using NEWS2 is a reasonable practical approach, and may possibly identify deteriorating patients. While NEWS2 did not perform significantly better than CURB65 and PSI, it is simple to use, include only easily accessible clinical parameters. While CURB-65 and PSI both include laboratory analyses. P/F-ratio assessment require blood gas analysis, and may therefore be difficult to perform outside of the ICU. It must be emphasized that COVID-19 is mainly a pulmonary disease. We suggest that the NEWS2 score in COVID-19 patients is supplemented with monitoring the degree of oxygen requirement and more detailed grading of respiratory rate and signs of patient exhaustion, as well as the P/F-ratio if feasible.

This study has some limitations. It is a single centre study which may limit the generalizability, and the relatively small number of participants limits the certainty of our analysis. The study was conducted in the early phase of the pandemic, and there may have been changes in clinical practice and routines during the study period. However, we do not think this affected data collection. Notable strengths are that all participants are well characterized in a large prospective quality register that comprise a complete consecutive patient set, including the first COVID-19 patients admitted to our hospital.

In conclusion, our study revealed that NEWS2 was equivalent to CURB65, PSI and P/F-ratio, but more accurate than SIRS and qSOFA, in predicting severe disease among patients hospitalized for COVID-19. However, the value of single assessments is limited, and hospitalized patients must be adequately monitored for signs of deterioration.

Funding: The study was funded by Oslo University Hospital. It did not receive any external funding Registration: The study was registered as an observational study in ClinicalTrials.gov. The ClinicalTrials.gov Identifier is NCT04345536. Conflicts of interest: Dr Olasveengen has received research grants from Zoll Foundation and Laerdal Foundation. No other disclosures were reported. Ethics approval: Informed consent was waived in accordance with the data protection officer (case number 20/07119) due to the status as a quality register with reporting of aggregated patient data with no risk of identification of personal sensitive information. Consent for publication: The publication was approved by data protection officer at Oslo University Hospital (case number 20/08822, April 04, 2020) Availability of data and material (data transparency): Due to the nature of this research, with data from a quality register with waived consent, data is not available do to ethical and legal restrictions. All the authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. J o u r n a l P r e -p r o o f 

Outbreak of pneumonia of unknown etiology in Wuhan, China: The mystery and the miracle

Covid-19: how doctors and healthcare systems are tackling coronavirus worldwide

Potential association between COVID-19 mortality and health-care resource availability. The Lancet Global health

Triage of Scarce Critical Care Resources in COVID-19 An Implementation Guide for Regional Allocation: An Expert Panel Report of the Task Force for Mass Critical Care and the American College of Chest Physicians

Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases From the Chinese Center for Disease Control and Prevention

Baseline Characteristics and Outcomes of 1591 Patients Infected With SARS-CoV-2 Admitted to ICUs of the Lombardy Region

Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the

Critically ill SARS-CoV-2-infected patients are not stratified as sepsis by the qSOFA

COVID-19: Symptoms, course of illness and use of clinical scoring systems for the first 42 patients admitted to a Norwegian local hospital. Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke

Predictive value of National Early Warning Score 2 (NEWS2) for intensive care unit admission in patients with SARS-CoV-2 infection

Prognostic Accuracy of the SIRS, qSOFA, and NEWS for Early Detection of Clinical Deterioration in SARS-CoV-2 Infected Patients

Comparison of CRB-65 and quick sepsis-related organ failure assessment for predicting the need for intensive respiratory or vasopressor support in patients with COVID-19

Applicability of the CURB-65 pneumonia severity score for outpatient treatment of COVID-19

Performance of Pneumonia Severity Index and CURB-65 in Predicting 30-day mortality in patients with COVID-19

National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 -a prospective cohort study

Critical care crisis and some recommendations during the COVID-19 epidemic in China

Assessment of clinical criteria for sepsis: For the third international consensus definitions for sepsis and septic shock (sepsis-3)

Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis

National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party

Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

CRB-65 predicts death from communityacquired pneumonia

A prediction rule to identify low-risk patients with community-acquired pneumonia

Diagnosis and Treatment of Adults with Communityacquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. American journal of respiratory and critical care medicine

Acute respiratory distress syndrome: the Berlin Definition

The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure

STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration

NEWS2 is Superior to qSOFA in Detecting Sepsis with Organ Dysfunction in the Emergency Department

Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. American journal of respiratory and critical care medicine

Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal

Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease

Towards an Artificial Intelligence Framework for Data-Driven Prediction of Coronavirus Clinical Severity

PPV % (95 % CI) NPV% (95 % CI)