key: cord-1047787-1e9wn54a
authors: Bellos, Ioannis; Lourida, Panagiota; Argyraki, Aikaterini; Korompoki, Eleni; Zirou, Christina; Kokkinaki, Ioanna; Pefanis, Angelos
title: Development of a novel risk score for the prediction of critical illness amongst COVID‐19 patients
date: 2020-12-22
journal: Int J Clin Pract
DOI: 10.1111/ijcp.13915
sha: fa5c5b284eaa83d9e23729fa0c3b6de7e572193b
doc_id: 1047787
cord_uid: 1e9wn54a

OBJECTIVES: Coronavirus disease‐19 (COVID‐19) is associated with various clinical manifestations, ranging from asymptomatic infection to critical illness. The aim of this study is to evaluate the clinical and laboratory characteristics of hospitalised COVID‐19 patients and construct a predictive model for the discrimination of patients at risk of disease progression. METHODS: A single‐centre cohort study was conducted including consecutively patients with COVID‐19. Demographic, clinical and laboratory findings were prospectively collected at admission. The primary outcome of interest was the intensive care unit admission. A risk model was constructed by applying a Cox's proportional hazard's model with elastic net penalty. Its diagnostic performance was assessed by receiver operating characteristic analysis and was compared with conventional pneumonia severity scores. RESULTS: From a total of 67 patients 15 progressed to critical illness. The risk score included patients’ gender, presence of hypertension and diabetes mellitus, fever, shortness of breath, serum glucose, aspartate aminotransferase, lactate dehydrogenase, C‐reactive protein and fibrinogen. Its predictive accuracy was estimated to be high (area under the curve: 97.1%), performing better than CURB‐65, CRB‐65 and PSI/PORT scores. Its sensitivity and specificity were estimated to be 92.3% and 93.3%, respectively, at the optimal threshold of 1.6. CONCLUSIONS: A10‐variable risk score was constructed based on clinical and laboratory characteristics in order to predict critical illness amongst hospitalised COVID‐19 patients, achieving better discrimination compared with traditional pneumonia severity scores. The proposed risk model should be externally validated in independent cohorts in order to ensure its prognostic efficacy.

from asymptomatic disease to acute lung injury with severe hypoxemia requiring mechanical ventilation, although the subgroup of patients prone to develop rapid respiratory deterioration remains still a matter of debate.

Early risk stratification is essential to guide decision making regarding clinical management and patient allocation, especially in resource-limited settings. To this end, significant research effort has been devoted to the identification of factors associated with disease progression and the construction of predictive models, including clinical, laboratory or radiological parameters. 5 Nonetheless, inconsistent results have been reported leading to remarkable heterogeneity of the existing risk scores. In addition, the risk of bias and concerns about overfitting often exist, rendering the proposed models overoptimistic and limiting their direct clinical applicability. 6 As a result, the optimal screening model for the prediction of critical illness remains still under investigation.

The present study aims to comprehensively assess the clinical and laboratory characteristics at the admission of patients with COVID-19 and identify those linked to worse prognosis and disease progression. A novel risk score is constructed by applying machine learning methodology in order to improve variable selection and effectively recognise patients at higher risk of severe disease. At the same time, traditional pneumonia severity scores are estimated and their predictive performance is compared with the proposed risk model.

All consecutive adult patients with COVID-19 admitted to our department in "Sotiria" General and Chest Diseases Hospital of Athens from 11 March to 1 June 2020 were prospectively enrolled. The diagnosis was based on the detection of SARS-CoV-2 by real-time polymerase chain reaction (RT-PCT) analysis of nasopharyngeal swabs. Non-laboratory-confirmed cases were excluded. The study was approved by the institutional review board of the hospital and all patients or their next of kin gave written informed consent.

Reporting of outcomes was performed in accordance with the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) guidelines. 7

Pre-specified variables about clinical history, comorbidities, symptoms, laboratory tests on admission and treatment were registered in a comprehensive database. Variables of interest were baseline characteristics, comorbidities (diabetes mellitus, hypertension, coronary artery disease, heart failure, chronic obstructive pulmonary disease, asthma, liver disease, cancer, hematologic malignancy and immunodeficiency), clinical symptoms (fever, chills, cough, shortness of breath, sputum production, hemoptysis, fatigue, headache, diarrhoea and nausea/vomiting) and laboratory tests (complete blood count, coagulation tests, renal and liver function tests, serum electrolytes, glucose, lactate dehydrogenase, amylase, troponin, triglycerides, total cholesterol, C-reactive protein, procalcitonin, ferritin and blood lactate levels). Derived neutrophil-to-lymphocyte ratio (dNLR) was defined as: dNLR = Neutrophil count/(White blood cell count − Lymphocyte count)). 8 10 and PSI/PORT (Pneumonia Severity Index/ Pneumonia Outcome Research Trial-PSI/PORT). 11 The main outcome of interest was set to be the transfer from the isolation ward to the intensive care unit (ICU). Secondary outcomes included the occurrence of acute respiratory distress syndrome (ARDS), 12 systemic inflammatory response syndrome (SIRS) 13 and acute kidney injury. 14 In addition, the maximum values of SOFA (Sequential Organ Failure Assessment), 15 APACHE II (Acute Physiology and Chronic Health Evaluation II) 16 and MEWS (Modified Early Warning Score) 17 scores during hospital stay were calculated.

Statistical analysis was conducted in R-3.6.3 ("survival," 18 "glmnet," 19 and "pROC" 20 packages). Statistical significance was defined as P value < .05. Normality of continuous variables was tested by the Shapiro-Wilk test because of the moderate sample size. 21 The possible linear correlation amongst laboratory variables was assessed by the Spearman correlation coefficient, because of the presence of skewed distributions. Strong correlations were detected by Spearman ρ < −0.6 or > 0.6. 22 Potential missing values were planned to be statistically imputed by the k-nearest neighbour method. 23 

• A 10-variable model including demographical and laboratory parameters has been developed by elastic net regularisation, outperforming the conventional pneumonia severity scores.

Cox's proportional hazard's model with elastic net penalty was implemented. The elastic net penalty was defined by the following equation:

where β represents the regression coefficient, λ the shrinkage parameter and α the elastic net mixing parameter, with 0 ≤ α ≤1.

Values of α equal to 0 and 1 correspond to the ridge and the Least Absolute Shrinkage and Selection Operator (LASSO) regression models, respectively. 24 The values of α and λ were selected by 10-fold cross-validation. Specifically, the λ value providing minimum deviance (λ min ) was chosen for the analysis and the α value providing the lowest mean cross-validation error at λ min was selected. A risk prediction model was constructed by including the parameters with nonzero coefficients; hence, a risk score was calculated according to the following equation:

where β i refers to the regression coefficient and χ i the value of the parameter. The Harrell concordance C-index was used to evaluate the discrimination of the model. The estimated risk score was compared between patient subgroups based on age (≤65/>65 years), day from symptom onset (≤7/>7 days) and PaO 2 /FiO 2 ratio on admission (≤200, 201-300, >300) using the Mann-Whitney U test and was incorporated in a multivariate Cox's proportional hazard's model including the above parameters. Moreover, the diagnostic accuracy of the risk score was tested by plotting the receiver operating characteristics (ROC) curve and calculating the area under the curve (AUC).

The optimal threshold was specified by estimating the Youden index 25 and the corresponding sensitivity and specificity were reported. The diagnostic accuracy of the risk score was compared with those of CURB-65, CRB-65 and PSI/PORT scores. Kaplan-Meier survival curves stratified by the outcomes of the four clinical scores were also constructed, using the log-rank test to evaluate statistically significant differences.

Concerning secondary outcomes, the accuracy of the risk score in predicting the occurrence of ARDS, SIRS and acute kidney injury was estimated by performing ROC analysis and calculating the respective AUCs. In addition, the potential correlation of the risk score with the worst SOFA, APACHE II and MEWS scores was assessed by the Spearman rank correlation test.

A total of 67 patients were included in the present study. Fifteen of them were transferred to the ICU, needing intubation and mechanical ventilation, while three of them subsequently died.

Their demographic and clinical characteristics are summarised in 

The outcomes of laboratory tests are presented in Table 2 (1) Table 3) . As a result, a risk score was calculated for each patient (median: 1.29, interquartile range: 0.94 to 1.61). The C-index of the model was estimated to be 0.856 (standard error: 0.037). The risk score was significantly higher in patients that were subsequently admitted to ICU (P value < .0001), as well as to those developing ARDS (P value < .0001), SIRS (P value < .0001) and AKI (P value < .0001).

Subgrouping indicated that high-risk score was significantly associated with ICU admission both in patients younger and older than 65 years (P value < .001), as well as in patients presenting both before and after the first week of symptoms (P value < .001). In addition, higher risk score was linked to ICU transfer both in patients with PaO 2 /FiO 2 ≤ 200 mm Hg (P value < .05) and PaO 2 /FiO 2 of 200-300 mm Hg (P value < .01) at presentation, although no significant difference was observed for patients with initial PaO 2 /FiO 2 > 300, as only one patient was subsequently admitted to ICU in this subgroup ( Figure 2 ). The outcomes of the multivariate Cox regression model demonstrated that the association of risk score with disease progression remained significant after adjustment for age, day of symptom and PaO 2 /FiO 2 ratio at admission (adjusted hazard ratio: 23.14, 95% confidence intervals: 2.43 to 220.37, P value: .006) ( Table 4 ).

The AUC of the risk score for the prediction of ICU admission was calculated to be 97.1%, while it was estimated to provide a sensitivity of 92.3% and specificity of 93.3% at the threshold of 1.6. The diagnostic accuracy of the risk score was calculated to be MEWS score (ρ = 0.58, P value < .001) (Figures S3-S5) . 

The present prospective study included a total of 67 patients, The findings of the present study are in accordance with other prediction models concerning the negative prognostic value of male gender, hypertension, elevated glucose, C-reactive protein and lactate dehydrogenase. [27] [28] [29] Importantly, raised lactate dehydrogenase has been also recognised as a severity marker in different treatment options could also not be assessed since the majority of patients initially received similar therapeutic regimens, consisting mainly of azithromycin and hydroxychloroquine. Moreover, the present study adopted a prospective design with pre-specified variables and end-points, minimising thus the risk of selection bias. 

This study was carried out as part of our routine work

https://orcid.org/0000-0001-5088-5458

A Novel coronavirus emerging in China-key questions for impact assessment

COVID-19 pathophysiology: a review

COVID-19: towards understanding of pathogenesis

COVID-19 update: Covid-19-associated coagulopathy

Risk factors of severe disease and efficacy of treatment in patients infected with COVID-19: a systematic review, meta-analysis and meta-regression analysis

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement

Baseline neutrophil-to-lymphocyte ratio (NLR) and derived NLR could predict overall survival in patients with advanced melanoma treated with nivolumab

Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis

A prediction rule to identify lowrisk patients with community-acquired pneumonia

Acute respiratory distress syndrome

Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis

KDIGO clinical practice guidelines for acute kidney injury

The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine

APACHE II: a severity of disease classification system

Validation of a modified early warning score in medical admissions

A Package for Survival Analysis in R. 2020. https:// cran.r-proje ct.org/packa ge=survival

Regularization paths for generalized linear models via coordinate descent

pROC: an open-source package for R and S+ to analyze and compare ROC curves

Descriptive statistics and normality tests for statistical data

User's guide to correlation coefficients

Introduction to machine learning: k-nearest neighbors

Data integration by multi-tuning parameter elastic net regression

Index for rating diagnostic tests

Co-infections in people with COVID-19: a systematic review and meta-analysis

Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score

A novel risk score to predict diagnosis with Coronavirus Disease 2019 (COVID-19) in suspected patients: a retrospective, multi-center, observational study

A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: an observational cohort study

Hospitalized adult patients with 2009 influenza A(H1N1) in Beijing, China: risk factors for hospital mortality

Epidemiological, demographic, and clinical characteristics of 47 cases of Middle East respiratory syndrome coronavirus disease from Saudi Arabia: a descriptive study

The procoagulant pattern of patients with COVID-19 acute respiratory distress syndrome

Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients With COVID-19

PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration

Regularization methods for fitting linear models with small sample sizes: fitting the Lasso estimator using R

Quantitative detection and viral load analysis of SARS-CoV-2 in infected patients

A precision medicine approach to SARS-CoV-2 pandemic management

Development of a novel risk score for the prediction of critical illness amongst COVID-19 patients