key: cord-0927078-cyqhnfud
authors: Ena, J.; Segura-Heras, J.V.; Fonseca-Aizpuru, E.M.; López-Reboiro, M.L.; Gracia-Gutiérrez, A.; Martín-Oterino, J.A.; Martin-Urda Diez-Canseco, A.; Pérez-García, C.; Ramos-Rincón, J.M.; Gómez-Huelgas, R.
title: Derivation and validation of a risk score for admission to the Intensive Care Unit in patients with COVID-19()
date: 2021-09-14
journal: Rev Clin Esp (Barc)
DOI: 10.1016/j.rceng.2021.06.001
sha: 36936afc89a2a4a4aec7435161ad78d0b81e3ee5
doc_id: 927078
cord_uid: cyqhnfud

BACKGROUND: This work aims to identify and validate a risk scale for admission to intensive care units (ICU) in hospitalized patients with coronavirus disease 2019 (COVID-19). METHODS: We created a derivation rule and a validation rule for ICU admission using data from a national registry of a cohort of patients with confirmed SARS-CoV-2 infection who were admitted between March and August 2020 (N = 16,298). We analyzed the available demographic, clinical, radiological, and laboratory variables recorded at hospital admission. We evaluated the performance of the risk score by estimating the area under the receiver operating characteristic curve (AUROC). Using the β coefficients of the regression model, we developed a score (0 to 100 points) associated with ICU admission. RESULTS: The mean age of the patients was 67 years; 57% were men. A total of 1,420 (8.7%) patients were admitted to the ICU. The variables independently associated with ICU admission were age, dyspnea, Charlson Comorbidity Index score, neutrophil-to-lymphocyte ratio, lactate dehydrogenase levels, and presence of diffuse infiltrates on a chest X-ray. The model showed an AUROC of 0.780 (CI: 0.763-0.797) in the derivation cohort and an AUROC of 0.734 (CI: 0.708-0.761) in the validation cohort. A score of greater than 75 points was associated with a more than 30% probability of ICU admission while a score of less than 50 points reduced the likelihood of ICU admission to 15%. CONCLUSION: A simple prediction score was a useful tool for forecasting the probability of ICU admission with a high degree of precision.

Fundamento: Identificar y validar una escala de riesgo de ingreso en las unidades de cuidados intensivos (UCI) en pacientes hospitalizados con enfermedad por coronavirus 2019 (COVID-19).

Métodos: Realizamos una regla de derivación y otra de validación para ingreso en UCI utilizando los datos de un registro nacional de cohortes de pacientes con infección confirmada por SARS-CoV-2 ingresados entre marzo y agosto del año 2020 (N= 16.298 ).

Analizamos variables demográficas, clínicas, radiológicas y de laboratorio disponibles en el ingreso hospitalario. Evaluamos el rendimiento de la escala de riesgo mediante estimación del área bajo la curva de característica operativa del receptor (AROC).

Utilizamos los coeficientes β del modelo de regresión para elaborar una puntuación (0 a 100 puntos) asociada con ingreso en UCI.

Resultados: La edad media de los pacientes fue 67 años; 57% varones. Un total de 1.420 (8,7%) pacientes ingresaron en la UCI. Las variables independientes asociadas con el ingreso en UCI fueron: edad, disnea, índice de comorbilidad de Charlson, cociente neutrófilos-linfocitos, lactato deshidrogenasa e infiltrados difusos en la radiografia de tórax. El modelo mostró un AROC de 0,780 (IC: 0,763-0,797) en la cohorte de derivación y un AROC de 0,734 (IC: 0,708-0,761) en la cohorte de validación. Una puntuación > 75 se asoció con una probabilidad de ingreso en UCI superior a un 30%, mientras que una puntuación < 50 redujo la probabilidad de ingreso en UCI al 15%.

Conclusión: Una puntuación de predicción simple proporcionó una herramienta útil para predecir la probabilidad de ingreso en la UCI con un alto grado de precisión. 

As of April 16, 2021 , the number of COVID-19 cases caused by SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) in Spain was 3,407,283. Of them, 340,130 (9.98%) required hospital admission and 31,054 (0.91%) were admitted to intensive care units (ICU). 1 The clinical spectrum of SARS-CoV-2 infection varies from mild symptoms of fever and cough followed by expectoration and fatigue to severe symptoms in critical patients, including sepsis, coagulopathy, respiratory failure, and onset of severe acute respiratory distress syndrome. 2 Cumulative mortality in Spain as of April 16, 2021 was 76,981 (2.25%) patients, but among patients admitted to the ICU, mortality was 31%. 3 Various studies concur that the risk factors associated with mortality include advanced age, presence of kidney failure, low oxygen saturation levels, and high C-reactive protein values. 4, 5 Early identification of patients who are going to need critical care may be important for providing more appropriate treatment and optimizing available resources.

The purpose of this study is to use an extensive national registry of patients with COVID-19 in order to develop a risk scale for ICU admission applicable to patients who have just been admitted to the hospital.

The data source was the SEMI-COVID-19 registry, an ongoing retrospective cohort that includes the majority of patients who were discharged or died following hospitalization for confirmed COVID-19 in 150 Spanish hospitals from March 1, 2020 until September 1, 2020. 6 The sample comprised 16,298 patients.

The outcome evaluated was admission to the ICU, measured from the time of admission to the hospital.

P r e d i c t o r v a r i a b l e s To develop the prediction rule, we used variables that were routinely available at the time of hospital admission and which have been associated with ICU admission in other studies. These variables included: (1) demographics, age, and sex; (2) comorbidities and Charlson Comorbidity Index; (3) signs or symptoms, including dyspnea, confusion, hemoptysis; (4) laboratory data, oxygen saturation in ambient air, LDH levels, the neutrophil-lymphocyte ratio, and C-reactive protein levels.

In the derivation model, we randomly selected two-thirds (n = 10,865) of the sample. To create the derivation model, we conducted a multiple logistic regression analysis with ICU admission as the primary outcome and the previously described clinical parameters as predictor variables. Using the  coefficients, we created a scoring system that divided patients into different risk categories. S t a t i s t i c a l a n a l y s i s We followed the recommendations of the TRIPOD guidelines in developing the multivariate analysis model. 7 Continuous variables are expressed as means and standard deviations. Categorical variables are expressed as frequencies and percentages. The chisquare test was used to compare categorical variables and Student's t test was used to compare continuous variables between groups. Values of p<.05 were considered statistically significant. A multiple logistic regression analysis was conducted in the derivation sample to estimate probability of ICU admission.

To build the derivation model, variables with a p value lower than .10 on the univariate model were selected. The odds ratio (OR) and 95% confidence intervals (95% CI) were estimated based on the regression coefficients. Various regression models were built and the one that was simplest and most explicative based on the Hosmer-Lemeshow goodness of fit test was selected for application to the validation cohort. Nagelkerke's R 2 was used to estimate the proportion of variation explained by the model.

The model's final performance was evaluated by means of the receiver operating characteristic (ROC) curve and calculation of the area under the ROC curve. Finally, based on the  coefficients, we created a scoring system for establishing different risk levels for ICU admission.

The statistical analysis was conducted using the free R software, version 4.0.2 (Free Software Foundation, Inc. Boston, MA). The SEMI-COVID-19 Registry was approved by the Research Ethics Committee of the Province of Málaga. The Department of Medicines for Human Use of the Spanish Agency of Medicines and Health Products (AEMPS, for its initials in Spanish), in accordance with the applicable precepts, has designated the study to be a "non-postauthorization observational study."

A total of 1,420 (8.7%) patients were admitted to the ICU (table 1 table 1) . Patients who were admitted to the ICU were younger (mean age 63.3 ± 12.3 years) than those who were admitted to conventional hospitalization wards (67.7 ± 16.4 years) (p<.001). Male sex was associated with greater risk of ICU admission (69.5% vs. 56.3%; p<.001).

There were comorbidities that were associated with greater risk of ICU admission, including obesity, among others, whereas others were associated with a lower frequency of ICU admission, such as chronic kidney disease, COPD, or chronic heart failure.

Smoking was a risk factor for ICU admission. A greater proportion of patients who were admitted to the ICU presented with dyspnea at the time of admission (75.4% vs. 56%, p<.001) as well as a lower rate of altered consciousness (8.4% vs. 12.2%, p<.001), a greater frequency of presence of diffuse infiltrates on a chest x-ray on admission than patients who were admitted to the ICU (43% vs. 29.3% p<.001), and lower oxygen saturation levels (SpO2) than patients admitted to conventional hospitalization wards (89% vs. 93.3%, p<.001).

Regarding laboratory variables, patients hospitalized in the ICU had a greater neutrophil-lymphocyte ratio (9.7% vs. 7.0%, p<.001) and higher D-dimer (2, A total of 568 (40%) of patients admitted to the ICU died whereas those among those admitted to conventional hospitalization wards, there were 2,851 (19.2%) deaths.

To create the predictive model, we randomly divided the sample into two parts: twothirds were the derivation cohort and the remaining one-third was the validation cohort. The characteristics of patients in both cohorts are described in table 2 table 2. In the derivation cohort, variables that had a greater association with ICU admission on the univariate analysis were age younger than 75 years, a lower Charlson Comorbidity Index, presence of dyspnea or tachypnea, a neutrophil-lymphocyte ratio greater than five, lactate dehydrogenase values greater than 250 U/L, urea values greater than 40 mg/dL, and presence of diffuse infiltrates on the chest x-ray. The presence of cancer (solid tumor, metastases, hematologic tumor) and altered levels of consciousness were factors that reduced the risk of ICU admission (table 3 table 3) .

According to the multivariate analysis, the variables of age, presence of dyspnea, Charlson Comorbidity Index score, neutrophil-lymphocyte ratio, lactate dehydrogenase values, and chest x-ray data on admission remained in the model (table 4 table 4 ).

The p value for goodness of fit for the multivariate model, calculated using the Hosmer-Lemeshow statistic, was 0.154, indicating a good fit. Nagelkerke's R 2 value showing the model's explanatory value was 0.196. The evaluation of the predictive model's performance showed an area under the receiver operator characteristic curve (AUROC) of 0.780 (95% CI 0.763 -0.797) ( fig. 1 fig. 1 ). For a probability of ICU admission of J o u r n a l P r e -p r o o f 0.102 (cut-off point), the model showed a sensitivity of 67.8% and a specificity of 74.6% in the derivation sample. In the validation cohort, the AUROC was 0.734 and the model showed a sensitivity of 62.1% and a specificity of 72%.

We built a scoring system based on the  coefficients estimated in the logistic regression model (table 4 ). The total individual score (range from 0 to 100 points) was obtained by adding the points of each patient characteristic. This scoring system allows for estimating probability of ICU admission (positive predictive value) with the data obtained at the time of admission. For scores greater than 75 points, the probability of ICU admission is greater than 30% and for scores lower than 50 points, the probability of ICU admission is less than 15% ( fig. 2 fig. 2 ).

We developed and validated a clinical prediction model to identify patients with SARS-CoV-2 infection who will require ICU admission. The predictive model's performance showed an AUC of 0.780 for a probability of ICU admission of 0.102.

Risk of ICU admission increased progressively according to the number of independent variables, which included age younger than 75 years, presence of dyspnea, Charlson Comorbidity Index score of less than 3, a neutrophil-lymphocyte ratio greater than 5, lactate dehydrogenase values greater than 250 U/L, and presence of diffuse infiltrates on the chest x-ray. Using these variables, which are easily available at the time of the patient's hospital admission, we built a scoring system that allows for determining the probability of ICU admission. This study allows for anticipating the necessary intensive care resources for patients hospitalized due to COVID-19.

The percentage of patients admitted to the ICU in the derivation cohort was 8.9% and in the validation cohort it was 7.7%. These values are similar to what has been reported in other studies conducted in Spain. 8 The proportion of hospitalized patients admitted to the ICU in other countries was 16.8% in Italy, 17% in the United Kingdom, 11% in France, 32% in the USA, and 8.2% in China. 9-13 A higher percentage of patients admitted to the ICU applied to our model would entail an increase in the positive predictive value of the scoring system.

The demographic characteristics of the study population also show differences according to country, with a mean age of around 70 years for studies conducted in Europe and a mean age around 50 years for studies conducted in China. All studies showed the determining factors for ICU admission to be older age and male sex, greater comorbidity, and presence of dyspnea. Among the laboratory data, signs of poor prognosis that have been found in all studies include a greater neutrophil-lymphocyte ratio and a greater increase in lactate dehydrogenase levels. [9] [10] [11] [12] [13] Likewise, a diffuse, bilateral pattern with ground glass opacity on the chest x-ray has also been associated with greater risk of severe disease. 14 The performance of our predictive model for ICU admission is in line with data found by other authors, who report AUROC values of 0.74 to 0.88 in studies with derivation and validation models. 12, 13 J o u r n a l P r e -p r o o f Nevertheless, we must be cautious when extrapolating the performance of models when applying them to other countries, given that they can have differences that, in some instances, are notable. 15 These differences can be due to the demographic characteristics of the subjects studied, the incidence of cases at one particular moment in the pandemic, or the structure of the healthcare system, among others.

Among the strengths of this study are the large sample size, which allowed for considering a large number of variables in the predictive model, as well as a robust analysis of the derivation and validation models.

It is advisable to interpret this study as an aid for determining intensive care unit bed requirements for patients with severe COVID-19. Although the risk score refers to clinical deterioration related to COVID-19, the benefit of ICU admission may be conditioned by other aspects. The patient's comorbidity, presence of advanced cognitive decline, or inability to perform basic activities of daily living can constitute limitations for ICU admission in subjects with severe COVID-19. 16 On the other hand, the probability of ICU admission may be influenced by the percentage of beds occupied during different periods of the pandemic. It is possible that during periods when there is good access to supportive treatment, it is decided to provide this treatment to patients with moderate risk of mortality. On the contrary, in periods or areas with more limited resources, the decision could be made to limit intensive supportive treatment in patients with moderate risk in order to optimize available resources. Likewise, the admission of patients with severe COVID-19 in respiratory intensive care units was not taken into consideration; the availability of these facilities could alter the scale validation results.

Finally, the development and validation of the predictive model was carried out in Spain. This could limit the generalization of the scoring system to other areas of the world with different healthcare systems.

In conclusion, we developed a clinical prediction rule for ICU admission for patients with COVID-19. The predictor variables are easy to obtain at the time of admission. Our prediction rule has been validated in a large cohort of patients and has been demonstrated to be reproducible. The incorporation of the rule into the electronic medical record would facilitate its implementation and help clinicians appropriately adapt the patient admissions department. 

The authors declare that they have no conflicts of interest.

J o u r n a l P r e -p r o o f Figure 1 Receiver operating characteristic curve for ICU admission in the derivation and validation cohort. The area under the receiver operating characteristic curve was 0.781 in the derivation cohort and 0.747 in the validation cohort. The cut-off point which optimizes sensitivity and specificity was 0.087 in both the derivation and validation cohort. gr1 Figura 1. Curva de característica operativa del receptor para ingreso en UCI en la cohorte de derivación y validación. El área bajo la curva de característica operativa del receptor fue 0,781 en la cohorte de derivación y 0,747 en la cohorte de validación. El punto de corte que optimiza sensibilidad y especificidad fue 0,087, tanto en la cohorte de derivación como en la de validación.

English Sensibilidad Sensitivity 1-Especificidad 1-Specificity Cohorte derivación Derivation Cohort Cohorte Validación Validation Cohort J o u r n a l P r e -p r o o f Figure 2 Relationship between individual score and probability (positive predictive value) that a patient will be admitted to the ICU. gr2 Figura 2. Relación entre la puntuación individual y la probabilidad (valor predictivo positivo) de que un paciente ingrese en UCI.

Positive predictive value (%) Puntos asignados Points assigned J o u r n a l P r e -p r o o f 

Actualización nº 350

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Patient characteristics, clinical course and factors associated to ICU mortality in critically ill patients infected with SARS-CoV-2

Characteristics and predictors of death among 4035 consecutively hospitalized patients with COVID-19 in Spain

Mortality risk assessment in Spain and Italy, insights of the HOPE COVID-19 registry

Clinical characteristics of patients hospitalized with COVID-19 in Spain: Results from the SEMI-COVID-19 Registry

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement

COVID-19 in hospitalised patients in Spain: a cohort study in Madrid

Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores

Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study

Development of a multivariate prediction model of intensive care unit transfer or death: A French prospective cohort study of hospitalized COVID-19 patients

Prediction model and risk scores of ICU admission and mortality in COVID-19

Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19

Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study

The COVID-GRAM Tool for Patients Hospitalized With COVID-19 in Europe

The contribution of frailty, cognition, activity of daily life and comorbidities on outcome in acutely admitted patients over 80 years in European ICUs: the VIP2 study

We would like to thank the investigators who participated in data collection for the SEMI-COVID-19 Registry.