key: cord-0837802-crybusq3
authors: Yang, Yun; Zhu, Xiao-Fei; Huang, Jian; Chen, Cui; Zheng, Yang; He, Wei; Zhao, Ling-Hao; Gao, Qian; Huang, Xuan-Xuan; Fu, Li-Juan; Zhang, Yu; Chang, Yan-Qin; Zhang, Huo-Jun; Lu, Zhi-Jie
title: Nomogram for prediction of fatal outcome in patients with severe COVID-19: a multicenter study
date: 2021-03-17
journal: Mil Med Res
DOI: 10.1186/s40779-021-00315-6
sha: 4d283c5426fd838399eb3c02c07372a202fadc67
doc_id: 837802
cord_uid: crybusq3

BACKGROUND: To develop an effective model of predicting fatal outcomes in the severe coronavirus disease 2019 (COVID-19) patients. METHODS: Between February 20, 2020 and April 4, 2020, consecutive confirmed 2541 COVID-19 patients from three designated hospitals were enrolled in this study. All patients received chest computed tomography (CT) and serological examinations at admission. Laboratory tests included routine blood tests, liver function, renal function, coagulation profile, C-reactive protein (CRP), procalcitonin (PCT), interleukin-6 (IL-6), and arterial blood gas. The SaO(2) was measured using pulse oxygen saturation in room air at resting status. Independent high-risk factors associated with death were analyzed using Cox proportional hazard model. A prognostic nomogram was constructed to predict the survival of severe COVID-19 patients. RESULTS: There were 124 severe patients in the training cohort, and there were 71 and 76 severe patients in the two independent validation cohorts, respectively. Multivariate Cox analysis indicated that age ≥ 70 years (HR = 1.184, 95% CI 1.061–1.321), panting (breathing rate ≥ 30/min) (HR = 3.300, 95% CI 2.509–6.286), lymphocyte count < 1.0 × 10(9)/L (HR = 2.283, 95% CI 1.779–3.267), and interleukin-6 (IL-6) > 10 pg/ml (HR = 3.029, 95% CI 1.567–7.116) were independent high-risk factors associated with fatal outcome. We developed the nomogram for identifying survival of severe COVID-19 patients in the training cohort (AUC = 0.900, 95% CI 0.841–0.960, sensitivity 95.5%, specificity 77.5%); in validation cohort 1 (AUC = 0.811, 95% CI 0.763–0.961, sensitivity 77.3%, specificity 73.5%); in validation cohort 2 (AUC = 0.862, 95% CI 0.698–0.924, sensitivity 92.9%, specificity 64.5%). The calibration curve for probability of death indicated a good consistence between prediction by the nomogram and the actual observation. The prognosis of severe COVID-19 patients with high levels of IL-6 receiving tocilizumab were better than that of those patients without tocilizumab both in the training and validation cohorts, but without difference (P = 0.105 for training cohort, P = 0.133 for validation cohort 1, and P = 0.210 for validation cohort 2). CONCLUSIONS: This nomogram could help clinicians to identify severe patients who have high risk of death, and to develop more appropriate treatment strategies to reduce the mortality of severe patients. Tocilizumab may improve the prognosis of severe COVID-19 patients with high levels of IL-6. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40779-021-00315-6.

Previous studies have indicated that in all coronavirus disease 2019 (COVID-19) patients, the incidence of severe cases is approximately 15% [1, 2] . The mortality rate of severe COVID-19 patients varies from 8.0 to 61.5% and significantly increases among older patients [3] [4] [5] [6] [7] [8] . Early medical intervention is very important to reduce the mortality of severe patients. Thus, it is of great importance to screen out severe patients with a high risk of death promptly and accurately at the initial admission [9] . However, this is particularly difficult because of limited medical resources and staff and the large number of patients. Therefore, elucidating the independent risk factors and establishing an accurate model for predicting severe COVID-19 patients at high risk of death is necessary.

A previously established nomogram suggested five prognostic factors for predicting the outcome: Acute Physiology and Chronic Health Evaluation II (APACHE II), creatine kinase (CK), C-reactive protein (CRP), immunoglobulin A (IgA), and the interaction between CK and APACHE II [10] . However, risk factors associated with fatal outcomes in severe patients are unclear. This study aimed to provide a model to help clinicians identify patients with severe COVID-19 at high risk of death, which may be beneficial for decision making regarding treatment strategies.

Between February 20, 2020 and April 4, 2020, consecutive confirmed COVID-19 patients were assessed to enter into this study from three designated hospitals of COVID-19 in China: the Guanggu Branch of the Women and Children's Hospital of Hubei Province, Tongji Taikang Hospital and Huoshen Mountain Hospital. The diagnosis of COVID-19 was based on the World Health Organization (WHO) interim guidance and guidelines for diagnosis and treatment of novel coronavirus pneumonia (5 th version) released by the National Health Commission of China [11, 12] . The presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in respiratory specimens was confirmed by a positive result of quantitative real-time reverse transcriptase-polymerase chain reaction (qRT-PCR) assay from nasal or pharyngeal swab specimens. Swab samples were collected and tested for SARS-CoV-2 with the Chinese Center for Disease Control and Prevention (CDC) recommended Kit (BioGerm, Shanghai, China), following WHO guidelines for qRT-PCR [13] [14] [15] . All samples were tested for SARS-CoV-2 by use of qRT-PCR with the CDC recommended Kit. The test results were confirmed by nested RT-PCR with designed primers. The nested RT-PCR assay was performed according to the previous report [16] .

Severe COVID-19 group was defined if meeting at least one of the following criteria: (1) Shortness of breath, breathing rate ≥ 30/min, (2) Arterial oxygen saturation (SaO 2 , resting status) ≤ 93%, or (3) the ratio of partial pressure of arterial oxygen (PaO 2 ) to fraction of inspired oxygen (FiO 2 ) ≤ 300 mmHg.

During the study period, a total of 2541 patients were enrolled into this study. Of 2541 cases included in this study, 1056 patients were from the Guanggu Branch of the Women and Children's Hospital of Hubei Province, 726 were from Tongji Taikang Hospital, and 759 were from Huoshen Mountain Hospital. According to the definition of severe COVID-19 described above, there were 124, 71, and 76 severe cases from the three hospitals, respectively. Therefore, 124 patients with severe diseases from the Guanggu Branch of the Women and Children's Hospital of Hubei Province were included in the training cohort, while 71 and 76 severe cases in other two hospitals formed validation cohort. The selection of the study population was shown in Supplemental Fig. 1 . The study was approved by the Ethics Committee of all centers. Written informed consent was waived by the Ethics Commission of each hospital for emerging infections.

All patients received chest computed tomography (CT) and serological examinations at admission. Laboratory tests included routine blood tests, liver function, renal function, coagulation profile, CRP, procalcitonin (PCT), interleukin-6 (IL-6), and arterial blood gas. The SaO 2 was measured using pulse oxygen saturation in room air at resting status. Comorbidity was defined as having at least one of the followings: Hypertension, diabetes, cardiovascular disease, cerebrovascular disease, chronic lung disease, and malignant tumor for at least 6 months.

All data were collected, re-checked for accuracy independently by at least two researchers.

Continuous variables were expressed as mean (SD). Categorical variables were expressed as frequency (percentage). Categorical variables were compared by the χ 2 test or Fisher's exact test. Continuous variables were compared by the Student's t test or Mann-Whitney U test. Survival curves were analyzed using the Kaplan-Meier method. Differences between curves were assessed using the Log-rank test.

Univariate and multivariate Cox proportional regression analysis was used for investigating the independent risk factors of death. The independent risk factors associated with the risk of mortality of patients with severe COVID-19 were used to build the nomogram in the training cohort. The performance and accuracy of the established nomogram were assessed by receiver operating characteristic (ROC) curve and calibration with 1000 bootstrap samples. The area under ROC (AUC) and optimal cut-off values were determined. Decision curve analysis (DCA) based on the net benefit was depicted by the package of Rmda in R. The nomogram was validated in the validation cohorts 1 and 2, respectively. The nomogram was constructed and evaluated using the R software version 3.4.1 package with the Rms and Hmisc. All statistical analysis was performed using R version 3.4.1, a P < 0.05 in two-tailed was the significance threshold.

Of 2541 patients included in this study, 271 patients had severe COVID-19 and 2270 were non-severe cases. Supplemental Table 1 indicated the basic characteristics of the study cohorts. Compared with non-severe cases, patients were older (P < 0.001) and more males (P < 0.001) were found in the severe disease cohort. In addition, there is more comorbidity in severe cases (P < 0.001). The proportion of patients with panting was higher in the severe cases (P < 0.001). Higher levels of white blood cell (WBC) count, neutrophil count, CRP, PCT, IL-6, total bilirubin (TBIL), alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), γ-glutamyl transpeptidase (γ-GT), and creatinine (Cr) were identified in severe cases (P < 0.001), while the levels of lymphocyte count, platelet (PLT), and SaO 2 of patients with severe disease were lower than those in non-severe cases (P < 0.001). There were 186 patients with no manifestations of chest CT in non-severe COVID-19 cohort, and no significant differences of the findings of chest CT between severe and non-severe cases were found (P > 0.05). By the end of April 4, 2020, there were 58 and 39 patients died in the severe and non-severe disease group, respectively (Supplemental Table 1 ).

The basic characteristics of the severe patients are listed in Supplemental Table 2 . The median age of the patients was 68 (range 20-100) years. One hundred and fifty-two (56.1%) patients were males and there were one hundred nineteen (43.9%) female patients. There were 73 (26.9%) patients who had a high WBC count of > 10 × 10 9 /L, and 148 (54.6%) patients with lymphopenia defined as lymphocyte count of ≤1.0 × 10 9 /L. Ninety-five (35.1%) patients had a high neutrophil count of > 6.3 × 10 9 /L, while 26 (9.6%) patients had a low PLT count of < 100 × 10 9 /L. Ground-glass opacity and consolidation were found in 122 (45.0%) and 114 (42.1%) patients, respectively. In addition, twenty-eight (10.3%) and seven (2.6%) patients had thickened interlobular septa and nodular lesions, respectively in chest CT. Of 271 severe patients, 58 died during the study period (Supplemental Table 2 ).

Comparison of baseline characteristics between patients in training and validation cohorts can be seen in Table 1 . There were significant differences of age, proportion of smokers, incidence of panting, WBC, neutrophil count, CRP and SaO 2 at admission between the three cohorts (P < 0.05). There was no significant difference in the other variables between the three cohorts (P > 0.05). By the end of April 4, 2020, 22 severe COVID-19 patients died in the training group, and 22 and 14 patients died in the validation group 1 and validation group 2, respectively ( Table 1 ).

The baseline characteristics of patients in the training cohort were shown in Table 2 . There were no significant differences in gender, TBIL, ALT, AST, LDH, γ-GT, Cr, PLT, and the proportion of smokers between survivors and non-survivors (P > 0.05). Survivors were significantly younger than the non-survivors in the training cohort [(70.4 ± 12.3) years vs. (81.6 ± 7.3) years, P < 0.05], however, the proportion of patients with multiple comorbidities and panting (breathing rate ≥ 30/min) was significantly higher in non-survivors (P < 0.05). In addition, WBC and neutrophil count, CRP, D-dimer, PCT, and IL-6 were also significantly higher in nonsurvivors (P < 0.05). The lymphocyte count was significantly lower in non-survivors (P < 0.05).

All variables listed in Table 1 were analyzed by univariate and multivariate Cox regression analysis. Multivariate Cox analysis indicated that age ≥ 70 years (HR = 1.184, 95% CI 1.061-1.321), panting (breathing rate ≥ 30/min, HR = 3.300, 95% CI 2.509-6.286), lymphocyte count < 1.0 × 10 9 /L (HR = 2.283, 95% CI 1.779-3.267), and IL-6 > 10 pg/ml (HR = 3.029, 95% CI 1.567-7.116) were independent risk factors associated with fatal outcomes (Table 3) .

Survival analysis in the patients with the high level of IL-6

Due to the high level of IL-6 correlating with poor outcomes in severe COVID-19 patients, the therapeutic effect of tocilizumab in the patients with high IL-6 was further analyzed. In the training cohort, it was demonstrated that the prognosis of patients receiving tocilizumab was better than that of patients not receiving tocilizumab, but without significance (P = 0.105, Supplemental Fig. 2a) . Similar results were also observed in the validation cohort 1 and validation cohort 2, respectively (P = 0.133, P = 0.210, Supplemental Fig. 2b-c) .

Four independent risk factors found to be associated with the risk of mortality of patients in the multivariate analyses were incorporated into the nomogram (Fig. 1) . The ROC curve was employed to assess the predictive ability of the established nomogram, and the result demonstrated that the AUC was 0.900 (95% CI 0.841-0.960) in the training cohort, with a sensitivity of 95.5% and specificity of 77.5% (Fig. 2a) . Moreover, the calibration curves for nomogram predicted mortality indicated that a good consistency between observed actual outcomes and predicted ones in the training cohort (Fig. 3a) .

In the validation cohort 1, the AUC was 0.811 (95% CI 0.763-0.961) for patients with a sensitivity of 77.3% and specificity of 73.5% (Fig. 2b) . In the validation cohort 2, the AUC was 0.862 (95% CI 0.698-0.924) for patients with a sensitivity of 92.9% and specificity of 64.5% (Fig.  2c) . The calibration curves also showed good agreement between prediction and observation in the risk of mortality in the two validation cohorts (Fig. 3b-c) .

Clinical application of the nomogram DCA based on the net benefit and threshold probabilities was performed to assess the clinical applicability of the risk prediction nomogram. The DCA showed that our risk prediction nomogram had a superior net benefit with a wide range of threshold probabilities in the training cohort and validation cohorts (Fig. 4 ).

Our study revealed the clinical characteristics and risk factors for fatal outcomes in confirmed severe COVID-19 patients based on multicenter cohorts. Multivariate Cox analysis in this study indicated that age, lymphopenia, respiratory rate ≥ 30/min, and IL-6 were independent high-risk factors associated with poor prognosis. Older age has been proven to be a risk factor for the virus infection and survival in many previous studies [17] [18] [19] [20] . Elderly patients with severe COVID-19 were more likely to develop fatal outcomes because of the rapid progression of the disease, which reminded us of providing early intervention for elderly severe patients. Similarly, lymphopenia was more common in the nonsurvivors and severe cases according to previous reports, suggesting dysregulation of the immune response in patients with COVID-19 [21] [22] [23] . Nevertheless, most of these were only descriptive studies. A study clarified that lower lymphocyte count was predictive of COVID-19 progression [24] , whereas the impact of lymphocyte count on the survival of severe COVID-19 was unclear. This study demonstrated that a lymphocyte count < 1.0 × 10 9 /L was independently associated with death in severe cases.

Recent studies have found that the cytokine storm is an important factor leading to rapid disease progression and poor prognosis [25, 26] . IL-6 is one of the major cytokines involved in cytokine storms [27, 28] . A previous univariate analysis showed that the IL-6 level was associated with worse survival without significance [18] .

Our study showed that a high level of IL-6 was a predictor of death in severe COVID-19 patients. Additionally, the survival curve showed that the outcome was better in patients with tocilizumab than in patients without tocilizumab in the training cohort. Nonetheless, no significant difference was found. Similar results were also found in validation cohort 1 and validation cohort 2.

The reason for this result may be attributable to the small sample size. Although the previous study has shown that tocilizumab can reduce mortality, highquality studies are still needed to verify the effectiveness of tocilizumab on the survival of patients with severe COVID-19 [29] . Increasing respiratory rate is an important clinical feature of acute respiratory distress syndrome (ARDS), which is a major cause of death in severe COVID-19 patients [8, 17, 30] . A previous study showed that a respiratory rate ≥ 24/min was a risk factor for death in the univariate analysis [18] , whereas no significance was found after the multivariate regression analysis. Our multivariate regression analysis clarified that a respiratory rate ≥ 30/min was a predictor of death. For patients with increasing respiratory rates, especially those with respiratory rate ≥ 30/min, it is necessary for physicians to be aware of the potential progression of ARDS.

Many previous studies have shown that comorbidity was significantly associated with high mortality rate and disease progression [24, 31] . Nevertheless, in this study, the significance of comorbidity was only indicated in the univariate analysis, but not in the multivariate regression analysis, which may be ascribed to different patients enrolled in these studies. All the patients included in this study had severe COVID-19, and the proportion of comorbidities was approximately 70%, which was significantly higher than those in other studies.

Previous studies have shown several prediction models with different parameters [1, 24, 31, 32] . Compared with other studies, the predictive effect of age, panting, and lymphopenia has been described in previous reports, while the main feature of this study is the analysis of the prognostic value of IL-6 in severe COVID-19 patients for the first time. IL-6 plays an important role in the pathophysiological changes of severe cases. However, the prediction value of IL-6 was not shown in other prediction models. In addition, our study also indicated that tocilizumab, a monoclonal antibody against the IL-6 receptor, may improve the prognosis of severe COVID-19 patients with the high level of IL-6.

Our study has some limitations. First, this is a retrospective study, and there may be potential biases in the selection of patients. Second, the sample size of the study was relatively small, and the results need to be further validated in a larger cohort.

This study firstly developed a nomogram for predicting fatal outcomes in the severe COVID-19 patients. The four predictors included in the model are easy to obtain. The prediction risk of the model indicated a good consistency with the observed one. Hence, this nomogram may be conducive to more effective treatment to reduce the mortality of those severe cases at high risk of death. 

The online version contains supplementary material available at https://doi. org/10.1186/s40779-021-00315-6.

Additional file 1: Supplemental Figure 1 Selection of the study population.

Additional file 2: Supplemental Figure 2 Survival curves of severe COVID-19 patients with the high level of IL-6 receiving tocilizumab and not receiving tocilizumab treatment. a The survival curve of severe COVID-19 patients with the high level of IL-6 receiving tocilizumab and not receiving tocilizumab treatment in the training cohort (P = 0.105). b The survival curve of severe COVID-19 patients with the high level of IL-6 receiving tocilizumab and not receiving tocilizumab treatment in the validation cohort 1 (P = 0.133). c The survival curve of severe COVID-19 patients with the high level of IL-6 receiving tocilizumab and not receiving tocilizumab treatment in the validation cohort 2 (P = 0.210). Table 1 Comparison of Baseline characteristics between severe patients and non-severe patients n(%). WBC. White blood cell; PLT. Platelet; CRP. C-reactive protein; PCT. Procalcitonin; IL-6. Interleukin-6; SaO 2 . Oxygen saturation; TBIL. Total bilirubin; ALT. Alanine aminotransferase; AST. Aspartate aminotransferase; LDH. Lactate dehydrogenase; γ-GT. γ-glutamyl transpeptadase; Cr. Creatinine; CT. Computed tomography. Supplemental Table 2 Baseline characteristics of the patients [n(%)]. a Ages are shown as median (range). WBC. White blood cell; PLT. Platelet; CRP. C-reactive protein; PCT. Procalcitonin; IL-6. Interleukin-6; SaO 2 . Oxygen saturation; TBIL. Total bilirubin; ALT. Alanine aminotransferase; AST. Aspartate aminotransferase; LDH. Lactate dehydrogenase; γ-GT. γ-glutamyl transpeptadase; Cr. Creatinine; CT. Computed tomography

A tool to early predict severe corona virus disease 2019 (COVID-19): a multicenter study using the risk nomogram in Wuhan and Guangdong, China

Clinical characteristics of 2019 novel coronavirus infection in China

Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan

Efficacy of tocilizumab in patients with COVID-19 ARDS undergoing noninvasive ventilation

Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

COVID-19 in critically ill patients in the Seattle region -case series

Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study

Analysis of clinical characteristics and laboratory findings of 95 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a retrospective analysis

A predictive nomogram for predicting improved clinical outcome probability in patients with COVID-19 in Zhejiang Province

Clinical management of severe acute respiratory infection when novel coronavirus (nCoV) infection is suspected: Interim guidance

National Health Commission (NHC) of the PRC and National Administration of Traditional Chinese Medicine of the PRC. Guidance for corona virus disease 2019: Prevention, control, diagnosis and management. China: Peoples Medical Publishing House

Clinical management of severe acute respiratory infection (SARI) when COVID-19 disease is suspected: Interim guidance

World Health Organization. Laboratory testing for 2019 novel coronavirus (2019-nCoV) in suspected human cases: Interim guidance

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Clinical characteristics and intrauterine vertical transmission potential of COVID-19 infection in nine pregnant women: a retrospective review of medical records

Clinical characteristics of 113 deceased patients with coronavirus disease 2019: retrospective study

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

The origin, transmission and clinical therapies on coronavirus disease 2019 (COVID-19) outbreak -an update on the status

Critical patients with coronavirus disease 2019: risk factors and outcome nomogram

Characteristics of peripheral lymphocyte subset alteration in COVID-19 pneumonia

Dysregulation of immune response in patients with COVID-19 in Wuhan, China

Hematological findings and complications of COVID-19

Prediction for progression risk in patients with COVID-19 pneumonia: the CALL score

The pathogenesis and treatment of the 'cytokine storm' in COVID-19

COVID-19: consider cytokine storm syndromes and immunosuppression

Cytokine release syndrome in severe COVID-19: Interleukin-6 receptor antagonist tocilizumab may be the key to reduce mortality

Rational use of tocilizumab in the treatment of novel coronavirus pneumonia

Chemoprophylaxis, diagnosis, treatments, and discharge management of COVID-19: an evidence-based clinical practice guideline (updated version)

Analysis of 92 deceased patients with COVID-19

Medical Treatment Expert Group for COVID-19, et al. Risk factors of fatal outcome in hospitalized subjects with coronavirus disease 2019 from a nationwide analysis in China

Development and validation of a risk factor-based system to predict short-term survival in adult hospitalized patients with COVID-19: a multicenter, retrospective, cohort study

We would like to thank the staff of the Department of Critical Care Medicine of Huoshen Mountain Hospital, Wuhan Hubei Province, who contributed to this study by collecting the required data in the hospital data system.

Authors' contributions YY, XFZ, JH, and CC designed this study; YZ, WH, LHZ, QG, XXH, LJF, YZ, and YQC collected the data; YY and JH were responsible for the statistical analysis; YY wrote the draft; XFZ, YQC, and HJZ revised this draft; ZJL finalized this manuscript. The authors read and approved the final manuscript.

This work was supported by grants from the National Natural Science Foundation of China (81502416) to Dr. Yun Yang.

The patient level data has been uploaded to the Figshare database. Dataset. https://doi.org/10.6084/m9.figshare.12866060.v1.

Ethics approval and consent to participate The study was approved by the Ethics Committee of all the study centers. Written informed consent was waived by the Ethics Commission of each hospital for emerging infectious.

No individual participant data is reported that would require consent to publish from the participant (or legal parent or guardian for children).

The authors declare that they have no competing interests.