key: cord-0977164-nifu7dxl authors: Zheng, Bowen; Cai, Yong; Zeng, Fengxia; Lin, Min; Zheng, Jun; Chen, Weiguo; Qin, Genggeng; Guo, Yi title: An Interpretable Model-Based Prediction of Severity and Crucial Factors in Patients with COVID-19 date: 2021-03-01 journal: Biomed Res Int DOI: 10.1155/2021/8840835 sha: 3f7df51a0cdcb3e80c61bc1e5b55d41501414583 doc_id: 977164 cord_uid: nifu7dxl This study established an interpretable machine learning model to predict the severity of coronavirus disease 2019 (COVID-19) and output the most crucial deterioration factors. Clinical information, laboratory tests, and chest computed tomography (CT) scans at admission were collected. Two experienced radiologists reviewed the scans for the patterns, distribution, and CT scores of lung abnormalities. Six machine learning models were established to predict the severity of COVID-19. After parameter tuning and performance comparison, the optimal model was explained using Shapley Additive explanations to output the crucial factors. This study enrolled and classified 198 patients into mild (n = 162; 46.93 ± 14.49 years old) and severe (n = 36; 60.97 ± 15.91 years old) groups. The severe group had a higher temperature (37.42 ± 0.99°C vs. 36.75 ± 0.66°C), CT score at admission, neutrophil count, and neutrophil-to-lymphocyte ratio than the mild group. The XGBoost model ranked first among all models, with an AUC, sensitivity, and specificity of 0.924, 90.91%, and 97.96%, respectively. The early stage of chest CT, total CT score of the percentage of lung involvement, and age were the top three contributors to the prediction of the deterioration of XGBoost. A higher total score on chest CT had a more significant impact on the prediction. In conclusion, the XGBoost model to predict the severity of COVID-19 achieved excellent performance and output the essential factors in the deterioration process, which may help with early clinical intervention, improve prognosis, and reduce mortality. Coronavirus disease 2019 (COVID- 19) , pneumonia caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), is a highly infectious respiratory disease with a variable incubation period ranging from 1 to 14 days, and people are generally vulnerable to the virus. Reverse transcription-polymerase chain reaction (RT-PCR) for SARS-CoV-2 is the standard for diagnosing COVID-19. However, RT-PCR takes 1-2 days to complete and may report false-negative results. Some areas even faced a shortage of RT-PCR testing kits [1, 2] . Under these circumstances, chest computed tomography (CT) played a vital role in detecting and assessing patients with COVID-19, especially in detecting patients with COVID-19 in the early stage [3] . According to clinical presentation, patients with COVID-19 were classified into four categories: mild type, moderate type, severe type, and critical type [4] . Most patients were classified as the mild type and moderate type with mild symptoms, whereas a small group of patients may experience acute respiratory distress syndrome (ARDS), septic shock, coagulation dysfunction, and multiple organ failure. These patients required ventilators and extracorporeal membrane oxygenation during an expensive treatment and had a high death rate [5] . Previous researchers showed that up to 5.0% of the patients were admitted to the intensive care unit (ICU), 2.3% of the patients needed invasive mechanical ventilation, and 1.4% of patients died eventually [6] . It is unclear why some patients develop into severe or critical cases, while others only get mild or no symptoms. The crucial factors in the deterioration process remain unknown. Early identification of severity and crucial factors are of great value, prompting early clinical intervention and preventing deterioration of patients' condition. However, it is hard for the doctor to identify those patients under the human limitation on information processing. Hence, artificial intelligence has been widely applied in the medical domain, enabling radiologists to make full use of data, including imaging information, and explore the images' biological nature. Since the initial outbreak, attempts have been made to detect COVID-19 using chest CT. In this study, we established a machine learning model, combining clinical information, laboratory tests, and chest CT features for early prediction of the severity and crucial factors of patients with COVID-19. Our model may help identify patients who require early clinical intervention to improve prognosis and reduce mortality. Participants. This retrospective study evaluated de-identified data and involved no potential risk to the patients. Therefore, the institutional review board waived the requirement of obtaining written informed consent. This study included patients with COVID-19, as confirmed by RT-PCR, admitted to the People's Hospital of Honghu and Honghu Xiaotangshan Hospital from January 1 to March 27, 2020. The inclusion criteria were as follows: (a) a positive RT-PCR result for SARS-CoV-2 infection, (b) patients who underwent a chest CT scan and laboratory tests at admission in the two hospitals mentioned above, and (c) no other viral infection or serious complication. The exclusion criteria were as follows: (a) patients who underwent a chest CT scan and laboratory tests in other hospitals and (b) patients whose chest CT images showed no lesion in the lungs. Patients' triage, sex, age, symptoms, pre-existing diseases, the temperature at admission, and laboratory tests, such as white blood cell (WBC), neutrophil, and lymphocyte counts, were collected. Patients with COVID-19 were classified into four categories [4] : (1) The mild type includes those who have mild clinical symptoms and no pneumonia manifestations found in imaging. (2) The moderate type includes the patients who have symptoms such as fever and respiratory tract symptoms with pneumonia manifestations seen on imaging. (3) The severe type fulfilled the following criteria: respiratory frequency ≥ 30/minute, blood oxygen saturation ≤ 93%, arterial partial pressure of oxygen (PaO 2 )/oxygen concentration (FiO 2 ) ratio < 300, and lung infiltrates > 50% within 24-48 hours. (4) The critical type meets any of the following criteria: occurrence of respiratory failure requiring mechanical ventilation and the presence of shock and other organ failures that require monitoring and treatment in the ICU. In this study, all patients were classified into four clinical types according to the criteria mentioned above during treatment. The mild type was excluded because of no pneumonia manifestations found in imaging. The moderate type was classified into the mild group. Concerning the rareness of the critical type, the severe type and critical type were classified into the severe group in this study ( Figure 1 ). Chest CT scanning (Go Now, Siemens Healthcare, Germany; GE optima 680, GE Healthcare, USA) was performed at the end of full inspiration in the supine position. The images were acquired and reconstructed with 80-130 kV tube voltage and automatic tube current modulation (up to 400 mA). The slice thicknesses were 0.6 mm (GE optima CT680) and 1.5 mm (Go Now), respectively. The lung window setting was at a window level of -600 Hounsfield units (HU) and a window width of 1500 HU. The scanning range was from the apex to the lung base. 2.3. Image Interpretation. All chest CT images were reviewed by two radiologists with over five years of clinical experience in the respiratory system independently. Any disagreement was resolved by discussion and consensus. The following aspects were reviewed for each patient: (1) stage (early stage, progress stage, or restoration stage); (2) distribution (subpleural, scatter, or diffuse) and shape (nodular, patchy, or large patchy); (3) number of lung lobes involved; (4) presence of ground-glass opacity (GGO); (5) presence of consolidation, fibrotic lesions, reticular shadow, crazy paving pattern, air bronchogram, pleural effusion, pleural thickening, and mediastinal lymphadenopathy (axil diameter > 10 mm); and (6) CT scores of the percentage of lung involved [7, 8] . Each lobe was evaluated for the percentage involved on a scale of 0-4 (0: 0% involvement, 1: <25% involvement, 2: 25%-50% involvement, 3: 50%-75% involvement, and 4: ≥75% involvement). The total score on the chest CT was the summation of all five lobes. The maximum possible score was 20. Statistical analyses were performed using SPSS (version 26.0). Continuous variables are expressed as means and standard deviations and compared by an independent-sample t-test; categorical variables are expressed as counts and frequencies (%) and compared using Fisher's exact test between the mild and severe groups. Statistical significance was set at p < 0:05. The area under the curve (AUC) of different models was compared by the DeLong test using MedCalc (version 19.4.1). A dataset was built, including clinical information, laboratory tests, and chest CT features, from 198 patients with COVID-19, as confirmed by RT-PCR. The machine learning model was established using Python 3.7. We randomly split the dataset into a 70% training and validation set and a 30% test set. All quantitative features were normalized to the range of 0 to 1. The categorical features were transformed into a onehot numerical array. Six machine learning models, including logistic regression (LR), k-nearest neighbor (KNN), decision tree (DT), random forest (RF), support vector machine (SVM), and eXtreme gradient boosting (XGBoost), were built based on the features after preprocessing. After parameter tuning, the model's performance was assessed using the AUC. The receiver operating characteristic (ROC) curve of each model was further evaluated using DeLong's test on MedCalc ( Figure 2 ). Based on Shapley values from coalitional game theory, Shapley Additive explanations (SHAP) were used to explain the model [9, 10] . The SHAP explains the model prediction by computing each feature's contribution individually or jointly to the prediction. With kernelSHAP, treeSHAP, and deepKernal subclasses, SHAP can explain any machine learning model's output. This study enrolled 198 patients (mild group: 162 cases and severe group: 36 cases), including 80 males and 118 females. The average age of the mild (46:93 ± 14:49 years) and severe (60:97 ± 15:91 years) groups was significantly different. Patients in the mild group were admitted to the hospital 10:40 ± 5:58 days after the onset, which is longer than that in the severe group (8:00 ± 4:88 days, p = 0:038). However, the temperature of patients in the severe group was higher than that of those in the mild group (37:42 ± 0:99°C vs. 36:75 ± 0:66°C). Fever, cough, shortness of breath, and dyspnea were significant features associated with the severe group. In terms of basic diseases, 22.22% (8/36) and 6.79% (11/162) of patients in the severe and mild groups, respectively, had high blood pressure (p = 0:008) ( Table 1 ). There were 9:35 ± 7:44 and 6:44 ± 4:08 days between the first CT scan and onset of chest CT features in the mild and the severe groups, respectively. However, the total CT score and the number of different lobes involved in the severe group were significantly higher than those in the mild group. Patients with diffuse (23/36, 63.89%) and large patchy (18/36, 50.00%) appearances were likely to deteriorate. In contrast, patients with diffuse location and patchy shape of the mild group were 35.80% and 81.48%, respectively. Moreover, 80.6% of severe group patients showed lung lesions that had invaded five lobes at admission, compared to 39.5% of 3 BioMed Research International the mild patients (p = 0:001). The manifestations of pleural effusion, consolidation, crazy paving, and air bronchogram played an essential role in predicting COVID-19 deterioration, indicating that these patients were more likely to develop into severe and critically ill patients ( Table 2) . As for laboratory tests, the severe group had a higher WBC count, neutrophil count, and neutrophil ratio and a lower lymphocyte count and lymphocyte ratio than the mild group. Furthermore, the neutrophil-to-lymphocyte ratio (NLR) in the severe group was significantly higher than that in the mild group (8:12 ± 9:69 vs. 3:04 ± 2:75) ( Table 1) . Interpretability. A dataset was built, including enrolled patients' clinical information, laboratory tests, and chest CT features. We randomly split the dataset into a 70% training The days from symptom onset to hospital admission was seven days, and the temperature at admission was 37.4°C. The neutrophil was 11 × 10 9 /L, with a neutrophil ratio of 92.5% and an NLR of 17.46. 6 BioMed Research International the XGBoost and RF models was not statistically significant (p = 0:192). The sensitivity of the two models remained the same; however, XGBoost had higher specificity. Although the AUC between XGBoost and LR, KNN, and RF showed no statistical difference, XGBoost acquired the highest Youden index, sensitivity, and specificity. In general, XGBoost was the best model in this dataset. We further explored the interpretability of XGBoost using the TreeExplainer of SHAP [11] . Figure 3(a) shows the top 19 features that influenced the severe group prediction in descending order. The early stage of chest CT, total CT score of the percentage of lung involvement, and age were the top three contributors to the prediction of deterioration (Figure 3(a) ). Patients in the early stage of chest CT at admission were more likely to deteriorate. Moreover, a higher chest CT total score meant that a broader area of the lung was involved; the patients had an increased risk of becoming severe or critically ill (Figure 3(b) ). Specifically, injury to the inferior lobe of the right lung (IOR) and upper lobe of the left lung (UOL) had a more significant impact on the prediction than the other lobes. The high neutrophil count, neutrophil ratio, and NLR were also useful in predicting severe and critically ill patients. We can take one step further to explore the feature contribution in individual predictions. The model outputs the probability of a patient becoming severe or critically ill, followed by the specific weight of contribution in the single prediction. Figure 4 shows an example of a SHAP. While the conventional machine learning model merely outputs the prediction, SHAP was able to show the details of how AI concluded. The universal manifestation of COVID-19, such as GGO, has low specificity, making it difficult to distinguish COVID-19 from other types of pneumonia solely based on chest CT appearance [12, 13] . It would be even harder, more time-consuming, and often unfeasible for radiologists to assess the disease severity based on the lobar extent, type of pulmonary opacities, clinical information, and laboratory tests, especially in urgent situations or high demand [8, 14, 15] . Since the COVID-19 outbreak, attempts using AI have been made to integrate the information from molecular, medical, and epidemiological scales [16, 17] . The cluster computing power of AI can help with early and improved disease detection and diagnosis, treatment monitoring, and contact tracing of infected individuals, which may help predict the future course of COVID-19 [18] . Moreover, AI can help with designing and developing vaccines and drugs [19] [20] [21] . This study took a step further and established six machine learning models to predict COVID-19 patients' prognosis; XGBoost ranked first in performance. Homayounieh et al. [22] performed multiple logistic regression tests combined with the radiomics of chest CT, clinical information, and laboratory tests on 115 RT-PCR positive patients to predict the possibility of ICU admission, i.e., severe patients. They achieved a 0.84 AUC (95% CI (0.78, 0.85), p < 0:02). In comparison, the XGBoost model showed a 0.924 AUC (95% CI (0.826, 0.976), p < 0:0001), 90.91% sensitivity (95% CI (58.7%, 99.8%)), and 97.96% specificity (95% CI (89.10%, 99.90%)) based on the clinical information, laboratory tests, and chest CT features. Another issue with AI applications is interpretability. Most AIpredicted models are a "black box"; that is, it is not possible to know further details about each feature's contribution towards model prediction, an important issue with AI applications in clinical settings. Therefore, we established an interpretable XGBoost-based module called SHAP. This interpretable module outputs the contribution of important features. Patients with features on the list have a higher possibility of deteriorating to severe or critically ill condition. In this cohort, the early stage of chest CT manifestation made the most significant contribution to the prediction, followed by the total score of chest CT and age. Lesions in the severe and critically ill patients seem to be more extensive than mild cases, meaning a higher total score of chest CT and presence of diffused patchy and large patchy appearances on the CT image. Similar to MERS-CoV, patients in the severe group were usually older than those in the mild group, indicating that the elderly tends to develop severe or critical forms of COVID-19, possibly due to comorbidities such as hypertension and underlying immune response [23] . Fever was a typical symptom of COVID-19, and those with a higher temperature at admission were more likely to worsen in the future. The cough was another common symptom, whereas fatigue, shortness of breath, and dyspnea were more common in the severe group, which is consistent with previous research [24, 25] . Furthermore, higher neutrophil count, neutrophil ratio, and NLR ratio increased the possibility of deterioration. Lymphocytopenia is a characteristic of COVID-19 [26] . The virus proliferates in the respiratory system, causing a series of immune responses, leading to changes in lymphocytes and other immune cells [25] . The lower lymphocyte count and lymphocyte ratio, higher WBC and neutrophil counts, and higher neutrophil ratio and NLR may be related to the severity and mortality rate of COVID-19 [27] . Similar to the days from onset to admission, the days from symptom onset to the first CT scan for the severe group were shorter than those for the mild group, meaning that the initial symptoms were serious, resulting in early hospital presentation. In contrast, the lesions appeared to be more extensive in the severe group, suggesting the rapid progression of COVID-19 in these patients. It is worth noting that the more extensive injuries in the IOR and UOL, the more significant their contribution to the deterioration. With the interpretable machine learning model's application, the medical institutions could identify the potential severe type and critical type patients, hence applying the main observation since admission. Once the crucial factors change during treatment, the doctors could take the early clinical intervention to stop deterioration in the early stage. Our study has some limitations. First, the small sample size and differences in the number of mild and severe patients may have affected the statistical power of our study. In this study, we applied stratified sampling in data segmentation to reduce the influence brought by imbalanced numbers. Second, the prognostic prediction model may be further 7 BioMed Research International improved by combining chest CT radiomics or deep learning models. The application of radiomics and deep learning models may eliminate subjective bias and improve performance. Attempts have been made in a previous study on the detection, outcome, and prognosis prediction of COVID-19 [2, 28, 29] . Third, this was a retrospective study, indicating uncontrollable data loss in the collection, such as procalcitonin and C-reactive protein. In order to ensure a sufficient data size, we had to give up some laboratory results, which may have decreased the performance of the model. Given the limited scale and data, the established XGBoost model requires further clinical validation. In conclusion, this study established an interpretable machine learning model based on the XGBoost algorithm combined with clinical information, laboratory tests, and chest CT features, aimed at predicting the possibility of COVID-19 patients becoming severe and critically ill, which achieved excellent performance. Furthermore, we explored the most important features in the deterioration process using the interpretable SHAP module, which enabled us to determine the factors that put the patients at risk of developing ARDS and dying from respiratory failure and take necessary clinical interventions to improve the patient prognosis and reduce mortality among the severe and critically ill patients. All data used to support the findings of this study are restricted by the Ethics Committee of Honghu People's Hospital in order to protect patient privacy. How might AI and chest imaging help unravel COVID-19's mysteries? Artificial intelligenceenabled rapid diagnosis of patients with COVID-19 CT imaging features of 2019 novel coronavirus (2019-nCoV) Chemoprophylaxis, diagnosis, treatments, and discharge management of COVID-19: an evidence-based clinical practice guideline (updated version) Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China Clinical characteristics of coronavirus disease 2019 in China Severe acute respiratory syndrome: temporal lung changes at thin-section CT in 30 Patients Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection Consistent individualized feature attribution for tree ensembles A unified approach to interpreting model predictions From local explanations to global understanding with explainable AI for trees Performance of radiologists in differentiating COVID-19 from non-COVID-19 viral pneumonia at chest CT Erratum: extension of coronavirus disease 2019 (COVID-19) on chest CT and implications for chest radiograph interpretation The clinical and chest CT features associated with severe and critical COVID-19 pneumonia Temporal changes of CT findings in 90 patients with COVID-19 pneumonia: a longitudinal study Mapping the landscape of artificial intelligence applications against COVID-19 Artificial intelligence (AI) applications for COVID-19 pandemic Artificial intelligence (AI) applications in orthopaedics: an innovative technology to embrace Clinical considerations for patients with diabetes in times of COVID-19 epidemic Potential T-cell and B-cell epitopes of 2019-nCoV COVID-19 coronavirus vaccine design using reverse vaccinology and machine learning Computed tomography radiomics can predict disease severity and outcome in coronavirus disease 2019 pneumonia Prevalence of comorbidities in the Middle East respiratory syndrome coronavirus (MERS-CoV): a systematic review and meta-analysis Thin-section chest CT imaging of COVID-19 pneumonia: A comparison between patients with mild and severe disease Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China Radiographic and CT features of viral pneumonia The acute respiratory distress syndrome Classification of COVID-19 patients from chest CT images using multiobjective differential evolution-based convolutional neural networks Serial quantitative chest CT assessment of COVID-19: deep-learning approach We would like to thank Editage (https://www.editage.com) for English language editing. This study was funded by the National Key The authors declare no conflict of interest. Bowen Zheng and Yong Cai contributed equally to this work.