key: cord-325170-50oy9qqy authors: Bai, Xiang; Fang, Cong; Zhou, Yu; Bai, Song; Liu, Zaiyi; Chen, Qianlan; Xu, Yongchao; Xia, Tian; Gong, Shi; Xie, Xudong; Song, Dejia; Du, Ronghui; Zhou, Chunhua; Chen, Chengyang; Nie, Dianer; Tu, Dandan; Zhang, Changzheng; Liu, Xiaowu; Qin, Lixin; Chen, Weiwei title: Predicting COVID-19 malignant progression with AI techniques date: 2020-03-23 journal: nan DOI: 10.1101/2020.03.20.20037325 sha: doc_id: 325170 cord_uid: 50oy9qqy Background: The coronavirus disease 2019 (COVID-19) has become a worldwide pandemic since mid-December 2019, which greatly challenge public medical systems. With limited medical resources, it is a natural strategy, while adopted, to access the severity of patients then determine the treatment priority. However, our work observes the fact that the condition of many mild outpatients quickly worsens in a short time, i.e. deteriorate into severe/critical cases. Hence, it has been crucial to early identify those cases and give timely treatment for optimizing treatment strategy and reducing mortality. This study aims to establish an AI model to predict mild patients with potential malignant progression. Methods: A total of 133 consecutively mild COVID-19 patients at admission who was hospitalized in Wuhan Pulmonary Hospital from January 3 to February 13, 2020, were selected in this retrospective IRB-approved study. All mild patients at admission were categorized into groups with or without malignant progression. The clinical and laboratory data at admission, the first CT, and the follow-up CT at severe/critical stage of the two groups were compared with Chi-square test, Fisher's exact test, and t test. Both traditional logistic regression and deep learning-based methods were used to build the prediction models. The area under ROC curve (AUC) was used to evaluate the models. Results: The deep learning-based method significantly outperformed logistic regression (AUC 0.954 vs. 0.893). The deep learning-based method achieved a prediction AUC of 0.938 by combining the clinical data and the CT data, significantly outperforming its counterpart trained with clinical data only by 0.141. By further considering the temporal information of the CT sequence, our model achieved the best AUC of 0.954. The proposed model can be effectively used for finding out the mild patients who are easy to deteriorate into severe/critical cases, so that such patients get timely treatments while alleviating the limitations of medical resources. In mid-December 2019, the ongoing coronavirus disease 2019 broke out in Wuhan and spread rapidly in the mainland of China (80813 cases, updated through March 12, 2020) . So far, the infection had burst in countries outside China, evolving into a pandemic 4,5 . According to the Chinese epidemic data, the mild, severe, and critical types of COVID-19 were 81%, 14%, and 5% separately 6 . More seriously, as of 12 March, the mortality of COVID-19 was 3.93% (3176/80813) in the mainland of China, even reached 4.87% (2436/49991) in Wuhan City, which was much higher than that of other influenza 7, 8 . In addition, the clinical course of COVID-19 varied individually. In order to prevent malignant progression and reduce the mortality of COVID-19, it is vital to identify mild patients who are easy to deteriorate into severe/critical cases and give them active treatment earlier. However, most studies focused on cross-sectional description and comparison of clinical, laboratory and CT imaging findings [9] [10] [11] [12] . Some studies focused on seeking risk factors for death outcome 3, 13 . None of them used AI-based methods for progression prediction of mild COVID-19 patients up to date. To solve this problem, we aimed to apply AI techniques to study multivariate heterogeneous data (clinical data and serial chest CT imaging) and to further develop an accurate and effective prediction model. Specifically, we employed a deep learning-based model to effectively mine the complementary information in static clinical data and serial quantitative chest CT sequence. Since deep learning-based methods had been widely adopted and had achieved great performance in cancer outcome prediction 14 , head CT scans detection 15 and antibiotic discovery 16 investigation. The inclusion criteria were: 1) respiratory rate < 30 breaths per min; 2) resting blood oxygen saturation > 93%; 3) the ratio of arterial oxygen partial pressure to fraction of inspiration oxygen > 300mmHg; 4) non-ICU patients without shock, respiratory failure, mechanical ventilation, and failure of other organs. The clinical and laboratory data at the time point of admission, together with serial chest CT images of all patients were retrospectively analyzed. Based on the presence or absence of the severe/critical progression during the hospitalization, all patients were categorized into two groups. The diagnostic criteria for severe/critical progression were: 1) respiratory rate radiology) independently blinded to the clinical information, and the discrepancy was resolved by consulting another radiologist (WC, 15 years' experience in radiology). Lesions and imaging features were assessed in each lung segment of each patient. The number of involved segments was counted not only for each patient or each lobe but also for each imaging feature. If more than one type of imaging features present in a segment, the segment was counted for every involved feature. The imaging features assessed in this study included 1) ground glass opacity (GGO); 2) consolidation; 3) air bronchogram; 4) paving stone sign; 5) fibrosis; 6) nodule; and 7) halo sign. The first available CT after symptoms onset, the follow-up CT, the first available CT of the severe stage were assigned as CT 1 , CT 2 ~ CT n , and CT severe separately. In order to compare the longitudinal variation of CT features during the period of CT 1 and CT severe between the two patient groups, we chose CT 2 instead of CT severe for those patients without severe/critical progression. Our raw COVID-19 dataset contained all the clinical data and the quantitative chest CT data. After excluding invalid and duplicate information, each sample contained 75 clinical data characteristics and a quantitative CT sequence obtained at different times. Since the sequence length of each sample varied from zero to seven, we adjusted the data structure of each sample to the same shape by zero-filling the uncollected or missing chest CT data. The original quantitative chest CT data contained twelve infection distribution features, eight infection sign type features, the thickness of thoracic diaphragm, and CT course. The lung was medically divided into 18 segments, and the infection sign characteristics at each checkpoint can be formatted as a matrix. This matrix composed of infection distribution features and sign type features was flattened into a vector and then concatenated with the original quantitative chest CT data. The pipeline of the prediction model is shown in Figure 3 . The input data includes the static data and the dynamic data, where the static data is a 75-dimensional vector, containing the clinical data and personal information of patients. Dynamic data is a series of quantitative chest CT data collected at different times. Each CT data at different checkpoint consists of a 3 × 6 matrix and a 22-dimensional vector. In order to merge these two parts, we directly flattened the All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. . https://doi.org/10.1101/2020.03.20.20037325 doi: medRxiv preprint matrix into an 18-dimensional vector and concatenated it with the 22-dimensional vector to form a 40-dimensional CT feature vector. According to the checkpoints, the CT data sequence with a length of seven and a dimension of 40 was formed. For the sake of combining static and dynamic data as the input of long short term memory (LSTM), a multi-layer perceptron (MLP) was applied to the static data to obtain a 40-dimensional feature vector, which is used as the input data of the first timestamp of the LSTM, followed by the other seven CT feature vectors 18 . The LSTM model employed in this study is a single-layer network with the embedding dimension of 40 and the hidden dimension of 32. The output of the LSTM, a 32 × 8 feature sequence, was then fed into fully connected layers. A Softmax layer was added at the top of the network to output the probability of the patient conversion to the severe/critical stage. A total of 133 samples were included in the COVID-19 dataset. The robustness of the model was evaluated by five-fold cross-validation repeating five times, and each fold was obtained by category-wise sampling. All the statistical analysis was performed using SPSS (Version 26) with statistical significance set at 0.05. Statistical optimization of the deep learning model was done through iterative training using Python (Version 3.6 with scipy, scikit-learn, and pytorch packages). The differences of clinical and laboratory data and imaging features between the patient with and without severe/critical progression were compared using Chi-square test, Fisher's exact test, independent t test and paired t test. AUC, accuracy, specificity, and sensitivity were compared among different AI methods and multivariable logistic regression. Two-sided 95% CIs were used to summarize the sample variability in the estimates. Specifically, the normal approximation CIs was used for accuracy, sensitivity, and specificity. The CI for the AUC was estimated using the bootstrap method with 2000 replications. 133 patients with mild COVID-19 pneumonia at admission included 66 male and 67 female, age ranged from 18 to 82 (52.82 ± 12.59) years, the interval from symptoms onset to admission ranged from 1 to 20 (8.76 ± 4.05) days. 54 patients (54/133, 40.6%) malignantly progressed to severe/critical periods during the All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. . hospitalization, while the remaining 79 patients (79/133, 59.4%) did not ( Figure 4 ). The whole clinical course of all patients, including assessment at admission, the severe or critical progression, and the outcome, was plotted in Figure 4 . The age, sex, exposure, comorbidity, signs and symptoms, laboratory results measured at admission, and serial CT imaging features of patients with and without severe/critical progression were separately summarized in Tables 1, 2, and 3. In brief, comparing to the patients without severe/critical progression, the patients with severe/critical progression showed older age, more comorbidities, higher respiratory rate, inflammatory cell factors, lower albumin and fewer counts of lymphocyte, T cell, and its subsets. The patients with severe/critical progression were more likely to involve organs other than the lung. On the first available CT, no difference was found in either the distribution of involved lung or the other CT imaging features, except paving stone sign and the presence of fibrosis. However, the patients with severe/critical progression showed significantly more lesions in all lobes, more lesions of consolidation, paving stone sign and halo sign than patients without severe/critical progression when they progressed to the period of severe/critical stage. were the risk factors for severe/critical progression. However, the presence of fibrosis at CT 1 (OR 0.656, 95%CI 0.473-0.910) was the protective factor for severe/critical progression. The accuracy of the prediction is 79.2%. We conducted comprehensive experiments to validate our hypotheses and compared the performance of various models. Table 4 summarized the performance of traditional multi-stage and deep learning-based methods. Static clinical data including personal information, dynamic quantitative chest CT data or both of them were used for predictive experiments. For traditional multi-stage methods, PCA was used for data dimensionality reduction, and SVM or LDA was used for classification. The results indicate that quantitative chest CT data without time series modeling is also beneficial for All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. The above results clearly supported the significance of complementary information from different medical data and time-series information from the chest CT sequence. Finally, our proposed method had a high probability of stabilizing at a high confidence interval, which is very important for clinical applications. With the worldwide outbreak of COVID-19, early prediction and early aggressive treatment of mild patients at high risk of malignant progression to severe/critical stage are important ways to reduce mortality. In this work, we found that the complementarity of clinical data and quantitative chest CT sequence is important for predicting patients with malignant progression. In particular, the rich series information of the chest CT sequence, which has not been considered by other studies so far, is critical for this specific task. We also demonstrated that our method can effectively fuse these two complementary data and handle time-series information in the quantitative chest CT sequence, which achieved an AUC of 0.954 (95% CI Although lots of clinical, laboratory, and imaging parameters varied significantly between patients with and without severe/critical progression, seven predictive All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. Unlike the traditional predictive model using a hand-crafted feature extractor and shallow classifiers, our deep learning-based method using a multilayer perceptron combined with an LSTM to this predictive task, which attempts to learn high-level hierarchical features from mass data, and expands the search space of the features for specific tasks. Moreover, this method jointly optimizes the feature extraction network and classifier through an end-to-end manner. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Our study has several limitations. First, samples available for malignant progression prediction were limited. The diverse data in the large scale dataset will allow deep learning-based methods to gain a more comprehensive understanding of what causes the malignant progression of mild patients. Second, the quantitative information of CT data is not detailed enough. Using the richer original features included in pixel-wise segmentation results of the CT scans, the predictive model may perform better. In conclusion, the deep learning-based method using clinical and quantitative CT data to predict malignant progression to severe/critical stage. We modeled the spatial information in the quantitative CT data and organized the static clinical data and dynamic chest CT data into a time series form. We validated the significance of complementary data and its special formatting form for this particular prediction task. Compared with traditional multi-stage methods, we demonstrate that our deep learning-based method can extract spatial and temporal information efficiently and improve the prediction performance significantly. The ability to identify patients with potentially severe and critical COVID-19 outcomes using an inexpensive, widely available, the point-of-care test has important practical implications for preventing mild patients from becoming severe, effectively improving cure rate, and reducing mortality. Our future work will focus on mining richer spatial information from the CT scan sequence and using AI technologies to screen the risk factors of potential severe/critical patients. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. . https://doi.org/10.1101/2020.03.20.20037325 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. . https://doi.org/10.1101/2020.03.20.20037325 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 23, 2020. . https://doi.org/10.1101/2020.03.20.20037325 doi: medRxiv preprint All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Estimates of global seasonal influenza-associated respiratory mortality: a modelling study Influenza-associated excess respiratory mortality in China, 2010-15: a population-based study Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease (COVID-19):A multi-center study in Wenzhou city Outbreak of novel coronavirus (COVID-19): What is the role of radiologists? Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan Deep learning for prediction of colorectal cancer outcome: a discovery and validation study Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition Critical care crisis and some recommendations during the COVID-19 epidemic in China Pathological findings of COVID-19 associated with acute respiratory distress syndrome. The Lancet Respiratory Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer Analysis of factors associated with disease outcomes in hospitalized patients with 2019 novel coronavirus disease Laboratory abnormalities in patients with COVID-2019 infection The Clinical and Chest CT Features Associated with Severe and Critical COVID-19 Pneumonia RadiologyEssentials for Radiologists on COVID-19: An Update-Scientific Expert Panel