key: cord-0706183-g1ez2l6u
authors: Zhou, Tao; Lu, Hui-ling; Yang, Zaoli; Qiu, Shi; Huo, Bing-qiang; Dong, Yali
title: The ensemble deep learning model for novel COVID-19 on CT images
date: 2020-11-06
journal: Appl Soft Comput
DOI: 10.1016/j.asoc.2020.106885
sha: 6018788a995616dfbb72479dfac14388138aadac
doc_id: 706183
cord_uid: g1ez2l6u

The rapid detection of the novel coronavirus disease, COVID-19, has a positive effect on preventing propagation and enhancing therapeutic outcomes. This article focuses on the rapid detection of COVID-19. We propose an ensemble deep learning model for novel COVID-19 detection from CT images. 2933 lung CT images from COVID-19 patients were obtained from previous publications, authoritative media reports, and public databases. The images were preprocessed to obtain 2500 high-quality images. 2500 CT images of lung tumor and 2500 from normal lung were obtained from a hospital. Transfer learning was used to initialize model parameters and pretrain three deep convolutional neural network models: AlexNet, GoogleNet, and ResNet. These models were used for feature extraction on all images. Softmax was used as the classification algorithm of the fully connected layer. The ensemble classifier EDL-COVID was obtained via relative majority voting. Finally, the ensemble classifier was compared with three component classifiers to evaluate accuracy, sensitivity, specificity, F value, and Matthews correlation coefficient. The results showed that the overall classification performance of the ensemble model was better than that of the component classifier. The evaluation indexes were also higher. This algorithm can better meet the rapid detection requirements of the novel coronavirus disease COVID-19.

: image features of CT image of COVID-19 Table 1 shows the current clinical value of lung CT images on COVID-19. Ultra-high-resolution CT image can identify the terminal bronchiole in normal lungs. U-HR-CT can be used to detect abnormal lung volume reduction, which is essential for the early diagnosis and timely treatment of critical illness in COVID-19 patients. Song F [15] The most common feature of COVID-19 on CT images are pure ground-glass opacity (GGO). If patients present GGO in the peripheral and posterior lungs on chest CT images as well as cough and/or fever, a history of epidemic exposure, and normal or decreased white blood cells, then COVID-19 infection is highly suspected. K. Wang [16] The author made a summary of the location, distribution, morphology, and density of the lesions in CT images of 114 COVID-19 patients. SPO2 and lymphocytes can reflect lung inflammation. The diagnostic sensitivity and accuracy of spiral CT testing was higher than nucleic acid detection. This method can be applied to early diagnosis and treatment of COVID-19 patients. Huanhuan Liu [17] Chest CT image features of pregnant women with COVID-19 pneumonia were atypical. It was observed from the CT images that the lungs of pregnant patients were more susceptible to the disease. The CT image features of children were nonspecific. Therefore, the combination of other diagnostic methods can be used to diagnose children. Agostini A [18] Because of radiation exposure and motion artifacts in CT images, patients need to be imaged multiple times. The author performed ultra-low-dose, dual-source, rapid CT imaging on 10 patients with confirmed COVID-19. This image method can provide a reliable diagnosis and can reduce motion artifacts and dose.

At present, COVID-19 has already spread to all over the world. With the sharp increase in COVID-19 patients, existing medical resources and diagnostic capabilities are insufficient. In addition, staff density in hospitals in core epidemic areas has increased, along with the risk of cross-infection. The development of COVID-19 computer-aided diagnosis models based on CT images of lungs is thus increasingly important. According to the National Health Commission of the People's Republic of China (2020), "Diagnosis and protocol of COVID-19", trial version 6 [19] , CT detection is not only one of the diagnostic standards for COVID-19, but also has an important significance in the treatment of COVID-19.

Ai, T. [20] found that chest CT had a higher sensitivity for diagnosing COVID-19 over RT-PCR. Chest CT may thus be considered as a primary tool for COVID-19 detection in epidemic areas. Himoto Y. [21] pointed out that CT images can distinguish COVID-19 from other similar respiratory diseases. Lin Li [22] proposed a COVNet model based on deep learning to distinguish COVID-19 from community-acquired pneumonia (CAP) in chest CT images. CT images were segmented via U-net, then COVNet distinguished between COVID-19 and CAP. Wang [23] proposed a fully automatic deep learning system for COVID-19 diagnosis and prognosis based on CT image analysis. First, they made a DenseNet121-FPN for lung segmentation in chest CT images and the proposed novel COVID-19Net for COVID-19 diagnosis and prognosis. This system can categorize patients into low-and high-risk groups according to the severity of disease, and can automatically identify the lesion area. COVID-19 computeraided diagnosis combined with deep learning and lung CT images plays an important role in quickly classifying and identifying COVID-19, improving diagnosis efficiency, saving doctors' energy, and optimizing medical resources.

2 Basic knowledge

Convolutional neural networks (CNNs) are multi-layer networks composed of overlapping convolutional layers for feature extraction and down sampling layers for feature processing. Figure 2 shows the structure of a typical convolutional neural network. CNNs can automatically extract features from images, and thus have become a research hot topic. CNNs are perceptron-based models [24] . Their robust translation, rotation, and scaling. Classic CNNs include: residual neural network (ResNet), AlexNet, and GoogleNet.

AlexNet [25] was designed by the 2012 ImageNet competition winners, Hinton and Alex Krizhevsky. AlexNet uses 5 convolutional layers, including three pooling layers and two norm layers. Three of them are fully connected layers with a total of 60 million parameters. Figure 3 shows the specific network parameters. Each input image is scaled to 256×256, and square blocks (of 224×224) are randomly intercepted from them, and then input in three dimensions of RGB. Due to GPU performance limitations at the time, Alex Krizhevsky et al. Processed AlexNet in two GPUs in parallel, therefore the hidden layer in the figure is shown as two simultaneous calculations. The five layers in front of the AlexNet network are convolutional layers. Taking the first layer as an example, 96 feature maps of 55×55 nodes are generated. Each feature map is composed of a convolution kernel with a size of 11×11 and step size of four. After convolution filtering, the output excitation of the convolution layer is obtained through a ReLU activation function and then output to the next convolutional layer after the local response normalization and maximum pooling down-sampling. A three-layer fully connected network is added as a classifier for the five-layer convolutional network, the highdimensional convolutional features are classified to obtain a class label. The fully connected network finally outputs a response with a dimension of 1000, corresponding to the 1000 categories of images to be classified. 

GoogleNet [26] consists of a cascade of multiple basic inception modules, having a depth of 22 layers. Figure 4 shows the structure of GoogleNet. Features are extracted at different scales from the previous input layer with three different convolution kernels of different sizes. The information is combined and passed to the next layer. Inception has 1×1, 3×3, and 5×5 convolution kernels, of which the 1×1 convolution kernel has a lower dimension than the previous layer, which is mainly used for data dimensionality reduction. This is passed to the latter 3×3 and 5×5 convolutional layers to reduce their convolution calculations and avoid large calculations caused by increasing the network scale. After combining the features of the four channels, the next layer can extract more useful features from different scales. 

The residual neural network (ResNet) is composed of a convolutional layer for feature extraction and a pooling layer for feature processing. A multi-layer neural network solves the problem of degradation and gradient disappearance. With the deepening of the network, the gradient of the convolutional neural network gradually disappears, and the shallow parameters cannot be updated. The structure of shortcut connections ensures the update of the backpropagation parameters and avoids the problem of gradient disappearance caused by backpropagation. The residual neural network makes it easier to optimize deep models. After the input image undergoes several convolution and pooling operations, the classification effect is achieved in the fully connected layer network. Layer connection implements identity mapping [27] . The identity mapping ensures that the network performance will not degrade, so that the network learns new features based on the input features, and the identity mapping does not add an extra to the network. The parameters and calculation amount can accelerate the training speed of the J o u r n a l P r e -p r o o f model and optimize the training effect.

Transfer learning [28] is a machine learning method that uses existing knowledge to solve problems in different, albeit related fields. It relaxes two basic assumptions in traditional machine learning with the purpose is to transfer existing knowledge to solve the learning problem in the target field where there is none, or only a small amount of labeled sample data. Transfer learning exists widely in all human activities. The more factors are shared between two different fields, the easier transfer learning is. Otherwise, it can be more difficult, or even negative transfer can occur to a deleterious effect. The purpose of this technique is to solve the learning problem of insufficient training samples in the target domain and make it possible to transfer some knowledge acquired in other related source domains to the target domain. According to whether the samples are marked in the source and target fields and whether the tasks are the same [29] , transfer learning can be divided into inductive and unsupervised transfer learning. According to the content of transfer learning, methods can be divided into feature representation, transfer, instance transfer, parameter transfer, and association relationship transfer. According to whether the feature space of the source domain and the target domain are the same, it can be divided into homogeneous transfer learning and heterogeneous transfer learning.

Ensemble learning [30] methods mainly include bagging, boosting, and stacking. Ensemble learning can significantly improve the generalization ability of the learning system. At present, the common methods for generating base classifiers can be divided into two categories: one is to apply different types of learning algorithms to the same data set [31] . The base classifiers obtained by this method are usually different. A heterogeneous classifier. The other is to apply the same learning algorithm to different training sets. The base classifier obtained by this method is called a homogeneous classifier. The combination strategy of integrated learning on classifiers mainly includes average, voting, and learning methods. According to the different uses of integrated learning, different combination methods are usually selected. For example, if the purpose of integrated learning is regression estimation, the prediction results of each individual learner are usually averaged or weighted average. If integrated learning is used for classification, each individual classification result is voted to obtain the final classification result.

The voting method is divided into absolute and relative majority voting methods. Absolute majority voting method, that is, where more than half of individual learners output the same classification result, the result is the final classification result of integrated learning output. In the relative majority voting method, the number of individual learners that output a certain classification result is the largest, and the result is the final classification result of the integrated learning output.

Some new methods are used for image classification, for example, enhanced learning, GA-SVM [32] , and dense-MobileNet models [33] . We propose an ensemble deep learning model for novel COVID-19 on CT images. The overview of the model is as follows:

(1) Data collection. 2933 lung CT images of COVID-19 patients were obtained from previous publications, authoritative media reports, and public databases. The images are preprocessed to obtain 2500 high-quality CT images. 2500 lung tumor CT images and 2500 normal lung CT images were obtained from general hospital of Ningxia medical university in China. Figure 5 shows examples of lung CT images collected for this study. 

Sample size is |Sample_Lung|=7500. According to the type of medical image (NormalLung, LungTumor, COVID), lung medical image sample set Sample_lung is divided into three sample subsets: Sample_NormalLung, Sample_LungTumor, and Sample_COVID. The sample sizes are |Sample_NormalLung|=2500, |Sample_LungTumor|=2500, |Sample_COVID|=2500. Sample_Lung={Sample_NormalLung Sample_LungTumor Sample_COVID} |Sample_NormalLung|=|Sample_LungTumor|=|Sample_COVID|=2500;

(3) Resize. Sample_Lung=resize(Sample_Lung); (4)The training and test sample sets were constructed using the 5-fold crossover method in the three sample subsets Sample_NormalLung, Sample_LungTumor, and Sample_COVID. Using a partition algorithm, the subset of each samples were divided into 5 uniform parts, 500 samples per part, obtaining sample sets of 5 fold cross. {Sample_Lung_TrainingSet Sample_Lung_TestingSet}=fiveCross(Sample_Lung) (5)Individual classifiers were generated by pretraining the network via transfer learning.

AlexNet, GoogleNet and ResNet are generated by pre_training the network with transfer learning while using these parameters in the pre_trained network as the initialization parameters of AlexNet, GoogleNet and ResNet respectively.

AlexNet_Softmax=TransferLearning(AlexNet,Softmax); GoogleNet_Softmax=TransferLearning(GoogleNet,Softmax); GoogleNet_Softmax=TransferLearning(GoogleNet,Softmax); (6) In the training sample set Sample_Lung_TrainingSet, Training AlexNet_Softmax, GoogleNet_Softmax, and ResNet_Softmax, respectively, Individual classifiers are obtained. AlexNet_Softmax=Training(AlexNet_Softmax,Sample_Lung_TrainingSet); GoogleNet_Softmax=Training(GoogleNet_Softmax,Sample_Lung_TrainingSet); ResNet_Softmax=Training(ResNet_Softmax,Sample_Lung_TrainingSet); (7) Ensemble ResNet_NRC classifier. Using the relative majority voting method to integrate three individual classifiers. EDL_COVID=Ensemble(AlexNet_Softmax,GoogleNet_Softmax,ResNet_Softma x) Figure 6 shows the algorithm flow chart.

Software environment: Windows 10 operating system, MatlabR2019a. Hardware environment: the hardware platform used for the simulation experiment was an Intel (R) Core (TM) i5-7200U CPU @ 2.50 GHz 2.70 GHz, 4.0 GB RAM, and 500 GB hard disk.

The measuring of the performances of the models, accuracy, sensitivity, specificity, F-score, Matthews correlation coefficient, is as follows: Accuracy is the most common evaluation index. The higher the accuracy, the better the classifier performance. The formulation is as follows:

(1) Sensitivity and specificity measure the classifier's ability to recognize positive and negative examples, respectively. The larger the value, the higher the recognition performance. The formulation is as follows:

F-score is the weighted harmonic average of recall rate and precision rate. It is used to weigh accuracy and recall rate. The formulation is as follows:

MCC is a correlation coefficient that describes the actual classification and the predicted classification. It comprehensively considers true positives, true negatives, false positives, and false negatives. It is a more balanced indicator. Its value range is [-1,1], and the closer the value is to 1, the more accurate the prediction of the test subject. The formulation is as follows:

where true positive (TP) represents the number of samples that are benign and correctly predicted, whereas true negative (TN) represents the malignant and correctly predicted number of samples. Also, false positive (FP) represents the number of samples that are actually malignant but incorrectly predicted while false negative (FN) represents the number of samples that are actually benign but incorrectly predicted.

2933 lung CT images of COVID-19 patients were obtained from previous publications, authoritative media reports, and public databases. Among them, 1752 cases were obtained from both domestic and foreign journals such as Sciencedirect, Nature, Springer Link, and China CNKI. 1012 cases were obtained from authoritative media reports such as the New York Times, Daily Mail (United Kingdom), The Times (United Kingdom), CNN, The Verge (United States), Avvenire (Italy), LaNuovaFerrara (Italy), People's Daily, Toutiao News, and Dr. Lilac. 68 cases were obtained from the sirm.org public database. 101 cases were obtained from a GitHub public database. The lung CT images of the COVID-19 patients obtained in this study were all from third-party platforms. Images from different platforms were different in size and format, and contained different degrees of noise, such as watermarks and mark instructions. The research directions were thus different. for example: some studies were statistical analyses of COVID-19 cases, others tracked and analyzed the same patient, others analyzed patients of different ages and genders. It is necessary the comparison of image characteristics for different clinical classifications. Hence, there are differences in data modalities, such as horizontal position or coronal position. CT images were preprocessed, for example, by deleting images with large noise and coronal position. The unified image format were .JPG. All images were normalized at the same time, converting the image to a size of 64×64. Finally, we obtained 2500 highe quality CT images of the novel COVID-19.

Five-fold cross-validation was used for training. Each experimental result was averaged to obtain the final experimental result. The number of training samples per time was 2000×3=6000. The number of test samples was 500×3=1500. Five experiments were used to calculate the average value. The experiments were carried out on CT image data sets of normal lungs, lung tumors, and COVID-19. Identification and classification was performed using AlexNet-Softmax, GoogleNet-Softmax, and ResNet-Softmax, respectively. Then, ensemble deep learning model EDL-COVID was used for classification. Finally, the accuracy, sensitivity, specificity, F-Score, and Matthews correlation coefficient were used for evaluation. Figure 7 shows the average value of five indexes, which indicates the differences of different algorithms on various index. Table 11 shows the experimental accuracy after the five_fold cross_validation and the final average. The average classification accuracy of the EDL_COVID is 99.054%, and running time is 2251.86 seconds. Deviation of classification accuracy is 0.1019, standard deviation is 0.357, deviation of running time is 295.0553. Deviation of classification accuracy of EDL_COVID model is same with ResNet_Softmax, and deviation of running time tends to be stable. Table 12 shows the classification evaluation index. The sensitivity(SEN), specificity(SPE), F_Score(F), and Matthews correlation coefficient (MCC) are 99.05%, 99.6%, 98.59%, and 97.89%, respectively. According to our experimental results on the three individual classifiers performance, accuracy, sensitivity, specificity, F_score, and MCC of the ResNet_Softmax are the highest, but the running time is also the highest. AlexNet_Softmax is the least time_consuming, with an average time of 354.475 second. The accuracy, sensitivity, specificity, F_Score, and MCC of AlexNet_Softmax are the lowest. Comparing with the single classifier, the classification accuracy of the EDL_COVID model is higher than the AlexNet_Softmax, GoogleNet_Softmax, and ResNet_Softmax, and classification accuracy i improved to 0.89%, 0.80%, and 0.49%, respectively. The training time is increased to 1897.39, 1323.12, and 1261.28 second, respectively. It can be seen that the classification accuracy of the EDL_COVID model is better than a single classifier.

Deep learning classifiers, such as AlexNet_Softmax, GoogleNet_Softmax, and ResNet_Softmax, have a faster detection speed than nucleic acid reagent detection with a higher detection accuracy. Ensemble learning such as EDL_COVID can improve the classification accuracy over individual classifier.

J o u r n a l P r e -p r o o f Because the COVID-19 is highly contagious, its transmission route is difficult to effectively control,it is vital to detect viruses quickly and accurately to prevent propagation and provide timely treatment for the disease.Computer tomography (CT) detection of COVID-19 possesses high sensitivity, low misdiagnosis rate, and high commercial availability.hence, Artificial intelligence, esp deep learning based on CT are used to detect COVID-19 patinent. That is a good approach. In this paper, We proposed an ensemble deep learning model (EDL_COVID) based on COVID_19 lung CT images to rapidly detect the novel coronavirus COVID-19. 2500 CT images of COVID-19 lungs were obtained from previous publications, news reports, public databases, and other channels. 2500 CT images of lung tumors and normal lung were obtained from three grade A hospitals in Ningxia, China. Transfer learning was used to pretrain three deep convolutional neural network models, namely, AlexNet, GoogleNet, and ResNet, and initialization parameters were obtained. Using softmax as the classification algorithm of the fully connected layer, three component classifiers, AlexNet_Softmax, GoogleNet_Softmax, and ResNet_Softmax were constructed. The ensemble classifier EDL_COVID was obtained by the method of relative majority vote algorithm. Our results showed that the overall classification performance of our EDL_COVID model was better than a single individual classifier with the fastest detection speed of 342.92 s and an accuracy of 97%, the ensemble accuracy can thus reach 99.05%. Evaluation indexes such as specificity and sensitivity were also high, outlining its potential use for the rapid detection of COVID-19.

First case of 2019 novel coronavirus in the United States

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia

Towards an artificial intelligence framework for data-driven prediction of coronavirus clinical severity

Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT?

Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing

Ct imaging of the 2019 novel coronavirus(2019-ncov) pneumonia. Radiology, 2020

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

Coronavirus disease 2019 (covid-19): role of chest ct in diagnosis and management

Ct imaging features of 2019 novel coronavirus (2019-ncov)

Time course of lung changes on chest CT during recovery from 2019 novel coronavirus(COVID-19) Pneumonia

Progress and prospect on imaging diagnosis of covid-19

Ultra-highresolution computed tomography can demonstrate alveolar collapse in novel coronavirus (COVID-19) pneumonia

Emerging Coronavirus 2019-nCoV Pneumonia

Imaging manifestations and diagnostic value of chest CT of coronavirus disease 2019 (COVID-19) in the Xiaogan area

Clinical and ct imaging features of the COVID-19 pneumonia: focus on pregnant women and children

Proposal of a low-dose, long-pitch, dual-source chest ct protocol on third-generation dual-source ct using a tin filter for spectral shaping at 100 kVp for CoronaVirus Disease 2019 (COVID-19) patients: a feasibility study

National Health Commission of the People's Republic of China. Diagnosis and protocol of COVID-19

Correlation of chest ct and rt-pcr Testing in coronavirus disease 2019 (covid-19) in China: a report of 1014 Cases

Diagnostic performance of chest ct to differentiate covid-19 pneumonia in non-high-epidemic area in Japan

Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct

A Fully Automatic Deep Learning System for COVID-19 Diagnostic and Prognostic Analysis. medRxiv, 2020, to be published

Research progress of computer-aided classification and diagnosis of cancer based on deep learning

Imagenet classification with deep convolutional neural networks

Going deeper with convolutions

Deep residual learning for image recognition

Research progress on the theory and algorithm of isomorphic transfer learning

Migration nearest neighbor propagation clustering algorithm

Ensemble learning method for imbalanced data based on sample weight updating

An ensemble learning method for wireless multimedia device identification

GA-SVM based feature selection and parameter optimization in hospitalization expense

A novel image classification approach via Dense-MobileNet models. Mobile Information Systems