key: cord-0925814-ahtak2or
authors: Deep Deb, Sagar; Kumar Jha, Rajib; Jha, Kamlesh; Tripathi, Prem S
title: A multi model ensemble based deep convolution neural network structure for detection of COVID19
date: 2021-09-03
journal: Biomed Signal Process Control
DOI: 10.1016/j.bspc.2021.103126
sha: 5d0fc9a9f377917ba4a85a1d66b966ef0f0b3d31
doc_id: 925814
cord_uid: ahtak2or

The year 2020 will certainly be remembered for the COVID-19 outbreak. First reported in Wuhan city of China back in December 2019, the number of people getting affected by this contagious virus has grown exponentially. Given the population density of India, the implementation of the mantra of the test, track, and isolate is not obtaining satisfactory results. A shortage of testing kits and an increasing number of fresh cases encouraged us to come up with a model that can aid radiologists in detecting COVID19 using chest Xray images. In the proposed framework the low level features from the Chest X-Ray images are extracted using an ensemble of four pre-trained Deep Convolutional Neural Network (DCNN) architectures, namely VGGNet, GoogleNet, DenseNet, and NASNet and later on are fed to a fully connected layer for classification. The proposed multi model ensemble architecture is validated on two publicly available datasets and one private dataset. We have shown that our multi model ensemble architecture performs better than single classifier. On the publicly available dataset we have obtained an accuracy of 88.98% for three class classification and for binary class classification we report an accuracy of 98.58%. Validating the performance on private dataset we obtained an accuracy of 93.48%. The source code and the dataset are made available in the github link https://github.com/sagardeepdeb/ ensemble-model-for-COVID-detection.

The 2019 novel Corona Virus Disease was first observed in the Wuhan city of the Hubei province of China in December last year. This rare novel virus belongs to the family of "Coronavirus" (CoV) and was called Severe Acute Respiratory Syndrome Coronavirus2 (SARS-CoV-2). On February 2020 it was named COVID-19 by the World Health Organization (WHO). As on 20th August during the time of writing this manuscript, 22.6 million people have already been infected by this virus (according to WHO updates on and 792,000 individuals have already scummed to this deadly virus.

The WHO declared the outbreak of a pandemic in March 2020. Since then we have seen the number of fresh cases going up exponentially. The virus has engulfed more than 210 countries and among them, the USA, Brazil, and India being the worst affected. As of 20th August, the USA alone has reported 5.61 million cases while 3.5 and 2.91 million cases were reported from Brazil and India respectively. A COVID-19 infected patient may develop symptoms like fever, cough, and respiratory illness. Some serious patients may experience pneumonia and (or) difficulty in breathing, multiorgan failure and death [1] . The radiographic findings of established COVID-19 pneumonia are bilateral multi focal, randomly scattered ground glass opacity present mainly in the peripheral sub pleural region with thickened pulmonary interstitium. The interstitium shows Broncho-vascular prominence and consolidation in moderate to severe individuals [2] . Many countries' health systems are on the verge of collapsing due to the rapid increase in COVID-19 incidents. Most of them are currently experiencing a scarcity of ventilators and testing kits. There is enough data to suggest that the R0 value which is a parameter to measures how contagious an infectious disease is higher than the deadly virus outbreak of 1918. The rapid spread of a disease of this kind demands a highly accurate point-of-care COVID19 screening [3] .

According to [4] , a report published in the Lancet, isolation, testing, contact tracing, and physical distancing increases the effect on reducing transmission of SARS-CoV-2 to a great extent. But the unavailability of testing kits hampers the process of large scale testing and thus contact tracing becomes nearly impossible. Moreover, the RT-PCR test, which is regarded as the gold standard screening process for detection of COVID-19 is not real-time.

Given the availability of X-Ray machines in all primary health care centers, detection of COVID-19 from chest X-Ray images will surely boost up the testing process, and thus at the end of the day more tests can also be performed. According to [5] and [6] , reports published in Radiology Journals, Chest X-Rays can be useful in detecting COVID-19. With the success of Deep Learning models in classification and detection, we can make use of them to help the radiologists in detecting COVID-19 from CXR images. And again the deep learning model can also deliver results in almost no time, thus making the system fast and robust at the same time.

Researchers all around the world are working day and night to counter this pandemic. Researchers of image processing and biomedical engineering fields are also not left behind. Every other day they are coming up with models based on deep learning architectures to detect COVID-19. Researchers in the image processing field have used either CT-Scan or ( proposed multi-objective differential evolution (MODE) based DCNN structure to identify COVID-19 infected patients by looking at their chest CT images. They obtained an accuracy of about 93%. Maghdid et al. [8] also used CT-Scan images for detection and classification of the COVID-19. They have introduced an AI-based mobile application that reads the smartphone sensors signal measurements and looks into the CT-Scan images to tell about the severity of pneumonia in the patient. In [9] the researchers have used Inception migration-learning model to detect COVID19 from 453 CT images and achieved an accuracy of 73.1%. Li et. al. [10] have also used Chest CT images for the detection of COVID19. An AUC (Area Under the ROC Curve) of 0.96 is reported by them. [11] have used an hybrid 3-D and full 3-D models, both based on Densenet-121 architecture to detect COVID-19 from CT images. They have obtained an accuracy of 90.8%. On CT images, it's worth noting that nearly half of patients with COVID-19 infection have a regular CT scan if screened soon after the onset of symptoms. [12] . Moreover the radiation dose, which is one of the important health concern, is much higher for CT scan in comparison to the X ray imaging. This is especially important for pregnant women and children who are much susceptible to high radiation dose [13] . Taking all these facts into consideration we have used CXR images for our experimentation.

Wang et al. [14] has proposed COVID-Net for classification of Chest Radiology images into four classes namely-normal, COVID-19, pneumonia (bacterial), and pneumonia (viral). They have proposed a tailored network and is one of the first open source designs for COVID-19 detection from Chest X-Ray(CXR) images. They obtained an accuracy of almost 83.5%. COVIDX-net [15] based on the Xception [18] model for classification of CXR into COVID-19, normal and pneumonia classes. They are also the first researchers to consider three different classes for pneumonia. Their 4-class classification obtained an accuracy of 89.6%, 3-class classification obtained 95% and the binary class-classification obtained a whooping 99% accuracy.

Developing a fast and accurate testing mechanism, which can assist the radiologists in detecting COVID-19 from Chest X-Ray images is our motivation. The CheXNet [19] , proposed by Rajpurkar et. al. for classification of Chest X-ray images into 14 different diseases also served as a motivation. The key contributions are listed below • A multi-model ensemble based DCNN structure is proposed to extract the low-level features from the CXR images and concatenate them before classification. The DCNNs used for building the proposed model namely VGGNet, GoogleNet, DenseNet, and NASNet are the most common DCNN architectures in literature. We have proved that employing such a scheme enhances performance.

• Till date most of the work presented in literature uses a very less number of test images for COVID-19 class. We aim to use a little large number of images for testing. For that the test images of the COVID-19 class in Kaggle dataset is added to to the original Cohen el. al. dataset. More details about this are presented in the next section.

• The performance of the proposed multi model ensemble architecture is validated on a private dataset collected from MGM Medical College and hospital.

• Lastly, to investigate the concept of transfer learning and to provide a competitive analysis with the state of the art model for COVID-19 detection.

III. DATASET DESCRIPTION Since the outbreak of COVID19 researchers has uploaded lot of clinically proven CXR dataset on Internet. For our experimentation we have used three different datasets. Cohen et. al [20] The dataset was acquired on 20th April 2020. Along with chest X-rays for COVID-19 it also contains examples from Community Acquired Pneumonia (CAP) and normal classes. Figure 1 shows some of the images from all three classes. The dataset was partitioned and compared as given in [14] .

Kaggle It is observed that in Cohen et. al. dataset the number of images in train/COVID-19 and test/COVID-19 is very less as compared to CAP and normal classes. So we have used the images from Kaggle's chest-xray-covid19-pneumonia dataset. All the images in the test and train section of COVID-19 are added to the original dataset for our experimentation. This is just done to increase the number of COVID-19 examples in train and test set. The number of images used for train, test and validation are given in Table I . For train/COVID-19 we have used 175 images from Cohen et. al. and 460 were used from Kaggle. Test 1 contains the number of samples from each category used for the first part of the experimentation.

Private Dataset The private dataset is prepared at MGM Medical College and hospital, Indore. It was retrospectively collected from patients seen at Radiodiagnosis center of MGM Medical College and hospital for the period of one week. Each images were reviewed by the radiologists and the ground truth was proved with biopsy. This small collection of private dataset is collected just to validate the performance of ensemble model trained using the datasets explained above. The dataset has three classes and the number of images are given in Table I against the column Test 2. A re-sampling approach known as under-sampling is used here since classifiers are often biased towards the majority class. Under-sampling involves randomly deleting instances from majority classes until the dataset is equal. This is done to remove the class imbalance present in the dataset.

For our experimentation we are using two publicly available dataset, one given in Cohen et. al and the other in Kaggle and Fig. 3 : Input lung image before and after pre-processing one private dataset. Zero mean Normalization is applied to get rid of the difference in lighting between them. Zero mean Normalization makes all pixel values between 0 and 1. This helps in fastens up the convergence of the model to be trained [21] . A sample input lung image before and after zero mean normalization is shown in figure 3 .

With merely 12,552 images for training (4836 for CAP, 7081 for normal and 635 for COVID-19), it will not be judicious to train a network from scratch. So we have used a technique called Transfer Learning. Transfer Learning is a process of re-using a network which have been trained for some other purpose on a dataset which is considerably large for our own use. The proposed architecture as shown in figure 2 uses four pre-trained network for extracting low level features from the input images. The four pre-trained networks namely -VGGNet [22] , DenseNet [23] , NASNet [24] and GoogleNet [25] and are trained on Imagenet [26] dataset.

It is proved that the best approach for various image classification and detection tasks is to create ensembles of multiple models. In [27] , the authors have used ensemble models for melanoma classification. In [28] , the authors proposed a bistream network, comprising two different sub branches of two network structures, namely ResNet50 and VGG16 for object detection and recognition. It is done with an aim to take advantage of rich semantic information present in their proposed [29] for salient object detection. In [30] , for solving the problem of object detection, the authors have successfully used VGG16 as the backbone architecture. Similarly for detection of objects from video sequence [31] proposed a novel plugand-play scheme to weakly retrain a pretrained image saliency deep model for video data by using the newly sensed and coded temporal information.

Initially individual DCNN structures are used for feature extraction. The extracted features are provided to a fully connected layer for final classification. Later on, we have used the ensemble of the four most common DCNN structures for feature extraction. The ensemble uses VGGNet, DenseNet, GoogleNet and NASNet which are the state-of-the-art Deep Convolutional Neural Networks and have performed significantly in object classification and recognition. As shown in figure 2 , the low-level features from the input images are extracted using an ensemble of the four pre-trained DCNNs. The details of the parameters and the architecture are given in table II .We have observed that employing such scheme enhances the classification performance. A brief description of all the four DCNNs used in our multi-model ensemble structure is given below. 1) VGGNet: With an architecture similar to that of AlexNet [32] , Visual Geometry Group Network [22] won the ImageNet LSVRC challenge in the year 2013. It uses 3 x 3 window size filter and 2 x 2 pooling network. The 3 convolution layer deep network performs better as compared to AlexNet, due to its simple architecture [33] .

2) DenseNet: Introduced in the year 2018 by Huang et al. [23] densely connected Covolutional Network connects each layer in a network to every other layer in a feed-forward style. This revolutionary work made possible to design more deeper and more accurate Convolutional Neural Network.

3) GoogleNet: A 22 layer deep network won the 2014 edition of ImageNet LSVRC challenge. This network revolutionized the idea of Deep Learning. With the introduction of inception model the networks could be more deeper and at the same time containing lesser parameter. This network contain 12 times lesser parameters than AlexNet [25] , [32] 4) NASNet: Introduced by Google Brain in the year 2018, the idea of transferability was first proposed by them [24] . The best convolutional layers, dubbed as cells, for solving the classification problem on the CIFAR10 dataset were found, and several stacks of those cells were used to solve the ImageNet [26] classification problem.

The ensemble model used for our experimentation is shown in 2 and the details are provided in table III. Transfer Learning scheme is employed to overcome the problem of overfitting as the training data is not sufficient. The experiments were implemented in Python using Keras package with Tensor-flow as a backend framework.

We implemented two scenarios using the proposed model to detect COVID-19 from CXR images.

In the first scenario the model is trained and validated using the images given in datasets obtained from Internet. Initially a three class multi-model ensemble architecture is trained. It classifies CXR images into three classes namely-Community-Acquired Pneumonia (CAP), Normal, and COVID-19. The number of images used for training is given in the Later on the same model is modified for binary classification. In binary classification the same model is retrained to classify the input CXR images into COVID and non-COVID cases.

2) Second Scenario: The efficiency of the ensemble model trained is evaluated on a private dataset obtained from MGM Medical College and hospital, Indore. Both three class and binary classification are performed. The number of images used to test the model is given in Table I against the column  Test 2. Both the scenarios was performed in Google Colaboratory which is equipped with a Tesla K80 graphics card. The training progress for the first experiment is shown in figure 4 .Adam optimizer with a learning rate of 0.0001 is used for both the cases. Early stopping was applied for fast implementation. 

The entire experiment is divided into two scenarios as given below.

The ensemble model for three-class classification achieves an accuracy of 88.92%. A comparison with the individual DCNNs in terms of accuracy is given in table III. The performance of the ensemble model is also compared with the two recent networks, namely ResNet [34] and ResNeXT [35] .The performance comparison of the proposed ensemble model is also done on dataset with and without the preprocessing steps as mentioned in section IV A. The results for the same are shown in table IV. The 95% confidence interval of the proposed model on three-class classification is 88.95% ± 1.52%. Similarly, the 95% confidence interval of the model on two-class classification is 98.58% ± 0.57%.

Precision, Recall and F-measure are the top metrics used to measure the performance of any classification algorithms. The class wise measures for the three-class classification are presented in table V . Training progress for the same is shown in figure 4 .

After carefully analyzing the results and after discussion with the experts we conclude that there is a confusion between the CAP and COVID-19 class. So in the second part of the experiment the proposed multi model ensemble structure is retrained for binary classification. In case of binary classification the input CXR images are classified either into COVID-19 or non COVID-19 class. For that the normal and CAP class are merged together to form non-COVID19 class. An accuracy of 98.58% is obtained. Class wise precision, recall and F-score measures are presented in Table VI. The results obtained by our model are superior to other models as most of the models are tested on a very less number of COVID-19 examples. A comparison with other models is given in Table IX . Narin et. al [36] used ResNet50 architecture for the detection of COVID-19. Though they have obtained an accuracy of 98% in binary classification but only 10 COVID data were used for testing their model. Similarly, Asif et. al. [17] , proposed Xception based CoroNet, used only 29 images both for binary and three class classification. Ozturk et. al. [37] proposed a darknet based COVID-19 detector but used only 127 images in total. They performed a 5-fold crossvalidation procedure. Sethy and Behra [16] performed binary classification using a DCNN based on ResNet50. They have [20] and rest 116 from Kaggle. The model proposed is more robust as it is tested on two publicly available dataset and one private dataset.

The second scenario is about classifying the dataset we have acquired from MGM Medical College, Indore. As the dataset is small so we have avoided retraining the model. Rather we have used the model trained in first scenario to test the private dataset. As far as three class classification is concerned we have obtained an accuracy of 93.48% . The 95% confidence interval of the proposed model on three-class classification is 93.48% ± 5.04%. Similarly, the 95% confidence interval of our model on two-class classification is 95.65% ± 4.16%. Precision, Recall, and F-measure is given in table VII. As shown in table VII, the precision for COVID-19 is 100%. This signifies that no patient is mis-classified into COVID-19 class. This is very important as this model has guaranteed that patients having pneumonia or are completely healthy is not mis-classified as COVID-19. Recall for normal class is also reported to be 100%. Significance of having cent percent recall of normal class is that no healthy patients are classified as COVID-19 or CAP. For classifying the input CXR images into COVID-19 and non COVID-19 class, we have merged the images from CAP and normal class into non COVID-19 class. An accuracy of 95.65% is obtained.

With the increasing number of cases, it is important to ensure that no single COVID-19 patient goes undetected. To make this possible we need to have a testing mechanism which is not only accurate but also real-time. The proposed method,which is based on four pre-trained DCNN structures can assist the radiologists to have a deeper understanding of the critical aspects related to COVID-19. The model achieved promising results on the prepared dataset and we strongly believe that once more training data becomes available, the accuracy will go up.

All the due ethical issues have been addressed and taken care properly. There is no ethical violation in this work.

The source code, dataset can be obtained from the following github link https://github.com/sagardeepdeb/ ensemble-model-for-COVID-detection

Coronavirus: covid-19 has killed more people than sars and mers combined, despite lower case fatality rate

Rational and practical use of imaging in covid-19 pneumonia

Covid-xpert: An ai powered population screening of covid-19 cases using chest radiography images

Effectiveness of isolation, testing, contact tracing and physical distancing on reducing transmission of sars-cov-2 in different settings

Chest ct findings in coronavirus disease-19 (covid-19): relationship to duration of infection

Chest ct for typical 2019-ncov pneumonia: relationship to negative rt-pcr testing

Classification of covid-19 patients from chest ct images using multi-objective differential evolution-based convolutional neural networks

A novel aienabled framework to diagnose coronavirus covid-19 using smartphone embedded sensors: Design study. arxiv

A deep learning algorithm using ct images to screen for corona virus disease

Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct

Artificial intelligence for the detection of covid-19 pneumonia on chest ct using multinational datasets

Essentials for radiologists on covid-19: an updateradiology scientific expert panel

Comparison of effective radiation doses from x-ray, ct, and pet/ct in pediatric patients with neuroblastoma using a dose monitoring program

A tailored deep convolutional neural network design for detection of covid-19 cases from chest radiography images

Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images

Detection of coronavirus disease (covid-19) based on deep features

Coronet: A deep neural network for detection and diagnosis of covid-19 from chest x-ray images

Xception: Deep learning with depthwise separable convolutions

Chexnet: Radiologistlevel pneumonia detection on chest x-rays with deep learning

Covid-19 image data collection: Prospective predictions are the future

Benign and malignant classification of mammogram images based on deep learning

Very deep convolutional networks for large-scale image recognition

Densely connected convolutional networks

Learning transferable architectures for scalable image recognition

Data-level recombination and lightweight fusion scheme for rgb-d salient object detection

Going deeper with convolutions

Imagenet: A large-scale hierarchical image database

Solo or ensemble? choosing a cnn architecture for melanoma classification

A deeper look at image salient object detection: Bi-stream network with a small training dataset

Stage-wise salient object detection in 360 omnidirectional image via object-level semantical saliency ranking

A plug-and-play scheme to adapt image saliency deep model for video data

Imagenet classification with deep convolutional neural networks

A novel deep learning based framework for the detection and classification of breast cancer using transfer learning

Deep residual learning for image recognition

Aggregated residual transformations for deep neural networks

Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks

Automated detection of covid-19 cases using deep neural networks with x-ray images

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Intra-model variability in covid-19 classification using chest x-ray images

Covid-19 detection from chest x-ray images using ensemble of cnn models