key: cord-0691350-co3u6r6m
authors: Kumar, N.; Gupta, M.; Gupta, D.; Tiwari, S.
title: Novel deep transfer learning model for COVID-19 patient detection using X-ray chest images
date: 2021-05-15
journal: J Ambient Intell Humaniz Comput
DOI: 10.1007/s12652-021-03306-6
sha: 4b0f252fc3ec9b3ef85ea8feb787e720b8362407
doc_id: 691350
cord_uid: co3u6r6m

Around the world, more than 250 countries are affected by the COVID-19 pandemic, which is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This outbreak can be controlled only by the diagnosis of the COVID-19 infection in early stages. It is found that the radiographic images are ideal for the fastest diagnosis of COVID-19 infection. This paper proposes an ensemble model which detects the COVID-19 infection in the early stage with the use of chest X-ray images. The transfer learning enables to reuse the pretrained models. The ensemble learning integrates various transfer learning models, i.e., EfficientNet, GoogLeNet, and XceptionNet, to design the proposed model. These models can categorize patients as COVID-19 (+), pneumonia (+), tuberculosis (+), or healthy. The proposed model enhances the classifier’s generalization ability for both binary and multiclass COVID-19 datasets. Two popular datasets are used to evaluate the performance of the proposed ensemble model. The comparative analysis validates that the proposed model outperforms the state-of-art models in terms of various performance metrics.

COVID-19 was declared an epidemic by World Health Organization (WHO) on 18 April 2020, since this disease was highly infectious and was spreading rapidly across the world (Shereen et al. 2020) . It is still affecting the world with severe consequences. A severe acute respiratory syndrome, coronavirus 2 (SARS-CoV-2), is the cause of this disease and includes symptoms like cough, fever, dizziness, breathing shortness, and acute respiratory distress (Huang et al. 2020; .

Non availability of any medical treatment requires the detection of COVID-19 in early stages to control further spread and to treat the patient timely. For early classification of COVID-19 infection, numerous approaches have been proposed by researchers using radiological images like chest X-rays and CT-scans. These images can help in the early diagnosis of COVID-19 patients Singh et al. 2021b ) along with the severity assessment (Ng et al. 2020) , (Das et al. 2020b ). However, X-ray imaging is better than CT scanning due to low costing, availability of machines in hospitals, and portability of machines in ICUs and field hospitals (LINDA 2020; .

Recently deep learning models have been extensively used in the field of biomedical image processing and have proven their efficacy in classification of numerous diseases such as respiratory distress, tuberculosis, pneumonia, etc. (Jung et al. 2018; Kaur et al. 2019 Girdhar et al. 2021; Yeung et al. 2019) . Inspired from this, in this research work, deep learning models are considered for COVID-19 detection from x-ray images.

Due to the nonavailability of a large number of images corresponding to COVID-19 compared to other diseases, it is quite challenging to develop effective techniques for the detection of COVID-19 from chest X-rays. However, the researchers have proposed many techniques with the help of available data, such as CNN Darknet (Ozturk et al. 2020; , VGG19 (Simonyan and Zisserman 2014; Gianchandani et al. 2020) , MobileNet v2 (Howard et al. 2017; Aggarwal et al. 2021) , Inception (Szegedy et al. 2017 ), XceptionNet (Rahimzadeh and Attar 2020b; , etc.

In the image classification process, large and properly labeled dataset help in extracting the features in a better way. They provide much data to train the model and provide properly differentiated classes as results. When the model is trained on such heterogeneous data, they can be used universally irrespective of the discipline. Such models also help in learning and representing different classes of medical data, which is limited, unbalanced, and highly prone to generalization errors. The concept of transfer learning enables us to use pre-trained models on larger datasets (e.g., ImageNet) with the help of well-defined class boundaries. It also helps in getting classification results at faster speed. Therefore, very helpful to design an early diagnosis of COVID-19 suspected cases. These features have motivated us to incorporate transfer learning in the proposed ensemble model.

The main novelties of this paper are described below:

i.

For chest X-ray images, an ensemble model is developed for early detection of COVID-19 infection. ii. The proposed model is also able to classify the suspected patient as infected from pneumonia, tuberculosis, or as a healthy person. iii. The ensemble model utilize pre-trained models to extract the potential features and classify these features using pre-trained models. iv. The proposed model is applied on two popular datasets. v. Comparison among the proposed model and the competitive models is carried out by considering various performance metrics like accuracy, area under curve, f-measure, precision and recall.

The remaining paper is categorized as: The related work is discussed in Sect. 2. The preliminaries are described in Sect. 3. Section 4 presents the proposed model for early diagnosis of COVID-19 suspected cases. Comparative analysis are presented in Sect. 5. The proposed model is concluded in Sect. 6. Dadário et al. (2020) proposed a three-dimensional deep learning model for the diagnosis of COVID-19. 4356 chest CT scans were used to validate the performance of the proposed method. The experimental results have shown better sensitivity and high specificity for the detection of COVID-19 infection. An algorithm is developed for the classification of medical images. It has utilized transfer learning and its performance has been compared with various existing systems based on CNN. Two different datasets have been used i.e., one with 1427 X-ray images in which 224 images with confirmed COVID -19 cases. In the second dataset, 1442 X-ray images were used, out of these 224 images are of confirmed COVID -19 cases. It has been found that the deep learning with CNNs is very effective for the detection of COVID-19 from X-ray images (Apostolopoulos and Mpesiana 2020).

A deep learning-based CNN model, i.e., truncated InceptionNet, was proposed. Chest X-rays were used to classify the different infections such as COVID-19 positive, pneumonia positive, tuberculosis positive and healthy cases. The truncated InceptionNet has achieved an accuracy of 99.96% (Das et al. 2020a) . A system has been developed for the classification of chest CT images to detect the COVID-19 infection. Initial parameters of CNN were tuned with the help of differential evolution. It has been found that the proposed model achieved an accuracy as 98.24 % . A deep CNN based system has been designed for COVID-19 detection. Projectionexpansion-projection (PEP) patterns were also used. A total of 13,975 chest X-ray images from 13870 patients have been used. A dataset named COVIDx was collected by using the integration of five different public datasets. Classification accuracy of 93.3% and sensitivity of 91% has been achieved .

A model has been developed for the segmentation of COVID-19 infected regions. Deep learning based VB-Net model was proposed. VB-Net has shown significant performance for the detection of COVID-19 infected regions (Shan et al. 2020 ). An automatic CNN model has been proposed for the detection of COVID-19. Experiments were also drawn on multiclass (COVID vs. No-Findings vs. Pneumonia) classification dataset. DarkNet classifier was utilized for YOLO architecture. 17 convolutional layers have been used. The DarkNet model has shown significant results in the initial screening of the patient (Ozturk et al. 2020 ).

We propose an ensemble model which is designed by collaborating three different transfer learning models, i.e., EfficientNet, GoogLeNet, and XceptionNet. Such ensemble leads to a powerful approach and provides better results with reduction in errors. The proposed model uses CNN, pretrained transfer learning model and ensemble learning to achieve better diagnostic results.

CNN models have shown better performance in various applications such as agriculture, industry, and diagnosis of medical diseases (Rahimzadeh and Attar 2020a; Ghosh et al. 2020; Dekhtiar et al. 2018) . The architecture of CNN imitates the visual cortex system of humans (Majeed et al. 2020; Basavegowda and Dagnew (2020) ). CNN architecture is shown in Fig. 1 . It consists of three layers, wherein the first layer is named as convolution layer, the second layer as pooling layer, and the last layer is so-called fully connected layer (Guo et al. 2017; Gupta et al. (2019) ). The convolution and pooling layers are responsible for learning of the model and a fully connected layer does the classification.

It is often difficult to obtain a large dataset in the medical imaging field and as stated earlier, the data for COVID-19 is even lesser, depth models cannot provide the desired results for small datasets (Ozturk et al. 2020; Wang et al. 2019) , Osterland and Weber 2019) . The deep learning models require a larger dataset to train the model. For smaller datasets, deep learning models suffer from the overfitting issue. All these problems can be resolved by using transfer learning (Zhou et al. 2020 ; Wiens 2019) model. In transfer learning, pretrained models are utilized. These pretrained models were trained on different datasets with a large volume of images. Transfer learning solves the problem of data training by transferring the existing knowledge in the target field where very less or no sample data is available. The use of transfer learning also facilitates data training with less model building cost.

EfficientNet (Tan and Le 2019) was introduced to deal with the scalability issues of CNN. CNNs are required to be scaled up width wise and depth wise to provide better accuracy. However, this scaling leads to an increase in the training and testing time cost. EfficientNet resolves the scaling problem with the use of a compound scaling method which scales the network with a fixed ratios of all dimensions to make it wider, deeper, and to provide high resolution (see Fig 2) . This type of scaling provides better accuracy and performance. The model comprises of eight models named from B0 to B7, with B0 representing the most compact and B7 representing the most scaled configuration of EfficientNet.

GoogLeNet (Szegedy et al. 2015) evolves in nine inception modules and has twenty-two concealed layers. The inception modules enable to select from the available filter size in each block. From a previous input layer having different size (i.e., 1 × 1, 3 × 3, and 5imes5), three convolution kernels, a feature extraction process is carried out at different scales and further passed to the next layer as depicted in Fig. 3 . The overfitting and gradient vanishing problems are overcome by making three groups of inception modules and by adding three objective functions for each group.

Xception (Chollet 2017 ) is a deep CNN which stretches the inception concept to extremes. It can be thought of as an extension to the Inception architecture. It introduces new inception layers which are created firstly by depth-wise convolution layers and then from a point-wise convolution layer as shown in Fig. 4 . The architecture Fig. 1 Architecture of convolutional neural networks resembles a linear stack of convolution layers which are separable depthwise. Being separable, depthwise separates the cross-channel and spatial features' learning. It also reduces the memory requirements and computational cost. It has thirty-six layers concealed in fourteen modules. The modules except the first and last have linear residual connections.

Ensemble learning (Chen et al. 2018 ) is a method of combining various deep learning models to obtain an ensembled predictive model. This process of combining several techniques is believed to reduce bagging (i.e., variance), boosting (i.e., bias), and enhances stacking (i.e., prediction). It also enhances the learning system's generalization ability. In ensemble learning, base classifiers can be generated in two ways (i) the data set is the same, but the learning algorithms are different, (ii) different data sets are used with the same learning algorithm. In the former case, a heterogeneous classifier is obtained and in the latter case, a homogeneous classifier is obtained. The purpose of using ensemble learning is to prevent overfitting by the combination of different methods. If ensemble learning is being used for classification, then the decisions from multiple models are combined and voted so that the final result can be obtained. Voting can be relative or absolute.

This section presents the steps followed to design the proposed ensemble model to classifying COVID-19 suspected cases. The flow of the work is represented in Fig. 5 . The MBConv block is an Inverted Residual Block (used in MobileNetV2) with a squeeze and excite block injected sometimes. Novel deep transfer learning model for COVID-19 patient detection using X-ray chest images 1 3

Step 1: Initially, multiclass classification dataset is loaded.

Step 2: The partitioning of the obtained dataset of chest x-ray images referred as CXR S ample is then achieved. The size of sample is |CXR_Sample| = 9300 . CXR_Sample is divided into four subsets depending on the type of Step 3: Tend (10)-fold crossover is implemented on four subsets created in Step 2 to obtain the training and testing sample sets. To obtain the 10-fold crosssample set, a partition algorithm is used which divides each sample subset in 10 uniform parts.

Step 4: Pre-trained models are used to train the network to generate individual classifiers. EN_SFX, GN_SFX , and Xception_SFX represent the softmax function of EfficientNet, GoogLeNet, and Xception, respectively.

Step 5: Obtain the individual classifiers by training EN_SFX, GN_SFX and Xception_SFX using the training sample set CXR_Sample_TrainingSet as follows:

Step 6: Implement ensemble learning to obtain the resultant classifier (Ensemble_EGX) by integrating the above three classifiers and by applying the relative voting.

The proposed model is compared with several transfer learning models like ResNet152V2, VGG16, DenseNet201, and InceptionResnetV2 to validate its performance. The preceding subsection discusses the dataset and comparative analysis.

Two popular datasets have been utilized for experimental purpose. Dataset 1 is obtained from Kaggle dataset resource (Gianchandani et al. 2020) . This dataset comprises of the X-ray images of pneumonia, tuberculosis, COVID +ve and COVID -ve patients. For binary classification, COVID +ve and COVID -ve images are used from dataset 1.

Dataset 2 is obtained form dataset (Mporas and Naronglerdrit 2020), Qatar University, and University of Dhaka (Chowdhury et al. 2020) . The size of the dataset is |CXR_Sample| = 9300 . CXR_Sample is divided into four subsets depending on the type of image as CXR_Healthy , CXR_Tuberculosis , CXR_Pneumonia , and CXR_COVID each having size as 2400, 2350, 2375, and 2175, respectively. For multiclass classification, dataset 2 has been used. Figure 6 shows the sample image datasets 1 and 2.

Firstly, resizing (224×224× 3 RGB) of the X-ray images is achieved. The lack of data availability is dealt with the use of transfer learning wherein pretrained models on larger datasets are reused. Data augmentation is also used to achieve the better generalization. Data in proportional order is favorable for a neural network since it has millions of parameters. Data augmentation is achieved using horizontal and vertical flipping, sheer transformation (using a slant angle of 0.2), and 45 o degree rotation, the validation of the model is done by a variety of inputs, therefore, the data augmentation is also done on the validation dataset. Image normalization is also done to achieve better convergence rate while network training and to get the data in [0, 1] range. This normalization is done with the division of images using 255. Finally, the dataset is divided into training, validation, and testing purposes. 15% of the actual dataset is used as testing data. From the remaining 85%, 17% is used for validation and 68% is used for training the proposed model. Maximum data is utilized for training purpose because the learning and weight assignment of the model is done using the training data. The results may differ if even slight variations (increased/ decreased) are done in the training data proportions.

The quantitative metrics based on confusion matrix are considered for testing the performance of the proposed model. These metrics are accuracy, precision, recall, and f1-Score.

The comparison between the proposed and competitive models like ResNet152V2, VGG16, DenseNet201, and 

For training purposes, the base models took ten minutes approximately. For binary classification, ResNet152V2 and VGG16 both attained the best accuracy value, however, considering the criticality of COVID-19, efforts can be made for enhancing the sensitivity and precision scores. Hence, the ensemble model used for multiclass diagnosis is introduced. The proposed model outperformed the basing of models with 96.15 % accuracy, which is approximately 1.2% higher. It also obtained a precision value as 0.959, which implies the correctness of the predicted results. It can be noted from the results that a high specificity rate is obtained using the proposed model indicating no falsepositive predictions. High specificity makes the system more reliable. It helps the health care system in using the testing kits correctly and providing facilities and kits to people who are in genuine need.

It is observed from the experimental results that the proposed model significantly provides a rapid solution with low cost for COVID-19 detection using chest X-ray images. The state-of-the-art models' performance along with the proposed model is presented in Table 3 . The non-availability of data to train the model results in models which are not good in generalization. To deal with this problem, we have used transfer learning which reuses the models trained on a larger dataset. We also have tried to get as minimum as possible false predictions. Keeping in view the statistical metrics shown in Table 2 , better generalization, better accuracy, and less false predictions are achieved with the proposed model by outperforming state-of-the-art models.

For multiclassification, the state-of-the-art models' performance along with the proposed model is presented in Table 4 . Table 4 demonstrates the comparative analysis of the proposed model for multiclass dataset. The proposed ensemble model outperforms other techniques with higher accuracy for multiclassification as well.

A deep transfer learning-based ensemble model was designed by integrating EffientNet, GoogLeNet, and Xcep-tionNet for early diagnosis of COVID-19 infection. The proposed model is capable of detecting COVID-19 as well (Simonyan and Zisserman 2014) 98.75% MobileNet v2 (Howard et al. 2017) 96.78% Inception (Szegedy et al. 2017) 86.13% XceptionNet (Rahimzadeh and Attar 2020b) 85.57% Proposed Ensemble model 98.95% (Simonyan and Zisserman 2014) 93.48% MobileNet v2 (Howard et al. 2017) 94.72% Inception (Szegedy et al. 2017) 92.85% XceptionNet (Rahimzadeh and Attar 2020b) 92.85% Proposed Ensemble model 99.21% as differentiating normal, COVID-19 (+), pneumonia, and tuberculosis infected cases. Two datasets were used to test the proposed model. The proposed model has shown an accuracy of 99.21% for multiclass and 98.95% for binary classification problem, respectively. Hence, the proposed model has emerged as recent solution which can be used by health officials in this critical situation for early diagnosis of COVID-19.

In the near future, the proposed work can be extended in a way that it also predicts the degree of risk and survival chances of the COVID-19 (+) patients, which in turn will be very helpful for medical practitioners in the management and healthcare planning of infected patients.

Two-way feature extraction using sequential and multimodal approach for hateful meme classification

Correlation of chest ct and rt-pcr testing in coronavirus disease 2019 (covid-19) in china: a report of 1014 cases

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Deep learning approach for microarray cancer data classification

Ensemble learning method for imbalanced data based on sample weight updating

Xception: Deep learning with depthwise separable convolutions

Can ai help in screening viral and covid-19 pneumonia?

Regarding" artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct

Truncated inception net: Covid-19 outbreak screening using chest x-rays

Automated deep transfer learning-based approach for detection of covid-19 infection in chest x-rays

Deep learning for big data applications in cad and plm-research review, opportunities and case study

Graphology based handwritten character analysis for human behaviour identification

Rapid covid-19 diagnosis using ensemble deep transfer learning models from chest radiographic images

Effect of covid-19 outbreak on urban health and environment

Simple convolutional neural network on image classification

Visibility improvement and mass segmentation of mammogram images using quantile separated histogram equalisation with local contrast enhancement

Mobilenets: Efficient convolutional neural networks for mobile vision applications

Clinical features of patients infected with 2019 novel coronavirus in

Classification of lung nodules in CT scans using three-dimensional deep convolutional neural networks with a checkpoint ensemble method

Metaheuristic-based deep covid-19 screening model from chest x-ray images

Fusion of medical images using deep belief networks

Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks

Multiobjective evolutionary optimization techniques based hyperchaotic map and their applications in image encryption

Color image encryption using minimax differential evolution-based 7d hyper-chaotic map

Drug synergy prediction using dynamic mutation based differential evolution

Parallel strength pareto evolutionary algorithm-ii based image encryption

Coronet: A deep neural network for detection and diagnosis of covid-19 from chest x-ray images

A tailored deep convolutional neural network design for detection of covid-19 cases from chest radiography images

Problems of deploying cnn transfer learning to detect covid-19 from chest x-rays

Covid-19 identification from chest x-rays

Imaging profile of the covid-19 infection: radiologic findings and literature review

Analytical analysis of single-stage pressure relief valves

Automated detection of covid-19 cases using deep neural networks with x-ray images

Introduction of a new dataset and method for detecting and counting the pistachios based on deep learning

A modified deep convolutional neural network for detecting covid-19 and pneumonia from chest x-ray images based on the concatenation of xception and resnet50v2

Lung infection quantification of covid-19 in ct images with deep learning

Covid-19 infection: Origin, transmission, and characteristics of human coronaviruses

Very deep convolutional networks for large-scale image recognition

Classification of covid-19 patients from chest CT images using multi-objective differential evolutionbased convolutional neural networks

Densely connected convolutional networks-based covid-19 screening model

Deep neural networkbased screening model for covid-19-infected patients using chest x-ray images

Inception-v4, inception-resnet and the impact of residual connections on learning

Going deeper with convolutions

Efficientnet: Rethinking model scaling for convolutional neural networks

Covid-net: a tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Study on the dynamic and static characteristics of gas static thrust bearing with micro-hole restrictors

Engine speed reduction for hydraulic machinery using predictive algorithms

A computer vision system for deep learning-based detection of patient mobilization activities in the icu

The ensemble deep learning model for novel covid-19 on CT images

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.