key: cord-0812445-rhyxj0yk
authors: Abraham, Bejoy; Nair, Madhu S.
title: Computer-aided detection of COVID-19 from X-ray images using multi-CNN and Bayesnet classifier
date: 2020-09-02
journal: Biocybern Biomed Eng
DOI: 10.1016/j.bbe.2020.08.005
sha: acbe65e5d64c3fc15c0a79302e7ba2bc43715da7
doc_id: 812445
cord_uid: rhyxj0yk

Corona virus disease-2019 (COVID-19) is a pandemic caused by novel coronavirus. COVID-19 is spreading rapidly throughout the world. The gold standard for diagnosing COVID-19 is reverse transcription-polymerase chain reaction (RT-PCR) test. However, the facility for RT-PCR test is limited, which causes early diagnosis of the disease difficult. Easily available modalities like X-ray can be used to detect specific symptoms associated with COVID-19. Pre-trained convolutional neural networks are widely used for computer-aided detection of diseases from smaller datasets. This paper investigates the effectiveness of multi-CNN, a combination of several pre-trained CNNs, for the automated detection of COVID-19 from X-ray images. The method uses a combination of features extracted from multi-CNN with correlation based feature selection (CFS) technique and Bayesnet classifier for the prediction of COVID-19. The method was tested using two public datasets and achieved promising results on both the datasets. In the first dataset consisting of 453 COVID-19 images and 497 non-COVID images, the method achieved an AUC of 0.963 and an accuracy of 91.16%. In the second dataset consisting of 71 COVID-19 images and 7 non-COVID images, the method achieved an AUC of 0.911 and an accuracy of 97.44%. The experiments performed in this study proved the effectiveness of pre-trained multi-CNN over single CNN in the detection of COVID-19.

Coronavirus disease 2019 (COVID-19) is a kind of viral pneumonia which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). It is one among the three respiratory disease outbreak caused by the coronavirus, other two being severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS). As on 7th of May 2020, more than 3.5 million cases of COVID-19 and 250,000 deaths due to the disease have been reported by the World Health Organization (WHO) [1] . WHO has listed COVID-19 as a Public Health Emergency of International Concern (PHEIC) [2] . There is an urgent need for early diagnosis of the disease to prevent further spreading and control the death toll. The gold standard for diagnosing COVID-19 is reverse transcription-polymerase chain reaction (RT-PCR) test [2, 3] . However, the RT-PCR testing facility is inadequate in most of the areas hit by the COVID-19 outbreak [3] . COVID-19 is characterized by a lung infection in the post of the patients [4] . Easily available modalities like Xray and CT can be used for detecting lung infections [4] . It is proven that X-ray and computed tomography (CT) scan can be used effectively for the diagnosis of COVID-19 [5] . However, manual reading of X-ray and CT scan of a large number of patients could be time-consuming. A computeraided diagnosis method could assist the radiologists in predicting COVID-19 from X-ray and CT-scan images [3] . Convolutional neural network (CNN) has shown promising results in the area of computer-aided detection and diagnosis of various diseases. CNN requires a large amount of data for training from scratch. In the case of medical images, it is difficult to obtain a huge number of labelled images. In such cases pre-trained CNNs trained on a large number of natural images like ImageNet can be used [6] . Pre-trained CNNs were earlier used successfully in diagnosis of prostate cancer [7, 8] , breast cancer [9] , brain diseases [10] , leukemia [11] , etc. to name a few. Pre-trained CNN is also found successful in predicting COVID-19 [12] [13] [14] . This paper presents a method for the prediction of COVID-19 using features extracted from multiple pre-trained networks.

The paper is organized as follows. Section 1.1 discusses the related works in the area of computer-aided detection of COVID-19. Section 1.2 describes the contributions of the proposed method. Section 2 describes the proposed multi-CNN, feature selection technique and classifier. Section 3 discusses the results achieved using various combinations of multi-CNN. The section also analyzes the results achieved using various classifiers in comparison with the proposed classifier. Section 4 explains the conclusions reached based on the experimental analysis.

Recently, a number of papers were published in the area of computer-aided detection of COVID-19 using pre-trained CNNs from X-ray and CT images. Shi et al. [3] performed a detailed review of the state-of-the-art computer-aided techniques for the detection of COVID-19 from X-ray and CT scans. Narin et al. [12] The above-mentioned methods use X-ray images for the computer-aided detection of COVID-19. Few recent works also prove the effectiveness of CT scans in the detection of COVID-19. He et al. [18] performed transfer learning on a CT dataset, containing 349 COVID-19 CT scans and 397 normal CT scans. The method proposed a novel transfer learning technology called self-trans that learns features that are robust to overfitting. Mei et al. [19] used two CNN models (one for slice selection and another one for diagnosis) in combination with clinical data to predict COVID-19 using CT scans. Shan et al. [20] proposed a deep learning based scheme that uses 'VB-Net', a modification of V-Net architecture for the segmentation of COVID-19 affected areas in chest CT scans. The method used CT scans of 249 subjects for training and 300 subjects for validation. Chen et al. [21] proposed a Residual Attention U-Net for the segmentation of COVID-19 affected areas in CT scans. The dataset used for the method contains 110 CT images. Chen et al. [22] further proposed a contrastive learning technique to train an encoder for the detection of COVID-19 from CT scans. Fan et al. [23] proposed a model that employs implicit reverse attention and explicit edge-attention for the segmentation of COVID-19 infected areas in CT scans.

In this work, we have chosen to use X-ray for COVID-19 detection as X-ray is cost-effective compared to CT scans.

Contribution of the proposed method

The aforementioned methods have used a single pre-trained CNN to predict COVID-19 from a balanced dataset. The discriminatory features extracted from each pre-trained network will be different. Combination of features extracted from pre-trained CNN is expected to improve the performance of computer-aided diagnosis systems. The proposed method explores a combination of features from multiple CNNs for the diagnosis of COVID-19.

The proposed method has the following contributions. iii. Most of the existing works have used a small dataset whereas the proposed method used a relatively large number of COVID-19 cases.

The method was implemented using two COVID-19 public datasets. The first dataset created by Cohen et al. [24] consists of 560 chest X-ray images. The 560 chest X-ray images were composed of 453 COVID-19 and 107 non-COVID images. 107 non-COVID images consists of either bacterial pneumonia or viral pneumonia. Images with no findings were excluded. 390 chest X-ray images of viral and bacterial pneumonia taken from a Kaggle dataset [25, 26] were added to the 107 non-COVID images to make the dataset a balanced one. The combined dataset (DATASET-1) consists of 950 images of which 453 are COVID-19 and 497 are non-COVID. The second dataset consisting of 71 COVID-19 and 7 non-COVID chest X-ray images was taken from Kaggle [27] . The sub classification of non-COVID images are not available. For performing experimental analysis Dataset-1 was used due to its larger size. To confirm robustness of the method, the results of both Dataset-1 and Dataset-2 are considered. Fig. 1 show sample COVID-19 X-ray images.

The multi-CNN architecture used in the proposed method consists of a set of pre-trained CNN. Each CNN used in the method was pre-trained using Imagenet [28] which has more than a million natural images belonging to 1000 different classes. Different combinations of pre-trained CNNs are used for feature extraction. CNN consists of three basic layers: convolution, pooling and fully connected layers [29] . Convolution layers perform feature extraction by convolving the input image with a set of learned kernels. The layer typically consists of a combination of convolution operation and activation function. The convolution operation between an image I of dimension p  q and a kernel W of size x  y that produces a feature map s is defined by the following dot product:

The output of the convolution layer is then passed through a non-linear activation function. The most common nonlinear activation function used is the rectified linear unit (ReLU) and its variant Leaky ReLu. ReLu is represented as

where s ij kn is the input at location (i, j) on the nth feature map at kth layer. Leaky ReLu is represented as

where s ij kn is the input at location (i, j) on the nth feature map at kth layer and b is the slope of negative linear function [30] .

The pooling layer is used to reduce the spatial resolution of the activation map and thereby reducing the number of parameters. Pooling helps to decrease the computational cost Fig. 1 -The four X-ray images in first row corresponds to COVID-19 and the images in second row corresponds to non-COVID. b i o c y b e r n e t i c s a n d b i o m e d i c a l e n g i n e e r i n g x x x ( 2 0 1 9 ) x x x -x x x and over-fitting. Max-pooling and average pooling are the most common methods of pooling.

Every neuron in the previous layer are connected to a fully connected (FC) layer. Features generated by the previous layer are flattened in a feature vector by the FC layer. It then performs weight updates to improve the predicting ability of feature vector.

Densenet201 [31] , InceptionResnetV2 [32] , Shufflenet [33] , Resnet-101 [34] , Darknet-53 [35] , MobilenetV2 [36] , NasnetLarge [37] , Xception [38] , VGG-19 [39] and Squeezenet [40] are the pretrained networks used for experimental analysis in this work. The input X-ray images used in this study are of different formats. The dimension of input images also varies. The input size of Densenet-201, Shufflenet, MobilenetV2, VGG-19 and Resnet-101 are 224  224 whereas that of InceptionResnetV2 and Xception are 299  299. The input size of Nasnetlarge, Darknet-53 and Squeezenet are 331  331, 256  256 and 227  227, respectively. The size of the X-ray images present in the datasets vary. Before passing the images to the pretrained networks a preprocessing is done to make the size of images uniform and also to replicate the colour channels of the grayscale images in the dataset. The images are scaled using bilinear interpolation to make them compatible with input size of the pre-trained network. Features are extracted from the last fully connected layer of the pre-trained CNNs with 1000 neurons. Each pre-trained CNN produces a feature matrix of size n  1000, where n is the number of X-ray images. The feature matrices of the muti-CNN are combined together to form a feature matrix of dimension n  1000m, where m is the number of pre-trained networks used in the multi-CNN. The best performing multi-CNN of this study used 5 pre-trained CNNs (Squeezenet, Darknet-53, MobilenetV2, Xception, Shufflenet) to produce a feature matrix of dimension 950  5000.

Sample activations using last convolutional layer of Squeezenet, Darknet-53, MobilenetV2, Xception and Shufflenet is shown in Fig. 2. 

Feature matrix of size n  1000m is passed to a feature selection unit for selecting the most distinguishing features. A correlation-based feature selection (CFS) algorithm [41] in combination with subset size forward selection (SSFS), a linear forward selection based search technique [42] was utilized to determine the optimal feature subset. CFS evaluates the merit of a subset of features by considering the individual predictive ability of each attribute along with the degree of redundancy among them. Merit of a subset of features is given by where n is the number of features present in the subset, r fc is the mean feature correlation and r ic is the average value of feature intercorrelation. The numerator of the equation represents ability of a set of features in predicting a class whereas the denominator indicates redundancy among them. After computing the merit of subset of features, SSFS based search is performed. SSFS performs an interior cross-validation to determine the effectiveness of feature subsets. A linear forward selection (LFS) is performed on each fold. To estimate the optimal subset size, scores achieved on the test data for each subset size are averaged and subset size with highest average is chosen. Search terminates at the optimal subset size. Finally, a linear forward selection up to the optimal size of subset is conducted on the whole data. CFS in combination with SSFS reduces the dimensionality of features from n  1000m to n  p, where p is the reduced number of features. The algorithm reduced the dimension of feature vector corresponding to the best performing multi-CNN of this study from 950  5000 to 950  45.

Consider a set of variables, V = m 1 , . . ., m n , where n ≥ 1. A Bayesian network G over a set of variables V represents a network structure G S , and a set of probability tables G P [43] . G S is a directed acyclic graph (DAG) over V.

where paðÞ is the set of parents of in G S . A Bayesian network indicates the following probability distribution [43] .

Let m = m 1 , m 2 , m 3 , . . ., m n be a set of attribute variables. A classifier q : m ! c is a function that maps an instance of m to a class c. For using Bayesian network as a classifier, argmax c P(c|m) is computed using the distribution P(V) [43] .

The feature matrix of dimension n  p is passed to a Bayesnet classifier which classifies the images into COVID-19 and non-COVID categories.

Architecture of the proposed method is shown in Fig. 3 .

The experiments were performed using a core i7, GTX 1060 6GB GPU. Feature extraction was performed using MATLAB 2020a and classification using Weka 3.6. Area under the receiver operating characteristic curve (AUC) along with accuracy are used as the major performance metrics. Precision, recall and Fmeasure in predicting COVID-19 class are used as the auxilliary performance measures.

A ten-fold cross-validation was performed on both the datasets. The results achieved are shown in Table 1 . Confusion matrices corresponding to the results are given in Fig. 4 . ROC curves corresponding to results are shown in Fig. 5 . The method achieved considerable performance in both the datasets. 446 among the 453 instances of COVID-19 cases were classified correctly in Dataset-1, achieving a recall of 98.5%. One except all among the 71 COVID-19 cases were predicted correctly in Dataset-2 achieving a recall of 98.6%. The recall obtained in predicting non-COVID cases were 84.5% and 85.7% respectively, in Dataset-1 and Dataset-2. The method achieved a precision of 

Default parameter settings were used for all the pre-trained networks used for creating multi-CNN. Tables 2 and 3 displays the parameter settings of the feature selection method and Bayesnet classifier in WEKA. The algorithm used to determine the subset size was set to sequential minimal optimization (SMO). The number of crossvalidation folds used for subset size determination was set to 5. The algorithm for determining the conditional probability tables of the Bayes network was set to simple estimator algorithm. Parameter alpha used for determining the conditional probability tables of the Bayes network was set to 0.3. The search algorithm for searching the network structures was set to hill-climbing algorithm, K2.

Selection of search algorithm associated with feature selection technique Three algorithms, SSFS, best first and greedy stepwise were compared to select the best performing one. Best first and greedy stepwise algorithms achieved the same results in combination with CFS. Best performing algorithm among the three was SSFS. Even though SSFS achieved a slightly lower recall compared to other algorithms, it achieved better precision, F-measure, AUC and accuracy than the other two algorithms. The results achieved using the three algorithms are displayed in Table 4 .

Comparison of the proposed pre-trained multi-CNN with single pre-trained CNN Experiments were conducted on features extracted using different combinations of pre-trained networks. The results of various combinations of pre-trained CNNs are shown in Table 5 . The experiments show that combinations of multiple pretrained CNNs outperform that of single CNN. Among the combinations of pre-trained CNNs we experimented with, two multi-CNNs composed of 5 pre-trained networks and three multi-CNN composed of 4 pre-trained networks achieved an AUC above 95% and accuracy above 90%. A multi-CNN which uses a combination of features extracted from 5 different pretrained networks (Squeezenet, Darknet-53, MobilenetV2, Xception, Shufflenet) achieved the best performance. The multi-CNN achieved an AUC of 96.3% and accuracy of 91.1579%. Results achieved using single pre-trained CNNs were less compared to most of the multi CNNs composed of 3 or more pre-trained CNNs. All single pre-trained networks could achieve an AUC of less than 95% and accuracy less than 90% only. The best performing single CNN was MobilenetV2. Even though Mobile-netV2 achieved an AUC of 94.2%, its accuracy was less than 90% only. The experimental analysis proves the efficiency of pretrained multi-CNN over single pre-trained CNN.

Comparison of results achieved using various classifiers Table 6 displays the results achieved using various classifiers in combination with the best performing multi-CNN composed of 5 pre-trained CNNs. Only Bayesnet achieved an accuracy above 90%. All other classifiers could achieve an accuracy below 90% only. Bayesnet, NaiveBayes, LogisticRegression, Random Forest, ADTree and NBTree achieved an AUC above 90%. SVM and AdaBoostM1 could achieve AUC below 90% only. Bayesnet achieved better precision and Fmeasure than the other classifiers. However, the recall achieved by Bayesnet was slightly lower than that achieved by NaiveBayes, SVM and AdaBoostM1. The experimental analysis proves the efficiency of Bayesnet classifier in combination with multi-CNN and correlation-based feature selection technique for the detection of COVID-19.

Comparison with other state-of-the-art methods

Most of the existing methods used a small dataset for classification whereas the proposed method used a relatively large dataset. The method was further tested in a second dataset and achieved promising results in both the datasets. The number of images and validation techniques used by the various state-of-the-art methods are different. Table 7 shows the number of images and validation method used by the various authors. Table 8 displays the results of other major state-of-the-art methods along with the proposed best performing multi-CNN (Squeezenet+Darknet-53+Mobilenet +Xception+Shufflenet). A fair comparison of results is not possible due to the difference in the datasets, performance metrics and the validation techniques. However, it is noteworthy that the proposed method has proven its effectiveness in a relatively large dataset consisting of 453 COVID-19 images. b i o c y b e r n e t i c s a n d b i o m e d i c a l e n g i n e e r i n g x x x ( 2 0 1 9 ) x x x -x x x obtained by partitioning the dataset into 80% training data and 20% test data. Unlike cross-validation, testing on a held-out test dataset does not ensure the robustness of the method.

Even though the method produced good results in a considerably larger dataset, it has few limitations worth mentioning. The method performs classification between COVID-19 and non-COVID X-ray images only. It is not tested in a multi-class classification scenario where the images can be classified as COVID-19, normal and pneumonic. The method does not perform segmentation of the infected region. The combinations of all multi-CNNs are not explored in the paper. It is left to the readers and other researchers to explore more combinations of pre-trained CNNs for the prediction of COVID-19. Being more of images and validation techniques used by the various state-of-the-art methods.

Multi-CNN 453 COVID-19 vs. 497 non-COVID (Dataset-1) 10-fold CV Multi-CNN 71 COVID-19 vs. 7 non-COVID (Dataset-2) 10-fold CV [17] 231 COVID-19 vs. 500 no-findings 10-fold CV [15] 142 COVID-19 vs. 142 normal 70% data for training and 30% testing [12] 50 COVID-19 vs. 50 normal 5-fold CV [13] 250 COVID-19 vs. 250 non-COVID 10-fold CV [13] Training: 250 COVID-19 vs. 250 non-COVID Testing: 74 COVID-19 vs. 36 non-COVID [44] 100 COVID-19 vs. 1431 non-COVID 2-fold CV [14] 25 COVID-19 vs. 25 non-COVID 80% data for training and 20% testing b i o c y b e r n e t i c s a n d b i o m e d i c a l e n g i n e e r i n g x x x ( 2 0 1 9 ) x x x -x x x economical and easily available modality compared to CT scans, the proposed method focused on COVID-19 detection using Xray images. As a future research direction we propose the use of multi-CNN to extract features from CT scans for the detection of COVID-19 and other lung infections.

In this paper, the effectiveness of pre-trained multi-CNN in predicting COVID-19 from X-ray images is investigated. A combination of features extracted from several pre-trained networks in combination with Correlation-based Feature Selection technique and Bayesnet classifier is employed in the method. The best performing multi-CNN used in this study employs a combination of 5 pre-trained CNNs: Squeezenet, Darknet-53, MobilenetV2, Xception and Shufflenet. Results prove the effectiveness of pre-trained multi-CNN over pretrained single CNNs. Experimental analysis performed using two public datasets show that pre-trained multi-CNN in combination with CFS and Bayesnet is effective in the diagnosis of COVID-19.

Bejoy Abraham and Madhu S. Nair: Conception and design of study, acquisition of data, analysis and/or interpretation of data, drafting the manuscript, revising the manuscript critically for important intellectual content, approval of the version of the manuscript to be published. -0.96 -0.95 - [14] 0.83 1.00 0.91 --b i o c y b e r n e t i c s a n d b i o m e d i c a l e n g i n e e r i n g x x x ( 2 0 1 9 ) x x x -x x x

WHO. WHO situation report-108

Molecular immune pathogenesis and diagnosis of covid-19

Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for covid-19

Radiological findings from 81 patients with covid-19 pneumonia in Wuhan, China: a descriptive study

Portable chest X-ray in coronavirus disease-19 (covid-19): a pictorial review

Computer-aided grading of prostate cancer from MRI images using convolutional neural networks

Automated grading of prostate cancer using convolutional neural network and ordinal class classifier

Computer-aided diagnosis of clinically significant prostate cancer from MRI images using sparse autoencoder and random forest classifier

Automated invasive ductal carcinoma detection based using deep transfer learning with whole-slide images

Convolutional neural networks for multi-class brain disease detection using MRI images

Label-free leukemia monitoring by computer vision

Automatic detection of coronavirus disease (covid-19) using X-ray images and deep convolutional neural networks

Artificial intelligence applied on chest X-ray can aid in the diagnosis of covid-19 infection: a first experience from Lombardy

Covidx-net: a framework of deep learning classifiers to diagnose covid-19 in X-ray images

Application of deep learning for fast detection of covid-19 in X-rays using ncovnet

Covid-19 identification in chest X-ray images on flat and hierarchical classification scenarios

Convolutional capsnet: A novel artificial neural network approach to detect covid-19 disease from X-ray images using capsule networks

Sample-efficient deep learning for covid-19 diagnosis based on CT scans

Artificial intelligence-enabled rapid diagnosis of patients with covid-19

Lung infection quantification of covid-19 in CT images with deep learning

Residual attention u-net for automated multi-class segmentation of covid-19 chest CT images

Momentum contrastive learning for few-shot covid-19 diagnosis from chest CT images

Inf-net: automatic covid-19 lung infection segmentation from CT images

Covid-19 image data collection

Identifying medical diagnoses and treatable diseases by image-based deep learning

Covid-19 X rays

Imagenet: a large-scale hierarchical image database

Convolutional neural networks: an overview and application in radiology

Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review

Densely connected convolutional networks

Inception-v4, inception-resnet and the impact of residual connections on learning

Shufflenet: an extremely efficient convolutional neural network for mobile devices

Deep residual learning for image recognition

Yolov3: an incremental improvement

Mobilenetv2: inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Learning transferable architectures for scalable image recognition

Deep learning with depthwise separable convolutions

Very deep convolutional networks for large-scale image recognition

Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5 MB model size

Correlation-based feature subset selection for machine learning. Thesis submitted in partial fulfillment of the requirements of the degree of Doctor of Philosophy at the University of Waikato

Large-scale attribute selection using wrappers

Bayesian network classifiers in weka for version 3-5-7

Covid-19 screening on chest X-ray images using deep learning based anomaly detection

The authors declare that there is no conflict of interest.r e f e r e n c e s