key: cord-297517-w8cvq0m5
authors: Toğaçar, Mesut; Ergen, Burhan; Cömert, Zafer
title: COVID-19 detection using deep learning models to exploit Social Mimic Optimization and structured chest X-ray images using fuzzy color and stacking approaches
date: 2020-05-06
journal: Comput Biol Med
DOI: 10.1016/j.compbiomed.2020.103805
sha: 
doc_id: 297517
cord_uid: w8cvq0m5

Coronavirus causes a wide variety of respiratory infections and it is an RNA-type virus that can infect both humans and animal species. It often causes pneumonia in humans. Artificial intelligence models have been helpful for successful analyses in the biomedical field. In this study, Coronavirus was detected using a deep learning model, which is a sub-branch of artificial intelligence. Our dataset consists of three classes namely: coronavirus, pneumonia, and normal X-ray imagery. In this study, the data classes were restructured using the Fuzzy Color technique as a preprocessing step and the images that were structured with the original images were stacked. In the next step, the stacked dataset was trained with deep learning models (MobileNetV2, SqueezeNet) and the feature sets obtained by the models were processed using the Social Mimic optimization method. Thereafter, efficient features were combined and classified using Support Vector Machines (SVM). The overall classification rate obtained with the proposed approach was 99.27%. With the proposed approach in this study, it is evident that the model can efficiently contribute to the detection of COVID-19 disease.

The new Coronavirus (COVID- 19) is an acute deadly disease that originated from Wuhan province, China in December 2019 and spread globally. COVID-19 outbreak has been of great concern to the health community because no effective cure has been discovered [1] . The biological structure of COVID-19 comprises of a positive-oriented single-stranded RNA-type, and it is difficult to treat the disease owing to its mutating feature. Medical professionals globally are undergoing intensive research to develop an effective cure for the disease. Presently, COVID-19 is the primary cause of thousands of deaths globally, and major deaths are in the USA, Spain, Italy, China, the UK, Iran, etc.

Many types of coronavirus exist, and these viruses are commonly seen in animals. COVID-19 has been discovered in human, bat, pig, cat, dog, rodent, and poultry. Symptoms of COVID-19 include sore throat, headache, fever, runny nose, and cough. The virus can provoke the death of people with weakened immune systems [2, 3] . COVID-19 is transmitted from person to person mostly by physical contact. Generally, healthy people can be infected through breath contact, hand contact, or mucous contact with people carrying COVID-19 [4] .

Recently, artificial intelligence (AI) has been widely used for the acceleration of biomedical research. Using deep learning approaches, AI has been used in many applications such as image detection, data classification, image segmentation [5, 6] . People infected by COVID-19 may suffer from pneumonia because the virus spreads to the lungs. Many deep learning studies have detected the disease using a chest X-ray image data approach [7] . A previous study has classified the pneumonia X-ray images using three different deep learning models [8] namely the fine-tuned model, model without fine-tuning, and the model trained from scratch. By using the ResNet model, they classified the dataset into multiple labels such as age, gender, etc. They also used the Multi-Layer Perceptron (MLP) as a classification method and achieved an average of 82.2% accuracy. Samir Yadav et al. [9] performed a classification algorithm using pneumonia data, SVM as a classification method, and InceptionV3, VGG-16 models as a deep learning approach. In their study, the dataset is divided into three classes: normal, bacterial pneumonia, and viral pneumonia to improve the contrast and brightness zoom settings with the augmentation method for each image in the dataset. The best classification achievement was 96.6%. Rahib Abiyev et al. [10] used the Backpropagation Neural Network and Competitive Neural Network models to classify pneumonia data. Using pneumonia and normal chest X-ray images, they set 30% of the dataset as test data and compared the proposed approach with the existing CNNs. They achieved 89.57% classification success. Okeke Stephen et al. [11] proposed a deep learning model to classify the pneumonia data from scratch to train the data.

Their proposal consists of convolution layers, dense blocks, and flatten layers. The input size of the model was 200 × 200 pixels to determine the possibilities of classification using the sigmoid function.

The success rate was 93.73% in pneumonia from X-ray images. Vikash Chouhan et al. [12] detected the images of pneumonia using deep learning models, three classes of the dataset: normal, virus pneumonia, and bacterial pneumonia images. In the first instance, they carried out a set of preprocessing procedures to remove noise from the images. Then, they applied the augmentation technique to each image and used a transfer learning to train the models. The overall classification accuracy was 96.39%.

In this study, we used COVID-19 chest images dataset, pneumonia chest images, and normal chest images. We preprocessed each image before being trained with deep learning models. In this preprocessing, the dataset was reconstructed using the Fuzzy technique and Stacking technique. Then, we trained the three datasets using the MobileNetV2 and SqueezeNet deep learning models and classified the models by the SVM method. The remainder of this study is structured as follows: In Section 2, we discuss the structure of the dataset, technique, method, deep learning models, and optimization algorithm. Experimental analysis is mentioned in Section 3. Section 4 and 5 present the discussion and conclusion, respectively.

In the experimental analysis, we use the three classes of datasets that are accessible publicly. These classes are normal, pneumonia, and COVID-19 chest images. All datasets are X-ray images, and each image is converted to JPG format. Since COVID-19 is a new disease, the number of images related to this virus is limited. For this study, we combined two publicly accessible databases consisting of COVID-19 images. The first COVID-19 dataset was shared on the GitHub website by a researcher named Joseph Paul Cohen from the University of Montreal. After the experts checked the images, they were made available to the public. In the Joseph Paul Cohen dataset, image types are MERS, SARS, COVID-19, etc. The data of 76 images labeled with COVID-19 were selected for this study [13] . The second COVID-19 dataset consists of the images created by a team of researchers from Qatar University, medical doctors from Bangladesh, and collaborators from Pakistan and Malaysia. The second COVID-19 dataset is available on the Kaggle website, and the current version has 219 X-ray images [14] . For this study, two datasets containing COVID-19 images were combined, and a new dataset consisting of 295 images was created. COVID-19 virus causes pneumonia in the affected individuals and can provoke death if the lungs are permanently damaged [15] . The second dataset is important in this study to compare COVID-19 chest images using deep learning models. The second dataset consists of normal chest images and pneumonia chest images. Pneumonia chest images include both virus and bacteria types, and these images are taken from 53 patients. The images were created by experts and shared publicly [16] .

The combined dataset consists of three classes. Information about the classes of the dataset and the number of images in the classes are as follows: We collect a total of 295 images in COVID-19 class. Normal class X-ray images are 65 in total, and pneumonia class X-ray images are 98. The total number of images of the dataset is 458. In the experimental analysis, 70% of the dataset was used as training data, and 30% was used as test data. In the last step of the experiment, the k-fold crossvalidation method was used for stacked images. Sample images of the dataset are shown in Fig. 1 . 

MobileNet is a deep learning model intended to be used in low hardware cost devices. Object MobileNetV2 model uses the ReLU function between layers [18] . Thus, it enables the nonlinear outputs from the previous layer to be linearized and transferred as input to the next layer. The model continues its training process until a comfortable step is reached. In this model, the convolutional layers circulate filters over input images and create activation maps. The activation maps contain the features extracted from the input images, and these features are transferred to the next layer. Pooling layers are also used in the MobileNetV2 model. The matrices obtained through these layers are converted into smaller dimensions [19] . [20] .

MobileNetV2 model was used as pre-trained, and the SVM method was used in the classification phase. Besides, other important parameters of the MobileNetV2 model are given in Table 1 and Table 2 . All default parameter values were used for the MobileNetV2 model without any change. 

SqueezeNet, an in-depth learning model of input size of 224 × 224 pixels, comprises of convolutional layers, pooling layers, ReLU, and Fire layers. The SqueezeNet does not have fully connected layers and dense layers. However, Fire layers perform the functions of these similar layers. The major benefit of this model is that it performs analyses successfully by reducing the number of parameters, thereby decreasing the model size capacity. SqueezeNet model produced more successful results, approximately 50 times fewer parameters than the AlexNet model, thereby reducing the cost of the model [21] . Fig. 3 presents the model design. In the Expansion part, the depth is increased [21, 22] . Table 3 presents the layers and default parameter values of the model, these values are used without changes. Other important parameters of the SqueezeNet model are given in Table 4 . (2) and Eq. (3). Here, and represent the coordinate points of the features in the hyperplane.

parameter represents margin width, and parameter represents bias value [23, 24] . 

Deep learning models use the optimization methods, and these facilitate learning trends of models. Stochastic Gradient Descent (SGD) optimization is the method that updates the weight parameters in the model structure at every iteration. The models provide better training during each iteration. However, the SGD does not use all the images input into the model while updating the parameters. It performs this operation using only the images it randomly determines. This lowers the cost of the model, and it offers a faster training process. Eq. (4) shows the mathematical function that performs the weight parameter updates in the SGD. Where represents the weight parameter. This represents the coordinates of the features extracted in the and input images and represents α learning rate [25] .

In this study, the SVM method was preferred because:

i. it has a strong potential to provide solutions to the data analysis problems encountered in daily life,

ii. it is widely used for the remote pattern recognition and classification problems to successfully execute multiple classification processes [26, 27] , and

iii. it gives the best classification performance among other machine learning methods (discriminant analysis, nearest neighbor, etc.).

Moreover, the Linear SVM was preferred owing to its best performance such as cubic, linear, quadratic, etc. Preferred parameter values in the Linear SVM method were the kernel scale that was parameter automatically selected. The box constraint level parameter value was chosen, and the multiclass method parameter of one-vs-one was selected.

The Fuzzy concept is accepted based on its degree of accuracy, and its next degree is uncertain. Fuzzy

Color algorithms play an important role in image analysis, and the obtained results depend on the similarity/difference functions used for color separation. In the fuzzy color technique, each of the input images contains three input variables (red, green, and blue -RGB). As a result of this process, a single output variable is passed. The input and output values are determined according to the training data [28, 29] .

The logic behind the Fuzzy Color technique is to separate the input data into blurred windows.

Each pixel in the image has a membership degree to each window, and membership degrees are calculated based on the distance between the window and the pixel. Image variance is obtained with membership degrees. The Fuzzy Color technique is to create the finishing output-input. In this step, the weights of the images of each blurred window are summed, and the output image is created from the average. Here, the weight value of each pixel is expressed as the degree of membership [28, 30] .

We recreated the original dataset using the Python codes with the Fuzzy Color technique [31] . Fig. 5 shows the structure of the data image. 

Image stacking is a digital image processing technique that combines multiple images shot or is reconstructed at different focal distances. This is a technique used to improve the quality of the images in the dataset. This technique aims to eliminate the noises from the original image by combining at least two images in a row and dividing the image into two parts. These parts are background and overlay. While the first images are processed in the background, the second is overlaid on the image placed in the background. Here, parameters such as opacity, contrast, brightness, and combining ratio of the two images are important. The more accurately these ratios, the more amount of noise is reduced in the images, and the higher the quality ratio [32] .

In this study, Python and Pillow library was used for the Stacking Technique [33] . Here, the original dataset was stacked on the reconstructed dataset using the Fuzzy technique. The successful result of the Fuzzy technique will contribute to the success of the stacking technique. The parameter values preferred in the stacking technique were the opacity value, with the value of 0.6, the contrast value was 1.5, the brightness value was set to -80, and the combined ratio was chosen as 50%. These values can be varied for another dataset. We evaluate the various stages for dataset images and determine that these values are the most efficient of the dataset. Hence, we used them in the experimental analysis. Moreover, the original dataset was placed in the background, and the structured dataset was placed in the overlay. A combined representation of the original dataset using the stacking technique and dataset structured with the Fuzzy technique is shown in Fig. 6 . 

Emotions such as morale and happiness can be transfer from person to person. Emotional imitation happens consciously or unconsciously. This situation is related to neurons in the human brain, that is, [34] . For this study, the population size of 20 was selected, the maximum iteration parameter was selected as 10. The best global value parameter was taken as 1000 for the start, and the % parameter value of 10 was chosen.

&1, () = + + -& + − + ); ( = 1, 2, … , %

The proposed approach is designed to perform the classification of chest images based on the dataset types. This is an approach to separate chest images of COVID-19 virus infection from normal breast images and pneumonia. The original dataset is passed through the front image processing steps. In the first step, the original dataset was recreated with the Fuzzy Color technique, aimed to remove the noise in the original images. In the second step, using the original dataset Stacking technique, each fuzzy color image was combined with original images, and a new dataset was created. The aim was to create a better data quality image. The two deep learning models were used, and the stacked dataset was trained with MobileNetV2 and SqueezeNet deep learning models. Using the SMO algorithm, the 1000-features obtained by the models were used to extract efficient features. By combining the efficient features, the SVM method that produced successful results in multiple classifications was used as the classification process. Fig. 7 shows the overall design of the proposed approach. 

Python 3.6 is used to structure the original data set using the Fuzzy technique and Stacking technique.

Besides, the SMO algorithm was compiled in Python, and detailed information about the source codes and analysis used in this study are given in the web link specified in the Open Source Code section.

Jupyter Notebook is the interfaces program used in compiling Python. Using deep learning models, MATLAB (2019b) software was used for classification. The hardware features to compile software are the Windows 10 operating system (64 bit) with a 1 GB graphics card, 4 GB memory card, and an

Intel © i5 -Core 2.5 GHz processor. 

The experiment consists of three steps The 30% of the data set was used as test data and the remaining 70% as training data. In the steps related to the stacked dataset, the k-fold cross-validation method was used. The SVM method was used as a classifier in the last layers of SqueezeNet and MobileNetV2 models. Fig. 8 shows the training success graphs of the three stages performed with the SqueezeNet model, and Fig. 9 shows the confusion matrices. The results of the experimental analysis are given in Table 5 . To acknowledge the validity of these analyses, we conducted a new analysis using another deep learning model, MobileNetV2. In the first stage of the MobileNetV2 model, we trained the original dataset, and the overall accuracy rate obtained with the SVM method was 96.32%. In the second stage, the dataset structured with the Fuzzy Color technique was classified and the overall accuracy rate obtained in this classification was 97.05%. In the third stage of the MobileNetV2 model, the model was trained with the stacked data set, and the overall accuracy rate in the classification was 97.06%. Fig. 10 shows the graphs of the training success of the three stages performed with the MobileNetV2 model, and Fig. 11 shows the confusion matrices. The results of the experimental analysis of this model are given in Table 6 . In the second step of this experiment, the k-fold cross-validation method was used for the stacked dataset, classified by using the SVM method. In the first step of the experiment, 30% of the dataset was used as test data. To confirm the validity of the results of the first step, the dataset was separated using a k-fold cross-validation method. For two CNN models, the k-fold value was adjusted to five. In the second step, the overall accuracy rate achieved with the SqueezeNet model was 95.85%.

In the first step of this model, the classification rate with the stacked dataset (with 30% test data) was 97.06%. In the analyzes performed in the second step, the SqueezeNet model produced stable results in both steps. In the second step, the overall accuracy rate was 96.28%, obtained from the MobileNetV2 model trained with the stacked dataset. In the first step of the MobileNetV2 model, the classification rate was 97.06%, achieved with the stacked dataset (with 30% test data). Fig. 12 shows the confusion matrices of the analysis performed in the second step, and Table 7 gives the metric values obtained from the confusion matrices. As a result, all analyzes performed in the second step produce a stable result compared to the results obtained in the first step. The results obtained by the 5fold cross-validation confirmed the reliability of the proposed approach compared to the results obtained from the previous step. In the third step, the SMO algorithm was applied to the feature sets obtained from the stacked dataset trained by CNN models. The dataset contains 1000 features, extracted by two CNN models.

The dataset was obtained as a file with a '*.mat' extension from MATLAB. These feature sets were obtained from the Pool10 layer in the SqueezeNet model and the Logits layer in the MobileNetV2 model. Choosing efficient features with the SMO algorithm offers a total of 800 column numbers as we have set the maximum features selection to 800, and some of these column numbers are repeated using the SMO algorithm. The feature column numbers selected by CNN models are less than 800.

Using an efficient optimization method with a few possible features, this step aimed to contribute to the classification success. The third step consists of four stages. In the first stage, the SMO algorithm was applied to the 1000-feature set, obtained by training the stacked dataset using the SqueezeNet images. Fig. 13 shows the confusion matrices of the analysis performed in the third step of the experiment, and Table 8 gives the values of the metric parameters.

In the proposed approach, the contribution of the SMO algorithm was recorded for the improvement of classification performance. The codes and analysis of results of the SMO algorithm are available in the web address of the Open Source Code. Available in the web address is also feature sets obtained in the experiment and related source codes. 

The number of confirmed COVID-19 cases has exceeded millions worldwide with thousands of confirmed deaths. The World Health Organization has declared that COVID-19 is a global epidemic [37] . Using the proposed approach, we performed the detection of COVID-19 from the X-ray image data. We compared COVID-19 chest data with that of pneumonia and normal chest data since pneumonia is one of the symptoms of COVID-19. The major challenge we encountered is that the publication of COVID-19 images is still limited. Moreover, previous studies on the detection of COVID-19 using deep learning are non-existence. Hence, we fill the gap in the literature.

Although the limited dataset is used, this study contributes to the classification of the dataset, using the image preprocessing to determine the data classes. Other techniques can also be used instead of the Fuzzy Color technique. We paid attention to the similarity between the structured image and the original stack image. If we configured the image in a different format (such as resolution status or color pixel status), we would not have achieved the success achieved in this study.

The advantages of the proposed approach are as follows:

• It provides a 100% success rate in detecting the disease by examining the X-ray images of COVID-19 patients. • The analysis can be carried out using AI, and the proposed approach can be integrated into portable smart devices (mobile phones, etc.) • The deep learning models (MobileNetV2 and SqueezeNet) used in the proposed approach have fewer parameters compared to other deep models. This helps to gain speed and time performance. Besides, using the SMO algorithm, CNN models save time and speed during the process.

• It minimizes the interference in every image in the dataset and provides efficient features with stacking technique.

The disadvantages of the proposed approach are as follows:

• If the sizes of the input images in the dataset are different, a complete success may not be achieved. Irrespective of the resize parameter, it is still a challenge for the proposed approach to deal with very low-resolution images.

• In the Stacking technique, the resolution dimensions of the original images and the structured images must be the same.

We ensure that the number of pneumonia and normal chest images are close to that of COVID-19 chest images. Since we presume that the image classes found in an unbalanced number cannot contribute to the success of the model, we still achieved an overall 99.27% accuracy in the classification process. In the proposed model, the end-to-end learning scheme has been exploited, which is one of the great advantages of CNN models. The pathologic patterns were detected and identified by using the activation maps that kept the discriminative features of the input data. In this manner, the tedious and labor-intensive feature extraction process was isolated; a highly sensitive decision tool was ensured.

People infected with COVID-19 are likely to suffer permanent damage in the lungs, which can later provoke death. This study aimed to distinguish people with damaged lungs owing to COVID-19 from normal individuals or pneumonia (not infected by . The detection of COVID-19 was carried out using deep learning models. Since it is important to detect COVID-19 that spread rapidly and globally, AI techniques are used to perform this accurately and quickly. One of the novelty aspects of the proposed approach is to apply the pre-processing steps to the images. When using preprocessing steps, more efficient features are extracted from the image data. With the stacking technique, each pixel of equivalent images is superimposed, and the pixels with low efficiency is increased. With the proposed approach, efficient features were extracted using the SMO algorithm.

The model was aimed to produce faster and more accurate results. Another innovative aspect is that the feature sets obtained with SMO are combined to improve the classification performance. We also demonstrate the usability of our approach to smart mobile devices with the MobileNetV2 model as it can be analyzed on mobile devices without using any hospital devices. The 100% success was achieved in the classification of COVID-19 data, and 99.27% success was achieved in the classification of Normal and Pneumonia images.

In future studies, deep learning-based analyzes will be carried out using data images of other organs affected by the virus, in line with the views of COVID-19 specialists. We plan to develop a future approach using different structuring techniques to enhance the datasets. As the data related to the factors influencing the virus in human chemistry (e.g. blood group, RNA sequence, age, gender, etc.) are available, we will produce a solution-oriented analysis using AI.

Information about Python and MATLAB software source codes, datasets, and related analysis results used in this study are given in this web link. https://github.com/mtogacar/COVID_19

There is no funding source for this article.

This article does not contain any data, or other information from studies or experimentation, with the involvement of human or animal subjects.

The authors declare that there is no conflict to interest related to this paper.

Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany

Editorial COVID-19 : too little , too late ?

Coronavirus disease 2019 (covid-19): a guide for UK GPs

Transmission routes of 2019-nCoV and controls in dental practice

Application of breast cancer diagnosis based on a combination of convolutional neural networks, ridge regression and linear discriminant analysis using invasive breast cancer images processed with autoencoders

Recent progress in semantic image segmentation

Identifying pneumonia in chest X-rays: A deep learning approach

Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification

Deep convolutional neural network based medical image classification for disease diagnosis

Deep Convolutional Neural Networks for Chest Diseases Detection

An Efficient Deep Learning Approach to Pneumonia Classification in Healthcare

A novel transfer learning based approach for pneumonia detection in chest X-ray images

COVID-19 Chest X-Ray dataset or CT dataset

COVID-19 Radiography Database

Radiology Perspective of Coronavirus Disease 2019 (COVID-19): Lessons From Severe Acute Respiratory Syndrome and Middle East Respiratory Syndrome

Pneumonia sample X-Rays, GitHub

MobileNetV2: Inverted Residuals and Linear Bottlenecks

MobileNetV2: The Next Generation of On-Device Computer Vision Networks

BrainMRNet: Brain tumor detection using magnetic resonance images with a novel convolutional neural network model

A Dual-Path and Lightweight Convolutional Neural Network for High-Resolution Aerial Image Segmentation

A deep-learning-based approach for fast and robust steel surface defects classification

Real-Time Vehicle Make and Model Recognition with the Residual SqueezeNet Architecture

A multiseed-based SVM classification technique for training sample reduction

Multiple Kernel-Based SVM Classification of Hyperspectral Images by Combining Spectral, Spatial, and Semantic Information

Stochastic Gradient Descent and Its Variants in Machine Learning

Support Vector Machines for Classification BT -Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers

A unified view on multi-class support vector classification

Prediction of Wood Density by Using Red-Green-Blue (RGB) Color and Fuzzy Logic Techniques

Hybrid filter based on fuzzy techniques for mixed noise reduction in color images

Color comparison in fuzzy color spaces, Fuzzy Sets Syst

Fuzzy Color Image Enhancement algorithm, Github

Focus stacking technique in identification of forensically important Chrysomya species (Diptera: Calliphoridae), Egypt

Image stack: Simple code to load and process image stacks

Social mimic optimization algorithm and engineering applications

Çınar, a New Approach for Image Classification: Convolutional Neural Network

Fusing fine-tuned deep features for recognizing different tympanic membranes

Coronavirus disease 2019

he received his undergraduate degree from Fırat University, Department of Electronics and Computer Education

Clinical Decision Support Systems. Research Interests Disease detection in biomedical images (MR, X-ray etc

Education Information