key: cord-0883234-1q6jaftl authors: Turkoglu, M. title: COVID-19 Detection System Using Chest CT Images and Multiple Kernels-Extreme Learning Machine Based on Deep Neural Network date: 2021-01-27 journal: Ing Rech Biomed DOI: 10.1016/j.irbm.2021.01.004 sha: 9cc25749d3c6ac9d1b5894605f67352e17c545f4 doc_id: 883234 cord_uid: 1q6jaftl OBJECTIVES: Coronavirus disease is a fatal epidemic that has originated in Wuhan, China in December 2019. This disease is diagnosed using radiological images taken with the help of basic scanning methods besides the test kits for Reverse Transcription Polymerase Chain Reaction (RT-PCR). Automatic analysis of chest Computed Tomography (CT) images that are based on image processing technology plays an important role in combating this infectious disease. MATERIAL AND METHODS: In this paper, a new Multiple Kernels-ELM-based Deep Neural Network (MKs-ELM-DNN) method is proposed for the detection of novel coronavirus disease - also known as COVID-19, through chest CT scanning images. In the model proposed, deep features are extracted from CT scan images using a Convolutional Neural Network (CNN). For this purpose, pre-trained CNN-based DenseNet201 architecture, which is based on the transfer learning approach is used. Extreme Learning Machine (ELM) classifier based on different activation methods is used to calculate the architecture's performance. Lastly, the final class label is determined using the majority voting method for prediction of the results obtained from each architecture based on ReLU-ELM, PReLU-ELM, and TanhReLU-ELM. RESULTS: In experimental works, a public dataset containing COVID-19 and Non-COVID-19 classes was used to verify the validity of the MKs-ELM-DNN model proposed. According to the results obtained, the accuracy score was obtained as 98.36% using the MKs-ELM-DNN model. The results have demonstrated that, when compared, the MKs-ELM-DNN model proposed is proven to be more successful than the state-of-the-art algorithms and previous studies. CONCLUSION: This study shows that the proposed Multiple Kernels-ELM-based Deep Neural Network model can effectively contribute to the identification of COVID-19 disease. Recently, a novel coronavirus disease that has originated in Wuhan, China, has been categorized in December 2019 and subsequently has been named as COVID-19 (Corona Virus Disease-2019). The virus has then spread rapidly worldwide, and the coronavirus epidemic was declared a global pandemic by the World Health Organization on March 12, 2020. COVID-19 is an infectious respiratory disease that infects humans with Severe Acute Respiratory Syndrome Corona Virus 2 (SARS-CoV-2) [1] [2] [3] . Initial symptoms commonly include fever, a dry cough, and shortness of breath. The COVID-19 disease can be transmitted from person to person through small infected droplets expelled via an infected person's nose or mouth, which then enters the new host body through the nose, mouth, or eyes. Therefore, the virus has been proven to spread easily when a person with COVID-19 coughs or exhales within close proximity of others. At the time of writing this study, the global number of COVID-19 cases has exceeded 2.1 million, with over 145,000 dead. The detection of COVID-19 disease has mostly been performed using test kits, with samples taken from the respiratory tract such as via nasopharyngeal swab or phlegm taken from the person being tested. Testing processes vary, with results usually obtained in anywhere from a few hours to 2 days. These test kits possess certain disadvantages in that they produce results relatively slowly, require experienced personnel to administer, have limited availability worldwide, are time-consuming, and have a high error tendency [3, 4] . Therefore, a faster and more reliable scanning-based system is needed in addition to such kits [1, 2] . In clinical studies, it has been proven that the majority of patients with COVID-19 suffer from severe lung infection [2] [3] [4] . It is known that a chest CT scan can be an effective imaging technique for the diagnosis of lung-related diseases. The studies based on machine learning and deep learning methods for the detection of COVID-19 disease have been conducted using CT scan images. In some of these studies, Pathak et al. [1] presented an approach based on the pre-trained ResNet32 model using a transfer learning approach to detect COVID-19 disease. In the study, tests were conducted using chest CT scan images available in a dataset containing 852 images (413 COVID-19 and 139 normal). In their experimental studies, the model proposed achieved a training and a testing accuracy of up to 96.22% and 93.01%, respectively. Zhao et al. [5] developed a method based on the pre-trained DenseNet model using the fine-tuning process for COVID-19 detection. In their experimental works conducted using the COVID-CT dataset, an accuracy of 84.7% was obtained. Li et al. [6] presented a deep learning model in order to extract features from volumetric chest CT images for the classification of COVID-19. In their study, a dataset containing 4,356 chest CT images taken from 3,322 patients was collected. In their experimental studies using this dataset, 90% sensitivity was achieved using the proposed neural network (COVNet) model, based on the pre-trained ResNet50 architecture to detect COVID-19 disease. Asnaoui et al. [7] used deep learning for classification of pneumonia using X-ray and CT images. In their study, the performances of various pre-trained deep architectures, namely VGG16, VGG19, InceptionV3, DenseNet201, Xception, ResNet50, Inception-ResNetV2, and MobileNetV2, which were based on the transfer learning approach, were investigated. Tests were conducted in the study using chest X-rays and CT scans that were available in a dataset containing 5,856 images (4,273 pneumonia and 1,583 healthy). In their experimental results, the highest accuracy was obtained as 96.61% with the ResNet50 model. Xu et al. [8] proposed a pre-trained 3D CNN to extract potentially infected regions from CT scan images, with experimental results showing an overall accuracy of 86.7% using deep learning models for Influenza-A viral pneumonia, COVID-19, and also for healthy subjects. Similar to these studies, there are other studies that are based on Deep CNNs using CT scan images for the classification of COVID-19 disease [9] [10] [11] . Moreover, some studies have been based on image processing methods that use X-ray images. Wang and Wong [3] used pretrained ResNet50 architecture for the classification of COVID-19 disease from chest radiography images. In their study, the COVIDx dataset included data from healthy, pneumonia, and COVID-19 subjects. Based on their experimental results, an average accuracy score of 93.4% was achieved. Narin et al. [12] developed an approach based on using X-ray images and Deep CNNs for the classification of COVID-19 disease. Convolutional Neural Networkbased models (InceptionResNetV2, ResNet50, and InceptionV3,) were used to detect subjects infected with coronavirus pneumonia using chest X-ray radiographs. In their experimental results, the highest accuracy score achieved was 98% with the ResNet50 model. Similar to these studies, there are other research studies conducted that are based on Deep CNNs using X-ray images for the classification of COVID-19 disease [13] [14] [15] [16] [17] [18] . In the current paper, a new Multiple Kernels-ELM-based Deep Neural Network (MKs-ELM-DNN) method is proposed for the classification of COVID-19 from CT scan images. The proposed model consists of three main stages: (1) data augmentation, (2) feature extraction, and (3) classification. In the preprocessing phase, diseased lung areas are magnified by applying a scaling process, and then the dataset is expanded using data augmentation methods. In the feature extraction phase, deep features are extracted from CT scan images using the pre-trained CNN-based DenseNet201 architecture. For the final stage, classification is carried out using ELM-based ReLU, PReLU, and TanhReLU activation functions. In the experimental studies, a COVID-CT dataset containing COVID-19 viral and Non-COVID-19 viral classes was employed. In the experimental results, a success rate of 98.28% was achieved in COVID-19 disease recognition. These results have shown that the MKs-ELM-DNN model proposed is significantly successful in detecting the COVID-19 disease. In this study, the main contributions of the MKs-ELM-DNN model proposed are as follows. • In this study, different activation functions of the ELM classifier were used to classify COVID-19 disease and their performances were calculated. In the experimental results, it was determined that the best activation function for the ELM classifier was ReLU. • This study provides a low-cost, rapid, high-performance, automated system of diagnosis to detect COVID-19 disease. • The MKs-ELM-DNN model proposed for classification of COVID-19 disease from CT scan images will be useful in identifying positive cases of COVID-19 disease if implemented in a timely manner in the treatment of those living under quarantine restrictions. In this paper, a new Multiple Kernels-ELM-based Deep Neural Network (MKs-ELM-DNN) system has been developed to detect COVID-19 disease from CT scan images. The proposed MKs-ELM-DNN model includes three main processes: data augmentation, deep feature extraction, and Multiple Kernels classifier. The MKs-ELM-DNN model proposed is illustrated in Fig. 1 . The MKs-ELM-DNN model proposed has six key steps, as shown in Fig. 1 , as follows: a) Scaling process is implemented on CT scan images, with potential areas of COVID-19 disease magnified. The following subsections of this paper detail the theoretical background and dataset used in the MKs-ELM-DNN model proposed for detection of COVID-19 disease. In small datasets, various distortion effects can be applied in order to increase classification performance and to learn different conditions [19, 20] . In the current study, new images were created using data augmentation methods, such as taking symmetries of an image relative to its x and y axes and rotating images at certain angles. In this context, the reflection operation was performed relative to the x and y axes, and the rotation operation at the angle values of 90 • and 270 • . In this study, the scaling method was implemented on the CT scan images prior to data augmentation. Thus, image areas that show the potential presence of COVID-19 disease were magnified, and unnecessary areas were cropped. The scaling value applied on the CT scan images was determined as [1.2-1.2] with the help of an experimental method. Sample images for different values related to this process are given in Fig. 2 . Recently, Convolutional Neural Networks (CNNs), which are used to solve problems related to object recognition and classification, have produced some quite successful results. As a result, CNN-based deep models have been developed using large datasets. In the current study, the weights of pre-trained deep architectures were used instead of training a model from scratch. As there are only a few images in the dataset used in the current study, developing a model from scratch would prove more difficult to train [21] [22] [23] . For this purpose, the DenseNet201 architecture, which consists of ResNet residual blocks, was selected. The characteristic features of this architecture are as follows: • DenseNet201 architecture, proposed by Huang et al. [24] , is one of the latest versions developed based on the Dense network. • The architecture connects each layer in a feed-forward fashion to all other layers. Additionally, the DenseNet201 model uses a pooling layer and bottleneck structure. Therefore, this architecture reduces property parameters and model complexity, hence more efficient [24] [25] [26] . In the current study, the transfer learning method is used to the DenseNet201 architecture (see Fig. 3 ). This approach is based on extracting deep features using the fully-connected layer of the pre-trained architecture. For this purpose, a total of 1,000 features were obtained from the fc1000 layer of the DenseNet201 architecture. In Fig. 3 , a general representation of the DenseNet201-ELM model is presented based on the transfer learning method for the classification of COVID-19 disease from CT scan images. ELM classifier, developed by Huang [27] for single-hidden layer feed-forward neural networks, was selected as the learning method M. Turkoglu 3 . Architecture of the proposed DenseNet201-ELM model. [28, 29] . In the ELM structure, hidden layer weights were randomly generated, and the least squares algorithm was used to calculate the output weights. The ELM classifier has certain advantages such as faster learning and better generalization performance compared to traditional backpropagation algorithms [27, 28, 30] . The ELM classifier can be modeled as shown in Equation (1). where [x j , y j ], b i , w i and β i values represent the input-output, the bias of the hidden layer, weight, input weight, and output weight, respectively. In addition, M and N are the number of training samples and hidden neurons, respectively. Equation (1) can also be written in matrix form as shown in Equation (2): where β and H represent the output weight and the hidden layer output matrix, respectively. Y is the output layer of the network, and is written in the form of Y = Hβ matrix. The β parameter is calculated by β = H † Y . The Moore-Penrose generalized inverse of H is demonstrated with H † [27, 28, 31] . Considering all these equations, the pseudocode of the ELM algorithm is presented as shown in Fig. 4 . In this paper, the effects of different activation functions on performance for the ELM classifier were examined. In other studies carried out in the literature, sigmoid and Tanh functions are generally used. Recently, newly developed activation functions have been adapted for machine learning classifier methods. There are a few studies based on different activation functions for the ELM classifier [32] [33] [34] [35] . In the current study, sigmoid, Tanh, ReLU, PReLU, and TanhReLU [32] functions were adopted for the ELM method and their performances were evaluated. Information on these activation functions is detailed in Table 1 . In this paper, a COVID-CT dataset containing CT scan images was used to classify COVID-19 disease using the MKs-ELM-DNN model proposed. The dataset consists of two classes, Non-COVID-19 (no-findings) and COVID-19, and a total of 746 images; with 349 images of COVID-19 and 397 images of no-findings cases. In addition, please refer to reference [5] for detailed information on the COVID-CT dataset. Sample images relating to the COVID-CT dataset are presented in Fig. 5 . In this study, data augmentation with reflection and rotation was applied to expand the dataset and to improve its classifica- Table 1 Comparison of activation functions. Sigmoid tion performance. Thus, while the total number of images in the dataset was 746, an expanded total number of 3,730 images was obtained using data augmentation. In addition, the number of nofindings and COVID-19 images in the original dataset was 349 and 397, respectively; whereas the expanded dataset contained 1,745 images of COVID-19 and 1,985 images of Non-COVID-19 cases. The proposed Multiple Kernels-ELM-based Deep Neural Network (MKs-ELM-DNN) model was conducted using MATLAB software. During the experimental studies, a computer with an Intel Xeon Silver 2.19 GHz processor, 32 GB RAM, and NVDIA P4000 Quadro GPU card was utilized. In addition, for the ELM algorithm used in the classifier stage, the hidden layer neuron number was tuned in the range of [1,000 10,000] with a step size of 100. In addition, during the experimentation, 10-fold cross-validation algorithm was applied, which is known to have a high level of reliability. In this paper, the MKs-ELM-DNN model proposed, which uses the pre-trained DenseNet201 architecture, was further trained for classification of COVID-19 disease. The visual properties learned from the 64-channel convolution (conv1 | conv) layer using CT scan images are shown in Fig. 6 . Fig. 6 shows how a CT scan image is used to display the activations of 'conv1 | conv' of the DenseNet201 architecture to detect COVID-19 disease. In this study, 1,000 deep features learned from the fully-connected layer (fc1000) of the DenseNet201 architecture were extracted. Prior to this process, the COVID-CT dataset was expanded using data augmentation technique. Then, the individual accuracy of the proposed model was calculated using different activation functions based on ELM classifier, and the results are as given in Table 2 . In Table 2 , individual performances of different activation functions based on ELM classifier with the DenseNet201 model, with and without data augmentation, are given to detect COVID-19 disease from CT scan images. In these results, it was observed that individual performances of all ELM-based activation functions were increased using data augmentation methods. In addition, among the ELM-based activation functions, the highest accuracy score achieved was 96.75% with the ReLU-ELM classifier, whilst the second highest was 96.67% with PReLU-ELM. The lowest accuracy score obtained was 93.02% using the Sigmoid-ELM classifier. Deep features were extracted from CT scan images in the current study using the fine-tuned DenseNet201 architecture in the MKs-ELM-DNN model proposed. Then, the predictive results for each activation function were obtained using the ReLU-ELM, PReLU-ELM, and TanhReLU-ELM classifiers for each of these deep features. Lastly, the final class label was determined using the majority voting method on the estimation results. Comprehensive experimental studies based on different activation function methods were conducted on the MKs-ELM-DNN model proposed. The performance results obtained from experimental studies and the Positive Predictive values for each class are given in Table 3 and Table 4 , respectively. Table 3 shows the accuracy, specificity, precision, sensitivity, F1-Score, and AUC values for the MKs-ELM-DNN model proposed. As can be seen from both Table 2 and Table 3 , a high level of performance was achieved using the proposed model based on an Table 4 Positive predictive value (%) of the MKs-ELM-DNN model proposed for each class. Non-COVID-19 98.28 98.44 ensemble of different activation functions. As also can be seen in Table 4 , the MKs-ELM-DNN model achieved a 98.28% success rate in detecting the COVID-19 disease. Another experimental study was carried out and the dataset was divided into two parts, 70% and 30%. The training and testing phases of the proposed system were carried out on the 70% part of the dataset, using the 10-fold cross-validation algorithm. The purpose of this process is to select the parameters by which highest classification performances of the ELM method are identified. For this purpose, values of hidden layer neuron number and activation function, which determine the best performance, were determined. Then, the trained and optimized model was tested with remaining 30% of the data. According to these results, the model proposed, which is optimized and trained for detecting the COVID-19 disease has an accuracy score of 92.79%. This low result is an expected result. Moreover, the result obtained from this experimental study is more objective and reliable. In the current work, accuracy scores of MKs-ELM-DNN model proposed were compared with pre-trained deep models. In accordance with this purpose, deep features were extracted using fully-connected layers of AlexNet, VGG16, MobileNetv2, GoogleNet, ResNet18, and Inceptionv3 architectures; 'fc6', 'fc6', 'Logits', 'loss3classifier', 'fc1000' and 'predictions', respectively. Then, accuracy scores were achieved using the ELM classifier for each of these features, and comparison of the accuracies is described in Table 5 . In Table 5 , the accuracies of the model proposed that uses pre-trained deep architectures with or without data augmentation for classification of COVID-19 disease based on CT scan images are presented. In the results, it is observed that data augmentation methods positively affect the individual accuracy scores of the deep architectures. In addition, the MKs-ELM-DNN model proposed is observed to achieve a superior success rate when compared to other deep architectures. Using data augmentation methods, the highest accuracy achieved, among the deep architectures, was 93.19% for MobileNetv2, while the lowest accuracy score was 90.34% for the AlexNet architecture. The accuracy scores for the GoogleNet, VGG16, ResNet18, and Inceptionv3 models from other architectures were 92.86%, 92.65%, 92.22%, and 92.54%, respectively. In addition, when data augmentation was not applied, the highest accuracy scores were obtained using the TanhReLU-ELM classifier for the AlexNet, MobileNetv2, ResNet18, and Inceptionv3 models, whilst the highest accuracy scores for the GoogleNet and VGG16 models were obtained using the PReLU-ELM and ReLU-ELM classifiers, respectively. However, the highest accuracy score of all the deep architectures was achieved with the ReLU-ELM classifier with data augmentation applied. In this paper, machine learning classifiers such as Linear Discriminant Analysis (LDA), K-Nearest Neighbors (KNN), and Support vector machine (SVM) were used as well as the ELM based model proposed for the classification of COVID-19 disease. Accordingly, deep features obtained from DenseNet201 architecture were fed as input of classical machine learning classifiers such as LDA, SVM, and KNN. Therefore, the accuracy scores of other classifier methods were calculated for the classification of COVID-19 diseases and these results are given in Table 6 . The parameters in these classification algorithms are: • SVM algorithm: one-vs-one approach and Quadratic SVM classifier type were utilized. • KNN algorithm: The number of neighbors is set to 1, and Euclidean distance function was used. As can be seen in Table 6 , the proposed model is observed to be more successful than other classifier methods. Additionally, the highest accuracy, among other classifier methods, was 95.1% using the SVM method, while the lowest accuracy score was 91.5% with the LDA classifier. In this study, the proposed MKs-ELM-DNN approach was compared with the accuracy scores of previous studies that used CT images, and these accuracies are given in Table 7 . In Table 7 , the MKs-ELM-DNN model proposed has achieved better performance compared to other studies in the classification of COVID-19 and no-findings using CT images. In all studies given in Table 7 , classification processes were carried out using pre-trained deep architectures based on transfer learning approach. As a result, pre-trained deep models have been proven to be effective and to yield higher performance results for the detection of COVID-19. On the other hand, the proposed MKs-ELM-DNN model was compared to the original study [5] that used the COVID-CT dataset, and is confirmed to have achieved a significant performance improvement of approximately 14%. In addition, a significant performance improvement is demonstrated through a 98.28% success rate in detection of the COVID-19 disease as well as all in all classes. In contrast to the original study [5] , the method proposed achieved a significant performance improvement by using data augmentation applied along with the scaling method. In this study, a new Multiple Kernels-ELM-based Deep Neural Network (MKs-ELM-DNN) approach was proposed for the classifi-cation of COVID-19 and no-findings using chest CT scan images. In this study, the COVID-19 recognition success of different activation functions of the ELM classifier based on deep features for chest CT scan images was examined. The results showed that the MKs-ELM-DNN model proposed can successfully and effectively classify COVID-19 disease in CT scan dataset expanded using data augmentation methods. In future works, the aim will be to develop mobile-web platforms that are based on the approach proposed, which will be capable of helping radiologists in the detection of COVID-19. Table 7 Comparison of the accuracy scores of the proposed model with previous studies. Model/method Classes Accuracy (%) Pathak et al. [1] Fine-tuned ResNet32 2 93.01 Loey et al. [36] Fine-tuned Deep Models (AlexNet, VGG16, VGG19, GoogleNet, and ResNet50) 2 82.91 Zhao et al. [5] Fine-tuned DenseNet 2 84.7 Xu et al. [8] Fine-tuned ResNet18 3 86.7 Maghdid et al. [9] Modified AlexNet model 2 94.1 Shah et al. [37] Fine-tuned Deep Models (VGG-16, VGG-19, ResNet-50, InceptionV3, and DenseNet-169) 2 94.5 Yasar and Ceylan [38] Deep learning and machine learning methods 2 95.99 Jaiswal et al. [39] Fine-tuned DenseNet201 2 96.25 MKs-ELM-DNN (proposed) Multiple Kernels-ELM and Deep Neural Network 2 98. 36 Deep transfer learning based classification model for COVID-19 disease COVID-19 and frequent use of hand sanitizers; human health and environmental hazards by exposure pathways COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images A systematic review on recent trends in transmission, diagnosis, prevention and imaging features of COVID-19 CT scan dataset about COVID-19. arXiv preprint Artificial intelligence distinguishes Covid-19 from community acquired pneumonia on chest CT Automated methods for detection and classification pneumonia based on X-ray images using deep learning Deep learning system to screen coronavirus disease 2019 pneumonia. arXiv preprint Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms Lung infection quantification of COVID-19 in CT images with deep learning Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. medRxiv Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks deep learning framework for screening of COVID19 from radiographs COVID-19 diagnosis system based on X-ray images using features selected from pre-learned deep features ensemble Learning on chest X-ray images to detect and evaluate pneumonia cases at the era of COVID-19 Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks Deep MLP-CNN model using mixeddata to distinguish between COVID-19 and non-COVID-19 patients COVIDX-Net: a framework of deep learning classifiers to diagnose COVID-19 in X-ray images Differential data augmentation techniques for medical imaging classification tasks International conference on advanced concepts for intelligent vision systems Plant disease and pest detection using deep learningbased features Classification of COVID-19 patients from chest CT images using multi-objective differential evolution-based convolutional neural networks Multi-model LSTM-based convolutional neural networks for detection of apple diseases and pests Densely connected convolutional networks Chinesefoodnet: a large-scale image dataset for Chinese food recognition Image processing based plant species and diseases recognition Plant recognition system based on extreme learning machine by using shearlet transform and new geometric features Extreme learning machine: a new learning scheme of feedforward neural networks Forward-backward pursuit based sparse extreme learning machine Leaf-based plant species recognition based on improved local binary pattern and extreme learning machine OMP-AÖM: orthogonal matching pursuit-based extreme learning machine for regression Dual activation function-based extreme learning machine (ELM) for estimating grapevine berry yield and quality A design of convolutional neural network using ReLU-based ELM classifier and its application On equivalence of FIS and ELM for interpretable rule-based knowledge representation Classification of power quality disturbances using dual strong tracking filters and rule-based extreme learning machine A deep transfer learning model with classical data augmentation and CGAN to detect Covid-19 from chest ct radiography digital images Diagnosis of COVID-19 using CT scan images and deep learning techniques. medRxiv A novel comparative study for detection of Covid-19 on CT lung images using texture analysis, machine learning, and deep learning methods Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning The authors declare that the work described has been carried out in accordance with the Declaration of Helsinki of the World Medical Association revised in 2013 for experiments involving humans as well as in accordance with the EU Directive 2010/63/EU for animal experiments. The authors declare that this report does not contain any personal information that could lead to the identification of the patient(s).The authors declare that they obtained a written informed consent from the patients and/or volunteers included in the article. The authors also confirm that the personal details of the patients and/or volunteers have been removed. This work did not receive any grant from funding agencies in the public, commercial, or not-for-profit sectors. All authors attest that they meet the current International Committee of Medical Journal Editors (ICMJE) criteria for Authorship.CRediT authorship contribution statement M. Turkoglu: Investigation, Methodology, Software, Visualization, Writing -review & editing. The authors declare that they have no known competing financial or personal relationships that could be viewed as influencing the work reported in this paper. https://github .com /UCSD -AI4H /COVID -CT.