key: cord-0993523-ek0pk0bs
authors: Yasar, Huseyin; Ceylan, Murat
title: A novel comparative study for detection of Covid-19 on CT lung images using texture analysis, machine learning, and deep learning methods
date: 2020-10-06
journal: Multimed Tools Appl
DOI: 10.1007/s11042-020-09894-3
sha: 0960f1fc9bd367f7e5eb9dc820c5c5d34cc67851
doc_id: 993523
cord_uid: ek0pk0bs

The Covid-19 virus outbreak that emerged in China at the end of 2019 caused a huge and devastating effect worldwide. In patients with severe symptoms of the disease, pneumonia develops due to Covid-19 virus. This causes intense involvement and damage in lungs. Although the emergence of the disease occurred a short time ago, many literature studies have been carried out in which these effects of the disease on the lungs were revealed by the help of lung CT imaging. In this study, 1.396 lung CT images in total (386 Covid-19 and 1.010 Non-Covid-19) were subjected to automatic classification. In this study, Convolutional Neural Network (CNN), one of the deep learning methods, was used which suggested automatic classification of CT images of lungs for early diagnosis of Covid-19 disease. In addition, k-Nearest Neighbors (k-NN) and Support Vector Machine (SVM) was used to compare the classification successes of deep learning with machine learning. Within the scope of the study, a 23-layer CNN architecture was designed and used as a classifier. Also, training and testing processes were performed for Alexnet and Mobilenetv2 CNN architectures as well. The classification results were also calculated for the case of increasing the number of images used in training for the first 23-layer CNN architecture by 5, 10, and 20 times using data augmentation methods. To reveal the effect of the change in the number of images in the training and test clusters on the results, two different training and testing processes, 2-fold and 10-fold cross-validation, were performed and the results of the study were calculated. As a result, thanks to these detailed calculations performed within the scope of the study, a comprehensive comparison of the success of the texture analysis method, machine learning, and deep learning methods in Covid-19 classification from CT images was made. The highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values obtained as a result of the study were 0,9197, 0,9891, 0,9473, 0,9058, 0,9888; respectively for 2-fold cross-validation, and they were 0,9404, 0,9901, 0,9599, 0,9284, 0,9903; respectively for 10-fold cross-validation.

In December 2019, an outbreak occurred in Wuhan of China's Hubei province, caused by a new corona virus which is of zootonic origin and affecting acute respiratory tract in general [46] . With the spread of the virus in question in the following days, the disease began to seriously affect all countries in the world. The World Health Organization announced in March 2020 that the disease became a global pandemic. In addition, the World Health Organization named this new epidemic in question "Covid-19". Common symptoms of the disease are fever, cough, shortness of breath, muscle pain, and weakness [36] . The disease has quite severe and negative effects on the lungs in general. In this context, many literature studies have been realised in a short time in which these effects of the disease on lungs were shown by the help of lung CT imaging. These studies reveal that besides clinical symptoms, blood and biochemical tests, lung CT imaging is an important diagnostic tool for the diagnosis of the disease as well.

In a study conducted by Qin et al. [33] , a clinical examination images of lung CT belonging to four cases, two men and two women with Covid-19 disease, was performed. The results of the study reveal that lung lesions due to Covid-19 show a high degree of involvement in patients with pneumonia disease. In the study carried out by Albarello et al. [2] , the change in the images related to chest X-Rays and CT of lungs belonging to 2 Covid-19 cases in Italy in the disease process was investigated. In the context of the study, it was evaluated by the authors that monitoring the deterioration of lungs by radiological images is an important alternative for the early diagnosis of the disease, considering the clinical findings. In the study conducted by Lin et al. [24] , the change in CT images of lungs belonging to a 61-year-old male Covid-19 patient in accordance with the progress in the disease was examined. In the scope of the study, it is reported that there is an increase in lung involvement depending on the progress of the disease. In a similar study by Li et al. [21] , CT images of lung belonging to 5 Covid-19 patients, whose ages range from 10 months to 6 years were evaluated clinically. As a result of the study, it was reported that 2 of these patients had no signs of disease on CT images of lungs but 3 of them showed significant abnormalities.

In the study conducted by Xu et al. [41] , CT images of lungs belonging to 50 Covid-19 patients in total, of whose 9 are in mild, 28 moderate, 10 severe, and 3 critical level, were evaluated clinically. In the examinations carried out within the scope of the study, it was reported that no changes in radiological images occurred in 9 patients, while symmetrical lesions developed in 26 patients and asymmetrical lesions in 15 patients. In addition, in the discussion part of the study, it was evaluated that repeated CT scanning is a useful method for monitoring disease progress and timely treatment of Covid-19. In the study performed by Xia et al. [39] , CT images of 20 Covid-19 diagnosed pediatric patients were analyzed. The results of the study reveal that all patients have sub-pleural lesions. A similar study by Chen et al. [10] was carried out by analyzing the clinical data history of 9 women who were diagnosed with Covid-19 and were also pregnant. As a result of the study, it is stated that CT images of lungs have high diagnostic value in Covid-19 diagnosis. A study of clinical data of patients diagnosed with Covid-19 was performed by Huang et al. [18] . It was reported that lung abnormalities were detected in 40 of 41 Covid-19 patients examined within the scope of the study and that there was bilateral involvement.

In the study performed by Hu et al. [16] , the CT images of lung belonging to 2 Covid-19 patients were evaluated. In the scope of the study, although the symptoms of the disease decreased after two days of treatment, this reduction and recovery showed inconsistency with the CT images of lungs. In the study conducted by Liu et al. [25] , clinical evaluation of CT images of lungs belonging to 73 Covid-19 cases of various severities was performed. The results of the study show that all patients except the patient group of 8 percent who undergo the disease at the level of mild pneumonia had abnormal CT images of lungs. In the study conducted by Xu et al. [42] , the clinical data histories of 90 Covid-19 patients were examined. From the results of the study, it was reported that Covid-19 patients had multiple patchy ground glass opacities on CT images. A similar study was carried out by Pan et al. [31] , using the clinical data of 63 Covid-19 patients and abnormalities were detected in CT images of lungs belonging to the patients. As for the study conducted by Shen et al. [37] , the lesion levels in CT images of lungs belonging to 44 Covid-19 patients were labeled by radiologists and computers respectively. The study results show that computerized labeling is a reliable alternative method in detecting the severity and distribution of pneumonia due to Covid-19 disease. In a study conducted by Li et al. [22] , a clinical evaluation of CT images of lungs taken in the course of pneumonia due to Covid-19 disease was performed. In the study, attention was drawn to the importance of CT imaging of lungs in understanding the effect and progress of the Covid-19 disease. Clinical symptoms and CT images of lungs belonging to a 54-year-old male patient with Covid-19 disease in South Korea were investigated by Lim et al. [23] . A similar study was conducted by Cheng et al. [11] for the first Covid-19 case in Taiwan.

In a review study conducted by Long and Ehrenfeld [26] , it is emphasized that using artificial intelligence methods to reduce the effects of Covid-19 outbreak crisis is an essential requirement. In this context, some applications regarding the automatic classification of Covid-19 and Non-Covid-19 CT images with artificial intelligence methods were implemented. Detailed information about the methods used in these studies, number of images, training-test methods and study results are given in Table 1 .

In this study, Convolutional Neural Network (CNN), one of the deep learning methods, was used which suggested automatic classification of CT images of lungs for early diagnosis of Covid-19 disease. In addition, k-Nearest Neighbors (k-NN) and Support Vector Machine (SVM) were used to compare the classification successes of deep learning with machine learning. Within the scope of the study, besides the results obtained when the images are used directly with the SVM, k-NN, and CNN classifiers, the results obtained by using Gray-Level Co-Occurrence Matrix (GLCM), Local Binary Pattern (LBP), and Local Entropy (LE) as a pre-process were found out as well. In addition, the success of the proposed method was tested separately in case that all the image data were combined as double, triple, and quadruple.

Within the scope of the study, a 23-layer CNN architecture was designed and used as a classifier. In addition, training and testing processes were performed for Alexnet and Mobilenetv2 CNN architectures as well. The classification results were also calculated for the case of increasing the number of images used in training for the first 23-layer CNN architecture by 5, 10, and 20 times using data augmentation methods. In order to reveal the effect of the change in the number of images in the training and test clusters on the results, two different training and testing processes, 2-fold and 10-fold cross-validation, were performed and the results of the study were calculated. As a result, thanks to these detailed calculations performed within the scope of the study, a comprehensive comparison of the success of the texture analysis method, machine learning, and deep learning methods in Covid-19 classification from CT images was made. The results of the study indicate that analyzing CT images of lungs with the help of deep learning methods in diagnosing Covid-19 disease will speed up the diagnosis and significantly reduce the burden on healthcare workers. 

The CT images of lungs belonging to Covid-19 patients used in the study were taken from two data set created and accumulated in the form of metadata by Cohen et al. [13] and Zhao et al. [45] who then made it available to public through Github for open access. In this context, 386 CT images of lungs in total were used in the study. The sizes of the images in question are between 115×98 and 2024×1523 and vary widely. First of all, in order to clarify the area of interest on the images, framing was carried out so as to include the lung region. After this process, the dimensions of all the images were rearranged as 448×448. In the scope of the study, 1.010 Non-Covid-19 CT images of lungs were also used. The images in question were similarly taken from, LIDC-IDRI public access research database [1, 5, 6, 12], a previously published data set. The processes applied to the Covid-19 images were also applied to the Non-Covid-19 ones.

Local Binary Pattern (LBP) was first introduced by Ojala et al. [27] . This method is often used to reveal local spatial structures. LBP is a sequential comparison process of a center pixel withh its neighboring pixel values. So it is a non-parametric method. The first example of this comparison was made for a 3×3 square operator. Then, operations were realized for operators of different sizes too. In (1), there is a mathematical representation of this operation.

As can be seen from the (1), threshold measurement is made by realizing a comparison with neighbors of the central pixel. As a result of the threshold measurement, a LBP code with a total of 256 combinations comes out. This code is then mathematically converted to a local number, the new pixel value. A sample LBP application is shown in Fig. 1 .

In case LBP is applied to the images, it is seen that the image sizes decrease. For example, when an LBP operation with a radius 2 is applied to an image of 224×224, an image of 220×220 is obtained. The reason for this is that this operation cannot be applied to the pixels in the starting and ending rows and columns. The diameter of the LBP used in this study is 2. To prevent this size reduction from causing any problem in the operations mentioned above, Fig. 1 A sample LBP application the dimensions of the image obtained after LBP operations were restored to 448×448. The use of the LBP process in the study enable us to have new images from the original ones that reflect local features. Thus, the total image feature depth was increased.

Entropy can be explained as the measure of uncertainty or randomness in an image [43] . The global entropy of an image can be mathematically calculated with (2) . In this formula, p i is obtained from the normalized histogram of the image and is called the probability associated with the pixel value i. If local histograms are used instead of global histograms, local entropy is calculated. Local entropy is an effective feature extraction method used in texture analysis. CT lung images were included in the study to reveal the texture properties.

Gray-Level Co-Occurrence Matrix (GLCM) is used to reveal how often pairs of pixels with a specified value and shift occur in an image [40] . The GLCM conjugate of an image can be mathematically calculated with (3) .

In this equation, I denote the image in size n×m, of which the GLCM is calculated, i and j are the pixel values; x and y denote the spatial position of the image and C is the cooccurrence matrix. The size of the matrix C is p×p, with p the number of different pixels in I image. For this reason, as in LBP, the re-arrangement of the dimensions is realized after the GLCM process. The GLCM, being a method frequently used for texture analysis in image processing applications, was included in the study to reveal different image features of CT lung images and to increase the variety of features.

In Fig. 2 , the original versions of a Covid-19 and Non-Covid-19 image and the versions obtained by applying LBP, LE, and GLCM are given.

Within the scope of the study, the effects of the increase in the number of images used in the training set of the CNN classifier on the study results were also examined. For this purpose, data were augmented using classical data augmentation methods. The number of 1.396 CT lung images (386 Covid-19 and 1.010 Non-Covid-19) was augmented by 5 times, 10 times, and 20 times using conventional data augmentation methods. In this context, data augmentation was realized by using contrast changing, brightness changing, distortion, and noise adding methods on the original images. In some steps of performing these processes, the image augmentation tools created by the works of Bloice et al. [7] [8] [9] were used.

First, the pixel values related to the original image size being multiplied by 0,8 and 0,6 respectively and the contrast being changed, the second and third images were obtained. The brightness change was made by increasing the value of each pixel of the second and third images by 7 and the fourth and fifth images were created. After these processes, the number of images amounted to 5 times the previous number. Secondly, the original images were eroded by the distortion process. Later, the processes performed in the first stage were also applied to the eroded images. At the end of the second stage, the number of images amounted to 10 times the number of images obtained in the first stage.

A salt and pepper noise of a density of 0,03 was added to the original image and the first nine images obtained from this image, and the number of images was increased to 20. The selection of the type of noise to be added and the parameter sizes used in the data augmentation processes was made by taking into consideration the literature studies [29] and [28] having successful results. In this context, it is possible to choose different parameter sizes and noise types and to use other data augmentation methods.

By using the data augmentation methods, the amount of LBP, LE, and GLCM images as well as the original images were augmented by 5 times, 10 times, and 20 times. At this stage, data augmentation processes applied to the original images were performed using these images, instead of the original image as the source image.

Support Vector Machine (SVM) can be broadly described as a supervised learning approach. It is based on statistical learning theory. The foundations of SVM were demonstrated by the studies realized by Vapnik and Chervonenkis [38] . SVM can effectively perform linear and nonlinear classifications. Two-class SVM was used within the scope of the study. Radial Basis Function (RBF) was preferred as a kernel function in SVM. While the results were standardized, other parameters were used as the default (Matlab 2019a).

k-Nearest Neighbors (k-NN) is a non-parametric classification and regression method [3] . When classifying is made using k-NN, the object to be classified is voted on by its neighbors and assigned to the most common class among its closest neighbors. Two-class k-NN was used within the scope of the study. While the neighbor number of k-NN was set 21, other parameters were used as the default (Matlab 2019a).

One of the main objectives of the study is to compare the success of machine learning with deep learning methods in Covid-19 classification. For this purpose, SVM and k-NN, which are basic machine learning methods, were included in the study as classifiers.

Deep learning is realized using too many layers in the realization of learning process. Convolutional Neural Network (CNN) is the most frequently used model of deep learning. This model has come into use widely, especially in image processing applications in recent years. CNN consist of some layers such as convolutional layer, activation function, pooling and fully connected layer. Convolutional layers are often designed to follow each other and enable us to obtain feature patterns from low-level features of images to high-level features [15] . Activation functions in CNN architecture can be defined as the functions that bring incoming inputs to a certain range or accept some of the input values while eliminating some of them again. Pooling layers, on the other hand, let the size of feature matrices to be reduced, through sampling. As for fully connected layer, it is the layer where the classification process is performed according to the features obtained through convolution, activation function and pooling. This layer works like a classic artificial neural network. Before the classification process, the conversion of feature matrices into feature vectors, that is, flattening process is performed. In this context, Fig. 3 shows the general architecture of the CNN classifier.

Within the scope of the study, a CNN architecture consisting of 23 layers in total was designed. Table 2 shows the information and parametric features of the layers of this CNN architecture. In programming processes, Matlab 2019a program was used and the function names and parameters of the layers used in the program were written directly in the layer name and parameters sections. The reason why there are four different dimensions in the first layer, the image input layer, is that experiments were made for multiple and different sized input images within the scope of the study.

In addition to the 23-layer CNN architecture presented within the scope of the study, two more CNN architectures that were previously presented in the literature and proved to be successful in many studies, were used. In this context, Alexnet [20] and Mobilenetv2 [35] CNN architectures were also included in the study. While only the architectures of these CNN models were transferred, the previously trained coefficients were not transferred. Alexnet and Mobilenetv2 were trained from the beginning with the images used in the study. Alexnet is a CNN architecture that consists of 25 layers and has a serially arranged structure. While the input image dimensions of the original architecture of the Alexnet model are 227×227×3 the fully connected layer output size is 1000. In other words, a classification with 1000 classes is performed at the output of this architecture. Mobilenetv2, on the other hand, has an architecture consisting of 155 layers and 164 connection points that is, arranged both serially and parallelly. While the initial image dimensions of the original architecture of Mobilenetv2 model are 224×224×3, the size of the fully connected layer output size is 1000.

Some modifications need to be made on Alexnet and Mobilenetv2 architectures for them to be used in the study. In this context, the number of lines and columns in the input image was preserved as the values in the original architecture. However, the size of the third dimension of the input data was rearranged as 1, 2, 3, and 4 according to the requirements of the study. In this context, the new input image sizes of the modified Alexnet architecture are 227×227×1, 227×227×2, 227×227×3, 227×227×4 and the new input image sizes of the modified Mobilenetv2 architecture are 224×224×1, 224×224×2 is 224×224×3, 224×224×4. Within the scope of the study, since a two-class classification, Covid-19 and Non-Covid-19, was performed, the fully connected layer output size was re-adjusted as 2 in both CNN architectures. For this reason, the dimensions of the images used in the operations realized using Alexnet and Mobilenetv2 are different from the first CNN architecture and machine learning classifiers. With these architectures, the images were re-sized using the imresize (Matlab 2019a) function. This issue was also taken into consideration additionally, in the interpretation of CPU run-times within the scope of the study.

In this study, besides the parameters: TP, TN, FN, FP and the dependent variables such as sensitivity (SEN), specificity (SPE), F-1 Score (F-1), and accuracy (ACC) which were obtained by mathematical processing of the values of the above-mentioned parameters were used in the evaluation of the results. Within the scope of the study, Receiver Operating Characteristic (ROC) analysis was made and the area (AUC) sizes under the ROC curve were compared.

In this context, TP is the number of times that the actually patient data is labeled as patients as a result of classification as well. FP, on the other hand, is the number of times that non-patient data is labeled as patient in the same way. TN is the number of times that nonpatient data is labeled as not patient as a result of the classification. FN, on the other hand, is the labeling of the patient data as non-patient in the same way. Sensitivity (SEN), specificity (SPE), F-1 Score (F-1), and accuracy (ACC) values calculated using these parameters are defined mathematically between (4)- (7) . Within the scope of the study, SEN, SPE, F-1, and ACC values were calculated for the threshold (cut-off) value to be 0,5.

ROC analysis examines the change of sensitivity (SEN) (y-axis) relative to the discrimination threshold value, that is, the cut-off value [44] , in relation to the precision that is the complement of the specificity to one (1-SPE) (x-axis) in the classification processes performed in two groups. In this context, the area under the curve that reflects the change in question is called AUC. This field approaching to 1 indicates that the classification goes to perfection, while approaching 0 indicates that the classification deteriorates.

Within the scope of the study, machine learning-based and deep learning-based artificial intelligence applications were implemented for the automatic diagnosis of CT images of lung for early diagnosis of Covid-19 disease. In this study, a total of 1.396 lung CT images were used, 386 of which were belonging to patients diagnosed with Covid-19 and 1.010 to Non-Covid-19 subjects.

In the study, first of all, framing process was carried out to include the lung region in order to clarify the interests of the related images. Since the images used are of very different sizes, the images were rearranged in size and the image dimensions were set to 448×448. Later, the images in question were recorded in gray-scale format and in 8 bit depth in png format. In this context, it is possible to name these processes as pre-process applications.

In the second part of the study, new images were created by applying LBP, LE, and GLCM to the original images of Covid-19 patients and Non-Covid-19 patients and recorded in the same size, format and depth as the original images. In this context, since there was a decrease in image sizes after LBP and GLCM application, those images were rearranged in 448×448 dimensions. In this way, new images containing feature patterns of the relevant images were created and the features of texture were enriched. Within the scope of the study, the effects of the increase in the number of images used in the training set of the CNN classifier on the study results were also examined. In this stage of the study, data augmentation was performed by the help of changing contrast, distortion, and brightness as well as noise-adding methods described earlier. This data augmentation process was performed for the images applied LBP, LE, and GLCM as well as the original images. As in the first stage, these images were recorded in gray-scale format and 8 bit depth in png format. At the end of the second phase, the amount of data was augmented 5 times, 10 times, and 20 times, including first images on which were based derivation.

In the third stage of the study, first, the original images with LBP, LE, and GLCM images separately and secondly in the form of double, triple and quadruple combinations including the original image in each group were subjected to Covid-19 and Non-covid-19 classification using SVM and k-NN classifier. In order to observe the effects of the change in the training-test image number balance on the results, two different training-test processes, that is, 2-fold cross-validation and 10-fold cross-validation, were performed and the results were calculated.

In the fourth part of the study, a 23-layer CNN architecture, previously described, was developed and this architecture was used in all the experiments. In this section, the image groups created in the previous step were classified using this 23-layer CNN architecture. As in the previous stage, 2-fold cross-validation and 10-fold cross-validation were used for training-test processes in classification. Each experiment was repeated 5 times in itself in order to let the results reach stability due to the random assignment of some initial weights and parameters used in the structure of CNN.The average of the test results obtained as a result of five repetitions was given in the scope of the study.

In the fifth stage of the study, the study results were calculated for the case of increasing the number of images and using them in 23-layer CNN training with the help of the data augmentation techniques detailed previously. In this context, training-testing processes were realized to increase by three different amounts: 5, 10, and 20 times. In these training-test processes, only 2-fold cross-validation was performed due to the length of processing times.

The original images were divided into two, so as to be they are in the same group as the images derived from them. That is, when the original image is in the training group, it is prevented that a derived image obtained from this image is in the test group or vice versa. While the training process was performed by augmenting the data, the testing process was performed only for the original images.

In the sixth part of the study, the training and testing processes were realized using the Alexnet and Mobilenetv2 CNN architecture which were detailed previously and of which the modification processes were performed. At this stage, training and testing processes were realized using 2-fold and 10-fold cross-validation as in the third and fourth stages. However, unlike these stages, instead of 448×448, the dimensions of the input images were used as 227×227 for Alexnet and 224×224 for Mobilenetv2.

Within the scope of the study, the times (CPU time) needed to be able to classify an image were also measured. The experiments of this study were carried out using MATLAB 2019 (a) software running on 64 GB RAM and Intel(R) Xeon (R) CPU E5-2680 2.7 GHz (32 CPUs).

Within the scope of the study, first the Covid-19/Non-Covid-19 classification was performed using the SVM classifier. The dimensions of the input images for this experiment were 448×448×1 for Original, GLCM, LBP, and LE; 448×448×2 for Original+GLCM, Original+LBP, and Original+LE; 448×448×3 for Origi-nal+GLCM+LBP, Original+GLCM+LE, and Original+LBP+LE; 448×448×4 for Original+GLCM+LBP+LE. These image matrixes were given as input to SVM matrixes after being converted to image vectors. The study results obtained in this context were given in Table 3 for 2-fold cross-validation and Table 4 for 10-fold cross-validation. The highest parameter values in Table 3 and Table 4 are stated in bold. This marking is also made in the following tables.

Within the scope of the study, secondly, the Covid-19/Non-Covid-19 classification was performed using the k-NN classifier. The dimensions of the input images for this experiment are the same as those used for the SVM classifier. These image matrixes were given as input to k-NN matrixes after being converted to image vectors. The study results obtained in this context were given in Table 5 for 2-fold cross-validation and Table 6 for 10-fold cross-validation.

Within the scope of the study, thirdly, the Covid-19/Non-Covid-19 classification was performed using the 23-layer CNN classifier. The dimensions of the input images for this experiment were 448×448×1 for Original, GLCM, LBP, and LE; 448×448×2 for Original+GLCM, Original+LBP, and Original+LE; 448×448×3 for Orig-inal+GLCM+LBP, Original+GLCM+LE, and Original+LBP+LE; 448×448×4 for Origi-nal+GLCM+LBP+LE. The study results obtained in this context were given in Table 7 for 2-fold cross-validation and Table 8 for 10-fold cross-validation. Each experiment was repeated 5 times in itself in order to let the results reach stability due to the random assignment of some initial weights and parameters used in the structure of CNN. The average of the test results obtained in five repetitions were given within the scope of the study. This is true for all experiments performed after this test.

Within the scope of the study, fourthly, the Covid-19/Non-Covid-19 classification was performed using the 23-layer CNN classifier and data augmentation (×5, ×10, and ×20). The dimensions of the input images for this experiment were 448×448×1 for Original, Table 3 Results obtained by using SVM classifier for 2-fold cross-validation In this experiment, training and testing processes were performed using only 2-fold cross-validation due to the increase in the time to obtain results in parallel with the increase in the training set. The results obtained in the experiments were given in Table 9 for increasing the number of training images by 5 times, in Table 10 for increasing by 10 times, and in Table 11 for 20 times.

Within the scope of the study, fifthly, the Covid-19/Non-Covid-19 classification was performed using the Alexnet CNN classifier. The dimensions of the input images for this experiment were 448×448×1 for Original, GLCM, LBP, and LE; 448×448×2 for Original+GLCM, Original+LBP, and Original+LE; 448×448×3 for Orig-inal+GLCM+LBP, Original+GLCM+LE, and Original+LBP+LE; 448×448×4 for Origi-nal+GLCM+LBP+LE. The study results obtained in this context were given in Table 12 for 2-fold cross-validation and Table 13 for 10-fold cross-validation. Within the scope of the study, sixthly, the Covid-19/Non-Covid-19 classification was performed using the Mobilenetv2 CNN classifier. The dimensions of the input images for this experiment were 448×448×1 for Original, GLCM, LBP, and LE; 448×448×2 for Original+GLCM, Original+LBP, and Original+LE; 448×448×3 for Orig-inal+GLCM+LBP, Original+GLCM+LE, and Original+LBP+LE; 448×448×4 for Origi-nal+GLCM+LBP+LE. The study results obtained in this context were given in Table 14 for 2-fold cross-validation and Table 15 for 10-fold cross-validation.

Within the scope of the study, the results of the study are given with a total of 13 Tables  (Tables 3-15 ) in which there are experimental results for 11 different input image in each Tables 3 and 4 , which includes the results obtained by using the SVM classifier, are examined within the scope of the study, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values were found to be 0,8912, 1,0000, 0,9470, 0,9021, 0,9815; respectively for 2-fold cross-validation, 0,9171, 1,0000, 0,9585, 0,9233, 0,9874; respectively for 10-fold cross-validation. It was observed that although the highest mean specificity value was obtained as 1,0000, there were serious worsenings in sensitivity, accuracy, F-1 score, and AUC values in experiments where this value was obtained. The second highest specificity obtained using the SVM classifier is 0,9713 for 2-fold cross-validation and 0,9812 for 10-fold cross-validation. In obtaining these highest average values mentioned, it is seen that the combinations of input images such as GLCM and Original+GLCM+LE for 2-fold Table 9 Results obtained by using 23-layer CNN classifier and data augmentation (5 times cross-validation, the Original, GLCM, and LE were used for 10-fold cross-validation. When the processing times are examined, it is understood that results can be obtained for an image at times ranging from 1,5857 to 7,0295 seconds for 2-fold cross-validation and from 9,2526 to 41,9181 seconds for 10-fold cross-validation, in direct proportion to the image data sizes. When Tables 5 and 6 , which includes the results obtained by using the k-NN classifier, are examined within the scope of the study, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values were found to be 0,7772, 1,0000, 0,9205, 0,8403, 0,9888; respectively for 2-fold cross-validation, 0,8057, 0,9980, 0,9284, 0,8567, 0,9903; respectively for 10-fold cross-validation. As is with the SVM classifier, although the highest mean specificity values were obtained for the k-NN classifier as 1,0000 and 0,9980 as well, serious worsenings occur in the sensitivity, accuracy, F-1 score, and AUC Table 11 Results obtained by using 23-layer CNN classifier and data augmentation (20 times) for 2-fold cross-validation Table 12 Results obtained by using Alexnet CNN classifier for 2-fold cross-validation   Method  TP  FN TN  FP SEN  SPE  ACC  F-1  AUC CPU   Time   Original  334,4 51,6 932,2 values in the experiments where these values were obtained. The second highest specificity obtained using the k-NN classifier is 0,9891 for 2-fold cross-validation and 0,9901 for 10-fold cross-validation. In obtaining these highest average values mentioned, it is seen that the combinations of input images such as Original, Original+GLCM+LE, Orig-inal+LBP+LE, and Original+GLCM+LBP+LE for 2-fold cross-validation, the Original, Original+LBP+LE, and Original+GLCM+LBP+LE were used for 10-fold cross-validation. When the processing times are examined, it is understood that results can be obtained for an image at times ranging from 2,4588 to 11,0160 seconds for 2-fold cross-validation and from 3,9970 to 22,7328 seconds for 10-fold cross-validation, in direct proportion to the image data sizes. When Tables 7 and 8 , which includes the results obtained by using the 23-layer CNN classifier, are examined within the scope of the study, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values were found to be 0,8865, 0,9756, 0,9394, Table 13 Results obtained by using Alexnet CNN classifier for 10-fold cross-validation Table 14 Results obtained by using Mobilenetv2 CNN classifier for 2-fold cross-validation   Method  TP  FN  TN  FP SEN  SPE  ACC F-1  AUC CPU   Time   Original  355,0 31,0 964,8 Tables 9 and 11 for 2-fold cross-validation. When these tables are examined, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values are 0,8855, 0,9675, 0,9321, 0,8782, 0,9736; respectively for 5 times increment, 0,8959, 0,9699, 0,9410, 0,8917, 0,9780; respectively for 10 times increment Tables 12 and 13 , which includes the results obtained by using the Alexnet CNN classifier, are examined within the scope of the study, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values were found to be 0,9041, 0,9620, 0,9457, 0,9020, 0,9767; respectively for 2-fold cross-validation, 0,9301, 0,9814, 0,9587, 0,9234, 0,9820; respectively for 10-fold cross-validation. In obtaining these highest average values mentioned, it is seen that the combinations of input images such as LE and Origi-nal+LBP+LE for 2-fold cross-validation and 10-fold cross-validation. When the processing times are examined, it is understood that results can be obtained for an image at times ranging from 0,0516 to 0,1062 seconds for 2-fold cross-validation and from 0,4003 to 0,8628 seconds for 10-fold cross-validation, in direct proportion to the image data sizes.

When Tables 14 and 15 , which includes the results obtained by using the Mobilenetv2 CNN classifier, are examined within the scope of the study, the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values were found to be 0,9197, 0,9691, 0,9473, 0,9058, 0,9783 respectively for 2-fold cross-validation, 0,9404, 0,9782, 0,9599, 0,9284, 0,9860 respectively, for 10-fold cross-validation. In obtaining these highest average values mentioned, it is seen that the combinations of input images such as Original, LE, and Original+GLCM for 2-fold cross-validation, the Original, GLCM and LE were used for 10fold cross-validation. When the processing times are examined, it is understood that results can be obtained for an image at times ranging from 0,7366 to 0,7987 seconds for 2-fold cross-validation and from 6,5895 to 7,1508 seconds for 10-fold cross-validation, in direct proportion to the image data sizes. Table 16 contains the comparison of the results obtained in the literature studies, of which a piece of summary information was given in the introduction section, with the results obtained within the scope of the study. As can be seen from Table 16 , the highest mean sensitivity, specificity, accuracy, F-1 score, and AUC values obtained within the scope of 

Important results were obtained within the scope of this study, which recommends automatic classification of CT images of lungs for early diagnosis of Covid-19 disease and benefits from texture analysis methods, machine learning and deep learning. Within the scope of the study, the classification success of texture analysis methods, machine learning and deep learning methods were compared comprehensively for many method combinations.

Within the scope of the study, if the SVM and k-NN classifiers are used as machine learning methods are compared, it is seen that the SVM classifier is more successful than the k-NN classifier. As for the classification time per image, it is understood that the SVM classifier for 2-fold cross-validation and the k-NN classifier for 10-fold cross-validation produce faster classification results. When the results obtained by using only the original images for SVM and k-NN classifiers are compared with the highest classification results obtained by adding new texture feature images, it is understood that significant improvements were achieved in many comparison parameters. Using new texture feature images, improvements were observed in the sensitivity, specificity, accuracy, F-1 score, and AUC parameters for SVM classifier and 2-fold cross-validation by 0,5181%, 1,2871% (with reference to the second-highest result), 0,8596%, 1,3805%, 0,3817% respectively. These improvements were; 1,4851% (with reference to the second-highest result), 0,5731%, 0,8552%, 0,2096% respectively, in the specificity, accuracy, F-1 score, and AUC parameters for the SVM classifier and 10-fold cross-validation. The highest sensitivity values for 2-fold cross-validation and 10-fold cross-validation using the k-NN classifier were obtained using the original images. However, it is seen that the use of new texture feature images reflected positively on other comparison parameters. Using the new texture feature images, it is seen that improvements in the specificity, accuracy, F-1 score, and AUC parameters for the k-NN classifier and 2-fold cross-validation by 2,3762% (with reference to the second-highest result), 0,7163%, 0,8110%, 0,8871% respectively and 1,9802% (with reference to the second highest result), 0,3582%, 0,1163%, 0,9370% respectively for 10-fold cross-validation were achieved. In this context, it is understood that the direct use of GLCM and LE texture feature extraction methods in the improvement of the classification results using the SVM classifier, and the use of LBP and LE tissue feature extraction methods in combination with the original images in the improvement of the classification results using the k-NN classifier will provide the most effective benefit.

Within the scope of the study, the effect of increasing the number of images used in education by using data augmentation on the classification results for the CNN classifier was examined. Because of the significant increases in training load and classification time due to data augmentation, the experiments were realized only for the use of 23-layer CNN and 2-fold cross-validation. The results of the study show that data augmentation has partial positive contributions on the classification. However, it is understood that it causes the same rates of increases in classification times as well. Also, when the results obtained by using only the original images for 23-layer CNN and data augmentation are compared with the highest classification results obtained by adding new texture feature images, it is understood that significant improvements were achieved in many comparison parameters.

If a comparison of the CNN architectures used within the scope of the study is required, it is seen that the most successful results are obtained by using Mobilenetv2. In terms of classification times though, it is understood that the fastest results are achieved by using Alexnet and 23-layer CNN. There is a sevenfold difference between Alexnet and Mobilenetv2 in terms of the speed of achieving results. Similar to the cases of SVM and k-NN, it is understood that the inclusion of new texture feature images into the classification has positive effects on the classification results for CNN classifiers, as well. Some conclusions were obtained for CNN classifiers, by comparing the results obtained by using only the original images with the highest classification results obtained by adding new texture feature images. In this context, using new tissue feature images, it was seen that improvements were achieved in the sensitivity, specificity, accuracy, F-1 score, and AUC parameters by 0,4145%, 3,3069%, 1,3467%, 1,8929%, 0,6367% respectively, for 23-layer CNN classifier and 2-fold cross-validation. The highest sensitivity and F-1 score values for 10-fold cross-validation using a 23-layer CNN classifier were obtained using the original images. However, it is seen that the use of new texture feature images reflected positively on other comparison parameters. In this context, it is seen that by using new texture feature images, improvements in the specificity, accuracy, and AUC parameters for a 23-layer CNN classifier and 10-fold cross-validation by 1,7822%, 0,0430%, 0,5768%, respectively, were achieved. For the Alexnet classifier, those improvements were even higher. Using new tissue feature images, improvements in the sensitivity, specificity, accuracy, F-1 score, and AUC parameters for the Alexnet classifier and 2-fold cross-validation by 3,7824%, 3,9010%, 3,8395%, 6,3918%, 2,4969% respectively, and for 10-fold cross-validation, by 2,1762%, 2,5347%, 1,5903%, 2,5606%, and 0,8262% respectively, were achieved. The highest sensitivity value for 2-fold cross-validation using the Mobilenetv2 classifier was obtained using the original images. However, it is seen that the use of new texture feature images reflected positively on other comparison parameters. In this context, using new texture feature images, it is seen that improvements in the specificity, accuracy, F-1 score, and AUC parameters for Mobilenetv2 classifier and 2-fold cross-validation by 1,3861%, 0,1862%, 0,2644%, 0,2356%, respectively were achieved. The highest sensitivity, accuracy, and F-1 score value for 10-fold cross-validation using the Mobilenetv2 classifier was obtained using the original images. However, it is understood that using the new texture feature images provided improvements in the specificity and AUC parameters for the Mobilenetv2 classifier and 10-fold cross-validation by 1,0891% and 0,2682%, respectively.

If deep learning classifiers are compared with machine learning classifiers, it is seen that deep learning classifiers are more successful than machine learning classifiers. In addition, deep learning classifiers produce results at least 5 times faster than machine learning classifiers.

When the results of the study obtained within the scope of the study and summary information of which is given in Table 16 are compared with the results obtained in previous studies in the literature, it is seen that successful results were obtained. In this context, it is understood that higher results were obtained than the results obtained in the studies performed by Pathak et al. [23] , Ouyang et al. [24] , Sakagianni et al. [25] , and Hu et al. [26] . Study results are generally equivalent to the results obtained in the study by Jaiswal et al. [22] and behind the studies performed by Han et al. [20] and Ardakani et al. [21] . However, for making a fairer comparison, it should be taken into account that in the study conducted by Ardakani et al. [21] , the determination of 60×60 ROI areas on the image was done manually and dense areas were marked. Indeed, although similar transfer learning approaches were used by Jaiswal et al. [22] , poorer results were obtained because the image was used as a whole. In our study, images were classified as a whole as well. One of the most important implications of this study is that if the images obtained by texture feature methods are used directly or together with the original ones, significant improvements can be achieved in the study results. This situation is applicable to other studies with high results. In this way, it is considered that the results obtained in other studies can be enhanced even more.

The results of the study indicate that analyzing CT images of lungs by the help of deep learning methods in diagnosing Covid-19 disease will speed up the diagnosis and significantly reduce the burden on healthcare workers. In this context, it is critical to increase the number of radiological and clinical data of Covid-19 patients and to make them available to researchers through open access in order to improve these studies and obtain better results.

In the studies to be carried out after this stage, it is aimed to make automatic classification of X-Ray images of chest, which is an important diagnostic tool such as CT images of lung, by the help of methods based on deep learning, in Covid-19 disease diagnosis. In addition, it will be tried to increase the success of study by making use of complex-valued CNN and transfer learning approaches. Finally, it is evaluated that another important study topic is the realization of pipeline methods that will provide better results by combining the results obtained using different extraction property methods.

Conflict of interests Dr. Ceylan declares that he has no conflict of interest. Mr. Yasar declares that he has no conflict of interest.

Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors.

COVID 19 INMI Study Group (2020) 2019-novel Coronavirus severe adult respiratory distress syndrome in two cases in Italy: An uncommon radiological presentation

An introduction to kernel and nearest-neighbor nonparametric regression

Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks

Data From LIDC-IDRI. The Cancer Imaging Archive 10:K9

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans

Patch augmentation: Towards efficient decision boundaries for neural networks

Augmentor: An image augmentation library for machine learning

Clinical characteristics and intrauterine vertical transmission potential of COVID-19 infection in nine pregnant women: a retrospective review of medical records

First case of Coronavirus Disease 2019 (COVID-19) pneumonia in Taiwan

The Cancer Imaging Archive (TCIA): Maintaining and operating a public information repository

COVID-19 image data collection, arXiv, 2020

Accurate screening of COVID-19 using attention based deep 3D multiple instance learning

A novel comparative study using multi-resolution transforms and convolutional neural network (CNN) for contactless palm print verification and identification. Multimedia Tools and Applications

CT imaging of two cases of one family cluster 2019 novel coronavirus (2019-nCoV) pneumonia: inconsistency between clinical symptoms amelioration and imaging sign progression

Weakly supervised deep learning for COVID-19 infection detection and classification from CT images

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning

Imagenet classification with deep convolutional neural networks. In: In Advances in neural information processing systems

Chest computed tomography in children with COVID-19 respiratory infection

Coronavirus Disease (COVID-19): Spectrum of CT Findings and Temporal Progression of the Disease

Case of the index patient who caused tertiary transmission of COVID-19 infection in Korea: the application of lopinavir/ritonavir for the treatment of COVID-19 infected pneumonia monitored by quantitative RT-PCR

Asymptomatic novel coronavirus pneumonia patient outside Wuhan: The value of CT images in the course of the disease

CT manifestations of coronavirus disease-2019: a retrospective analysis of 73 cases by disease severity

The role of augmented intelligence (ai) in detecting and preventing the spread of novel coronavirus

A comparative study of texture measures with classification based on featured distributions

Comparison of traditional transformations for data augmentation in deep learning of medical thermography

Health status detection of neonates using infrared thermography and deep convolutional neural networks

Dual-sampling attention network for diagnosis of COVID-19 from community acquired pneumonia

Initial CT findings and temporal changes in patients with the novel coronavirus pneumonia (2019-nCoV): a study of 63 patients in Wuhan

Deep Transfer Learning based Classification Model for COVID-19 Disease

18 F-FDG PET/CT findings of COVID-19: a series of four highly suspected cases

Setting up an easy-to-use machine learning pipeline for medical decision support: Case study for COVID-19 diagnosis based on deep learning with CT scans

Mobilenetv2: Inverted residuals and linear bottlenecks

A review of coronavirus disease-2019 (COVID-19)

Quantitative computed tomography analysis for stratifying the severity of Coronavirus Disease

A note on one class of perceptrons

Clinical and CT features in pediatric patients with COVID-19 infection: Different points from adults

An improved thermal exchange optimization based GLCM for multi-level image segmentation

Clinical and computed tomographic imaging features of Novel Coronavirus Pneumonia caused by SARS-CoV-2

Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2

Image segmentation based on gray level and local relative entropy two dimensional histogram

A novel approach for reduction of breast tissue density effects on normal and abnormal masses classification

CT scan dataset about COVID-19

A novel coronavirus from patients with pneumonia in China