key: cord-0825017-lgay3cjv
authors: Kiziloluk, Soner; Sert, Eser
title: COVID-CCD-Net: COVID-19 and colon cancer diagnosis system with optimized CNN hyperparameters using gradient-based optimizer
date: 2022-04-08
journal: Med Biol Eng Comput
DOI: 10.1007/s11517-022-02553-9
sha: 6f233b2b46b9001233b77f5e9393993eba779650
doc_id: 825017
cord_uid: lgay3cjv

Coronavirus disease-2019 (COVID-19) is a new types of coronavirus which have turned into a pandemic within a short time. Reverse transcription–polymerase chain reaction (RT-PCR) test is used for the diagnosis of COVID-19 in national healthcare centers. Because the number of PCR test kits is often limited, it is sometimes difficult to diagnose the disease at an early stage. However, X-ray technology is accessible nearly all over the world, and it succeeds in detecting symptoms of COVID-19 more successfully. Another disease which affects people’s lives to a great extent is colorectal cancer. Tissue microarray (TMA) is a technological method which is widely used for its high performance in the analysis of colorectal cancer. Computer-assisted approaches which can classify colorectal cancer in TMA images are also needed. In this respect, the present study proposes a convolutional neural network (CNN) classification approach with optimized parameters using gradient-based optimizer (GBO) algorithm. Thanks to the proposed approach, COVID-19, normal, and viral pneumonia in various chest X-ray images can be classified accurately. Additionally, other types such as epithelial and stromal regions in epidermal growth factor receptor (EFGR) colon in TMAs can also be classified. The proposed approach was called COVID-CCD-Net. AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet architectures were used in COVID-CCD-Net, and the hyperparameters of this architecture was optimized for the proposed approach. Two different medical image classification datasets, namely, COVID-19 and Epistroma, were used in the present study. The experimental findings demonstrated that proposed approach increased the classification performance of the non-optimized CNN architectures significantly and displayed a very high classification performance even in very low value of epoch. GRAPHICAL ABSTRACT: [Image: see text]

COVID-19 broke out in the world in early December 2019 and rapidly turned into a pandemic. According to the World Health Organization (WHO) data, 227,940,972 people have been infected, while 4,682,899 people have been killed by the disease around the world until today [1] . Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the virus which has caused COVID-19 pandemic [2] . Common symptoms of COVID-19 pandemic can be listed as fever, muscle pain, dry cough, head ache, a sore throat ve chest pain [3, 4] . Due to these sypmtoms, COVID-19 has been accepted as a respiratory tract disease. It may take these symptoms 2 to 14 days to appear in a person who has been infected with the virus [5] . Despite recent attempts at finding a treatment method, such as a drug or vaccine, against the disease, no viable solutions to COVID-19 have been found yet. Various medical imaging techniques such as X-ray and computed tomography (CT) can be considered as important tools in the diagnosis of COVID-19 cases [6, 7] . Coronavirus usually causes lung infections. Therefore, chest X-ray and CT images are widely used by physicians and radiologists for an accurate and quick diagnosis in the patients infected with the virus.

Polymerase chain reaction (PCR) test method is widely used for the diagnosis of COVID-19. However, the test is not always accessible at all healthcare points. It must be also noted that compared to PCR tests, X-ray and CTbased imaging techniques are usually more reliable and accessible. When CT and X-ray methods are compared, X-ray machines are preferred more by radiologists and physicians because of their accessibility nearly in every location including remote rural areas, cost-effectiveness, and their capacity to perform imaging in a fairly short period of time [5] . However, it is also time-consuming for physicians and radiologists to evaluate the patients' X-ray images. Furthermore, it also runs the risk of inaccurate diagnosis because the detection of infected areas in an image requires technical know-how and medical experience. Therefore, an accurate and quick computer-assisted diagnosis system is needed for COVID-19 cases. The following literature review indicated that deep learning (DL) algorithms were used in order to diagnose COVID-19 in X-ray images succesfully [5, [8] [9] [10] [11] [12] .

Introduced by Kononen [13] in 1998, tissue microarray (TMA) is an innovative and high-performance technique used for the analysis of multiple tissue samples. It is a highend technology with a remarkable performance and has been used in the analysis of molecular identifiers recently. There is sufficient evidence to claim that epidermal growth factor receptor (EGFR) plays an important role in tumor development [14] . In parallel with this, it was also observed that EGFR played an important role in the initation and progress of colorectal cancer [15] .

The present study proposes a convolutional neural network (CNN) classification approach with optimized hyperparameters using gradient-based optimizer (GBO) algorithm [16] . CNN is the most widely used DL model. The proposed approach was used to classify COVID-19, normal, and viral pneumonia. In addition, it can be also used to classify other types such as epithelial and stromal regions in EFGR-colon in digitized tumor TMAs.

Real-world applications in many different fields such as medicine, agriculture, and engineering can be approached as an optimization problem. To this day, numerous optimization approaches have been developed in order to solve real-world problems in an effective way. However, highperformance optimization approaches are needed due to the fact that the difficulty of these optimization problems is increasing day by day. In this respect, metaheuristic algorithms (MAs), which are known as global optimization techniques, have been widely used to solve challenging optimization problems [17] [18] [19] [20] [21] [22] .

Artificial neural network (ANN) is an important machine learning approach inspired by the neural system in human mind. It involves an input layer, hidden layer, and output layer, and aims to adjust optimal values in relation with the weight of each neuron in ANN following a training process [23] . The performance of an ANN structure is heavily affected by the number and variety of training data. If an insufficient number of data is used in the training process, the performance of ANN is very likely to decrease.

Various changes have been so far applied to ANN structure to design feedback and multi-layer model structures, which paved the way for the solution of non-linear problems. With the advent of multi-layer neural network models, the number of layers in an ANN structure has also increased and led to the development of CNN, which is a high-performance version of ANN models. Introduced during the 1990s, CNN was not preferred due to computer hardware incapacity in this period [23] . However, thanks to the technological developments in computer hardware and graphical processing unit (GPU) in the following years, CNN performances have also increased remarkably in recent years, and it became one of the most widely used machine learning approaches in various fields such as health, transportation, security, stock exchange, and law.

Various CNN architectures have been so far proposed in the existing literature, as manifested by several examples such as MobileNet-V2, ShuffleNet, GoogleNet, VGG-16, VGG-19, and AlexNet. In these CNN architectures, hyperparameters such as learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold are known to affect CNN performance directly. Therefore, it is not surprising that various studies in the existing literature attempted to offer solutions to the optimization of these hyperparameters.

The present study benefited from AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet architectures for the proposed approach, i.e., a COVID-19 and colon cancer diagnosis system with optimized hyperparameters using GBO. In order to optimize hyperparameters such as learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold in these architectures, GBO algorithm proposed by Ahmadianfar et al. [16] was used in the present study. Inspired by Newton's method, GBO is one of the most recent metaheuristic optimization approaches. The present study aims to optimize hyperparameters in AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet and increase its classification performance.

The main contributions of the present study can be summarized as follows:

1) The present study proposes a high-performance approach which can classify both COVID-19 and colon cancer in TMAs. No approach which can classify both diseases has been so far proposed in the current literature.

2) The proposed COVID-CCD-Net approach benefits from GBO [16] algorithm proposed in 2020 in order to optimize hyperparameters in AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet.

3) The present study aims to obtain a high level of accuracy with a low value of epoch in AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet architectures in the proposed COVID-CCD-Net approach. On the other hand, the non-optimized CNN methods obtained a much lower level of accuracy with the same value of epoch.

The organization of the present study is as follows: "Section 2" describes the related works. "Section 3" presents gradient-based optimizer and convolutional neural networks. "Section 4" describes the proposed COVID-CCD-Net approach. "Section 5" presents experiments and results, and Section 6 concludes the study.

In order to optimize hyperparameters in CNN, various approaches such as adaptive gradient optimizer [24] , Adam optimizer [25] , Bayesian optimization [26] , equilibrium optimization [27] , evolutionary algorithm [28] , genetic algorithm [29] , grid search [30] , particle swarm optimization [31, 32] , random search [30, 33] , simulating annealing [33] , and tree-of-parzen estimators [33] , whale optimization algorithm [34] , and weighted random search [35] have been so far proposed. random search, simulating annealing, and tree-of-parzen estimators.

In addition to its comprehensivess as a searching algorithm, grid search aims to identify the most optimal values for hyperparameters through a manually specified subset of hyperparameter space [36] . However, since the grid of configurations grows exponentially depending on the number of hyperparameters during the hyperparameter optimization process, the algorithm is not often useful for the optimization of deep neural networks [36] . During the hyperparameter optimization in CNN, it may take a few hours or a whole day to evaluate a hyperparameter selection, which causes serious computational problems. Similar to grid search algorithm, random search algorithm too encounters various disadvantages in sampling a sufficient number of points to be evaluated [37] .

Bayesian optimization has been a popular technique for hyperparameter optimization recently [38] . One of the main advantages in Bayesian optimization-based neural network optimization is that it does not require running neural network completely. On the other hand, its complexity and high-dimensional hyperparameter space makes Bayesian optimization an impractical and expensive approach for hyperparameter optimization [36] .

One of the biggest disadvantages of genetic algorithm is that it usually becomes stuck in a local optimal value and, as a result, results in yielding early convergence and non-optimal solutions [39] . Therefore, hyperparameter optimization techniques which benefit from genetic algorithm-based approaches are also likely to be problematic.

Lima [33] compared various hyperparameter optimization algorithms such as random search, simulating annealing, and tree-of-parzen estimators in order to find the most effective CNN architecture in the classification of benign and malignant small pulmonary nodules. Kumar and Hati [24] proposed the adaptive gradient optimizer-based deep convolutional neural network (ADG-dCNN) approach for bearing and rotor faults detection in squirrel cage induction motor. Ilievski et al. [40] used radial basis function (RBF) as a surrogate of hyperparameter optimization in order to reduce the complexity of original network. Talathi [41] proposed a simple sequential model based optimization algorithm in order to optimize hyperparameters in deep CNN architectures.

Rattanavorragant and Jewajinda proposed an approach using an island-based genetic algorithm in order to optimize hyperparameters in DNN automatically [42] . This approach involves two steps: hyperparameter search and a detailed DNN training. Navaneeth and Suchetha proposed the optimized one-dimensional CNN with support vector machine (1-D CNN-SVM) approach in order to diagnose chronic kidney diseases using PSO algorithm [43] .

Compared to the literature review above, the main contribution of the present study is that the proposed COVID-CCD-Net approach can detect two important diseases: COVID-19 and colon cancer in TMAs. In addition, the proposed approach benefits from GBO, which is a metaheuristic approach, for the optimization of CNN models to overcome various problems mentioned in the existing literature.

In recent times, many studies focusing on the diagnosis of COVID-19 using CNN have been published [44] [45] [46] [47] [48] [49] [50] . The literature review indicates that some of these studies [45] [46] [47] focused on the diagnosis of COVID-19 in non-COVID cases. On the other hand, there are also studies which classified cases into three groups as COVID, normal, and pneumonia [48] [49] [50] . Within the framework of the present study, the proposed COVID-CCD-Net approach classifies chest X-ray images into three different groups as COVID, normal, and pneumonia.

Shi et al. [51] performed a detailed literature review regarding the state-of-the-art computer-assisted methods for the diagnosis of COVID-19 in X-ray and CT scans. Castiglioni et al. [52] benefited from two chest X-ray datasets containing 250 COVID-19 and 250 non-COVID cases in order to perform training, validation, and testing processes for Resnet-50.

Hemdan et al. [53] proposed a deep learning-based approach called COVIDX-Net in order to diagnose COVID-19 in chest X-ray images automatically. This study involved seven different deep architectures, namely MobileNetV2, VGG19, InceptionV3, DenseNet201, InceptionResNetV2, ResNetV2, and Xception. Khan et al. [54] proposed a CNN-based approach called CoroNet in order to diagnose COVID-19 using X-ray and CT scans based on Xception architecture. The experimental studies demonstrated that the proposed model yielded an overall accuracy rate of 89.6% in four different classes (COVID vs. pneumonia bacterial vs. pneumonia viral vs. normal) and an overall accuracy rate of 95% in three different classes (normal vs. COVID vs. pneumonia).

The proposed COVID-CCD-Net approach differs from other studies on the detection of COVID-19 using CNN models in that it improves classification performance by optimizing hyperparameters of CNN models thanks to GBO approach.

As can be seen in various studies in the existing literature, the number of studies dealing with automatic diagnosis of colon cancer in TMAs is limited. Nguyen et al. [55] analyzed different ensemble approaches for colorectal tissue classification using highly efficient TMAs and proposed an ensemble deep learning-based approach with two different neural network architectures called VGG16 and CapsNet. Thanks to this approach, they classified colorectal tissues in highly efficient TMAs into three different categories, namely tumor, normal, and stroma/others. Xu et al. [56] proposed a deep CNN approach in order to perform the segmentation and classification of epithelial ve stromal regions in TMAs. This study benefited from two different datasets containing breast and colorectal cancer images. Finally, Linder et al. [57] proposed an approach for an automatic detection of epithelial ve stromal regions in colorectal cancer TMAs thanks to texture features and a SVM classifier.

The proposed COVID-CCD-Net approach is superior to other studies on the detection of colon cancer in TMAs using CNN models in that it optimizes the hyperparameters of CNN models, which significantly increases the detection accuracy rates of colon cancer. The effective performance of CNN in image classification contributes to the present study to a higher extent compared to other studies using other approachs for the classification of colon cancer in TMAs in the existing literature.

Inspired by gradient-based Newton's method, GBO was proposed by Ahmadianfar et al. [16] as one of the most recent metaheuristic algorithms. This algorithm is based on two main operators: gradient search rule (GSR) and local escaping operator (LEO). Main steps of GBO are described below. 

In GBO, each member of the population is called a "vector" and, as seen in Eq. 1, the population consists of N number of vectors in a D-dimension search space.

(1)

As shown in Eq. 2, each vector in the initial population is created by assigning random values within the boundaries of search space.

Here, X min and X max are lower and upper boundaries in the search space, respectively, while rand(0,1) is a random number in a range of [0,1].

GSR operator is used in GBO in order to increase exploration ability, eliminate local minimum, and accelerate the convergence rate. Thus, optimal solutions can be obtained within the search space [16] .

(2) The position of a vector in the next iteration (x n m+1 ) is calculated using Eqs. 3 and 4 with: X1 n m , X2 n m , and x n m , which denotes the current position of the vector. Here, x n m and x best are the current position and the best vector in the population, respectively. GSR denotes the gradient search rule, while DM represents the direction of movement. GSR enables GBO to assign randomly, improve its exploration ability and eliminate local minimals. GSR can be calculated as shown in the following equations [16] :

Here, rand(1:N) is an N-dimensional random number, r 1 , r 2 , r 3 , and r 4 denote random integer numbers selected from a range of [1, N] , and, finaly, step represents the step size.

DM shown in Eq. 11 helps the current position of the vector (x n ) move along the direction of x best -x n and thus provides local searching in order to improve convergence speed of GBO [16] .

Global exploration and local exploitation must be balanced in an algorithm in order to find solutions closer to a global optimal value. p 1 and p 2 parameters in Eqs. 4, 7, and 11 are used to balance exploration and exploitation in GBO [16] . These parameters are calculated using the following equations:

Here, β min and β max are 0.2 and 1.2, respectively, and m denotes the current number of iteration. M represents the maximum number of iteration. Here, f 1 and f 2 are random numbers generated in a range of [−1, 1], and u 1 , u 2 , and u 3 are three randomly generated and different numbers, while x k m is a newly generated vector. u 1 , u 2 , u 3, and x k m are defined as shown in the following equations:

Here, rand, μ 1 , and μ 2 are random numbers in a range of [0, 1], x rand denotes a randomly generated new vector, and x p m is a vector randomly selected from the population [16] . Flowchart of the GBO is shown in Fig. 1. 

Convolutional neural networks (CNN) is a special type of neural network inspired by the biological model of animal visual cortex [58, 59] . They are particularly used in the field (20) of image and sound processing due to their main advantage: the extraction of automatic and adaptive features during a training process [60] . In CNNs, the variable of the network structure (kernel size, stride, padding, etc.) and the network trained (learning rate, momentum, optimization strategies, batch size etc.) are known as hyperparameters [29] , which must be adjusted accurately for a more effective CNN performance.

In the present study, learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold value, which are among network trained hyperparameters of AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet, were optimized using GBO algorithm. Learning rate, which is also known as step size, is decisive in terms of updating weights [61, 62] . Solver, on the other hand, represents the optimization method to be used such as Adam, Sgdm, or Rmsprop [63] . The L2 regularization, which is also called weight decay, is a simple regularization method that scales weights down in proportion to their current size [64, 65] . Gradient threshold method and gradient threshold value are parameters related to gradient clipping. If the gradient increases exponentially in magnitude, it means that the training is unstable and can diverge within a few iterations. Gradient clipping helps avoid the exploding gradient problem. If the gradient exceeds the value of gradient threshold, then the gradient is clipped according to gradient threshold method [66, 67] .

Input image size in AlexNet architecture, developed by Krizhevsky et al. [68] , is 227×227. It consists of 5 convolution and 3 fully connected layers, thus reaching a depth of 8 layers. DarkNet-19 has a depth of 19 layers and its input image size is 256×256 [69] . Introduced by Szegedy et al. [70] , Inception-v3 model has a depth of 48 layers with an input image size of 299×299. ResNet-18, which has a depth of 18 layers and an input image size of 224×224, was developed by He et al. [71] . Zhang et al. [72] proposed ShuffleNet model with a depth of 50 layers and an input image size of 224×224. Finally, MobileNet, which was proposed by Sandler et al. [73] , has a depth of 53 layers and an input image size of 224×224.

In the present study, hyperparameters of AlexNet, Dark-Net-19, Inception-v3, MobileNet, ResNet-18, and Shuf-fleNet CNN models such as learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold value were optimized using GBO algorithm in order to classify COVID-19, normal, and viral pneumonia in chest X-ray images. In addition, other types such as epithelial and stromal regions in epidermal growth factor receptor (EFGR) colon in TMAs can also be classified. The proposed approach is called COVID-CCD-Net, as shown in the flowchart in Fig. 2 .

In the proposed COVID-CCD-Net approach, initial parameters of GBO such as ε, the number of population and maximum number of iteration are adjusted. Then, an initial population is created by using vectors with randomly assigned values. Each vector consists of 5 dimensions which represent learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold parameters of CNN models. Lower boundary (LB) and upper boundary (UB) values of these parameters are given in Table 1 . Learning rate, L2 regularization, and gradient threshold are real values which are randomly generated between LB and UB values. If the solver value is 1, 2, or 3, "sgdm," "adam," and "rmsprop" optimization method is selected, respectively. If the gradient threshold method value is 1, 2, or 3, "l2norm," "global-l2norm," and "absolute-value" method is selected, respectively. In parallel with these boundaries, each vector in the initial population is generated using the formula in Eq. 22:

The following steps are taken in order to calculate the fitness value of each vector: Firstly, X n vector whose fitness (22) X n,i = LB i + rand (0, 1) × UB i − LB i , n = 1, 2, … , N and i = 1, 2, … , 5 value will be calculated is sent to CNN model and the values of X n vector are assigned to learning rate, solver, L2 Fig. 9 Confusion matrices of Epistroma dataset regularization, gradient threshold method, and gradient threshold parameters of CNN model. Later, CNN model is trained using the training dataset. Following the training processes, validation accuracy value obtained from the training is sent back to GBO and assigned as the fitness value of X n vector.

As shown in Fig. 2 , each step of the algorithm is iterated until it reaches a maximum number of iterations. At the end, the vector with the most optimal fitness value is accepted as the solution of the problem.

The present study proposes the COVID-CCD-Net approach in which learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold parameters of AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet were optimized using GBO. The classification performance of the proposed approach was tested using two different medical image classification datasets. Additionally, the results of this test were compared with those obtained from non-optimized AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet CNN models. In addition, Quasi-Newton (Q-N) algorithm [74] , one of the most fundamental optimization methods, was also used to optimize the hyperparameters of CNN models and compared with the proposed COVID-CCD-Net approach. The following sub-sections describe medical image classification datasets, experiment setup, and present comparative experimental findings. 76] and Epistroma [77] datasets were selected for the experimental studies. COVID-19 dataset consists of three classes, namely "Covid-19," "Normal," and "Viral Pneumonia," with a total of 3829 images. Epistroma dataset, on the other hand, consists of two classes, namely "epithelium" and "stroma," with a total of 1376 images. In both datasets, 80% and 20% images were used for training and testing processes, respectively and we have performed 5-fold cross-validation. Ten percent of the training data in each data set was also used for validation. Samples images from both datasets are shown in Fig. 3 .

All experimental studies were carried out on MATLAB R2020a platform. The number of vectors in GBO population and the maximum number of iterations were selected as 10 in the proposed COVID-CCD-Net approach. In other words, the fitness function is called 100 times. Q-N algorithm performs search starting at a single point instead of a population-based search. For a healthier comparison with the proposed approach, the number of maximum iterations was selected as 100 in Q-N algorithm to call the fitness function 100 times. In addition, default MATLAB values for solver, L2 regularization, gradient threshold method, and gradient threshold parameters were selected as "sgdm," "0.0001," "l2norm," and "Inf," respectively for non-optimized CNN models. Values of epoch for all CNN models were selected as 2 for COVID-19 dataset as 5 for Epistroma dataset. Mini batch size was set to 25. Twenty independent experimental studies were conducted on these datasets for all CNN models, and the obtained mean accuracy, maximum accuracy, F1-score, and standard deviation values were compared to measure the performances of all models.

Mean accuracy, maximum accuracy, F1-score, and standard deviation values obtained from 20 different independent studies on COVID-19 and Epistroma datasets are given in Tables 2 and 5 , respectively. The findings were also shown in bar charts in Figs. 4 and 5 to give a clearer picture of the overall findings.

The findings related to COVID-19 dataset demonstrated that in the training process, COVID-CCD-Net (ResNet-18) reached the highest mean validation accuracy, maximum validation accuracy, and F1-score values with 97.977, 98.532, and 98.063, respectively. The second highest values were yielded by COVID-CCD-Net (DarkNet-19) with 97.553, 98.532, and 97.654, while non-optimized MobileNet displayed a lower performance with 82.007, 86.134, and 81.716. In the testing process, COVID-CCD-Net (ResNet-18) classified test images with a mean accuracy rate of 98.107%, followed by Darknet-19 with a mean accuracy rate of 97.369%. MobileNet displayed the lowest performance in terms of training and testing. validation and test accuracy for COVID-19 dataset before and after optimization with COVID-CCD-Net are given in Table 3 and the results demonstrated that COVID-CCD-Net increased the classification performance of the non-optimized CNN models by 6.22-13.29%. The performance was improved when Q-N algorithm was used to optimize the hyperparameters of non-optimized CNN models. However the performance increased between 2.92 and 8.40%, demonstrating that GBO displays a higher performance in the hyperparameter optimization in COVID-19 dataset.

It can understand from the findings related to Epistroma dataset that in the training process, the highest mean accuracy, maximum accuracy, and F1-score values were obtained by COVID-CCD-Net (Inception-v3) Table 4 and the results demonstrated that COVID-CCD-Net increased the classification performance of the non-optimized CNN models by 2.11-6.81%. The performance was improved when Q-N algorithm was used to optimize the hyperparameters of non-optimized CNN models. It can be seen in Table 5 that the performance increased between 1.81 and 5.43%, demonstrating that GBO displays a higher performance in the hyperparameter optimization in Epistroma dataset.

As shown in Tables 2 and 5 , GBO algorithm remarkably improves the performance of non-optimized CNN models in COVID-19 and Epistroma datasets. Additionally, experimental studies indicated that GBO algorithm displayed a higher performance in hyperparameter optimization in both datasets compared to Q-N algorithm.

Mean training accuracy curves of all models obtained from COVID-19 dataset are shown in Fig. 6 . While COVID-CCD-Net (ResNet-18) displayed a faster convergence, nonoptimized MobileNet displayed a slower convergence. Mean training accuracy curves of all models obtained from Epistroma dataset are shown in Fig. 7 , COVID-CCD-Net (Inception-v3), COVID-CCD-Net (ResNet-18), and COVID-CCD-Net (DarkNet-19) displayed a fast convergence in the first 20 iterations and a lower convergence in the remaining iterations.

Maximum and mean confusion matrix values of all models obtained from the testing processes for COVID-19 and Epistroma datasets are shown in Fig. 8 and Fig. 9 . A confusion matrix is a table which is used to describe the performance of a model by referring to its accuracy rates in each class. Rows and columns in a confusion matrix correspond to the predicted class (output class) and true class (target class), respectively.

The receiver operating characteristic (ROC) curves of COVID-19 and Epistroma datasets are provided in Fig. 10 and Fig. 11 respectively, which showing the relationship between the false positive rate (FPR) and the true positive rate (TPR). It can be clearly seen, in COVID-19 dataset COVID-CCD-Net (ResNet-18) and in Epistroma dataset COVID-CCD-Net (Inception-v3) have higher true positive rates. Table 6 and Table 7 compare the performance of the COVID-CCD-Net with several state-of-the art methods on COVID-19 and Epistroma datasets. It can be seen obviously; the the COVID-CCD-Net has the highest classification accuracy among the compared methods for both datasets. 

In order to classify Covid-19, normal, and viral pneumonia in chest X-ray images as well as epithelial and stromal regions in TMA images accurately, the present study proposed the COVID-CCD-Net approach with the optimized hyperparameters of AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet CNN models using GBO, which is one of the most recent metaheuristic optimization algorithms. Network-trained parameters of these CNN models such as learning rate, solver, L2 regularization, gradient threshold method, and gradient threshold were optimized and tuned using GBO algorithm. In the GBO, each vector of the population represents a set of CNN's hyperparameters, and the algorithm searches for the hyperparameter values that help the model display the highest classification performance. Two different medical image classification datasets, i.e., COVID-19 and Epistroma, were used in the experimental study. While GBO hyperparameter optimization improved the performance of non-optimized CNN models in COVID-19 dataset by 6.22% to 13.29%, the contribution of Q-N algorithm did not exceed 2.92% to 8.40%. Similarly, GBO hyperparameter optimization improved the performance of non-optimized CNN models in Epistroma dataset by 2.11% to 6.81%, Q-N algorithm improved it only 1.81% to 4.53%. These results demonstrated that the proposed approach significantly improved the classification performance of AlexNet, DarkNet-19, Inception-v3, MobileNet, ResNet-18, and ShuffleNet CNN models and displayed a better performance compared to non-optimized CNN models. One of the main problems in CNN-based classification approaches is their need for a high number of high-quality images for a succesful classification performance and optimal values for the hyperparameters of CNN architecture. In the present study, a sufficient number of images was used to complete training process for CNN architecture, and the proposed COVID-CCD-Net approach was used to optimize the hyperparameters of CNN architectures to overcome the above-mentioned problems. Future studies will focus on the optimization of different hyperparameters such as filter size, filter number, stride, and padding using various metaheuristic optimization algorithms.

WCOVID-19 Weekly epidemiological update data as received by WHO from national authorities

The species severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2

Clinical features of patients infected with 2019 novel coronavirus in

Using X-ray images and deep learning for automated detection of coronavirus disease

Application of deep learning techniques for detection of COVID-19 cases using chest X-ray images: a comprehensive study

The role of imaging in the detection and management of COVID-19: a review

Essentials for radiologists on COVID-19: an update-radiology scientific expert panel

Lightweight deep learning models for detecting COVID-19 from chest X-ray images

CVDNet: a novel deep learning architecture for detection of coronavirus (Covid-19) from chest x-ray images

Deep-covid: predicting covid-19 from chest x-ray images using deep transfer learning

CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images

InstaCovNet-19: a deep learning classification model for the detection of COVID-19 patients using chest X-ray

Tissue microarrays for high-throughput molecular profiling of tumor specimens

Impact of EGFR expression on colorectal cancer patient prognosis and survival

EGFR and KRAS in colorectal cancer

Gradient-based optimizer: a new metaheuristic optimization algorithm

Nature-inspired metaheuristic algorithms

A review of populationbased meta-heuristic algorithms

Source characterization of airborne pollutant emissions by hybrid metaheuristic/gradientbased optimization techniques

Metaheuristics to solve grouping problems: a review and a case study

A new metaheuristic based on vapor-liquid equilibrium for solving a new patient bed assignment problem

Combined Metaheuristic Algorithm and Radiomics Strategy for the Analysis of Neuroanatomical Structures in Schizophrenia and Schizoaffective Disorders

Optimal hyperparameter tuning of convolutional neural networks based on the parametersetting-free harmony search algorithm

Deep convolutional neural network based on adaptive gradient optimizer for fault detection in SCIM

Pandey HM (2020) An optimized dense convolutional neural network model for disease recognition and classification in corn leaf

Advances in neural information processing systems

EO-CNN: an enhanced CNN model trained by equilibrium optimization for traffic transportation prediction

Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms

Hyperparameter optimization in convolutional neural network using genetic algorithms

Random search for hyper-parameter optimization

cPSO-CNN: an efficient PSObased algorithm for fine-tuning hyper-parameters of convolutional neural networks

Hyper-parameters optimisation of deep CNN architecture for vehicle logo recognition

Toward classifying small lung nodules with hyperparameter optimization of convolutional neural networks

Texture classification using convolutional neural network optimized with whale optimization algorithm

Weighted random search for CNN hyperparameter optimization

EA-based hyperparameter optimization of hybrid deep learning models for effective drug-target interactions prediction

Convolutional neural networks-based lung nodule classification: A surrogate-assisted evolutionary algorithm for hyperparameter optimization

Taking the human out of the loop: a review of bayesian optimization

An automatic convolutional neural network optimization using a diversity-guided genetic algorithm

Efficient hyperparameter optimization for deep learning algorithms using deterministic rbf surrogates

Hyper-parameter optimization of deep convolutional networks for object recognition

A hyper-parameter optimization for deep neural network using an island-based genetic algorithm

PSO optimized 1-D CNN-SVM architecture for real-time detection and classification applications

COVIDetection-Net: A tailored COVID-19 detection from chest radiography images using deep learning

Explainable deep learning for pulmonary disease and coronavirus COVID-19 detection from X-rays

Application of deep learning for fast detection of COVID-19 in X-rays using nCOVnet

Classification of COVID-19 patients from chest CT images using multi-objective differential evolution-based convolutional neural networks

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

COVID-19 detection from chest X-ray images using deep learning and convolutional neural networks

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19

Machine learning applied on chest x-ray can aid in the diagnosis of COVID-19: a first experience from

Covidx-net: a framework of deep learning classifiers to diagnose covid-19 in x-ray images

CoroNet: a deep neural network for detection and diagnosis of COVID-19 from chest x-ray images

Classification of colorectal tissue images from high throughput tissue microarrays by ensemble deep learning methods

A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images

Identification of tumor epithelium and stroma in tissue microarrays using texture analysis

Simple convolutional neural network on image classification

Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network

Convolutional neural networks: an overview and application in radiology

Optimization of hyper-parameter for CNN model using genetic algorithm

Accuracy enhancement of a blind image steganalysis approach using dynamic learning rate-based CNN on GPUs

Image segment classification using CNN

Convolutional neural networks for human activity recognition using mobile sensors

A deep convolutional neural network model for intelligent discrimination between coal and rocks in coal mining face

On the difficulty of training recurrent neural networks

Deep gradient compression: Reducing the communication bandwidth for distributed training

ImageNet classification with deep convolutional neural networks

Darknet: Open source neural networks in C

Rethinking the inception architecture for computer vision

Deep residual learning for image recognition

Shufflenet: an extremely efficient convolutional neural network for mobile devices

Mobilenetv2: inverted residuals and linear bottlenecks

Numerical optimization

COVID-19 chest X-ray database. Kaggle Data, v3

Can AI help in screening viral and COVID-19 pneumonia

EGFR colon TMA stroma LBP classification

Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images

Automated detection of COVID-19 cases using deep neural networks with X-ray images

A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19)

COVID-19 classification of X-ray images using deep neural networks

COVID19XrayNet: a two-step transfer learning model for the COVID-19 detecting problem based on a limited number of chest X-ray images

Opt-CoNet: an optimized convolutional neural network for an automatic diagnosis of COVID-19

Histological image classification using deep features and transfer learning

Dimensionality reduction strategies for cnn-based classification of histopathological images

Epitheliumstroma classification via convolutional neural networks and unsupervised domain adaptation in histopathological images

Improved opponent color local binary patterns: an effective local image descriptor for color texture classification