key: cord-1037032-3oo8p9cp authors: Bandyopadhyay, Rajarshi; Basu, Arpan; Cuevas, Erik; Sarkar, Ram title: Harris Hawks optimisation with Simulated Annealing as a deep feature selection method for screening of COVID-19 CT-scans date: 2021-07-14 journal: Appl Soft Comput DOI: 10.1016/j.asoc.2021.107698 sha: 268876eafa676294ebb51cfe92a3a964e7bf468e doc_id: 1037032 cord_uid: 3oo8p9cp Coronavirus disease 2019 (COVID-19) is a contagious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). It may cause severe ailments in infected individuals. The more severe cases may lead to death. Automated methods which can detect COVID-19 in radiological images can help in the screening of patients. In this work, a two-stage pipeline composed of feature extraction followed by feature selection (FS) for the detection of COVID-19 from CT scan images is proposed. For feature extraction, a state-of-the-art Convolutional Neural Network (CNN) model based on the DenseNet architecture is utilised. To eliminate the non-informative and redundant features, the meta-heuristic called Harris Hawks optimisation (HHO) algorithm combined with Simulated Annealing (SA) and Chaotic initialisation is employed. The proposed approach is evaluated on the SARS-COV-2 CT-Scan dataset which consists of 2482 CT scans. Without the Chaotic initialisation and the SA, the method gives an accuracy of around 98.42% which further increases to 98.85 % on the inclusion of the two and thus delivers better performance than many state-of-the-art methods and various meta-heuristic based FS algorithms. Also comparison has been drawn with many hybrid variants of meta-heuristic algorithms. Although HHO falls behind a few of the hybrid variants, when Chaotic initialisation and SA are incorporated into it, the proposed algorithm performs better than any other algorithm with which comparison has been drawn. The proposed algorithm decreases the number of features selected by around 75% , which is better than most of the other algorithms. * Corresponding author Email addresses: rajarshibanerjee03@gmail.com (Rajarshi Bandyopadhyay), arpan0123@gmail.com (Arpan Basu), erik.cuevas@cucei.udg.mx (Erik Cuevas), ramjucse@gmail.com (Ram Sarkar) been widely adopted for the purpose of automatic feature 19 extraction from data as opposed to manual feature en- algorithms so that the shortcomings of one algorithm is 95 overcome by the other algorithm(s) [10] . Suppose if an al-96 gorithm with good exploration properties is coupled with 97 an algorithm that exhibits good exploitation, it will en-98 hance the performance of the entire system as a whole 99 and establish a trade-off between the two properties in the 100 system. Here, the local optima problem that HHO suffers proposed an end-to-end pipeline for the same task. The 2. Singer Chaotic Map: 4. Chebyshev Chaotic Map: 5. Tent Chaotic Map: 6. Logistic Chaotic Map: 7. Iterative Chaotic Map: In this phase, all the Harris Hawks are considered as possible solutions to the FS problem. The Harris Hawks' are one of the most intelligent birds and they can effortlessly trace a prey with their keen eyes but at times, the prey cannot be seen. In HHO, each Harris Hawk is considered a possible solution and the prey is taken to be the Harris Hawk which yields the minimum value when passed to the objective function (the best possible solution amongst the current set of Harris Hawks, one which yields maximum classification accuracy selecting the minimum number of features possible). There are two possible ways to imitate the exploration strategy of Harris Hawks as stated in Equation 8. (8) Here, P (i) denotes the position of a Harris Hawk in the i th iteration. P (i + 1) is the new position of the Harris Hawk at the end of the i th iteration. P prey (t) is the location of the prey. P rand (t) signifies an arbitrary solution chosen from the current population of the Harris Hawks. Here, r, f 1 , f 2 , f 3 , f 4 are numbers that are randomly chosen from the range (0, 1).LL denotes the lower limit or bound and U L stands for the upper limit or bound.P m stands for average position of the current generation of hawks. And the formula for P m is given by equation 9. where, the size of the current population is M and the 385 position of every hawk in the i th iteration is given by P j (i). The HHO algorithm can exhibit a changeover between the exploration and exploitation aspects. It can also show alteration between various exploitative actions based on the escaping energy of the rabbit and the probability of the rabbit escaping. As the prey attempts to escape, its escaping energy diminishes. If the initial energy of the prey is depicted by X 0 , the current iteration be denoted by m and the total iterations be denoted by M , then the escaping energy of prey in the m th iteration (say, X) can be depicted by equation 10 . The value of X 0 varies in the range [-1, 1] randomly. When Soft besiege is opted for when t ≥ 0.5 and |X| ≥ 0.5. The prey or the rabbit has ample energy but the hawks surround the rabbit when it tries to deceive the Harris Hawks and get away. Finally, the hawks carry out the surprise dive on the prey. This can be drafted mathematically as given in equation 11 to equation 13. Here, f 5 is a random number between 0 and 1, and JS Hard besiege is selected when t ≥ 0.5 and |X| < 0.5. The rabbit is worn out and the amount of energy it has to escape is low. The Harris Hawks barely need to perform the surprise dive. Soft besiege with continuous or progressive rapid dives is the next technique. When the prey has |X| ≥ 0.5 and the value of t < 0.5, it has more energy to get away, and also good chances of escaping. Here the Harris Hawks carry out the surprise dive in two moves. In the first move, the Hawks surround the prey and move to a location after assessing the next move of the prey. (17) where the value of β is set to 1.5 and the other two parameters, v 1 and v 2 are generated randomly in the range 0 to 1 (both inclusive). In the method of soft besiege with progressive rapid dives, the position of the Harris Hawk at the end of the iteration can be updated according to the formula stated in equation 18. where f itness(x) denotes the value obtained when x is 425 passed as parameter to the fitness or the objective func-426 tion. The last technique is Hard besiege with continuous 428 or progressive rapid dives, when t < 0.5 and |X| < 0.5. There is not much energy left in the prey to escape and where P m (i) is obtained as described in equation (3). Table 4 ). if |X| ≤ 1 then The values of the constraint are tabulated in Table 4 565 and as evident from the table, discarding the solutions that Table 7 . 610 been found out to be fifteen by tuning the hyperparame-617 ter), whichever is more. The percentage increase column 618 in Table 6 is with respect to the classification accuracy The results obtained by the various algorithms are high-717 lighted in Table 10 The results indicate that the inclusion of the feature ex- g 2 (x) = p rz V sr − V sr,max p max ≤ 0 Generalized fisher 824 score for feature selection A hybrid genetic 827 algorithm for feature selection wrapper based on mutual infor-828 mation A practical approach to feature 830 selection A bpso-svm algo-834 rithm based on memory renewal and enhanced mutation mech-835 anisms for feature selection An advanced 838 aco algorithm for feature subset selection Regression shrinkage and selection via 841 the lasso A gen-844 eralized uncorrelated ridge regression with nonnegative labels 845 for unsupervised feature selection Said Jadid Abdulkadir, Helmi Md Rais Approaches to multi-850 objective feature selection: A systematic literature review No free lunch theo-853 rems for optimization A taxonomy of hybrid metaheuristics Hybrid genetic 858 algorithms for feature selection Feature selection for facial emotion recognition using 862 late hill-climbing based memetic algorithm Investigating memetic 865 algorithm in solving rough set attribute reduction Ehhm: Electrical harmony based 871 hybrid meta-heuristic for feature selection A hybrid swarm and gravitation-875 based feature selection algorithm for handwritten indic script 876 classification problem. Complex & Intelligent Systems Densely connected convolutional networks Harris hawks opti-884 mization: Algorithm and applications. Future Generation Com-885 puter Systems Simulated 887 annealing An 891 improved harris hawks optimization algorithm with simulated 892 annealing for feature selection in the medical field Binary optimization 896 using hybrid grey wolf optimization for feature selection Hybrid whale opti-899 mization algorithm with simulated annealing for feature selec-900 tion An improved cuckoo 902 search optimization algorithm for the problem of chaotic sys-903 tems parameter estimation A hybrid particle 906 swarm optimization for feature subset selection by integrating 907 a novel local search strategy Sars-cov-2 ct-scan dataset: A 911 large dataset of real patients ct scans for sars-cov-2 identifica-912 tion. medRxiv Covidgan: Data augmentation using auxil-915 iary classifier gan for improved covid-19 detection Vijay Ku-918 mar, and Manjit Kaur. Classification of the covid-19 infected Biomolecular Structure and Dynamics Iteratively pruned deep learning en-924 sembles for covid-19 detection in chest x-rays Optconet: an optimized convolutional neural 941 network for an automatic diagnosis of covid-19 An 944 optimized deep learning architecture for the diagnosis of covid-945 19 disease based on gravitational search optimization Adam: A method for 948 stochastic optimization Performance anal-950 ysis of chaotic multi-verse harris hawks optimization: A case 951 study on solving engineering problems. Engineering Applica-952 tions of Artificial Intelligence Developing and applying chaotic Cooperative hunting harris' hawks (parabu-959 teo unicinctus) Pitch-962 ford Scaling laws 966 of marine predator search behaviour Lévy flights in random searches. Physica A: Statistical Me-971 chanics and its Applications Nature-inspired metaheuristic algorithms Optimization 975 by simulated annealing An introduction to kernel and nearest-977 neighbor nonparametric regression A genetic algorithm tutorial The whale optimization 982 algorithm Advances in engineering software Particle swarm optimization Joong Hoon Kim, and Gobichettipalayam Va-990 sudevan Loganathan. A new heuristic optimization algorithm: 991 harmony search Seyed Mohammad Mirjalili, and Xin-She Binary bat algorithm. Neural Computing and Appli-994 cations Hossein Nezamabadi-pour, and Saeid Saryazdi GSA: A gravitational search algorithm A mayfly 999 optimization algorithm Sca: a sine cosine algorithm for solving 1002 optimization problems. Knowledge-based systems Mostafa Hajiaghaei-1005 Red deer algorithm 1006 (rda): a new nature-inspired meta-heuristic. Soft Computing Extremal optimiza-1009 tion: Methods derived from co-evolution. arXiv preprint 1010 math/9904056 Mayfly in 1013 harmony: A new hybrid meta-heuristic feature selection algo-1014 rithm Late 1017 acceptance hill climbing based social ski driver algorithm for 1018 feature selection A new hybrid whale opti-1020 mizer algorithm with mean strategy of grey wolf optimizer for 1021 global optimization. Mathematical and Computational Appli-1022 cations Hybrid of harmony search 1025 algorithm and ring theory-based evolutionary algorithm for fea-1026 ture selection Vijay Ku-1028 mar, and Manjit Kaur. Classification of the COVID-19 infected 1029 patients using DenseNet201 based deep transfer learning Contrastive cross-site 1033 learning with redesigned net for covid-19 ct classification COVID-19 de-1038 tection in CT images with deep learning: A voting-based scheme 1039 and cross-datasets analysis Efficient deep network architecture for 1043 COVID-19 detection using computed tomography images A test-suite of non-convex constrained optimization 1048 problems from the real-world and some baseline results • A two-stage pipeline for the detection of COVID-19 in CT-scan images is proposed DenseNet, is trained as a feature extractor via transfer learning. • Feature selection is performed using the Harris Hawks Optimization (HHO) algorithm • Evaluation on SARS-COV-2 CT-Scan Dataset gives better results than some past methods It is also noted that this stage also reduces the number of