key: cord-0891111-u0h5sgli
authors: Saleh, Hager; Abd-el ghany, Sara F.; Alyami, Hashem; Alosaimi, Wael
title: Predicting Breast Cancer Based on Optimized Deep Learning Approach
date: 2022-03-19
journal: Comput Intell Neurosci
DOI: 10.1155/2022/1820777
sha: 13706dc9de3e7e82240cbdea3c676fc74b7c4851
doc_id: 891111
cord_uid: u0h5sgli

Breast cancer is a dangerous disease with a high morbidity and mortality rate. One of the most important aspects in breast cancer treatment is getting an accurate diagnosis. Machine-learning (ML) and deep learning techniques can help doctors in making diagnosis decisions. This paper proposed the optimized deep recurrent neural network (RNN) model based on RNN and the Keras–Tuner optimization technique for breast cancer diagnosis. The optimized deep RNN consists of the input layer, five hidden layers, five dropout layers, and the output layer. In each hidden layer, we optimized the number of neurons and rate values of the dropout layer. Three feature-selection methods have been used to select the most important features from the database. Five regular ML models, namely decision tree (DT), support vector machine (SVM), random forest (RF), naive Bayes (NB), and K-nearest neighbor algorithm (KNN) were compared with the optimized deep RNN. The regular ML models and the optimized deep RNN have been applied the selected features. The results showed that the optimized deep RNN with the selected features by univariate has achieved the highest performance for CV and the testing results compared to the other models.

Breast cancer (BC) is one of the most frequent malignant tumors in the world, accounting for 10.4% of all cancer deaths in women aged between 20 and 50 [1] . According to the World Health Organization figures, 2.3 million women will be diagnosed with BC in 2020. BC has been diagnosed in 7.8 million women in the previous 5 years, making it the most frequent malignancy worldwide. BC causes more disability-adjusted life years (DALYs) in women worldwide than any other type of cancer. BC strikes women at any age after puberty in every country on the planet, with rates rising as they become older. For all of these reasons, there is an ongoing need for a reliable and accurate system that can be used to help in the early detection and diagnosis of BC diseases to reduce the number of deaths. In the field of medical analysis, machine-learning (ML) algorithms can be applied extensively [2] , for example, predicting COVID-19 [3] , predicting Alzheimer's progression [4] , predicting chronic diseases [5] , predicting liver disorders [6] , heart disease [7] , cancer [8] , and others [9, 10] . ML and deep learning (DL) play a significant role in solving health problems and identifying diseases, such as cancer prediction. Many researchers have applied ML and DL techniques to develop models and systems to predict BC. For example, Asri et al. [11] applied ML algorithms, namely, decision tree (DT), support vector machine (SVM), naive Bayes (NB), and K-nearest neighbor (KNN) algorithm on the Breast Cancer Wisconsin (Diagnostic) Data set (BCWD) to predict BC. e result indicated that the SVM classifier was the best. Naji et al. [12] applied five ML algorithms: SVM, random forest (RF), logistic regression (LR), DT, and KNN on BCWD to predict BC. e results demonstrated that SVM had registered the highest accuracy. On their part, Amrane et al. [13] applied ML algorithms, KNN, and NB on BCWD database to predict BC. e results showed that KNN achieved the highest accuracy. Bayrak et al. [14] have applied SVM and artificial neural network on BCWD to predict BC. e results revealed that the best model is registered by SVM. Islam et al. [15] applied five ML techniques: SVM, KNN, RF, LR, and ANNs on BCWD to predict BC. e results showed that the ANNs registered the highest performance. Abdel-Zaher et al. [16] proposed deep neural networks (DNNs) that consist of three hidden layers and two dropout layers to predict BC. ey used the BCWD to make the experiment.

e results proved that DNN model has achieved the best performance. Also, Prananda et al. [17] proposed a DNN model that consists of three hidden layers and two dropout layers to classify BC. ey applied DNN model, SVM, NB, and RF on BCWD data set. e results revealed that the DDN model has registered a significant performance. Soliman et al. [18] designed a hybrid approach based on DNN to improve the classification accuracy. Karaci [19] proposed a DNN model with four hidden layers and two output layers that classify women with or without breast cancer. Nahid et al. [20] proposed the DNN technique for breast cancer picture classification using convolutional neural networks (CNNs), long short-term memory (LSTM), and a combination of CNN and LSTM. Also, Darapurredy et al. [21] used the deep neural network for classifying breast cancer.Feature-selection methods are used to reduce the number of features and selected the subset of features that improve the performance of classification algorithms. For example, Habib et al. [22] applied the genetic programming (GP) as the feature-selection method to select the important features from the BCWD database. ey applied nine ML algorithms, namely SVM, KNN, RF, LR, DT, NB, gradient boosting (GB) classifier, AdaBoost (AB) classifier, and linear discriminant analysis (LDA) to select features to predict BC. e results showed that LR, LDA, and GNB algorithms fit best compared to the other methods. Luo et al. [23] used two feature-selection methods: forward selection (FS) and backward selection (BS) to reduce the number of features and improve accuracy. ey applied SVM, DT, and ensemble techniques on the BCWD database to predict BC. e results indicated that the ensemble technique with the feature-selection methods had achieved the best performance. Emina et al. [24] used GA feature-selection methods to select the best subfeatures from the BCWD database. ey applied different algorithms, namely RF, LR, DT, SVM, DT, and multilayer perceptron (MLP) to select the features, the full features, and the ensemble techniques on BCWD to predict BC. e results showed that RF with GA had recorded the highest performance. is study used featureselection methods, ML algorithms, DL algorithms, and optimization methods to predict BC. e main contribution is to propose an optimized deep RNN model to predict BC and enhance the results based on recurrent neural networks (RNNs) and the Keras-Tuner optimization technique. ree feature-selection approaches have been employed to select the essential features from the database. e optimized deep RNN is compared to five regular ML algorithms: DT, SVM, RF, NB, and KNN. e remainder of the paper is structured as follows. Section 2 describes the proposed models and methodologies of predicting BC. Section 3 presents the experimental results of using the proposed model. Finally, section 5 concludes the paper.

e proposed system of predicting BC consists of two approaches: regular machine-learning (ML) approach and deep learning (DL) approach. In regular ML approach, five ML models are used, namely DT, SVM, RF, NB, and KNN to train and evaluate the BCWD data set. Grid search with cross-validation is used to optimize ML algorithms. In the DL approach, an optimized deep RNN model is proposed and optimized using Keras-Tuner optimization technique. e steps of the proposed system include feature-selection method, spitting database, optimization and training the models, and evaluating the models as shown in Figure 1 .

We used Breast Cancer Wisconsin (Diagnostic) Data set (BCWD) to train and evaluate the models [25] . e data set includes 30 features and one class label. ese features describe the cell nuclei detected in the breast picture clip. e class label has two possible values: 0 or 1. Breast cancer can be classified as benign or malignant, with 0 indicating benign and 1 indicating malignant.

e description of features is presented in Table 1 .

e key advantages of employing feature-selection algorithms are that they allow us to identify the most essential features in a data set.

We used correlation to reduce the number of features in this study and then applied two types of feature-selection algorithms to the data that remained after correlation: univariate feature selection and recursive feature elimination (RFE).

(i) Correlation methods: we studied the correlation between features using a correlation matrix [26] . 

e BCWD data set is divided into two parts: a training set and a testing set. We employed stratified CV to train and optimize the models with the training set, and the results of CV were recorded for each model. Models are evaluated using a testing set, and the results of the testing set were recorded for each model.

2.4.1. Regular ML Approach. In regular ML approach, five ML algorithms, such as decision tree (DT) [29] , support vector machine (SVM) [30] , K-nearest neighbor algorithm (KNN) [31] , random forest (RF) [32] , and naive Bayes (NB) [33] were used to compare with the optimized deep RNN. Grid search with cross-validation is used to optimize ML algorithms and improve ML algorithms performance. Gridsearch is used to determine the best hyper-parameters for ML algorithms in order to get the best results. Grid search specifies a set of values for each parameter and then tests each value and chooses the best values for the parameters that yield the best results. CV separates the data set into k subsets in order to train ML algorithms on k−1 subsets (the training set). e remainder is used to test ML algorithms [29] .

We proposed an optimized deep RNN model for breast cancer diagnosis based on recurrent neural networks (RNNs) and the Keras-Tuner optimization technique. Figure 2c displays the architecture of the optimized deep RNN model that consists of input layer, five hidden layers, five dropout layers, and one output layer. e input layer consists of the number of neurons, input_dim that equals the number of features, kernel_initializer is he_uniform and the activation function is relu. Each hidden layer consists of the number of neurons, the activation function is relu and kernel initializes the_uniform [34] . e output layer consist of two neurons, sigmoid is the activation function and kernel initializes is glorot_uniform.

e Keras-Tuner optimization technique [35] is used to optimize the deep RNN model. It is a scalable, easy-to-use hyperparameter optimization system that alleviates the problems associated with hyperparameter search. With a define-by-run syntax, you can easily build your search space and use one of the available search algorithms to identify the optimum hyperparameter values for your models. Keras-Tuner optimization technique has built-in Bayesian optimization, hyperband, and random search algorithms, as well as the ability for researchers to enhance it to try out new search methods. Table 2 presents the values of the hyperparameters that have been adapted for the optimized deep RNN. Dropout has been applied to hidden layers with the probability of retaining from 0.1 to 0.9. e number of neurons have adapted from 50 neurons to 700 neurons. (1) 

PR � TP TP + FP ,(1)RE � TP TP + FN ,(2)FM � 2 · precision · recall Precision + recall .(3)

is paper's experiments were run on Python 3 and a GPU. e optimized deep RNN was implemented using the Keras package. e ML models were implemented using the scikit-learn package. e data set was divided into two parts: an 80% training set for optimizing the models and registering cross-validation (CV) results and a 20% testing data set (unseen data) for evaluating the models and registering the testing results. First, we studied the correlation between features and removed features that have high correlation above 90% with other features. After that we applied two feature-selection methods on the selected features by correlation to select eight features. Next, the regular ML models and the optimized deep RNN models were applied to the selected features by correlation, selected features by univariate, and selected features by RFE. We adapted some parameters of the optimized deep RNN for each experimental batch size � 10 and epochs � 100. All of the trials were repeated four times in total. e results of CV and the testing of each experiment will be discussed in detail.

Features, and ML, and DL Approaches. As seen in heat map Figure 3 , ra_m, per_m, and ar_m are correlated, so ar_m is selected. Com_m, con_m, and con_po_m are correlated with each other. erefore con_m is selected. Apart from these, ra_se, per_s, and ar_s are correlated, so ar_s is selected. Ra_w, per_w, and ar_w are correlated, so ar_w is selected. Com_w, con_w, and concave po_w are correlated, so con_w is selected. Com_s, con_s, and con_po_s, are correlated, so con_s is selected. tex_m and tex_wo are correlated, and tex_m is selected. ar_w and ar_m are correlated, so ar_m is selected. e final results of the selected features is 16 features. e results of applying ML models and the proposed model to the selected features by correlation are shown in Table 3 .

e results of CV performance and testing performance will be described in two subsections. Table 4 shows the number of neurons and dropout value in each layer for the optimized deep RNN that is applied on selected features by correlation matrix.

After selecting 17 features of applying correlation matrix, the univariate feature-selection method is applied to 17 features, and 11 features that have the highest scores will be selected. e scores of all features of applying univariate to 17 features are shown in Table 5 . We can see that ar_m has the highest score at 53,991.65592, which is the most important feature for breast cancer diagnosis, while fr_di_m has the lowest score at 7.43E−05. We selected 11 features that have the highest score: area_m, ar_s, tex_m, con_w, con_m, sym_w, con_s, smo_w, sym_m, fra_dim_w, and smo_m. e results of applying ML models and the proposed model to select features by univariate are shown in Table 6 . e results of CV performance and the testing performance will be described in two subsections. Table 7 shows the number of neurons and dropout value in each layer for the optimized deep RNN that is applied on the selected features by univariate.

Approaches. RFE algorithm sets some of the rankings for each feature. We applied REF to 16 features after coloration and selected the 11 features which ranked the best. e ranking of features is shown in Figure 4 . te_m, a_m, smo_m, con_m, ar_s, con_s', fra_dim_s, smo_w, con_w, sym_w, and fra_dim_w have ranked the best, while sym_s and sym_m have registered the worst ranking as 5 and 6, respectively. e results of applying ML models and the proposed model to the selected features by RFE are shown in Table 8 .

e results of CV performance and the testing performance will be described in two subsections. Table 9 shows the number of neurons and dropout value in each layer for the optimized deep RNN that is applied on the selected features by REF.

In our work, first, features have been selected from the BCWD data set using correlation matrix. After that, two feature-selection algorithms, namely Univariate and RFE have been applied to features after correlation, and 11 features have been selected. Regular ML and the optimized deep RNN have been applied to the selected features, and the result of CV and the testing have been registered. Overall, the optimized deep RNN models have achieved the best performance for each feature-selection methods. Figure 5 displays CV results of the optimized deep RNN results for each feature-selection methods. As can be seen, the deep RF has achieved the best performance using univariate (AC � 99.89%, PR � 99.89%, RE � 99.89%, and FM � 99.89%). Correlation and REF have recorded the same performance. Figure 6 displays the testing results of the optimized deep RNN results for each feature-selection methods. As can be seen, the deep RNN has achieved the best performance using univariate (AC � 96.74%, PR � 96.39%, RE � 96.74%, and FM � 96.8%). Correlation and REF have recorded the same performance.

is paper used two approaches: the regular ML approach and the deep learning approach to predict breast cancer. In the DL approach, this paper proposes the optimized deep RNN model based on recurrent neural network (RNN) and the Keras-Tuner optimization technique. e optimized deep RNN consists of the input layer, six hidden layers, six dropout layers, and the output layer. In each hidden layer, we optimized the number of neurons and values of the dropout layer. In the regular ML approach, DT, RF, SVM, NB, and KNN were compared with the optimized deep RNN. ree feature-selection methods: correlation matrix, univariate, and REF were used to select the essential features from the database. e regular ML models and the optimized deep RNN are applied to selected features. e results show that the optimized deep RNN with selected features by univariate method has achieved the highest performance for crossvalidation and testing results. 

Breast Cancer Wisconsin (diagnostic) data set can be downloaded from https://www.kaggle.com/uciml/breastcancer-wisconsin-data, 2021.

e authors declare that they have no conflicts of interest.

CA: A

Designing disease prediction model using machine learning approach

Applying deep learning methods on time-series data for forecasting covid-19 in Egypt, Kuwait, and Saudi Arabia

e role of medication data to enhance the prediction of alzheimer's progression using machine learning

Applications of machine learning predictive models in the chronic disease diagnosis

Extracting information from the text of electronic medical records to improve case detection: a systematic review

Prediction of heart disease using machine learning algorithms: a survey

Predicting the recurrence of breast cancer using machine learning algorithms

Synchronization for fractional-order neural networks with full/under-actuation using fractional-order sliding mode control

Adaptive fuzzy synchronization of fractional-order chaotic (hyperchaotic) systems with input saturation and unknown parameters

Using machine learning algorithms for breast cancer risk prediction and diagnosis

Machine learning algorithms for breast cancer prediction and diagnosis

Breast cancer classification using machine learning

Comparison of machine learning methods for breast cancer diagnosis

Breast cancer prediction: a comparative study using machine learning techniques

Breast cancer classification using deep belief networks

Rapid assessment of breast cancer malignancy using deep neural network

A hybrid analytical hierarchical process and deep neural networks approach for classifying breast cancer

Predicting breast cancer with deep neural networks

Histopathological breast cancer image classification by deep neural network techniques guided by local clustering

Implementation of optimization algorithms on Wisconsin breast cancer dataset using deep neural network

Automated breast cancer diagnosis based on machine learning algorithms

Diagnosing breast masses in digital mammography using feature selection and ensemble methods

Breast cancer diagnosis using ga feature selection and rotation forest

Breast cancer wisconsin (diagnostic) data set

Correlation matrix

Breast cancer identification from patients' tweet streaming using machine learning solution on spark

What is a support vector machine?

k Nearest Neighbors Algorithm (Knn

Random forest Classifier Combined with Feature Selection for Breast Cancer Diagnosis and Prognostic

e optimality of naive bayes

Acknowledgments is research was supported by Taif University Researchers Supporting Project number (TURSP-2020/306), Taif University, Taif, Saudi Arabia.