key: cord-0970497-hkfl1nq9
authors: Khalifa, Nour Eldeen M.; Smarandache, Florentin; Manogaran, Gunasekaran; Loey, Mohamed
title: A Study of the Neutrosophic Set Significance on Deep Transfer Learning Models: an Experimental Case on a Limited COVID-19 Chest X-ray Dataset
date: 2021-01-04
journal: Cognit Comput
DOI: 10.1007/s12559-020-09802-9
sha: e0235a3d4279ee301f231a09dfb9dfaeae47ba2b
doc_id: 970497
cord_uid: hkfl1nq9

Coronavirus, also known as COVID-19, has spread to several countries around the world. It was announced as a pandemic disease by The World Health Organization (WHO) in 2020 for its devastating impact on humans. With the advancements in computer science algorithms, the detection of this type of virus in the early stages is urgently needed for the fast recovery of patients. In this paper, a study of neutrosophic set significance on deep transfer learning models will be presented. The study will be conducted over a limited COVID-19 x-ray. The study relies on neutrosophic set and theory to convert the medical images from the grayscale spatial domain to the neutrosophic domain. The neutrosophic domain consists of three types of images, and they are the True (T) images, the Indeterminacy (I) images, and the Falsity (F) images. The dataset used in this research has been collected from different sources. The dataset is classified into four classes {COVID-19, normal, pneumonia bacterial, and pneumonia virus}. This study aims to review the effect of neutrosophic sets on deep transfer learning models. The selected deep learning models in this study are Alexnet, Googlenet, and Restnet18. Those models are selected as they have a small number of layers on their architectures. To test the performance of the conversion to the neutrosophic domain, more than 36 trials have been conducted and recorded. A combination of training and testing strategies by splitting the dataset into (90–10%, 80–20%, 70–30) is included in the experiments. Four domains of images are tested, and they are, the original domain, the True (T) domain, the Indeterminacy (I) domain, and the Falsity (F) domain. The four domains with the different training and testing strategies were tested using the selected deep transfer models. According to the experimental results, the Indeterminacy (I) neutrosophic domain achieves the highest accuracy possible with 87.1% in the testing accuracy and performance metrics such as Precision, Recall, and F1 Score. The study concludes that using the neutrosophic set with deep learning models may be an encouraging transition to achieve better testing accuracy, especially with limited COVID-19 datasets.

Severe acute respiratory syndrome-related coronavirus (SARS-CoV-1) is a kind of B-coronavirus that infects bats and some other mammals. SARS-CoV-1 was a kind of coronavirus as a family of the B-coronavirus (B-CoV) subgroup and was title as SARSr-CoV. Historically, SARS-CoV-1, across 29 countries in the world, infected over 8000 humans and at least 750 died [1, 2] . In 2019, a coronavirus epidemic is an ongoing scourge of coronavirus malady 2019 (COVID-19) created by SARS-CoV-2 [3] . However, SARS-CoV-2 infected more than four million humans with more than 300,000 deaths, 1.6 million recovered cases, and 300,000 death cases [4] . It elucidates that the propagate rate of SARS-CoV-2 is greater than SRAS-CoV [5, 6] .

The theory of neutrosophic logic was proposed by Smarandache in 1995. Afterward, it has been unified and generalized by its founder in 1999 [7] . Since that date, neutrosophic logic has been used in many computer science fields including pattern recognition [8] , image segmentation, and processing [9] , and more. It contributes to solving many research and practical real-life problems in a lot of domains such as medicine [10] , economics [11] , space satellite [12] , and agriculture. Neutrosophy leads to a whole family of novel mathematical theories with an overview of not only classical but also fuzzy counterparts [13] . The term neutrosophy means knowledge of neutral thought, and this neutral represents the main difference between fuzzy and intuitionistic fuzzy logic and set [14, 15] . Neutrosophic set has the required potentials of being a general framework for uncertainty analysis in data sets [14] and especially with images in the field of Artificial Intelligence and deep learning. Deep Transfer Learning (DTL) is a type of Artificial Intelligence (AI) concerned with methods inspired by the functions of people's brain [16] . For the time being, DTLs like VGG, ResNet, and DenseNets [17] [18] [19] [20] [21] are quickly becoming an important method in image/video detection and diagnosis based such as. DL is used in medical x-ray/computed tomography diagnoses. DL upgrade the medical diagnosis system (MDS) to realize great results, and implementing an applicable real-time medical diagnosis system [22, 23] .

This part is dedicated to works on the recent x-ray academic researches for applying DL in the field of MDS in chest x-ray diagnosis. Ayan and Ünver [24] proposed an early medical diagnosis system for Pneumonia chest x-ray images based on DTL models. In this academic research, x-ray data [25] containing about 1600 healthy cases, 4200 un-healthy pneumonia cases. The trial score introduced that VGG DTL networks better than the X-ception DTL network with an error rate of 19%. In Stephen et al. [26] , it introduced a new method of diagnosing the existence of pneumonia from chest x-ray database samples based on a CNN architecture with augmentation algorithms trained based on an x-ray database [25] . The results the model improves medical x-ray diagnosis with a miss-classification rate of 12.88% in training miss-classification rate is 18.35% in the validation. Varshni et al. [27] introduced DTL architectures as feature extractors followed by various classifiers (k-nearest neighbors, naïve Bayes, support vector machine, and random forest algorithm) for the diagnose of healthy/unhealthy chest x-ray data. They used an x-ray database called ChestX-ray14 proposed by Wang et al. [28] . Islam et al. [29] introduced a Compressed Sensing (CS) with DTL architectures for automatic identification of pneumonia on the x-ray database to assist the medical physicians. The chest x-ray database used for this research contained about 5800 x-ray images of (healthy / unhealthy). The suggested simulation results have shown that the proposed DTL architectures diagnose pneumonia from a chest x-ray with an error rate of 2.66%. Finally, Chouhan et al. [30] proposed an ensemble DTL architecture that combines results from all DTL architectures for the identification of chest pneumonia x-ray based on the concept of DL. The suggested model based on Kermany et al. [25] database reached a miss-classification error of 3.6%. In this study, we introduced a neutrosophic study based on DTL to classify COVID-19 infection versus non-COVID-19 diseases. We hypothesized that the DTL with neutrosophic would help doctors in detecting COVID-19 x-ray scan images.

The selected COVID-19 x-ray dataset used in this research was acquired from [25, 31] . It was collected from different websites such as the Italian Society of Medical, Radiopaedia web, and online publications. The created dataset is organized into four categories normal, pneumonia bacterial, pneumonia virus, and COVID19. The dataset contains 306 x-ray images divided into 79 of normal, 69 of COVID-19, 79 of pneumonia bacterial, and 79 of pneumonia virus. Fig. 1 illustrates samples of images used for this research.

The methodology adopted in this research is to propose a model that can correctly classify the x-rays images with the different 4 classes. The proposed model includes two main components: the first component is neutrosophic domain conversion, while the second component is the transfer learning architectures. Figure 2 illustrates the proposed neutrosophic/DTL model for the study. The neutrosophic image domain conversion is used as a preprocessing step while the DTL architectures are used in the training and the testing steps.

Neutrosophy (NS) is a theory sophisticated and created by Florentin Smarandache [32] [33] [34] . NS is a useful and helpful theory in computing fuzzy situations. In NS theory, events are computed by subset them into three sets as true ( T ) significance, the status is percentage of true; as indeterminacy ( I ) significance, the status is percentage of indefinite; and as falsity ( F ) significance, the status is percentage of false, where t varies in T subsets. In image processing such as object and edge detection, all pixels of the image are subdivided into T, I, and F subsets. Then, the edge detection/object process of the image is performed through necessary operations on these subsets. The input image converts to the neutrosophic domain as shown in Eqs. 1-5. P(n, m) pixel in the image domain is converted to neutrosophic domain P2NS(n, m) [35, 36] :

where I − n, m is the local average value of related pixels − f min and − f max variables correspond to the last and first peaks measured from those pixels with a value higher than the maximum local average of the histogram.

where H(n, m) is the homogeneity value of T at (n, m) , which is measured by the absolute value of the difference between After the conversion of the image to the NS domain, the COVID-19 chest x-ray (object) is kept in the T n,m domain, the edges are in the I n,m domain, and the background is kept in the F n,m domain. Figure 3 presents samples of images after the conversion neutrosophic image domain in the different domains for every class in the dataset.

In this work, the procedure of the DTL follows algorithm 1. The procedure starts with the input image. The input image will be fed into the DTL model with a learning rate of 0.001, and the number of epochs equal to 50. The training procedure will start. If there is no enhancement for the validation accuracy for 10 epochs, the update rate will be changed else the weights of the network will be updated. After the completion of the training phase, the performance measurements will be calculated to evaluate the performance of the DTL model.

The introduced model for the evaluating of neutrosophic sets with deep transfer models was implemented using a software package (MATLAB). The experiments were conducted on a computer server with 96 GB of RAM, and an Intel Xeon processor (2 GHz). The development was CPU specific. About 36 recorded experiments were conducted in this study. The experiments included the following setups: •Different training and testing strategies • 70% for the training -30% for the testing. The authors of this research tried first to build their deep neural networks based on the works presented [37] [38] [39] , but the testing accuracy was not acceptable. So, the alternative way is to use deep transfer learning models. Using deep transfer models proved its efficiency in many types of research such as work presented in previous studies [40] [41] [42] [43] [44] . The Alexnet, Googlenet, and Restnet18 models are selected in this study as they have a small number of layers on their architectures which will reflect on decreasing the training time, consumed memory, and processing time. All the experimental results have been tested according to the following hyperparameters for the training, and the testing phases: The early stopping [47] plays an essential role in reducing training time and computational complexity. It triggers when the no improvement of the validation accuracy for a certain number of epochs. In the proposed model, the training stops if there no improvement for 10 epochs.

A large number of trials were performed to draw a full picture of the effectiveness and the significance of using neutrosophic sets in different experimental environments with different deep learning models. To evaluate the performance of the neutrosophic set in deep transfer learning models, performance matrices are needed to be investigated through this study. The most common performance measures in the field of deep learning are Accuracy, Precision, Recall, and F1 Score [48] , and they are presented from Eqs. 6-9. where True Positive is the number of True Positive samples, True Negative is the number of True Negative samples, False Positive is the number of False Positive samples, and False Negative is the number of False Negative samples from a confusion matrix. The experimental results will be presented in three subsections: the first subsection will discuss the experimental results for the original dataset. The second subsection will introduce the different neutrosophic domain experimental results. Finally, the third subsection will illustrate a comparative results analysis for the original, the neutrosophic domain according to the confusion matrix for the highest achieved testing accuracy. Table 1 illustrates interesting facts, and they are the following: (1) The more data the deep learning models have, the more testing accuracy they will achieve [49] . (2) The 80-20% and 70-30% strategy achieved close results for testing accuracy which means that those strategies are enough and reflect the accurate testing accuracy for the model.

(7) Precision = True Positive (True Positive + False Positive ) (8) Recall = True Positive (True Positive + False Negative ) (9) F1 Score = 2 * Precision * Recall (Precision + Recall)

The neutrosophic domains included three types, and they are the True (T) neutrosophic domain, the Indeterminacy (I) neutrosophic domain, and the Falsity (F) neutrosophic domain. Those neutrosophic domains will be experimented on in this section to measure their performance under different experimental conditions. Table 2 presents the testing accuracy and performance metrics for the True (T) neutrosophic domain. As illustrated in "Neutrosophic Image Domain Conversion," the True (T) image is the averaging of the original image as every pixel is averaged by it is neighbors with a window of choice. The choice of the window in the study is 5 pixels. Table 2 illustrates that in the 90-10% strategy, both Alexnet and Googlenet model achieves similar highest testing accuracy with 64.52% with an advantage for the Googlenet model in the achieved performance metrics. In the 80-20% strategy, also both Alexnet and Googlenet models achieve similar highest testing accuracy with 54.84% with an advantage for the Googlenet model in the achieved performance metrics, while in the 70-30% strategy, the Resnet18 model achieves the highest testing accuracy with 67.74%. Table 2 illustrates interesting facts, and they are the following: (1) In the True (T) neutrosophic domain, more data does not mean better accuracy in those deep learning architectures' as in the 70-30% strategy; the highest testing accuracy is achieved by 68.82% all over the other strategies.

(2) The images on the True (T) neutrosophic domain are averaged images, which means that some of the important features of images are concealed which negatively affect the achieved testing accuracy if it is compared with the original experimental results presented in Table 1 for the 90-10%, and the 80-20% strategy.

The second neutrosophic domain to be experimented on is the Falsity (F) neutrosophic domain. This domain is the opposite of the True (T) neutrosophic domain. In the Falsity (F) domain, all pixel's values are inverted, it is expected that some features will be concealed, and other features will be revealed in images. Table 3 presents the testing accuracy and performance metrics for the (F) Falsity domain. Table 3 illustrates that in the 90-10% strategy, both Alexnet and Googlenet models achieve similar highest testing accuracy with 64.52% with an advantage for the Googlenet model in the achieved performance metrics. In the 80-20% strategy, also both Alexnet and Googlenet models achieve similar highest testing accuracy with 56.45% with Table 3 also shows interesting facts, and they are the following: (1) In the Falsity (F) neutrosophic domain, more data does not mean better accuracy in those deep learning architectures' as in the 70-30% strategy, the highest testing accuracy is achieved by 65.59% all over the other strategies.

(2) The images on the Falsity (F) neutrosophic domain are the inversion of the True (T) domain, which means that some of the important features of images are concealed which negatively affect the achieved testing accuracy if it is compared with the original experimental results are presented in Table 1 for the 90-10%, and the 80-20% strategy. (3) The results presented in Table 3 are very close to results presented in Table 2 , which means the Falsity (F) neutrosophic domains do not add extra value for the grayscale images and can be discarded in some applications depending on their nature.

The third neutrosophic domain to be experimented on is the Indeterminacy (I) neutrosophic domain. This domain contains the absolute edges in the image. In the Indeterminacy (I) domain, all pixel values are resulted from subtracting the original pixel value from the average pixel value in the True (T) neutrosophic domain. Table 5 presents the testing accuracy and performance metrics for the Indeterminacy (I) domain. Table 5 illustrates that in the 90-10% strategy, Alexnet achieves the highest testing accuracy with 87.10% with the highest achieved performance metrics scores. In the 80-20% strategy, Googlenet achieved the highest testing accuracy with 66.13%, while in the 70-30% strategy, both Googlenet and Resnet18 models achieve similar highest testing accuracy with 73.12% with an advantage for the Googlenet model in the achieved performance metrics. Table 4 illustrates interesting facts, and they are the following: (1) In the Indeterminacy (I) neutrosophic domain, all the achieved testing accuracies are better than all the achieved testing accuracies in the Falsity (F), the True (T), and the original domain. (2) The images on Indeterminacy (I) neutrosophic domain are the absolute difference between the original and the averaged image in the True (T) domain. Those are very important features that are revealed in an image and helped deep transfer models to achieve higher testing accuracy. (3) More data does not mean achieving higher testing accuracy in Indeterminacy (I) neutrosophic domain; in the 70-30% strategy, the achieved testing accuracy was better than the achieved accuracy in 80-20% strategy with 6.99% better using Googlenet.

"Neutrosophic Domains Experimental Results"concluded that the Indeterminacy (I) neutrosophic domain achieved the highest possible testing accuracy in all experiment's trails. This section is dedicated to present a comparison result between the Indeterminacy (I) neutrosophic domain with the original domain with deeper performance metrics to evaluate the performance of the Indeterminacy (I) domain. Table 5 presents a comparative result of the achieved testing accuracy between the Indeterminacy (I) and the original domain. Table 5 shows that the Indeterminacy (I) neutrosophic domain achieved the highest testing accuracy in all training and testing strategies with 87.10% (Alexnet), 66.13% (Googlenet), and 73.12% (Googlenet) in the 90-10%, 80-20%, and 70-30% accordingly. Table 5 also shows interesting facts, and they are the following: (1) In the Indeterminacy (I) neutrosophic or the original domain, the Googlenet model is the most dominant in achieving the highest accuracy possible as it contains 20 layers in its architecture if it is compared with Alexnet and Resnet18 which contains 8, and 18 layers.

(2) The Indeterminacy (I) neutrosophic greatly affects the testing accuracy; in the 90-10% the Indeterminacy (I) neutrosophic domain achieves better accuracy with 12.91% more. In the 70-30% the Indeterminacy (I) neutrosophic domain achieves better accuracy with 10.75% more. (3) In the Indeterminacy (I) neutrosophic domain, deep transfer models can learn with fewer data as illustrated in the 70-30% strategy, the Googlenet achieves better accuracy than the 80-20% strategy. That means the model can generalize whatever the amount of the data existed. While in the original domain, more data means better testing accuracy.

All the experimental outcomes show that converting to the Indeterminacy (I) neutrosophic domain from the original domain grantees achieving better testing accuracy. The Indeterminacy (I) neutrosophic need further investigations to prove it is efficient for the detection of COVID-19 among the other classes. The confusion matrices for the Indeterminacy (I) neutrosophic domain for the different deep transfer models are presented in Figs. 4, 5, and 6.

The figures show that the testing accuracy for the COVID-19 class in the different training and testing strategies are acceptable. For the 90-10% strategy, The Alexnet model was able to detect COVID-19 with a testing accuracy of 100% and for the normal class with 100%. In the 80-20% strategy, The Googlenet model was able to detect COVID-19 with a testing accuracy of 77.8% and for the normal class with 100%. While in the 70-30% strategy, The Googlenet model was able to detect COVID-19 with a testing accuracy of 100% and for the normal class with 87.5%. 

According to the World Health Organization (WHO), coronaviruses are a family of viruses that lead to sicknesses ranging from the common cold to more severe diseases. With the advancements in computer science, detection of this type of virus is urgently needed. In this paper, a study of neutrosophic significance on the deep transfer learning model is presented. The neutrosophic domain consisted of three types of images and they are, the True (T) images, the Indeterminacy (I) images, and the Falsity (F) images. The dataset used in this research had been collected from different sources as there is no benchmark dataset for COVID-19 chest x-ray until the writing of this research. The dataset consisted of four classes, and they were COVID-19, Normal, Pneumonia bacterial, and Pneumonia virus. This study aimed to review the effect of neutrosophic sets on deep transfer learning models. The selected deep learning models in this study were Alexnet, Googlenet, and Restnet18. Those models were selected as they had a small number of layers on their architectures that will reflect on reducing the consumed memory and training time. To test the performance of the conversion to the neutrosophic domain, more than 36 trials had been conducted and recorded. A combination of training and testing strategies by splitting the dataset into (90-10%, 80-20%, 70-30) were included in the experiments. Four domains of images are tested, and they are the original images, the True (T) neutrosophic images, the Indeterminacy (I) neutrosophic images, and the Falsity (F) neutrosophic images. The four domains with the different training and testing strategies were tested using Alexnet, Googlenet, and Restnet18 deep transfer models. According to the experimental results, the Indeterminacy (I) neutrosophic domain achieved the highest accuracy possible in the testing accuracy and performance metrics such as Precision, Recall, and F1 Score. The study concluded that using the neutrosophic set with deep learning models might be an encouraging transition to achieve better testing accuracy, especially with limited COVID-19 x-ray datasets. One of the potential future works is trying the proposed model on bigger datasets. Also, include deeper transfer learning for experimental investigation such as Resnet50, and X-ception model with the neutrosophic theory.

Conflict of Interest The authors declare that they have no conflict of interest.

Ethical Approval This article does not contain any studies with human participants or animals performed by any of the authors.

Coronavirus disease 2019: coronaviruses and blood safety transfusion medicine reviews

Within the lack of chest COVID-19 X-ray dataset: a novel detection model based on GAN and deep transfer learning

A review of coronavirus disease-2019 (COVID-19)

COVID-19) map Available online

Novel coronavirus takes flight from bats?

SARS-CoV-2 and coronavirus disease what we know so far. Pathogens

A unifying field in logics: neutrosophic logic

The theory of neutrosophic cubic sets and their applications in pattern recognition

Basic structure of some classes of neutrosophic crisp nearly open sets and possible application to GIS topology

Neutrosophic Set in Medical Image Analysis

A review of seven applications of neutrosophic logic. Cultural Psychology, Economics Theorizing, Conflict Resolution, Philosophy of Science

Algorithm selection for edge detection in satellite images by neutrosophic WASPAS method

A. Introduction to neutrosophy and neutrosophic environment

Neutrosophic sets and its applications to decision making. In Computational intelligence for big data analysis

Neutrosophic set is a generalization of intuitionistic fuzzy set, inconsistent intuitionistic fuzzy set (picture fuzzy set, ternary fuzzy set), pythagorean fuzzy set, q-rung orthopair fuzzy set, spherical fuzzy set

Computer vision detection of foreign objects in walnuts using deep learning. Computers and Electronics in Agriculture

Very deep convolutional neural network based image classification using small training sample size

Going deeper with convolutions

Deep residual learning for image recognition

Computer Vision and Pattern Recognition (CVPR)

Xception: Deep learning with depthwise separable convolutions

Densely connected convolutional networks

An overview of deep learning in medical imaging focusing on MRI

A gentle introduction to deep learning in medical image processing

Diagnosis of pneumonia from chest x-ray images using deep learning

Identifying medical diagnoses and treatable diseases by image-based deep learning

An efficient deep learning approach to pneumonia classification in healthcare

Pneumonia detection using CNN based feature extraction

ChestX-Ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

Automatic detection of pneumonia on compressed sensing images using deep learning

A novel transfer learning based approach for pneumonia detection in chest x-ray images

COVID-19 image data collection

Neutrosophic masses indeterminate models. Applications to information fusion

Applications of neutrosophic logic to robotics: an introduction

Bipolar neutrosophic sets and their application based on multi-criteria decision making problems

CT liver tumor segmentation hybrid approach using neutrosophic sets, fast fuzzy c-means and adaptive watershed algorithm

Brain tumor detection based on Convolutional Neural Network with neutrosophic expert maximum fuzzy sure entropy

Deep iris: deep learning for gender classification through iris patterns

Deep bacteria: robust deep learning data augmentation design for limited bacterial colony dataset. International Journal of Reasoningbased Intelligent Systems

Artificial intelligence technique for gene expression by tumor RNA-Seq Data: a novel optimized deep learning approach

Deep transfer learning models for medical diabetic retinopathy detection

Insect pests recognition based on deep transfer learning models

Deep learning in plant diseases detection for agricultural crops: a survey Available online: www. igi-globa l.com/artic le/deep-learn ing-in-plant -disea ses-detec tionfor-agric ultur al-crops /24849 9

CNN for Handwritten Arabic digits recognition based on LeNet-5 BT -Proceedings of the International Conference on Advanced Intelligent Systems and Informatics

Arabic handwritten characters recognition using convolutional neural network

Don't decay the learning rate, increase the batch size

Text Mining with Machine Learning

Automatic early stopping using cross validation: quantifying the criteria

Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation

The effectiveness of data augmentation in image classification using deep learning