key: cord-0680037-bp8110do
authors: Apostolopoulos, Ioannis D.; Groumpos, Peter P.; Apostolopoulos, Dimitris I.
title: State Space Advanced Fuzzy Cognitive Map approach for automatic and non Invasive diagnosis of Coronary Artery Disease
date: 2020-04-03
journal: nan
DOI: nan
sha: 3d04c7f54af693fac8cb928e0805726bd787d828
doc_id: 680037
cord_uid: bp8110do

Purpose: In this study, the recently emerged advances in Fuzzy Cognitive Maps (FCM) are investigated and employed, for achieving the automatic and non-invasive diagnosis of Coronary Artery Disease (CAD). Methods: A Computer-Aided Diagnostic model for the acceptable and non-invasive prediction of CAD using the State Space Advanced FCM (AFCM) approach is proposed. Also, a rule-based mechanism is incorporated, to further increase the knowledge of the system and the interpretability of the decision mechanism. The proposed method is tested utilizing a CAD dataset from the Laboratory of Nuclear Medicine of the University of Patras. More specifically, two architectures of AFCMs are designed, and different parameter testing is performed. Furthermore, the proposed AFCMs, which are based on the new equations proposed recently, are compared with the traditional FCM approach. Results: The experiments highlight the effectiveness of the AFCM approach and the new equations over the traditional approach, which obtained an accuracy of 78.21%, achieving an increase of seven percent (+7%) on the classification task, and obtaining 85.47% accuracy. Conclusions: It is demonstrated that the AFCM approach in developing Fuzzy Cognitive Maps outperforms the conventional approach, while it constitutes a reliable method for the diagnosis of Coronary Artery Disease. Conclusions and future research related to recent pandemic of coronavirus are provided.

FCMs constitute a computational methodology able to examine situations which the human thinking process involves fuzzy or uncertain descriptions. FCMs consist of a graphical representation through a signed directed graph, which includes feedback consisting of nodes and weighted arcs [12] . Each node in the graph represents a concept used to describe the cause and effect relationship. The nodes are connected by weighted arcs representing the actual interconnections. Each concept Ci (i.e., the variable of the system) is characterized by a number representing its values, and is calculated through the transformation of a fuzzy value, or the fitting of a numeric value, to the desired interval [0,1]. The initial weight values are defined by the experts of the domain and, therefore, they are linguistic variables, or rule-type statements. Through a defuzzification procedure the linguistic variables are transformed into numeric weights. In this way, FCMs embody the accumulated knowledge and experience from experts [13] . An example is given in Figure 1 . The degree of influence between the two concepts is indicated by the absolute value of Wij. During the simulation, the value of each concept (Ci) is calculated as shown in [14] .

In the classic FCM representation, all concepts and parameters are treated and calculated regardless of their different nature. However, even when a system is described in fuzzy way, the main concept is the same. In the classic control theory, which shall be employed to address the matter, a separation of the concepts is suggested [15] [16] , as follows:  Input Concepts: The inputs of the system, (u)  State Concepts: The concepts describing the operation of the system, (x)  Output Concepts: The concepts describing the outputs of the system, (y).

A simple representation of the system is achieved by a block diagram, which is presented in [16] , and depicted in Figure 2 . In this way, more accurate knowledge of the system is obtained. The proposed separation facilitates the understanding of the system's operation, and the simpler calculation of the concepts' values.

Separating the concepts into categories enables the calculation of their values in a more distributed way. The initial weight matrix, which describes all relations, shall be divided into smaller ones in order to correspond them to each concept category. The classic equations (eq. 1, eq. 2)

Shall now be used to calculate the variation, caused by the change in the input and state concepts, to the state and output concepts, at each time step [k] . In equations (1), and (2), A, B, C and D are individual weight matrixes derived from the initial; The elements of matrix A depend on the states' weights, while the elements of matrix B show how each input concept affects the state concepts of the system. Matrix C embodies the relation between the states and the outputs, while matrix D incorporates the direct affection of input concepts to output concepts [15] . Since equations (1) and (2) (5) and (6):

One more advantage of the mentioned technique is the transparence and decomposability of the system. Concepts behaving as if they were clusters is reality, are allowed to do so, through their participation in state concepts. It is worth mentioning that the above equations' best use is when the system is described by timesteps, and by complex interconnections between the concepts.

In order to apply the AFCM methodology, the values of all the input concepts must lay between the interval [0,1], where 0 denotes that the value of the concept is very small and 1 that the value is very big. There are many proposed activation functions to perform the aforementioned task. In this work we utilize the newly proposed alteration to the classic sigmoid function, as introduced in [17] . We use this function for the inner calculations (i.e. the inputs and the states). The function is explained below:

In equation (7), m is the lower limit of the curve, M is the upper limit of the curve, r is the slope of the curve and to is the symmetry to the y axis. We refer to equation (7) as SigmoidN.

For the final classification between "Healthy" or "Diseased" ,we also use a Softmax classifier [18] , to evaluate and compare the different methodologies.

An overview of the fuzzification and the de-fuzzification process shall be presented. 

Step 1: The experts assign the weight values (verbal). Those weight values shall be de-fuzzified into crisp numbers in space [0,1]. 

Step 2: The concepts initial values may be verbal or numerical. As in step 1, the verbal inputs shall be defuzzified into crisp numbers in space [0,1]. 

Step 3: When reaching a stable state, after a specific amount of iterations, the model's output values are numerical in the desired space. Those values shall then be fuzzified and substituted with verbal ones (e.g. 0.85 equals to "Definitely Abnormal Situation") The process of fuzzification and de-fuzzification shall be done with different methods, such as the Center of Area (COA) [19] , or Center of Gravity (COG) [20] . For the normalization of the output values, both sigmoid or tanh functions may be utilised.

3.1. The overall procedure Two Nuclear Medicine doctors were pooled to define the concepts, the interconnections and the outputs of the system. The system was designed to meet the needs of the Department of Nuclear Medicine of Patras. Therefore, the input concepts of the system reflect the department's approach in diagnosing CAD. The relationships between the concepts of the system are described by the doctors in a linguistic way, thus allowing freedom to explain the interconnections the way experts prefer. The rest of the procedure, i.e., the defuzzification of the inputs and weights, is performed by the engineers. We can summarize the steps of the process as follows: (a) the experts decide the inputs of the system and their possible values, (b) each expert assigns a specific verbal weight between the concepts he/she believes that share a connection, (c) the doctors specify the specific rules describing the system and provide the necessary documentation that confirms the rules, (d) the doctors in collaboration with the engineers define the state concepts of the system, (e) the input values, the state and the weight values as well as the rules are transformed from nominal to numeric or binary form, following specific mathematical expressions, and (f) the numeric value of the output is calculated. The normalization function returns the score, or the probability of the subject.

Based on the aforementioned approach is designing FCMs, the concepts of the AFCM shall be divided into inputs, states and outputs.

The global input concepts, i.e., before the specific input and state concepts are defined, as well as their possible values as defined by the experts are given in the Table 1 . The proposed system classifies the instances to "Healthy" or "Diseased"; thus, the system shall have two possible classes as outputs. We propose two approaches for the final classification. The first approach suggests a single output concept (Out), the value of which describes the probability of infection. For this approach the SigmoidN is utilized. The second approach suggests two output concepts to be inserted, referred to as "out_healthy", and "out_diseased". They present each class's score. A softmax classifier is utilized to classify each instance, based on the score of each class.

The interconnection weights between nodes shall be undertaken by experts, in cooperation with each other. The possible linguistic values of the weights shall be: Very Weak (VW), Weak (W), Medium (M), Strong (S), Very Strong (VS). The weights may also take negative values, e.g. "-VS".

These values are then defuzzied and a corresponding numerical value will be assigned to each one [21] . We provide the equations' (5) and (6) A6  A22 -A31  +W, -W, +VW, -VW, +W, -W, +W, -W, +W, -W,  A12 A6 -A7 Minimize Weights (close to 0)

The AFCM's table A embodies the interconnection between the input concepts. In the proposed system, there are relations between the input concept A6 (female) and the concepts describing the diagnostic tests (A22 -A31). Also, there is relation between the input concept A12 (Known Cad) and the input concepts of the genders (A6 -A7). Table 3 presents the interconnections between the inputs and the states of the system. That is, which inputs may have an influence on the states of the model. In our case, the concepts A12, A13, A15, and A16-A19 are defining the state A32 (predisposing factors). The concepts A14, A20 and A21 define the state A33 (recurrent diseases). Concepts A6-A11 are constituting the state A34 (Demographic Characteristics). Finally, concepts A22-A31 are constituting the state A35 (Diagnostic Tests). 

Includes concepts Weights A32

A12 Table 4 , the direct relation between the inputs and the outputs is presented. Please note that in this case, the inputs defining the state concepts do not directly affect the output(s). 

In Table 5 , the system's Table D is presented. Table D , contains the connection of the state concepts with the output concepts. In the particular case, interconnections between the states do not exist. As mentioned above, modern decision-making problems involve complex relationships, and often a relation between concepts cannot only be explained with the casual FCM strategy (i.e., assigning a specific weight). In this study, a RE-AFCM model is designed, allowing the embedding of more complex expressions (rules).

To achieve this, the addition of a hidden concept, referred to as "rule-activator" is proposed. The hidden concept behaves like the other concepts (i.e., affecting the output), with a weight value or a specific action, which varies according to the suggestions of the experts.

The experts suggested a set of rules to be applied to the model, in order to increase its knowledge. Their suggestions were derived from either recent published guidelines [22] , or their experience. Those rules are presented in Table 6 . Table 6 . Rules and their effect on systems concepts, inputs and outputs

The weight of Scintigraphy is significantly increased

The weights of both tests are relatively increased during the next iteration step

Negate the Gender Discrimination and de-activate the attribute from the system, before the first iteration step

Negate the family history affection before first iteration step

Assign a negative weight to the rule_activator

Uncertainty increases over the trustworthiness of the symptom; assign a small positive weight to rule_activator

The database of the particular study consisted of 303 patient cases, all recorded at the Department of Nuclear Medicine of Patras, in Greece. Most of instances were recorded during the last 8 years. All the patient cases had been pointed to Surgical Coronary Angiography. For every instance, therefore, a final medical report confirming or denying the presence of the disease is available. For the characterization of the instances, the stenosis of the coronary artery is the only criterion, which is obtained by the above mentioned invasive diagnostic test. Patients with stenosis equal or above 70% were labeled as diseased, whereas the rest were labeled as healthy.

The dataset contains 116 healthy cases and 187 diseased cases, whereas male instances are 266. The attributes of the dataset used for the experiments are corresponding to the factors influencing the diagnosis of CAD. The dataset contains every possible diagnostic test a patient may undergo at the clinical section of the experts' laboratory.

The medical reports regarding the diagnostic test were translated in numeric values with the appropriate staging. This approach was supervised by medical staff. The attributes regarding the patient's history and condition (i.e. smoking) were also in need of preprocessing in order to express the linguistic values (i.e. "smoker") into binary format.

We extensively evaluate our system, therefore, more criteria besides accuracy will be employed. The evaluation criteria of the system shall be the following: (a) Accuracy based on the whole dataset, (b) True Positives, (c) False Positives, (d) False Negatives, (e) True Negatives, (f) Sensitivity, (g) Specificity.

For the experiments, we propose several cases. In each case, we employ modified architectures and parameters. A summary of the experiment setups is given below: [23] . Table 7 presents the results for the different cases. The results highlight the effectiveness of the new equations for computing the concepts' values, as it is demonstrated by the increase in accuracy for Case 3 to Case 10. Moreover, the results clarify that the new proposed activation function, SigmoidN, is more preferable for this task over the sigmoid. Also, the combination of single output with SigmoidN, results in an improvement of performance, compared to the combination of two class outputs, and Softmax Classifier. Finally, the addition of rules further improved the knowledge of the system.

Based on the results, we conclude that the proposed RE-AFCM outperforms the traditional FCM approach. What is more, for the specific task, the optimal parameters shall be an architecture consisting of three states, a single output, and the SigmoidN function. Table presents the confusion matrix of Case 9 (the optimal case). The confusion matrix corresponds to sensitivity of 89.3%, specificity of 79.3%, PPV of 87.43% and NPV of 82.14%. The overall accuracy is 85.47%. An overview of the RE-AFCM (case 9) is depicted in Figure 3 . In this section we compare the results of the proposed system with state-of-the-art classification algorithms. The crossvalidation was chosen to be 5-fold due to the small amount of training data. The Neural Network contains three depth layers of 1024, 512, and 256 nodes. Training was performed for 120 epochs, with a batch size of 32 and an initial learning rate of 0.01. Other Neural Networks with alterations on the prementioned parameters were tested and excluded due to inefficiency. Hence, the Neural Network we pick is the one with the better accuracy over all. The Support Vector Machine (SMO) has the following parameters: Complexity Parameter (c) is set at 1.0, the value of epsilon is set at 1E-12, the batch size is 32 and the calibrator method is tuned to Logistic. AdaBoostM1 meta classifier is trained for 50 iterations, with a batch size of 32 and the decision classifier Decision Stump is selected. Chirp is trained with a batch size of 32. Random Forest was trained for 100 iterations with a bag size of 20%, and batch size 32. Spaarc was trained with a batch size of 32. The rest of the classifiers were trained with the optimal parameters suggested by MATLAB Machine Learning Toolkit. The results demonstrate that the proposed model, outperforms every Machine Learning classifier which was employed for the classification task in this work. The shortage of large-scale dataset is impeding the traditional machine learning algorithms to learn the actual and significant relationships between the concepts. This is reflected to the accuracy each algorithm obtains.

It is demonstrated that the new approaches for the development of Advanced Fuzzy Cognitive Maps, as well as the rules mechanism inserted to the system, improve the results, compared to classic FCMs.

Fuzzy Cognitive Maps provide a distinguishable way to express the causeeffect relationship between phenomena, between numeric, nominal, binary, or categorical parameters. A complex system may include all the above-mentioned factors. Relationships between them, provided that are discovered, can be represented visually and mathematically through FCMs. In this way, not only the developers, but also the experts on the field, may observe and understand the FCM representations.

The interconnections between concepts learned by supervised learning algorithms are not ensured to reflect causal connections. That does not mean that unobserved causes may not exist; in fact, one target of supervised learning is to make assumptions regarding the relations between mutually affected concepts during the process of training and testing. Assumptions that can be confirmed or denied experimentally later. Compared to trainable artificial intelligence algorithms, ΑFCM do not intend to discover associations that may or may not exist.

Clinical researchers today are confronted with increasingly large, complex, and high-dimensional datasets [24] . Consequently, the application of interactive visual data exploration in combination with machine-learning techniques for knowledge discovery and data mining is indispensable.

Clinical researchers, or domain-experts are often not computer experts as well. They have high level medical domain-expert knowledge to perform their research, to interpret newly gained knowledge and patterns in their data. On the other hand, computer engineers lack the knowledge of medical data and their unique nature; Moreover, deep, dynamic and complex medical situations require a high level of expertise and experience for the decision making. A smooth interaction of the domain-expert with the data would greatly enhance the whole knowledge-discovery process chain [24] . This can be achieved by the cooperation of engineers and doctors.

That is the case in our work and, generally, in most of proposed models using AFCMs. The experts in a specific domain, not only have the overall supervision of the procedure, but also design the models in cooperation with the developers. Experts define the relationships, the concepts, the system's desired outputs. In our work, the doctor is in the loop, playing the most vital role in the development and the evaluation of the model.

In this paper, the automatic and non-invasive prediction of certain medical problems have been considered and studied. Medical decision-making problems are difficult, complex and have attracted many research studies. As a result, a good number of Clinical Decision Support Systems (CDSS) have been developed based on different methodologies. The majority of them are based on statistical approaches. Indeed all mathematical models been used in confronting the recent epidemic of Coronavirus are using statistical models. However all medical problems are non-linear and statistical models are based on the assumption that variables have linear correlation between them. All medical problems are dynamic and non-linear. Moreover physicians are requested to handle information of different nature, e.g. patient's history, clinical diagnostic tests, medical images, personal health problems and demographic characteristics. The interpretation of these results involve ambiguity and uncertainty, which plays a critical role for the decision-making to a wide and diverse set of medical problems. Therefore, new advanced scientific approaches and new mathematical models are needed. The challenge to physicians, engineers and decision makers is great and very important. The humankind, expects not to say it demands, serious answers from them.

In this research, a state approach of FCMs, as well as the cooperation of FCMs and rule-based decision mechanism, were applied and examined. The proposed model was evaluated on a dataset of Coronary Artery Disease (CAD) candidates and outmatched several Machine Learning Algorithms regarding the prediction of CAD infection. The new approach combines the classical state space approach of dynamic systems to help improve the existing method of FCMs.

Specifically, the separation of concepts in states, inputs and outputs was evaluated to increase the knowledge of the system. The presence of state vectors, helped clustering concepts of similar nature, rather than directly affecting the output in a static way. With the addition of rules, the system managed to explain more complex relations between the concepts; relations that often are conflicting with each other. This approach will be referred as the Advanced Fuzzy Cognitive Maps (AFCM) methodology. The results of this study show that the new state space AFCM achieve a slight better performance over the traditional FCM methodology namely: an increase of seven percent (+7%) on the classification task, and obtaining 85.47% accuracy a slight higher than the 78.21% of the traditional approach. Therefore, we can conclude that the AFCMs' decision-making mechanism is a unique and promising method of handling the parameters of a difficult medical problem.

To the best of our knowledge, the proposed State Space AFCMs has not been applied to medical problems before, except in [25] [26] . With this and the large interest in developing more effective and efficient CDSS, not only for diagnosis but also for therapeutic treatment, future research directions are quite open.

The State Space AFCM methodology needs further development from theoretical point of view: a) better separation of concepts, utilization of more experts, understand better the nonlinear behavior of the system, use learning methods to update the AFCM model, controllability and observability of the dynamic system, sensitivity of all concepts, b) the AFCM could be utilized to study other medical problems, especially the pandemic of coronavirus, c) develop appropriate software tools for the AFCM models, e) perform extensive simulation studies for as many as possible medical problems and validate these models, f) Develop Wise Learning (WL) methods and compare them with Deep Learning (DL) approaches of Artificial Intelligence (AI), g) employ the AFCM models, and feedback methodologies to develop Intelligent Control and Cognitive Control algorithms, and h) explore synergies between Neuroscience and fuzzy cognition.

A survey about fuzzy cognitive maps papers

Fuzzy cognitive maps. International journal of man-machine studies

Global stability of generalized additive fuzzy systems

Time dependent fuzzy cognitive maps for medical diagnosis

A new methodology for Decisions in Medical Informatics using fuzzy cognitive maps based on fuzzy rule-extraction techniques

A fuzzy cognitive map of the psychosocial determinants of obesity

Brain tumor characterization using the soft computing technique of fuzzy cognitive maps

State Feedback of Complex Systems Using Fuzzy Cognitive Maps

others: Global status report on noncommunicable diseases

Artificial intelligence in medicine and cardiac imaging: harnessing big data and advanced computing to provide personalized medical diagnosis and treatment

Predictive data mining for medical diagnosis: An overview of heart disease prediction

Modelling supervisory control systems using fuzzy cognitive maps

Intelligence and Fuzzy Cognitive Maps: Scientific Issues, Challenges and Opportunities. STUD INFORM CONTROL

Fuzzy cognitive maps. Handbook of Granular computing

Re-approaching fuzzy cognitive maps to increase the knowledge of a system

A revised approach in modeling fuzzy cognitive maps

Mediterranean Conference on Control and Automation (MED)

Categorical reparameterization with gumbel-softmax

Structure of decision: The cognitive maps of political elites

Neuro-fuzzy and soft computing-a computational approach to learning and machine intelligence

Extended defuzzification methods and their properties

ESC guidelines on the management of stable coronary artery disease: The Task Force on the management of stable coronary artery disease of the European Society of Cardiology

A Medical Decision Support System for the Prediction of the Coronary Artery Disease Using Fuzzy Cognitive Maps

Knowledge discovery and data mining in biomedical informatics: The future is in integrative, interactive machine learning solutions

A New Approach of Dynamic Fuzzy Cognitive Knowledge Networks in Modelling Diagnosing Process of Meniscus Injury. IFAC-PapersOnLine

A Novel Software Tool for Detection of Meniscus Injury using Dynamic Fuzzy Cognitive Networks

The dataset is recorded at the Department of Nuclear Medicine of the University Hospital of Patras, Greece and is not allowed to be publically available.

Not applicable