The role of AI classifiers in skin cancer images 750  |  wileyonlinelibrary.com/journal/srt Skin Res Technol. 2019;25:750–757.© 2019 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd 1   |   I N T R O D U C T I O N Malignant skin neoplasms, for example melanoma, basal cell carci‐ noma (BCC) and squamous cell carcinoma (SCC), have gained public attention over the last years. The increased interest in this matter, not only by population in general but also by scientific researchers, can be attributed to the growing incidence rates and respective mor‐ tality tolls of malignant skin tumours.1 Despite the common association of alarming prevalence rates to unsafe life‐xmlstyle choices, such as excessive and unprotected UV exposure,2 the introduction of preventative measures, like frequent screening and biopsy of suspected lesions, can also be accountable by the growth verified in these numbers.3 Moreover, the totality of deaths caused by malignant skin neoplasia has shown some stabiliza‐ tion, a consequence of early recognition strategies.1 To support this tendency and possibly improve the current rates, developments in the diagnosis field are required. The diagnosis of skin cancer is mostly based on the evaluation performed by a physician, being the decision of surgical excision only made when suspected malignity or inconclusive analysis occurs.4 To supply additional information to the one acquired by the naked eye, several imaging modality techniques have been used to assist in skin cancer diagnosis. Dermoscopy images are a clinical reality, represent‐ ing the number one aiding tool to evaluate melanocytic tumours,5 apart from other methods, as thermography and spectroscopy‐ based. Still, the understanding of the gathered information can be Received: 29 March 2019  |  Accepted: 28 April 2019 DOI: 10.1111/srt.12713 O R I G I N A L A R T I C L E The role of AI classifiers in skin cancer images Carolina Magalhaes  | Joaquim Mendes  | Ricardo Vardasca INEGI‐LAETA, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal Correspondence Ricardo Vardasca, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias S/N, 4200‐465 Porto, Portugal. Email: rvardasca@fe.up.pt Funding information Fundação para a Ciência e a Tecnologia, Grant/Award Number: LAETA ‐ UID/ EMS/50022/2013 Abstract Background: The use of different imaging modalities to assist in skin cancer diagnosis is a common practice in clinical scenarios. Different features representative of the le‐ sion under evaluation can be retrieved from image analysis and processing. However, the integration and understanding of these additional parameters can be a challeng‐ ing task for physicians, so artificial intelligence (AI) methods can be implemented to assist in this process. This bibliographic research was performed with the goal of assessing the current applications of AI algorithms as an assistive tool in skin cancer diagnosis, based on information retrieved from different imaging modalities. Materials and methods: The bibliography databases ISI Web of Science, PubMed and Scopus were used for the literature search, with the combination of keywords: skin cancer, skin neoplasm, imaging and classification methods. Results: The search resulted in 526 publications, which underwent a screening pro‐ cess, considering the established eligibility criteria. After screening, only 65 were qualified for revision. Conclusion: Different imaging modalities have already been coupled with AI meth‐ ods, particularly dermoscopy for melanoma recognition. Learners based on support vector machines seem to be the preferred option. Future work should focus on image analysis, processing stages and image fusion assuring the best possible classification outcome. K E Y W O R D S algorithms, image processing and computer vision, machine learning, skin cancer https://orcid.org/0000-0001-5602-718X https://orcid.org/0000-0003-4254-1879 https://orcid.org/0000-0003-4217-2882 mailto:rvardasca@fe.up.pt      |  751MAGALHAES Et AL. a challenging task for clinicians and different diagnosis can be found for the same skin neoplasia, due to the dependence of practitioner experience.5 Thus, the introduction of artificial intelligence (AI) com‐ putational methods can be of value to supply a second opinion. The use of AI for decision support systems in medicine has been present for over 50 years. Machine learning algorithms represent a key branch of this area, specially when dealing with medical deci‐ sions, due to its ability to learn over time, as the supplied information increases, before drawing a conclusion.6 In the case of skin cancer diagnosis, a set of input variables can be retrieved from image anal‐ ysis and processing. These parameters are feed to a classifier, for example support vector machine (SVM), decision trees and random forest, that delivers an output result, assigning each lesion to a given group.6 The classification performance can be evaluated by a set of selected measurements.7 High accuracy and efficiency rates are one of the main advan‐ tages of the application of classifiers in medical diagnosis, since the developed algorithms can capture and integrate information in ways in a fraction of time that the human brain cannot perform. Its suc‐ cessful implementation could reduce human error, providing early diagnosis and consequent cost reduction in skin cancer treatments.8 The goal of this review was to evaluate the current state of the application of machine learning algorithms as an assistive tool in skin cancer diagnosis, based on information retrieved from different im‐ aging modalities. The understanding of the parameters and strate‐ gies used to acquire the best results is of interest, in order to outline possible improvements for upcoming researches. 2   |   M AT E R I A L S A N D M E T H O D S 2.1 | Search strategy The presented bibliographic research was performed in the reference sources ISI Web of Science, PubMed and Scopus, with the following keyword combinations, respectively: TOPIC: (skin cancer) OR TOPIC (skin neoplasm) AND TOPIC: (imaging) AND (classification meth‐ ods); ((skin cancer [Title/Abstract]) OR (skin neoplasm)) AND (imag‐ ing) AND (classification methods [Title/Abstract])); (TITLE‐ABS‐KEY ("skin cancer") OR TITLE‐ABS‐KEY ("skin neoplasm") AND TITLE‐ ABS‐KEY (imaging) AND TITLE‐ABS‐KEY ("classification methods")). The selected fields of search, in each database, were used to assure uniformity in the results encountered, and the use of complex and unclear terms was avoided to guarantee a maximum number of re‐ sults. Since the terms “skin cancer” and “skin neoplasm” are common interchangeable expressions, the Boolean operator OR was applied. No data restriction was considered. After the database search, a du‐ plicate removal was performed. 2.2 | Screening and eligibility of results A title and abstract screening of the encountered articles was ini‐ tially performed to consider only those that reported the use of clas‐ sification methods in skin cancer images. The first eligibility criterion consisted in the removal of meeting abstracts and revision articles. Secondly, only publications written in English were kept, eliminating articles submitted in other languages. Considering that this review is focused on the use of classifiers for skin cancer classification, articles that described the use of classifica‐ tion methods for other purposes were excluded, as well as research studies that classified skin neoplasms with different approaches, for example ABCDE rule, making the third and fourth selection rule, respectively. Lastly, publications that did not report values of accu‐ racy, sensitivity or specificity of classification outcomes were also removed. The remaining publications were categorized based on the im‐ aging modality used to characterize the skin neoplasm: dermatos‐ copy, microscopy, digital photography, spectroscopy, combination and others. A full‐text review was conducted following the PRISMA rules for systematic reviews detailed in Ref.9,10. The revision process is described in Figure 1. 3   |   R E S U LT S A total of 526 publications were selected from the database search, with 251, 194 and 81 publications found in ISI Web of Science, PubMed and Scopus, respectively. The duplicate removal leads to the exclusion of 183 articles. The remaining 343 publications were submitted to a title and abstract screening process that resulted in the elimination of 195 records, since artificial intelligence compu‐ tational classification methods were not applied to skin cancer im‐ ages for classification purposes. Following, 83 articles were removed from the final results, due to the eligibility criteria defined, being 19, 7, 31, 15 and 9 eliminated due to the first, second, third, fourth and fifth criterion, respectively. The final set includes 65 publications eli‐ gible for revision with 34, 6, 7, 12, 3 and 3 articles concerning the use of dermoscopy, microscopy, spectroscopy, digital photography, F I G U R E 1   PRISMA flow diagram. Adapted from Moher et al10 752  |     MAGALHAES Et AL. histopathology, combination of several and other imaging modalities to analyse the neoplasia, respectively. 3.1 | Dermoscopy The successful use of artificial intelligence methods for the assess‐ ment of skin lesions represented by computerized digital dermatos‐ copy images is one of the most documented subjects of this field. The majority of authors reporting on this topic demonstrate the usefulness of several classification algorithms in the evaluation of le‐ sion’ malignancy, distinguishing benign from malignant melanocytic tumours. Grzesiak‐Kopec et al11 presented two different strategies for this purpose, using single classifiers: Naive Bayes, random forest and k‐NN; and a meta‐learning approach with bootstrap aggregating and vote ensemble classifier. The outcomes of the metaheuristics ex‐ ceeded the ones of single classifiers, having bagging applied to ran‐ dom forest presented the highest sensitivity value (0.851). Pairwise coupling (PWC) of SVM, k‐NN and Gaussian maximum likelihood (ML) was applied by Rahman et al12 to different colour and texture features extracted from skin lesion’ images. The implementation of fusion by sum of the single classification results obtained exceed the perfor‐ mance of the lone machine learning algorithms, delivering an accu‐ racy (AC) of 75.69%. Pennisi at al13 showed the ability of Naive Bayes, Adaptive Boosting (AdaBoost), k‐NN and random trees machine learning methods in the detection of melanomas among benign le‐ sions, segmented with Delaunay Triangulation. The best results were encountered with AdaBoost, with sensitivity (SN) and specificity (SP) values of 0.935 and 0.871, respectively. With the same purpose, Ruiz et al14 use k‐NN, ANN and Bayes learners in a collaborative method, improving the end results. A number of seven neighbours and seven hidden layers were the optimal parameters encountered for the k‐NN and ANN algorithm, respectively. A part from these, several other au‐ thors choose similar approaches, comparing single learners to select the classifier that best performs, depending on the provided data.15,16 The implementation of more complex approaches for the purpose of classifying pigmented skin lesions was performed by Masood et al17, with the development of a Deep belief network, using labelled and unlabelled data, in parallel with a self‐advised single vector ma‐ chine learning algorithm responsible for improving the classification results. This strategy deals with the frequent problem of insufficient training data, delivering classification errors lower than other com‐ mon classifiers. Schaefer et al18 presented another method to tackle this data imbalance, using an ensemble of various one‐class classifi‐ ers based on support vector data description. The final classification results showed its superiority in comparison with other ensemble classifiers based on SVM, commonly used in this situation. The com‐ bination of multilayer perceptron (ANN), Naive Bayes, decision tree, k‐NN and SVM classifiers documented by Castillejos‐Fernández et al19 also exceeds the performance of the single classifiers in the task of malignancy classification. The relevance of the features selected for input is stressed, as the ensemble accuracy decreases with the increase of feature number. This topic is also highlighted by Faal et al,20 after achieving better classification results with an ensemble of k‐NN, SVM and Linear discriminant analysis (LDA) algorithms with different feature inputs for the different classifiers, as oppose to the same shape, colour and texture components. The impact of input vectors was also tested by Rastgoo et al21 with the implementation of an ensemble learning with random forest (RF) and weighted com‐ bination constructed with RF, SVM and LDA, where the combination of several features achieve higher specificity results (94%), instead of the use of a single characteristic. This exact conclusion was reached by the same author,22 but applying single learners, that is, SVM, RF and Gradient boosting. In both articles, RF outperforms the others. Fengying et al23 build a meta‐ensemble model for the classification of melanocytic lesions, composed of three different ensembles based on NN, each feed with different inputs, with posterior combination of the outputs. Its overall sensitivity and accuracy exceeded RF, Gentle AdaBoost, SVM, k‐NN, Fuzzy NN and systems based on Bagging of Features (BoF) models. Lastly, Abbas et al24 manipulated image fea‐ tures representative of lesion' patterns to classify pigmented skin tumours, through the use of majority voting with support vector ma‐ chine, achieving accuracy, sensitivity and specificity values of 93%, 94% and 84%, respectively. Apart from ensemble models and meta‐learning approaches, the implementation of single learners for melanocytic lesion classi‐ fication is also fairly common. SVM is the favoured learning model, being used standardly by the majority of authors, that preferred to focus in elaborate feature extraction and selection methods.25‐28 Specifically, Jaworek‐Korjakowska et al29 used this type of learn‐ ing algorithm to develop a computer‐aided diagnosis (CAD) system for the detection of micro malignant melanoma that outdid other literature models, with SN of 90% and SP of 96%. La Torre et al30 tested the performance support vector machine classifiers with different function kernels, namely chi‐square, Gaussian and gen‐ eralized Gaussian. The latter showed remarkable results, detecting all cancerous lesions. Codella et al31 explored the application of SVM to whole and partitioned images, segmented using ensemble approaches, resulting in a area under the receiver operating char‐ acteristic (ROC) curve of 0.843. Support vector machine has also been used to deal with class imbalance situations for melanoma classification. Celebi et al implemented Random under‐sampling and Synthetic minority oversampling technique (SMOTE) and con‐ cluded that SMOTE is a better approach, since the first option can eliminate valuable data32 and reduce drastically the number of samples, hardening the learning process. Since SVM can ignore input samples that are not linearly separated during the training step, Masood and Al‐Jumaily33 opted for a Self‐Advising SVM (SA‐SVM) strategy, to retrieve information from the misclassified data, during this phase. SA‐SVM presented higher AC, SN and SP, followed by SVM with radial basis function, quadratic, poly‐ nomial, linear and multilayer perceptron kernel, decreasingly. The classification capacity of SVM with different kernel functions was also explored by Wahba,34 defending the importance of kernel selection, according to the type of features included in the input vector. Since the supplied dataset was non‐linearly separable, the quadratic polynomial function kernel delivered the top results.      |  753MAGALHAES Et AL. The same kernel was found to be the most adequate by Yuan et al,35 to prevent the increase of error rate. Few authors exceed the 1 level distinction with SVM, introducing a second classification step. Suganya36 developed a model to, firstly, categorize melano‐ cytic and non‐melanocytic lesions, followed by the differentiation of melanoma from naevus and BCC from seborrhoeic keratosis, respectively. Joseph and Panicker37 divided normal and abnormal skin lesions, further classifying the last into Atypical nevi or mela‐ noma. Both achieved SN, SP and AC values close to, or above 90%. Melanoma detection proved to be doable with the implemen‐ tation of back‐propagation neural networks (NN) classifiers, al‐ though inferior to the ones performed with SVM. Premaladha and Ravichandran achieved an accuracy of 87%, considering an error lower than e−0.5, while Messadi et al38 established a maximum error of 0.1, achieving a correct classification rate (TCR) of 76.76%. The implementation of additional algorithms for the optimization of ANN performance in skin cancer classification is also found. Common methods include the use of genetic algorithms (GA)39 and particle swarm optimization (PSO).40 Additionally, random forests (RF) learning methods have also been applied to dermatoscopy images for both melanoma41 and BCC42,43 classification. Ferris et al41 constructed a model of 1000 decision trees and a threshold for malignant diagnosis of 0.4, with sensitivity results higher than physicians and specificity lower, while Kharazmi et al42,43 explored the use of vascular features for basal cell carcinoma automatic detection with 100 trees. No reference is made to the reasons considered for tree number selection. When compared to the abovementioned classifiers, the imple‐ mentation of k‐NN outside of ensemble models or comparative approaches is more uncommon. For melanoma recognition in der‐ matoscopy images, Ganster et al44 choose a 24‐NN strategy, achiev‐ ing a better overall performance when only two classes (benign and malignant) were considered, as oppose to three (benign, dysplastic and malignant). A neighbourhood of 24 was selected, considering that it ensured the best results, for the available data set. 3.2 | Microscopy Machine learning algorithms have demonstrated their potential in the detection of melanoma tumours, analysed through microscopy techniques. The use of decision trees appears to be the preferred choice for the classification phase, whether the image collection is performed by confocal laser‐scanning microscopy (CLSM)45‐47 or re‐ flectance confocal microscopy (RCM).48 The reported sensitivity and specificity values of CLSM papers exceed the 90% mark, reaching 97% and 96%,46,47 respectively. The same classification and regres‐ sion tree (CART) software is implemented by all authors, although the parameters selected are not mentioned. The use of features extracted from fluorescence images, as inputs for machine learning classifiers, has also been approached. Odeh et al49 reported excellent results, for the k‐NN algorithm, in the classifi‐ cation of benign and malignant skin lesions and not so good outcomes in the differentiation of BCC and AK tumours. A Euclidean distance metric was used, and several k were tested (1, 3, 5, 7 and 9). The best results were achieved with k = 1; however, this can be misleading due to possible overfit of the date. The author accentuates the relevance of feature selection, testing the use of genetic algorithm and sequen‐ tial scanning selection technique for this purpose. Odeh and Baareh50 explored further one of the authors previous work and tested other options, namely ANNs with GA and an Adaptive Neuro‐Fuzzy Inference System for the same classification purposes. Nonetheless, k‐NN with GA outperformed all the others. 3.3 | Spectroscopy Artificial intelligence algorithms have achieved satisfactory results in the identification of the deadliest form of skin cancer, in spectros‐ copy images. However, these are often worse than the ones attained in dermoscopy and microscopy methods. Li et al51 trained and tested the k‐NN, ANN and Naïve Bayes classifiers, detailing the parameter choice. While default values were chosen for the application of the latter algorithm, ANN was used with back‐propagation and the num‐ ber of six hidden units was justified by the sum of the number of inputs and outputs divided by two. To balance noise and result robustness, a number of three neighbours was selected for k‐NN. SVM was the learner of choice for Liu et al52 that verified a particular improvement in classification results, when patient’ age was added to the input fea‐ ture vector. Tomatis et al53,54 focused their research on the implemen‐ tation of neural networks for classification of multispectral images, achieving SN and SP values higher than 70% in both works. The use of electrical impedance spectra as input for melanoma detection was successfully tested by Mohr et al55 with the SVM classifier, achieving high accuracy. With the same goal, Aberg et al56 combined the results of four different learners (partial least squares discriminant analysis (PLS‐DA), SVM, ANN and k‐NN) to obtain a sensitivity value of 95%. Lastly, k‐NN was used by Maciel et al57 to discriminate other skin disorders, for example psoriasis from neoplastic lesions, represented in spectral images. The increment of k had little to none influence on the great SN and SP results encountered. 3.4 | Digital photography Classification strategies based on support vector machine learning are favoured when evaluating features extracted from macroscopic im‐ ages. Similar to previously mentioned imaging methods, discrimination of melanoma among other lesions is the main objective, and accuracy values from 79%58 to 97%59 have been achieved. Takruri et al60 con‐ structed a multi‐classifier to improve the accuracy of melanoma detec‐ tion, based on three SVM algorithms with radial basis function (RBF) kernel. The top result of 88.9% ACC was achieved when Probability Averaging Fusion was used to combine the classifiers results, instead of Majority Voting. The RBF kernel was also selected by Oliveira et al,61 as well as the histogram intersection kernel, due to the non‐linearity of the data, to increase algorithm efficiency. Likewise, Spyridonos et al62 made the same kernel selection, but for the detection of AK among 754  |     MAGALHAES Et AL. healthy skin, attaining sensitivity and specificity values in the range of 63.7%‐80.2% and 65.6%‐82.3%, respectively. Has in other papers with different imaging techniques for skin lesion classification, some authors rather focus their work on the development of novel feature extraction and selection strategies to achieve the best classification results.63‐66 Jafari et al67 choose this approach, extracting different sets of colour features that were used as inputs for an ANN classifier. Neural networks performance in melanoma diagnosis, based on extracted colour features, was also evaluated by Przystalki et al.68 SVMs learners with linear, polyno‐ mial, quadratic and sigmoid kernel were used for comparison, with the highest accuracy (97.44%) corresponding to the linear function. The implementation of nearest neighbour methods in macroscopic images was described only by Cavalcanti and Scharcanski.69 A k of 1 and the Euclidean distance metric were selected to distinguish benign from malignant lesions. In order to reduce the number of false neg‐ atives obtained by this approach, a set of Bayes classifiers is applied afterwards, leading to an increase in sensitivity of 94.92%‐96.37%. 3.5 | Histopathological images The detection of non‐melanoma skin cancers and pre‐cancerous lesions has successfully been done with the analysis of histopatho‐ logical images. SVM is the go‐to machine learning algorithm, show‐ ing sensitivity and specificity values higher than 90%70 and 80%,71 respectively. Its use in semi‐advised learning models is also en‐ countered, but for melanoma recognition. Like in17 for dermoscopy images, Masood et al72 choose this strategy to address the issue of limited unlabelled data, using SVM to adjust the weight of each sample. The SA‐SVM outperformed the standard SVM learner. 3.6 | Other imaging modalities Lesion classification strategies based on information retrieved from un‐ common imaging modalities have also been studied by some authors. Parameters from terahertz pulse images of BCC lesions were retrieved and used for its distinction from healthy skin, using SVM algorithms.73 Kia et al74 classified healthy skin, melanoma, BCC and benign lesion' sonograms with multilayer perceptron (ANN), achieving SN of 98% and SP of 5%. Even though the authors emphasize the importance of high sensitivity, the low specificity obtained is not acceptable, since unnec‐ essary patient stress would be caused, due to the high number of false positives. Hence, the proposed classifier needs improvements. Lastly, Ding et al75 used 3D texture features and 2D ABCD parameters for melanoma diagnosis with support vector machine. A multilayer per‐ ceptron kernel was selected, reaching an accuracy of 87.8%. 3.7 | Statistical significance The acquirement of good classification results can have a greater im‐ pact when supported by an indicator of statistical significance, that is P‐value. Typically, this probability is set to be less than 5%, in order to achieve statistically significant results.76 When analysing the results of this bibliographic research, few are the authors that choose to include this indicator in their work. Some of them selected a P‐value smaller than 5%,53,73 while others preferred a lower threshold of 0.00125,52—Table 1. All papers used this probability during the feature selection stage for the construc‐ tion of input vectors, in order to guarantee the best possible classifi‐ cation outcome. Thus, only features that resulted in a P‐value lower than the set limit were considered for lesion classification. 4   |   D I S C U S S I O N O F T R E N D S A N D F U T U R E C H A L L E N G E S Most publications concerning the use of classifiers for skin cancer detection seem to highlight the importance of feature extraction and selection stages to attain the best results.19‐22,25,26,28,49,63‐67 Thus, it is expected further research in this area, focusing on image analysis and processing, in preference of new machine learning strategies. In fact, there is a loophole in the description of the classification task, since some papers lack any reference to the parameters se‐ lected for the implementation of the algorithm of choice.11,27,37,42,71 The decision of using platforms that already include pre‐written al‐ gorithms, that is WEKA, MATLAB and LIBSVM, could be the cause, Authors Imaging modality P‐value Application of P‐value Amelard et al25 Dermoscopy <0.001 Selection of features for dif‐ ferentiation between benign and malignant lesions Truong et al73 Terahertz pulse imaging <0.05 Selection of Debye parame‐ ters for distinction of normal skin and BCC lesions Liu et al52 Reflectance imaging + 3D geometric information <0.001 Selection of best combination of patient' meta‐data for melanoma detection Tomatis et al53 Multispectral imaging <0.05 Selection of lesion descrip‐ tors for distinction of mela‐ noma and non‐melanoma lesions T A B L E 1   Publications referring to the use of P‐value      |  755MAGALHAES Et AL. since a few authors prefer the implementation of standard models, instead of exploring other parameter options. Nonetheless, papers concerning the development of highly complex classification sys‐ tems can also be found17,33,72 and its increment should be pursued. The increase of available labelled data is of concern to improve the training task. However, the majority of studies rely on images available on databases13,49 or supplied by hospitals,21,44 being de‐ pendent of the number of given samples. Hence, strategies to ad‐ dress this issue are of interest for future work and have already been explored by some authors.17,72 The use of support vector machine (SVM) algorithms seem to be the prime choice for skin cancer classification, either on basic mod‐ els29,30 or on more complex ones,18,60 suggesting upcoming research with this tendency. The use of p‐value to attest the significance of the acquired re‐ sults is not a common practice; however, it should be considered in upcoming research.25,52,53,73 Lastly, new publications pertaining to classification models for skin cancer detection should focus on finding a good balance be‐ tween specificity and sensitivity values, to avoid faulty diagnosis for healthy and ill patients, respectively, opening space for different image modalities information fusion. 5   |   C O N C L U S I O N Machine learning algorithms are a useful tool to assist in medical diag‐ nosis, due to its ability to rapidly assimilate information. Its efficiency in melanoma detection, particularly in dermoscopy images, has been proved with extensive research in this area, using several classifiers. However, methodologies based on information retrieved from other imaging modalities, for example spectroscopy and sonograms, needs more improvements for its application in a clinical scenario. A note is made for future research, stressing the significance of the adopted image analysis and processing methods, due to the per‐ formance dependency of the classifier on this task. In addition, fur‐ ther work exploring different classifier parameter options is also of importance to assure its successful implementation. Thus, reducing the human/operator error and the associated health costs with skin cancer diagnosis. A C K N O W L E D G E M E N T S Authors gratefully acknowledge the funding of Project NORTE‐ 01‐0145‐FEDER‐000022 ‐ SciTech ‐ Science and Technology for Competitive and Sustainable Industries, cofinanced by Programa Operacional Regional do Norte (NORTE2020), through Fundo Europeu de Desenvolvi‐mento Regional (FEDER) and Project LAETA ‐ UID/ EMS/50022/2013. O R C I D Carolina Magalhaes https://orcid.org/0000‐0001‐5602‐718X Joaquim Mendes https://orcid.org/0000‐0003‐4254‐1879 Ricardo Vardasca https://orcid.org/0000‐0003‐4217‐2882 R E F E R E N C E S 1. Reichrath J, Leiter U, Garbe C. Epidemiology of Melanoma and Nonmelanoma Skin Cancer — The Role of Sunlight. In: Reichrath J, ed. Sunlight, Vitamin D and Skin Cancer. New York, NY: Springer; 2014:89‐103. 2. Narayanan DL, Saladi RN, Fox JL. Ultraviolet radiation and skin can‐ cer. Int J Dermatol. 2010;49(9):978‐986. 3. Apalla Z, Nashan D, Weller RB, Castellsagué X. Skin cancer: epide‐ miology, disease burden, pathophysiology, diagnosis, and therapeu‐ tic approaches. Dermatol Ther (Heidelb). 2017;7(1):5‐19. 4. Lubam MC, Bangs SA, Mohler AM, Common D. Benign skin tu‐ mors. Am Fam Physician. 2003;67(4):729‐738 http://www.aafp.org/ afp/2003/0215/p729.html. 5. Massone C, Di Stefani A, Soyer HP. Dermoscopy for skin cancer detection. Curr Opin Oncol. 2005;17(2):147‐153. 6. Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective. Artif Intell Med. 2001;23(1):89‐109. 7. Labatut V, Cherifi H. Accuracy measures for the comparison of classifiers. 5th Int Conf Inf Technol. 2011;(May):11 https ://doi.org/10.1.1.658. 1777. 8. Gui C, Chan V. Machine learning in medicine. Univ West Ont Med J. 2017;86(2):77‐78. 9. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta‐analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Medicine. 2009;6(7):e1000100. 10. Moher D, Liberati A, Tetzlaff J, Altman D, The PRISMA Group. Preferred reporting items for systematic reviews and meta‐analy‐ ses: The PRISMA statement. PLoS ONE. 2009;6(7):1‐6. 11. Grzesiak‐Kopeć K, Ogorzałek M, Nowak L. Computational clas‐ sification of melanocytic skin lesions. Artif Intell Soft Computing. 2016:9693;169–178. 12. Rahman MM, Bhattacharya P. An integrated and interactive deci‐ sion support system for automated melanoma recognition of der‐ moscopic images. Comput Med Imaging Graph. 2010;34(6):479–486. 13. Pennisi A, Bloisi DD, Nardi D, Giampetruzzi AR, Mondino C, Facchiano A. Skin lesion image segmentation using Delaunay Triangulation for melanoma detection. Comput Med Imaging Graph. 2016;52:89–103. 14. Ruiz D, Berenguer V, Soriano A, Sánchez B. A decision support sys‐ tem for the diagnosis of melanoma: A comparative approach. Expert Syst Appl. 2011;38(12):15217–15223. 15. Narasimhan K, Elamaran V. Wavelet‐based energy features for di‐ agnosis of melanoma from dermoscopic images. Int J Biomed Eng Technol. 2016;20(3):243. 16. Barata C, Ruela M, Francisco M, Mendonca T, Marques JS. Two sys‐ tems for the detection of melanomas in dermoscopy images using texture and color features. IEEE Syst J. 2014;8(3):965–979. 17. Masood A, Al‐Jumaily A, Anam K. Self‐supervised learning model for skin cancer diagnosis. Int IEEE/EMBS Conf Neural Eng NER. 2015;2015‐July:22‐24. 18. Schaefer G, Krawczyk B, Celebi ME, Iyatomi H, Hassanien AE. Melanoma Classification Based on Ensemble Classification of Dermoscopy Image Features. In: Hassanien AE, Tolba MF, Taher Azar A, eds. Advanced Machine Learning Technologies and Applications. AMLTA 2014. Communications in Computer and Information Science. Cham: Springer; 2014:291–298. https ://doi.org/10.1007/978‐3‐319‐13461‐1_28 19. Castillejos‐fernández H, López‐ortega O. An intelligent system for the diagnosis of skin cancer on digital images taken with dermos‐ copy. Acta Polytechnica Hungarica. 2017;14(3):169–185. https://orcid.org/0000-0001-5602-718X https://orcid.org/0000-0001-5602-718X https://orcid.org/0000-0003-4254-1879 https://orcid.org/0000-0003-4254-1879 https://orcid.org/0000-0003-4217-2882 https://orcid.org/0000-0003-4217-2882 http://www.aafp.org/afp/2003/0215/p729.html http://www.aafp.org/afp/2003/0215/p729.html https://doi.org/10.1.1.658.1777 https://doi.org/10.1.1.658.1777 https://doi.org/10.1007/978-3-319-13461-1_28 756  |     MAGALHAES Et AL. 20. Faal M, Miran Baygi MH, Kabir E. Improving the diagnostic accuracy of dysplastic and melanoma lesions using the decision template combination method. Ski Res Technol. 2013;19(1):113–122. 21. Rastgoo M, Morel O, Marzani F, Garcia R. Ensemble approach for differentiation of malignant melanoma. Proc SPIE ‐ Int Soc Opt Eng. 2015;9534. https ://doi.org/10.1117/12.2182799. 22. Rastgoo M, Garcia R, Morel O, Marzani F. Automatic differentia‐ tion of melanoma from dysplastic nevi. Comput Med Imaging Graph. 2015;43:44–52. 23. Xie F, Fan H, Li Y, Jiang Z, Meng R, Bovik A. Melanoma classification on dermoscopy images using a neural network ensemble model. IEEE Trans Med Imaging. 2017;36(3):849–858. 24. Abbas Q, Sadaf M, Akram A. Prediction of dermoscopy patterns for recognition of both melanocytic and non‐melanocytic skin lesions. Computers. 2016;5(3):13. 25. Amelard R, Glaister J, Wong A, Clausi DA. High‐level intuitive fea‐ tures (HLIFs) for intuitive skin lesion description. IEEE Trans Biomed Eng. 2015;62(3):820–831. 26. Almansour E, Jaffar MA. Classification of dermoscopic skin cancer images using color and hybrid texture features. IJCSNS Int J Comput Sci Netw Secur. 2016;16(4):135–139. 27. Adjed F, Faye I, Ababsa F, Gardezi SJ, Dass SC. Classification of skin cancer images using local binary pattern and SVM classifier. In: 4th International Conference on Fundamental and Applied Sciences (ICFAS 2016). Vol 1787. Kuala Lumpur, Malaysia: AIP Conference Proceedings; 2016. https ://doi.org/10.1063/1.4968145 28. Tan TY, Zhang L, Jiang M. An intelligent decision support system for skin cancer detection from dermoscopic images. In: 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC‐FSKD). IEEE; 2016:2194–2199. doi:10.1109/FSKD.2016.7603521. 29. Jaworek‐Korjakowska J. Computer‐aided diagnosis of micro‐malig‐ nant melanoma lesions applying support vector machines. Biomed Res Int. 2016;2016:1–8. 30. La Torre E, Caputo B, Tommasi T. Learning methods for melanoma recognition. Int J Imaging Syst Technol. 2010;20(4):316–322. 31. Codella N, Nguyen Q‐B, Pankanti S, et al. Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J Res Dev. 2017;61(4/5):5:1–5:15. 32. Celebi ME, Kingravi HA, Uddin B, et al. A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph. 2007;31(6):362–373. 33. Masood A, Al‐Jumaily A. SA‐SVM based automated diagnostic system for skin cancer. In: Wang Y, Jiang X, Zhang D, eds. In SPIE Sixth International Conference on Graphic and Image Processing (ICGIP 2014). Vol 9443. Sidney, Australia: International Society for Optics and Photonics; 2015. https ://doi.org/10.1117/12.2179094 34. Wahba MA, Ashour AS, Napoleon SA, Abd Elnaby MM, Guo Y. Combined empirical mode decomposition and texture features for skin lesion classification using quadratic support vector machine. Heal Inf Sci Syst. 2017;5(1):10. 35. Yuan X, Yang Z, Zouridakis G, Mullani N. SVM‐based Texture Classification and Application to Early Melanoma Detection. In: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE; 2006:4775‐4778. https ://doi.org/10.1109/ IEMBS.2006.260056 36. Suganya R. An automated computer aided diagnosis of skin lesions de‐ tection and classification for dermoscopy images. In: 2016 International Conference on Recent Trends in Information Technology (ICRTIT). IEEE; 2016:1‐5. https ://doi.org/10.1109/ICRTIT.2016.7569538 37. Joseph S, Panicker JR. Skin lesion analysis system for melanoma detec‐ tion with an effective hair segmentation method. In: 2016 International Conference on Information Science (ICIS). IEEE; 2016:91‐96. https ://doi.org/10.1109/INFOS CI.2016.7845307 38. Messadi M, Bessaid A, Taleb‐Ahmed A. New characterization methodology for skin tumors classification. J Mech Med Biol. 2010;10(03):467–477. 39. Aswin RB, Jaleel JA, Salim S. Hybrid genetic algorithm ‐ Artificial neu‐ ral network classifier for skin cancer detection. In: 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT). IEEE; 2014:1304–1309. https ://doi.org/10.1109/ICCIC CT.2014.6993162 40. Cheng B, Joe Stanley R, Stoecker WV, et al. Analysis of clinical and dermoscopic features for basal cell carcinoma neural network clas‐ sification. Ski Res Technol. 2013;19(1):217–222. 41. Ferris LK, Harkes JA, Gilbert B, et al. Computer‐aided classifica‐ tion of melanocytic lesions using dermoscopic images. J Am Acad Dermatol. 2015;73(5):769–776. 42. Kharazmi P, Lui H, Wang ZJ, Lee TK. Automatic detection of basal cell carcinoma using vascular‐extracted features from dermoscopy images. In: 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE; 2016:1‐4. https ://doi.org/10.1109/ CCECE.2016.7726666. 43. Kharazmi P, AlJasser MI, Lui H, Wang ZJ, Lee TK. Automated detec‐ tion and segmentation of vascular structures of skin lesions seen in dermoscopy, with an application to basal cell carcinoma classifica‐ tion. IEEE J Biomed Health Inform. 2017;21(6):1675–1684. 44. Ganster H, Pinz P, Rohrer R, Wildling E, Binder M, Kittler H. Automated melanoma recognition. IEEE Trans Med Imaging. 2001;20(3):233–239. 45. Gerger A, Koller S, Weger W, et al. Sensitivity and specificity of con‐ focal laser‐scanning microscopy for in vivo diagnosis of malignant skin tumors. Cancer. 2006;107(1):193–200. 46. Lorber A, Wiltgen M, Hofmann‐Wellenhof R, et al. Correlation of image analysis features and visual morphology in melanocytic skin tumours using in vivo confocal laser scanning microscopy. Ski Res Technol. 2009;15(2):237–241. 47. Gerger A, Wiltgen M, Langsenlehner U, et al. Diagnostic image anal‐ ysis of malignant melanoma in in vivo confocal laser‐scanning mi‐ croscopy: a preliminary study. Ski Res Technol. 2008;14(3):359–363. 48. Koller S, Wiltgen M, Ahlgrimm‐Siess V, et al. In vivo reflec‐ tance confocal microscopy: automated diagnostic image analy‐ sis of melanocytic skin tumours. J Eur Acad Dermatology Venereol. 2011;25(5):554–558. 49. Odeh SM, de Toro F, Rojas I, Saéz‐Lara MJ. Evaluating fluorescence illumination techniques for skin lesion diagnosis. Appl Artif Intell. 2012;26(7):696–713. 50. Odeh SM, Baareh A. A comparison of classification methods as diagnostic system: a case study on skin lesions. Comput Methods Programs Biomed. 2016;137:311–319. 51. Li L, Zhang Q, Ding Y, Jiang H, Thiers BH, Wang JZ. Automatic di‐ agnosis of melanoma using machine learning methods on a spectro‐ scopic system. BMC Med Imaging. 2014;14(1):1–12. 52. Liu Z, Sun J, Smith M, Smith L, Warr R. Incorporating clinical meta‐ data with digital image features for automated identification of cu‐ taneous melanoma. Br J Dermatol. 2013;169(5):1034–1040. 53. Tomatis S, Carrara M, Bono A, et al. Automated melanoma detec‐ tion with a novel multispectral imaging system: Results of a pro‐ spective study. Phys Med Biol. 2005;50(8):1675–1687. 54. Tomatis S, Bono A, Bartoli C, et al. Automated melanoma detection: Multispectral imaging and neural network approach for classifica‐ tion. Med Phys. 2003;30(2):212–221. 55. Mohr P, Birgersson U, Berking C, et al. Electrical impedance spec‐ troscopy as a potential adjunct diagnostic tool for cutaneous mela‐ noma. Ski Res Technol. 2013;19(2):75–83. 56. Åberg P, Birgersson U, Elsner P, Mohr P, Ollmar S. Electrical imped‐ ance spectroscopy and the diagnostic accuracy for malignant mela‐ noma. Exp Dermatol. 2011;20(8):648–652. https://doi.org/10.1117/12.2182799 https://doi.org/10.1063/1.4968145 https://doi.org/10.1117/12.2179094 https://doi.org/10.1109/IEMBS.2006.260056 https://doi.org/10.1109/IEMBS.2006.260056 https://doi.org/10.1109/ICRTIT.2016.7569538 https://doi.org/10.1109/INFOSCI.2016.7845307 https://doi.org/10.1109/INFOSCI.2016.7845307 https://doi.org/10.1109/ICCICCT.2014.6993162 https://doi.org/10.1109/CCECE.2016.7726666 https://doi.org/10.1109/CCECE.2016.7726666      |  757MAGALHAES Et AL. 57. Maciel VH, Correr WR, Kurachi C, Bagnato VS, da Silva SC. Fluorescence spectroscopy as a tool to in vivo discrimination of dis‐ tinctive skin disorders. Photodiagnosis Photodyn Ther. 2017;19:45–50. 58. Jafari MH, Samavi S, Karimi N, Soroushmehr S, Ward K, Najarian K. Automatic detection of melanoma using broad extraction of features from digital images. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE; 2016:1357‐1360. https ://doi.org/10.1109/EMBC.2016.7590959 59. Eslava J, Druzgalski C. Differential feature space in mean shift clustering for automated melanoma assessment. In: Jaffray DA, ed. IFMBE Proceedings. Vol 51. IFMBE Proceedings. Cham: Springer International Publishing; 2015:1401‐1404. https ://doi. org/10.1007/978‐3‐319‐19387‐8_341 60. Takruri M, Rashad MW, Attia H. Multi‐classifier decision fusion for en‐ hancing melanoma recognition accuracy. Int Conf Electron Devices, Syst Appl. 2017:0‐4. 61. Oliveira RB, Marranghello N, Pereira AS, Tavares J. A computational approach for detecting pigmented skin lesions in macroscopic im‐ ages. Expert Syst Appl. 2016;61:53–63. 62. Spyridonos P, Gaitanis G, Likas A, Bassukas ID. Automatic discrim‐ ination of actinic keratoses from clinical photographs. Comput Biol Med. 2017;88:50–59. 63. Abbes W, Sellami D, Control A, Departmement EE. High‐level fea‐ tures for automatic skin lesions neural network based classification. Int Image Process Appl Syst Conf. 2016;1–7. 64. Karami N, Esteki A. Automated Diagnosis of Melanoma Based on Nonlinear Complexity Features. In: Osman NAA, Abas WABW, Wahab AKA, Ting H, eds. 5th Kuala Lumpur International Conference on Biomedical Engineering 2011. Berlin, Heidelberg: Springer; 2011:270‐274. https ://doi.org/10.1007/978‐3‐642‐21729‐6_71 65. Sanchez I, Agaian S. Computer aided diagnosis of lesions extracted from large skin surfaces. In: 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE; 2012:2879‐2884. https ://doi.org/10.1109/ICSMC.2012.6378186 66. Tabatabaie K, Esteki A. Independent Component Analysis as an Effective Tool for Automated Diagnosis of Melanoma. In: 2008 Cairo International Biomedical Engineering Conference. IEEE; 2008:1‐4. https ://doi.org/10.1109/CIBEC.2008.4786081 67. Jafari MH, Samavi S, Soroushmehr S, Mohaghegh H, Karimi N, Najarian K. Set of descriptors for skin cancer diagnosis using non‐dermoscopic color images. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE; 2016:2638‐2642. https ://doi.org/10.1109/ICIP.2016.7532837 68. Przystalski K. Decision support system for skin cancer diagnosis. Oper Res. 2010;406–413. 69. Cavalcanti PG, Scharcanski J. Automated prescreening of pig‐ mented skin lesions using standard cameras. Comput Med Imaging Graph. 2011;35(6):481–491. 70. Noroozi N, Zakerolhosseini A. Computer assisted diagnosis of basal cell carcinoma using Z‐transform features. J Vis Commun Image Represent. 2016;40:128–148. 71. Noroozi N, Zakerolhosseini A. Differential diagnosis of squamous cell carcinoma in situ using skin histopathological images. Comput Biol Med. 2016;70:23–39. 72. Masood A, Al‐Jumaily A. Semi‐advised learning model for skin cancer diagnosis based on histopathalogical images. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE; 2016:631‐634. https ://doi. org/10.1109/EMBC.2016.7590781 73. Truong B, Tuan HD, Wallace VP, Fitzgerald AJ, Nguyen HT. The potential of the double debye parameters to discriminate be‐ tween basal cell carcinoma and normal skin. IEEE Trans Terahertz Sci Technol. 2015;5(6):990–998. 74. Kia S, Setayeshi S, Shamsaei M, Kia M. Computer‐aided diag‐ nosis (CAD) of the skin disease based on an intelligent classifi‐ cation of sonogram using neural network. Neural Comput Appl. 2013;22(6):1049–1062. 75. Ding Y, John NW, Smith L, Sun J, Smith M. Combination of 3D skin surface texture features and 2D ABCD features for improved mela‐ noma diagnosis. Med Biol Eng Comput. 2015;53(10):961–974. 76. Teixeira PM. Sobre o significado da significância estatística. Acta Med Port. 2018;31(5):238. How to cite this article: Magalhaes C, Mendes J, Vardasca R. The role of AI classifiers in skin cancer images. Skin Res Technol. 2019;25:750–757. https ://doi.org/10.1111/srt.12713 https://doi.org/10.1109/EMBC.2016.7590959 https://doi.org/10.1007/978-3-319-19387-8_341 https://doi.org/10.1007/978-3-319-19387-8_341 https://doi.org/10.1007/978-3-642-21729-6_71 https://doi.org/10.1109/ICSMC.2012.6378186 https://doi.org/10.1109/CIBEC.2008.4786081 https://doi.org/10.1109/ICIP.2016.7532837 https://doi.org/10.1109/EMBC.2016.7590781 https://doi.org/10.1109/EMBC.2016.7590781 https://doi.org/10.1111/srt.12713