key: cord-0060577-sbop95t2
authors: Kan, Maxim; Aliev, Ruslan; Rudenko, Anna; Drobyshev, Nikita; Petrashen, Nikita; Kondrateva, Ekaterina; Sharaev, Maxim; Bernstein, Alexander; Burnaev, Evgeny
title: Interpretation of 3D CNNs for Brain MRI Data Classification
date: 2021-02-20
journal: Recent Trends in Analysis of Images, Social Networks and Texts
DOI: 10.1007/978-3-030-71214-3_19
sha: 3bf233911ced2b4627451f5b492ca00e6aadbd9c
doc_id: 60577
cord_uid: sbop95t2

Deep learning shows high potential for many medical image analysis tasks. Neural networks can work with full-size data without extensive preprocessing and feature generation and, thus, information loss. Recent work has shown that the morphological difference in specific brain regions can be found on MRI with the means of Convolution Neural Networks (CNN). However, interpretation of the existing models is based on a region of interest and can not be extended to voxel-wise image interpretation on a whole image. In the current work, we consider the classification task on a large-scale open-source dataset of young healthy subjects—an exploration of brain differences between men and women. In this paper, we extend the previous findings in gender differences from diffusion-tensor imaging on T1 brain MRI scans. We provide the voxel-wise 3D CNN interpretation comparing the results of three interpretation methods: Meaningful Perturbations, Grad CAM and Guided Backpropagation, and contribute with the open-source library.

Deep learning and specifically Convolutional Neural Networks (CNNs) has recently found many applications in the area of medical diagnostics and image processing [15, 20] . For example, processing Magnetic Resonance Images (MRI) with CNN allow reducing the dose of gadolinium used for contrast by an order of magnitude [8] . Another example is the detection of cerebral microbleeds using a 3D-CNN [4] . In the following work [26] authors apply convolutional networks to multi-modal (T1, T2 and FA) MRI images in order to segment infant brain tissue images into Gray Matter (GM), White Matter (WM), and Cerebrospinal Fluid (CSF). CNNs are used in early-stage Alzheimer's disease detection in MRI and PET images [22] . Finally, these type of networks are applied to a variety of predictive regression tasks in brain imaging, see [16] .

Conventionally, the brain data is firstly processed to get the lower dimensional meaningful features [10] or goes with whole-brain statistical analysis or Voxel-Based Morphometry (VBM). For the diffusion tensor imaging (DTI) extracted features conventionally include fraction anisotropy (FA), mean, axial and radial diffusivity values [23] . For functional T2* MRI images functional connectivity features are commonly extracted, and for T1 structural imaging-morphometry features. Data analysis and machine learning usually follow this feature extraction step [19] .

From the other side, deep learning approaches, especially those for processing 3D data, are shown to be more accurate in many applications [23] as they use full-sized data without information loss during aforementioned extensive preprocessing and feature extraction.

However, the stable and reliable interpretation of 3D CNNs is still a big topic of discussion. For example, recent studies of age and gender brain differences point out that despite its high prediction quality "the localised predictions of age and gender do not yield easily interpretable insights into the workings of the neural network" [14] .

Deep learning models interpretation in MRI implies training on large databases of healthy subjects. One of the most common and highly explored databases available in open-access is a Human Connection Project (HCP) 1 . A conventional task being extensively explored within this database is a task of gender patterns recognition between men and women [1, 3] .

However, previous studies on morphological difference between specific brain regions show interpretable results only on the feature or region-of-interest level [23] . On the contrary, the state-of-the-art deep-learning interpretation methods allow visualization of the decision rule in a pixel-wise fashion. Or in the case of 3D convolution models-voxel-wise [14] . The contributions of the proposed paper are as follows:

-we reproduce and extend the state-of-the-art 3D CNN model [23] to investigate the difference between men and women brain on T1 images and confirm previous findings on DTI; -we apply several network interpretation methods to the 3D CNN model:

Meaningful Perturbations, Grad CAM and Guided Backpropagation to find gender-specific patterns and compare their performances; -we compare the obtained attention maps to the conventional machine learning classification models on morphometry data and discuss the differences as well as with previous findings; -lastly we provide the code for MRI 3D CNN interpretation as an open-source library.

The source code is open and available at https://github.com/maxs-kan/InterpretableNeuroDL.

The database Human Connectome Project (HCP) contains MRI data from 1113 subjects, including 507 men and 606 women of ages 22-36. We explored T1 images, preprocessed with HCP-pipelines 2 .

For the morphometry data analysis we used Freesufrer 3 preprocessed features from section Expanded FreeSurfer Data for the same 1113 subjects. The morphometry characteristics as number of vertices, volumes, surface areas, and others were computed for 34 cortical regions according to Desikan-Killiany Atlas and for 45 subcortical areas according to Automated Subcortical Segmentation Atlas [6] summing up in 935 vectorized features.

We used the morphometry data for gender classification with machine learning models in comparison as one of the most popular methods for data analysis in neuroimaging. The best performing model is chosen among different classifiers: XGBoost, k-Nearest Neighbors (KNN) and Logistic Regression (LR) with a grid-search. All considered models were validated with 10-fold cross-validation technique to give an understanding how the model will generalize to an independent dataset.

In this work we reproduced 3D CNN model architecture [23] for the images sized [58,70,58]. The proposed network consists of three hidden layers, thus it is light, fast in training, and easy to interpret. To ensure the model stability, we performed 10-fold cross-validation with stratification to estimate model performance.

As the baseline for the 3D CNN network results we chose the support vector machine classifier with rbf kernel (SVM) on full sized images. This classifier is conventionally used as the network reference for image classification, for example on MNIST. The SVM classifier from sklearn was trained on the full-size data reshaped to the 1-dimensional vector, as proposed in [23] . We performed all 3D CNN experiments with pytorch framework, on Google Colab work station (Tesla K80 GPU).

Meaningful Perturbations for 3D CNN Results Interpretation. The goal of the method [7] is to perturb the smallest possible region of the MRI such that the model significantly changes its output probability for MR image class, which means that this region is the most important for model decision and it is the most informative part of the image. In this work we perturbed original image x 0 by replacing the corresponding region with Gaussian blurring of the image. Let m : Λ → [0, 1] be a mask which map each voxel u ∈ Λ of MRI to m(u). Then the operation of perturbation of the MRI has the form:

where g σ0 is a 3D Gaussian kernel with standard deviation σ 0 . The closer m(u) is to 1, the less perturbed the input image. The goal of the algorithm is to find the mask m that leads to

is the probability of x 0 belonging to a class c, while perturbing the smallest part of the input image, i.e. with a bigger amount of values m(u) close to 1. To avoid the artifacts [9] , we paded the x 0 with j zeros and the mask m applied to x K 0 :

with integer K drawn from the discrete uniform distribution on [0, j), where H, W, D-size of the image. Also, we regularized m in total-variation (TV) norm in low-resolution version, to force it had a more simple structure, upsampled it by factor s to image size and applied a Gaussian filter on the full size mask.

where g σm is a 3D Gaussian kernel with standard deviation σ m . and Up(·, s) is a trilinear upsampling algorithm by factor of s. Finding m c for class c can be formulated as the following optimization problem:

We perform ablation study and find following hyper-parameters which provide stable convergence: σ 0 = σ m = 10, λ 1 = 3, λ 2 = 1, β = 7, s = 4, j = 5, and we drawn K from the discrete uniform distribution 10 times. The score is optimized by Adam optimizer with learning rate α = 0.3, with exponential decay rates equals β 1 = 0.9, β 2 = 0.99.

Interpretation. In order to obtain saliency map for each voxel importance we used Guided Backpropagation method [21] . This approach computes the gradient of the score y c for class c with respect to the input MRI image x based on the network:

The gradient is computed with specific backpropagation through the ReLU nonlinearity. In Guided Backpropagation, we backpropagate the positive values of the gradient and set the negative ones to zero. Let G l be a gradient backpropagated through layer l and f l+1 i = ReLU (f l i ), then the used backpropagation process is given by the equation:

After obtaining the saliency map in accordance with this specific backpropagation, we obtain an attention map m c = ReLU (m c ) showing the voxels, which has only positive impact to score y c .

Grad CAM for 3D CNN Results Interpretation. Grad CAM [18] interprets the model, assuming that the deep CNN layers capture higher-level visual constructs [2] . In the neural network terminal layers we expected features maps to capture higher-level patterns which could be responsible for class discrimination. Grad CAM computes the gradient for score y c of class c before terminal layer with respect to filter activations of the last convolutional layer F k . Then it computes the importance weights for each filter, i.e.

where H, W, D-size of a filter activation tensor. α c k captures the "importance" of filter k for a target class c. To obtain the class-discriminative localization mask m c , we computed a weighted combination of filter activations, and follow it by a ReLU:

Also we upsampled m c to the input image resolution using trilinear interpolation.

The results of accuracy metrics for 10-folds cross-validation and 1113 subjects are in Table 1 . Feature importances (β scores) for Logistic Regression model chosen via a grid search are provided in Fig. 1 . Parameters of model are follow: C = 1.3, penalty = 'l2', max iter = 10000. Importances were selected by coefficients of features in the decision function.

Here on x axis we provide the features scores and on y axis-names of features. The classification models were chosen to explore the classification baseline accuracy across several different machine learning methods. As can be seen from Fig. 1 , the most important features for the Logistic Regression model belong to the following brain regions volumes and intensities: corpus callosum, left and right insula and thalamic regions, as well as whole brian metrics for white matter hyperintensities.

The 3D CNN model yielded the mean accuracy of 0.92 ± 0.03 on 10-fold crossvalidation. SVM achieved 0.90 ± 0.02 accuracy on 10-fold cross-validation. So 3D-CNN model slightly outperformed standard SVM.

For the two target classes in the gender differences classification, we got two different feature maps correspondingly with the meaningful perturbations algorithm. Two feature masks for men and women appeared to highlight different regions of interest and were then explored separately.

We completed 10 fold cross-validation to check the 3D CNN performance with the images restricted on masks. Then we multiplied every validation sample by averaging men mask, by averaging women mask or by the sum of these masks voxel-wise. The accuracy for the mask belonging to the men group is 0.59 ± 0.18, for the women mask-0.59 ± 0.07 and for the conjoined mask-0.82 ± 0.12 respectively. Thus, we can conclude that all necessary information for classification task is both in men and women masks (in their conjunction). The difference in masks for man and women may be explained by the specifics of the algorithm: we need to find the smallest region in the input image, deletion of which will decrease the probability of being a specific class. In Fig. 2a we show the final mask which contains regions for men and women.

Next, to find the correspondence of the attention maps with specific brain regions we performed a segmentation of the MR images on 246 grey matter regions with the Human Brainnettome Atlas [5] , as well as 50 white matter regions with ICBM-81 Atlas [13] . For each region of each brain atlas, we estimated fractions of voxels of this region included into the mask, which we have obtained via Meaningful Perturbations. We normalized these fractions, so the values for all regions would sum up to 1. The top-5 scored regions of each atlas with the largest values are presented in Table 2 . These findings partially overlap with the morphometry results, showing common white matter regions in the corpus callosum (Anterior corona radiata), as well as the cerebellum (Cerebral peduncle). The grey matter region in common overlaps on frontal gyri (Middle frontal gyrus Right).

We computed a saliency map for every person in the dataset and then took mean over the dataset. As we have two classes in our dataset, the final map contains the regions of interest for each class, see Fig. 2b .

We computed corresponding localization masks, containing information about both men and woman discriminative regions of interest. The cross-sectional view of the result is shown in Fig. 2c . To compare the resulting masks, we calculated the DICE [12] score between them. For this, we chose a threshold value t to obtain binary masks in the way to nullify all values outside the brain. The threshold of t = 0.65 was chosen for Grad CAM and Meaningful Perturbations masks and t = 0.04 for Guided Backpropagation. For Guided Backpropagation, such small threshold is used, due to the fact that the resulting mask is the gradient of the input image, and the exact values in the mask are much lower compared to other methods. The cross-sectional DICE score is shown in Table 3 . 

In the current work we aimed at studying gender-related differences in human brain by creating a predictive model to solve a classification task and to interpret its decisions.

In order to localize the most informative brain areas for classification task, we created attention maps for 3D CNN output in three different ways. Using these maps we were able to denote which brain regions play the most important role in gender classification. According to Meaningful Perturbations method the brain regions with the highest classification accuracy Table 2 was Orbital Gyrus Right (A13 R and A11l R) what corresponds with the results obtained from [23] . Moreover we got that Inferior parietal lobule (A39RD R) is the region with high classification accuracy. That goes in line with the previous studies [17] , where it was shown that parietal lobe activity is biased to the right hemisphere in men. We got that the parahippocampal gyrus (TI L) plays part in gender differentiation. This result corresponds to the previous findings, where it was shown that parahippocampal gyrus activates more intensely in the female brain while executing large-scale spatial tasks [24] .

Right, and left corticospinal tracts showed high classification accuracy. This can be explained by the fact that the relative volume of the corticospinal tract on the left is larger in the female brain [11] . Anterior corona radiata right and cerebral peduncle are also responsible for gender differentiation what goes in line with [23] . That confirms the most discriminative brain regions in morphometry model results, showing the importance of Corpus colosseum posterior part voxels mean (FS CC Posterior Mean) as the most discriminative in the Logistic Regression model.

It is also worth noting, that attention maps in Fig. 2a show the spatial pattern of frontoparietal resting-state brain network, which was initially discovered from resting-state fMRI activity and is thought to be involved in a wide variety of tasks by initiating and modulating cognitive control abilities [25] . It might be interesting in future research to look specifically at this network and explore it in terms of gender-related brain differences.

We show that the classification result on T1 images is lower than the result on FA images comparing the cross-validation mean scores (0.933), thus suggesting that DWI imaging might be more predictive for these type of structural differences. However, it should be mentioned that in the original approach as well, as in current paper, all T1 images were compressed to smaller tensors from [260, 311, 260] to [58, 70, 58] addressing memory issues. This compression may result in information loss and can explain similar classification score in neural networks comparing to the baseline methods.

This work is an extension of the study on FA images from [23] . Here we create a 3D CNN model on T1 images of the same subjects and reveal similar genderrelated differences in brain structure. The model exhibits the mean accuracy of 0.92 ± 0.03, which corresponds to the morphometry data classification and higher than SVM classification (0.90 ± 0.02).

We apply several network interpretation methods to the 3D CNN model: Meaningful Perturbations, Grad CAM and Guided Backpropagation to find gender-specific patterns, and to compare their performances. We compare the results of interpretation in terms of the Dice coefficient. High scores on paired mask comparisons confirm that all three methods reveal similar patterns and thus belong to stable and trustworthy predictions. We found that Grad CAM and Guided Backpropagation are very sensitive to the slightest changes in the model weights, therefore, when using such layers as Dropout, it is necessary to fix the random seed. Then the Meaningful Perturbations method is less sensitive to small changes in model weights. We found that the Grad CAM method is the fastest one (executed within one minute) and ready plug-and-play method, Guided Backpropagation is slower (up to several minutes), as it propagates the gradient across the entire network, and has a more sparse attention map. The Meaningful Perturbations method is the slowest one (executes for several hours) yet showing the most anatomic-like attention maps. We recommend using the Grad CAM method for network interpretation at first in similar tasks.

Our deep learning model interpretation results are in line with the results from machine learning classification based on morphometry data. These findings are in line with previous studies as well, confirming the assumption that men's and women's differences more probably exist in the whole-brain range.

We also publish the code to the open-source library for public use. The proposed interpretation tool could be successfully used in various MRI pathology detection applications like epilepsy detection, Alzheimer's disorder diagnosis, Autism Spectrum disorder classification, and others.

We analyzed features obtained in First Hidden Layer of 3D CNN as in [23] . Even though we used T1 modality MRI images in contorary to DWI modality and fractional anisotropy (FA) images in previous studies. Similar to results shown on FA images, we found that mean voxel values for 31 features have a significant difference in men-women groups, with 10 features larger for women, and 21 features larger for men (see Fig. 3 ) accounting the multiple-comparisons correction. That reproduces the previously stated result, assuming that "men's brains likely have more complex features as reflected by significantly higher entropy." As well as that important gender-related patterns are likely to be spread in the whole-brain grey and white matter. That highlights the importance of the results discussed in the main paper, as the attention maps compared from different approaches are extracted from the whole brain imagery, without any region-of-interest removal, as in [23] .

Why sex matters for neuroscience

Microsoft COCO captions: data collection and evaluation server

Evolving knowledge of sex differences in brain structure, function, and chemistry

Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks

The human Brainnetome Atlas: a new brain atlas based on connectional architecture

Interpretable explanations of black boxes by meaningful perturbation

Deep learning enables reduced gadolinium dose for contrast-enhanced brain MRI

Adversarial examples in the physical world

Microanisotropy imaging: quantification of microscopic diffusion anisotropy and orientational order parameter by diffusion MRI with magic-angle spinning of the q-vector

Gender differences in language and motor-related fibers in a population of healthy preterm neonates at term-equivalent age: a diffusion tensor and probabilistic tractography study

V-Net: fully convolutional neural networks for volumetric medical image segmentation

MRI atlas of human white matter

Is texture predictive for age and sex in brain MRI?

Voxelwise 3D convolutional and recurrent neural networks for epilepsy and depression diagnostics from structural and functional MRI data

Ensemble of 3D CNN regressors with data fusion for fluid intelligence prediction

Gender dependent EEG-changes during a mental rotation task

Grad-CAM: visual explanations from deep networks via gradient-based localization

Pattern recognition pipeline for neuroimaging data

MRI-based diagnostics of depression concomitant with epilepsy: in search of the potential biomarkers

Striving for simplicity: the all convolutional net

Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis

Brain differences between men and women: evidence from deep learning

Gender differences in largescale and small-scale spatial ability: a systematic review based on behavioral and neuroimaging research

Fronto-parietal network: flexible hub of cognitive control

Deep convolutional neural networks for multi-modality isointense infant brain image segmentation

Acknowledgements. The reported study was funded by RFBR according to the research project №20-37-90149. Also we acknowledge participation of Ruslan Rakhimov in development of meaningfull perturbation method on MRI data.