key: cord-0015670-5du5t75i authors: Chabrun, Floris; Dieu, Xavier; Doudeau, Nicolas; Gautier, Jennifer; Luque‐Paz, Damien; Geneviève, Franck; Ferré, Marc; Mirebeau‐Prunier, Delphine; Annweiler, Cédric; Reynier, Pascal title: Deep learning shows no morphological abnormalities in neutrophils in Alzheimer's disease date: 2021-02-20 journal: Alzheimers Dement (Amst) DOI: 10.1002/dad2.12146 sha: ffd9fdf2dff0b58a33f4d168183ec82f74d4d3d1 doc_id: 15670 cord_uid: 5du5t75i INTRODUCTION: Several studies have provided evidence of the key role of neutrophils in the pathophysiology of Alzheimer's disease (AD). Yet, no study to date has investigated the potential link between AD and morphologically abnormal neutrophils on blood smears. METHODS: Due to the complexity and subjectivity of the task by human analysis, deep learning models were trained to predict AD from neutrophil images. Control models were trained for a known feasible task (leukocyte subtype classification) and for detecting potential biases of overfitting (patient prediction). RESULTS: Deep learning models achieved state‐of‐the‐art results for leukocyte subtype classification but could not accurately predict AD. DISCUSSION: We found no evidence of morphological abnormalities of neutrophils in AD. Our results show that a solid deep learning pipeline with positive and bias control models with visualization techniques are helpful to support deep learning model results. Over the past decade, inflammation has emerged as a prominent feature of Alzheimer's disease (AD) pathophysiology. 1 Several studies have provided evidence of a key role played by neutrophils in ADrelated inflammatory processes, 2 mediated by intracellular granules, surface expression of inflammatory proteins, higher extravasation, and vascular NETosis. [2] [3] [4] It is known that, on a blood smear, neutrophils exhibit abnormalities or morphologic particularities during infections and inflammatory morphology and AD, we trained state-of-the-art artificial intelligence architectures based on machine learning methods, so-called deep learning, to predict AD based on images of blood smears. Patients from the Department of Geriatrics of the University Hospital of Angers were recruited into two cohorts: the AD group for patients with AD and the subjective memory complaint (SMC) group for patients without dementia and without mild cognitive impair- Two pre-trained state-of-the-art architectures (VGG-16 and Inception v3) were trained after transfer learning for three learning prediction tasks, including two control tasks, to ensure the veracity of the results obtained for AD patients. The first task, referred to as "AD prediction," aimed at classifying patients with AD versus patients with SMC from neutrophil images from peripheral blood smears. The same deep learning architectures were used for a simpler task: Finally, to explore biases potentially responsible for overfitting, by learning by heart for example, we also trained the same architectures to predict the patient to whom each image belonged. This task is referred to as "patient identification." Details concerning the choice of those deep learning architectures and the training are presented in File S1, sections 3-7; see Figure S1 in supporting information. • There is no argument in favor of morphologically abnormal neutrophils in Alzheimer's disease. • Study is based on data allowing state-of-the-art leukocyte deep learning classification. • A pipeline allows its robustness to be checked regardless of sample size. 1. Systematic review: We reviewed literature using published and pre-print sources (eg, PubMed, arXiv). Several studies have recently provided evidence of a major role played by neutrophils in the pathophysiology of Alzheimer's disease. To our knowledge, however, no study has addressed the presence of morphological alterations among neutrophils on blood smears, despite the wellknown links between inflammation processes and neutrophilic morphological abnormalities. Our results confirm the absence of neutrophilic morphological abnormalities on blood smears. This is supported by deep learning control models achieving state-of-the-art accuracy for a known feasible task on the same data and visualization techniques exploring the behavior of those models. 3. Future directions: As neutrophils did not show any morphological alterations on blood smears carried out on routine blood formula counts, it may be interesting to analyze higher resolution images to search for finer alterations or focus on non-cell elements not routinely analyzed such as neutrophil extracellular traps. The cohort is described in Table 1 . Two patients had Waldenström's disease: one in the AD group and one in the SMC group. Because of the balance between both groups, those patients were kept in the analyses. No other patients in the AD group had any form of malignant hemopathy. In the SMC group, one patient had chronic lymphocytic leukemia, and one had a follicular lymphoma. Those patients were kept in our study, because our partitioning pipeline prevents models from being tested on images of patients they were trained with. Abbreviations: CRP, C-reactive protein; PME, polymorphonuclear eosinophil; PMN, polymorphonuclear neutrophil; SD, standard deviation. The The top model (Inception v3) achieved an accuracy on the test set of 95.3% (80.3% with random translation/rotation applied to the image). For all models, the median accuracy was 17.3% (73.8% and 12.2% median accuracy for models trained with soft and strong image augmentation, respectively). The heatmaps presented in Figure 1A -C depict the attention of neural networks on the image for the associated output prediction. This allows assessment of (1) which regions are useful for the predictions We also showed that the deep learning models used (mainly Inception v3) could achieve a high accuracy when predicting to which patient a cell image belongs. Grad-CAM showed that this ability is attained by focusing on more minor details, such as red blood cell density and spreading, or particularities on cells, like granulation density. Although this highlights the high sensitivity of those small details and thus their possible performance, this also highlights how those models are prone to overfitting. This is demonstrated by predictions that neural networks tend to change when translation/rotation is applied to the image. The best model trained for AD prediction showed an interesting AUC of 0.70 on the training set, confirmed on a test set of new images (AUC = 0.68). However, Grad-CAM visualization supports a tendency to overfit, showing a high versatility concerning the details on which the model focuses and a high instability when applying translation or rotation to the images. Furthermore, a grid search showed that most models could not achieve an AUC on a test set higher than 0.5, thus supporting this tendency. Our results draw attention to the overfitting risks of deep learning architectures. We hereby show that state-of-the-art deep learning architectures can efficiently learn to memorize to which patient an image belongs, based on a variety of details, some of which are minute. In our study, we carefully partitioned our samples to make sure that a model trained on images from a patient would not be tested on images of the same patient, to avoid any bias leading to an overestimation of its performance. However, to date, a number of articles recently applying deep learning to blood cell classification or prediction have not specified the stratification during partitioning. [12] [13] [14] This omission may lead to the overestimation of both the results of those models and particularly their ability to generalize to new patients. Our study was restricted on retrospective data analyzed during the routine care of patients seen in the University Hospital of Angers. For further confirmation of these results, multicentric prospective data, including blood smear images obtained with other coloration methods and other digital microscopes will be needed. Our data are accessible for further analysis upon reasonable request to the corresponding author. We used a solid prediction-based pipeline: (1) a positive control predictive model, trained to verify the feasibility of a known task on our dataset, that is, leukocyte subtype classification; (2) Inflammation as a central mechanism in Alzheimer's disease Neutrophil hyperactivation correlates with Alzheimer's disease progression Imaging of Leukocyte Trafficking in Alzheimer's The role of neutrophil granule proteins in neuroinflammation and Alzheimer's disease A prospective study of hospitalized patients with leukemoid reaction; causes, prognosis and value of manual peripheral smear review Morphological anomalies of circulating blood cells in COVID-19 Medical Image Analysis using Convolutional Neural Networks: a Review Human-Level Recognition of Blast Cells in Acute Myeloid Leukaemia with Convolutional Neural Networks Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning Recognition of peripheral blood cell images using convolutional neural networks Automatic white blood cell classification using pre-trained deep learning models: resNet and Inception Automatic classification of leukocytes using deep neural network Fine-grained leukocyte classification with deep residual learning for microscopic images SUPPORTING INFORMATION Additional supporting information may be found online in the Supporting Information section at the end of the article. How to cite this article The authors declare that there are no conflicts of interest. Floris Chabrun https://orcid.org/0000-0001-6871-8711