key: cord-0496150-cx2j6n7u
authors: Huang, Ling; Ruan, Su; Denoeux, Thierry
title: Covid-19 classification with deep neural network and belief functions
date: 2021-01-18
journal: nan
DOI: nan
sha: a0e1128f94edc0afd594819f38c068e5e364cd8e
doc_id: 496150
cord_uid: cx2j6n7u

Computed tomography (CT) image provides useful information for radiologists to diagnose Covid-19. However, visual analysis of CT scans is time-consuming. Thus, it is necessary to develop algorithms for automatic Covid-19 detection from CT images. In this paper, we propose a belief function-based convolutional neural network with semi-supervised training to detect Covid-19 cases. Our method first extracts deep features, maps them into belief degree maps and makes the final classification decision. Our results are more reliable and explainable than those of traditional deep learning-based classification models. Experimental results show that our approach is able to achieve a good performance with an accuracy of 0.81, an F1 of 0.812 and an AUC of 0.875.

Covid-19 case classification; 3) a semi-supervised training algorithm is proposed to train the framework with partial annotations. We show that this work can contribute to more reliable detection of Covid-19 cases and make the best use of relevant information from CT images.

Although a lot of research has been done on the Covid-19 pandemic, only a few datasets are publicly available, due to privacy protection concerns and the delay in collecting information. The Italian Society of Medical and Interventional Radiology (SIRM) provided chest X-rays and CT images of 68 Italian Covid-19 cases [1] . Murphy et al. released a dataset of chest Xrays and CT images from 99 Covid-19 cases at Radiopaedia [2]. Cohen et al [3] released a Covid-19 dataset that contains 45 patients with 84 Covid-19 X-rays images. Yang et al [4] built a COVID-CT dataset with 349 Covid-19 CT images from 216 patients and 397 non-Covid-19 CTs. These data are very useful for the development of automatic detection methods.

Since the Covid-19 outbreak, a lot of efforts have been devoted to the development of automatic deep learning methods to perform screening of Covid-19 cases from medical images [6] [7] [8] [9] [10] . For the analysis of CT scans, Yang et al. [4] [7] . Alternatively, Wang et al. [8] proposed a tailored deep convolutional neural network (COVID-Net) for detection of Covid-19 cases from chest X-ray images using EfficientNet and an additional lung dataset. Also, some researchers carried out deeper analysis of Covid-19 data. Zhou et al. [9] described an automatic Covid-19 CT segmentation framework using a U-Net integrating spatial and channel attention mechanisms. Amyar et al. [10] proposed a multi-task model for both Covid-19 CT classification and segmentation by putting together an encoder, a decoder and a classifier. All of these existing deep learning-based Covid-19 screen methods are mainly based on the probabilistic formalism, which is more limited than the theory of belief function introduced in the next section.

The theory of belief functions, also known as Dempster-Shafer theory or evidence theory, was first introduced by Dempster [11] and Shafer [12] and further popularized and developed by Smets [13] . This theory is a generation of the Bayesian theory, but it is more flexible and it allows for a wider range of uncertain and imprecise information. The great expressiveness belief function theory makes it possible to represent evidence in a more faithful way than using probability theory. Let Ω = {ω ! , ω " , … , ω # } be a finite set of hypotheses about some question. Evidence about Ω can be represented by a mapping from 2 $ to [0,1] such that ∑ ( ) %⊆$ = 1, called a mass function. For any hypothesis ⊆ Ω, the quantity ( ) represents the mass of belief specifically allocated to , and that cannot be allocated to any strict subset. Given a mass function , belief and plausibility functions from 2 $ to [0,1] can be defined, respectively, as

(2) for all ⊆ Ω. Quantity ( ) can be interpreted as the degree of evidence that totally support , while ( ) can be interpreted as the degree of evidence that does not contradict .

Two mass functions ! and " derived from two independent items of evidence can be combined by Dempster's rule [11] defined as

3 Methods

Inspired by the Evidential Neural Network (ENN) model introduced in [14] , we propose here an evidential Covid-Net for Covid-19 case classification. Figure 1 shows the proposed architecture. The input of this framework is a Covid-19 image. Firstly, ResNet50 [15] is used as for feature extraction; we take the extracted features from the last convolution layer. Secondly, an adaptive average pooling layer is used after feature extraction to get a 2048dimensional feature vector. Then a linear layer is added after the pooling layer in order to reduce the number of features from 2048 to 64. The selection of 64 is from experience. We took efficiency and computation cost into consideration and reduced the number of features from 2048 into 64. Lastly, an evidential module is added after the linear layer to map image features into mass functions used for classification. The structure of the evidential module is similar to that of the ENN model, comprised of three parts: a distance computation layer, a basic belief assignment layer and a mass fusion layer in which mass functions are combined by Dempster's rule. Details of the evidential module will be introduced in Section 3.2. Finally, the network output is a decision to classify the input image as a Covid-19 case or not. 

Neural networks have achieved remarkable performances in machine learning tasks and have become very popular, but their black-box nature makes them ill-suited to decision-aid in the medical domain. In [14] , Denoeux first proposed an Evidential Neural Network (ENN) classifier, which outputs mass functions for classification tasks. The ENN classifier first summarizes the input features as prototypes initialized, e.g., by the K-means algorithm. Each prototype / is a piece of evidence about the class of input , whose reliability decreases with the distance / between and / . Mass function induced by prototype / is

where /0 is the degree of membership of prototype / to class 0 , i = 1, … r, k = 1,2, and / and / are two tuning parameters. The evidences from the r prototypes are then combined by the Dempster's rule = ⨁ / 1 /.! .

(7) The parameters to be learned during training are the prototypes / , the membership degrees /0 , / and / . The learning process can be conducted by minimizing the cost function

where is the vector of all parameters, /0 is the output plausibility and the ground truth for instance and class , and is a regularization coefficient. The regularization coefficient can be either determined by cross-validation or fixed to a given value chosen from experience. Here we set =0.01 from experience as suggested in [14] . The details of the optimization algorithm are described in [14] .

The need for big annotated training data has become a bottleneck in deep learning, which limits its application in the medical domain where the labelling task requires not only careful delineation but also high-level professional knowledge. Thus, in this paper we propose a semi-supervised learning method for Covid-19 case classification. For each input image , we use several transformations to get new images, noted 3 . Similar images are expected to produce similar classification results even if some transformations have been performed on them.

Two loss functions are proposed for training images with label and without label. Here we train the network with 50% images and their corresponding labels using the following loss1 function,

which measures the difference between the output mass and the ground truth label y. For the remaining 50% images, we do not use label information. We calculate the mass functions for the image and the transformed image 3 and we minimize their distance using the following loss2 criterion, which measures the difference between the origin output and the transformed output:

where m is the mass function of the input image , y is the classification label, m 3 is the mass function of the transformed image 3 , and T is the number of transformed images.

In this paper we choose the COVID-CT dataset [4] to test our proposal because it contains diverse information from 216 patients. Out of the 746 instances, 425 were used for training (including 191 positive cases), 118 were used for validation (including 60 positive cases), and 203 were used for testing (including 98 positive cases). Table 1 and Table 2 show the composition of COVID-CT dataset. For the training images, we applied the following preprocessing operations: resizing to 256*256, random resizing and cropping to 224*224 and normalization. Similarly, validation and test images were preprocessed as follows: resizing to 256*256; center cropping to 224*224 and normalization. We also used some image transformations to generate new images for semi-supervised training. Based on the preprocessed image , we added Gaussian noise and flipped images with horizontal parameter 0.5 to get a new image 3 . During training, an early-stop mechanism is used during training. If the performance does not increase in 5 iterations, the training will be stopped. We used three metrics to evaluate our proposal: accuracy, F1, and area under ROC curve (AUC). For all three metrics, the higher the better. Fig. 2 shows some examples of images from the COVID-CT dataset. 

We compared our results with those of the baseline model ResNet and other state-of-the-art methods on the Covid-19 dataset. Table 3 shows the results after training the models with labeled images. We first compared our proposal with the baseline method ResNet50 and obtained 2.5%, 2.6% and 3.5% increase in, respectively, Accuracy, F1 and AUC. Compared with the DensNet-based model Covid-CT-Net, we get 1%, 0.3% and 0.6% increase in Accuracy, F1 and AUC, respectively. Our performance does not reach the performance of the best method Covid-Net, but the results are not strictly comparable because Covid-Net used additional information in the lung dataset for self-training. As for semi-supervised training, Table 4 shows the results of models trained with all learning images but only 50% of the labels. Compared with the baseline method ResNet50, our proposal obtained, respectively, 6.8%, 12.1%, 6.4% increase in Accuracy, F1 and AUC. Compared with Covid-CT-Net, our proposal obtained, respectively, 3.3%, 4.1%, 1% increase in Accuracy, F1 and AUC. For semi-supervised training, our proposal shows better performance than the baseline ResNet50 method and the state-of-the-art method Covid-Net. Compared with fully supervised training (using all training labels), semi-supervised training (with 50% of labels used) achieved comparable performance, with only 7.9%, 11%, 0.5% decrease in, respectively, Accuracy, F1 and AUC. Fig. 3 shows the ROC curves of our method with fully and partially supervised training. 

We have proposed an evidential Covid-Net model for automatic Covid-19 detection. Our model combines the ResNet architecture with an evidential layer to map high-level image features into mass functions. The belief function formalism offers a new approach for reasoning with pieces of evidence and make the results more easily interpretable than those of probabilistic methods. Moreover, we proposed a semi-supervised training method to train the evidential Covid-Net model. In future research, we will investigate the fusion of classification results from different image modalities as well as additional information in the belief function framework, to enhance the detection performance. Also, we will extend our approach to more powerful CNN architectures such as EfficientNet to obtain higher classification accuracy.

Covid-19 image data collection: Prospective predictions are the future

COVID-CT-Dataset: a CT scan dataset about COVID-19

Efficientnet: Rethinking model scaling for convolutional neural networks

Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans. medRxiv

2020. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Automatic COVID-19 CT segmentation using U-Net integrated spatial and channel attention mechanism

Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation

Upper and lower probability inferences based on a sample from a finite univariate population

A mathematical theory of evidence

The combination of evidence in the transferable belief model

A neural network classifier based on Dempster-Shafer theory

Inception-v4, inception-resnet and the impact of residual connections on learning