key: cord-1040956-ayek891u
authors: Munusamy, Hemalatha; JM, Karthikeyan; G, Shriram; S, Thanga Revathi; S, Aravindkumar
title: FractalCovNet architecture for COVID-19 Chest X-Ray image Classification and CT-scan image Segmentation
date: 2021-07-08
journal: Biocybern Biomed Eng
DOI: 10.1016/j.bbe.2021.06.011
sha: 5965ee5df586a27bd5b54aedcf0ea18f02283e3c
doc_id: 1040956
cord_uid: ayek891u

Precise and fast diagnosis of COVID-19 cases play a vital role in early stage of medical treatment and prevention. Automatic detection of COVID-19 cases using the chest X-Ray images and chest CT-scan images will be helpful to reduce the impact of this pandemic on the human society. We have developed a novel FractalCovNet architecture using Fractal blocks and U-Net for segmentation of chest CT-scan images to localize the lesion region. The same FractalCovNet architecture is also used for classification of chest X-Ray images using transfer learning. We have compared the segmentation results using various model such as U-Net, DenseUNet, Segnet, ResnetUNet, and FCN. We have also compared the classification results with various models like ResNet5-, Xception, InceptionResNetV2, VGG-16 and DenseNet architectures. The proposed FractalCovNet model is able to predict the COVID-19 lesion with high F-measure and precision values compared to the other state-of-the-art methods. Thus the proposed model can accurately predict the COVID-19 cases and discover lesion regions in chest CT without the manual annotations of lesions for every suspected individual. An easily-trained and high-performance deep learning model provides a fast way to identify COVID-19 patients, which is beneficial to control the outbreak of SARS-II-COV.

COVID-19 [1] is one of the major catastrophes in the history of mankind. It is important that suspected cases must go on with early diagnosis and detection so that they can recover from this disease. COVID-19 is caused by SARS-II Corona Virus [2] which was first discovered in China at late 2019. The virus majorly transmits through surfaces, human -human contact and through aerosols and droplets. The major impact of COVID-19 includes affecting the lungs with severe pneumonia, suffocation from loss of oxygen and it is still under research. There are few vaccines or drugs introduced to prevent or to cure this disease [3] , [4] . But still there are muted variants of corona virus are spreading across the globe [5] , [6] the effectiveness of the vaccine is not predicted for this modified virus strand [7] . So in this alarming pandemic situation it is our responsibility as individuals to be safe if any symptoms such as cough, sneeze, fever is present. As there is a probable transmission through aerosols, COVID-19 has significantly high 0 factor. We must be able to diagnose at earlier stage either through blood test with Reverse Transcription Polymerase Chain Reaction (RT-PCR) [8] or through antibody testing. A large crowd cannot exploit all the tests like RT-PCR. This is a golden method and its sensitivity and specificity is high according to the research. Priority must be given for the people who have all the symptoms. The main short-comes of RT-PCR kit is there are very few in terms of resource, the cost is too high, limited hospitals are having this facility. The time needed for report generation is high which does not meet the isolation of patients rapidly. It is important to find an alternative solutions like antibody testing using CT-scan images by practitioners to diagnose patients with COVID-19. Though they reduce the exploit of resource, there is a high chance of error and cost of testing also crucial.

In order to reduce cost and increase efficiency many so-ORCID(s):

lutions were proposed that deals with classifying the chest Xray images and CT-scan images for diagnosing whether the patient is COVID-19 positive or negative. It would be much helpful if COVID-19 positive cases can be identified at the earlier stage, so that isolating of the patients will be easy. These action can limit the rapid spread of the disease. To perform automatic detection of COVID-19 patients we have designed a decision support tool which helps to automatically detect, classify, and segment the lesion region of lungs of the COVID-19 patients. In this work, we implemented a Fractal COVID-19 Network (FractalCovNet) architecture for classification of X-Ray images and segmentation of the CT-scan images. The FractalNet [9] was first proposed for classification of the images. Here the fractal blocks are used to build a very deep neural network for classification of the ImageNet dataset. Later, it was adopted for segmentation models. In this work, we propose a FractalCovNet using the Fractal modules. The overall framework of the proposed model is shown in Figure 1 . The proposed approach performs segmentation of CT-scan images using the FractalCovNet architecture. We have also used the FractalCovNet for classification of chest X-Ray images using transfer learning.

In this work, we have implemented various deep learning models such as ResNet50, VGG16, Xception, DenseNet201 and InceptionResNetV2 using transfer learning for classification of chest X-Ray images and their performances are compared with our proposed model. The various segmentation models such as FCN (Fully Convolutional Network), U-Net, DenseUNet, and ResNetUNet are implemented for segmentation of CT-scan images and results are compared with the proposed approach. Metrics such as precision, recall, F-measure, and accuracy are used to evaluate the models. We hope using the models that we created will help in fast and early diagnosis of COVID-19 patient and help in saving life. We are proposing a cost and time effective mea- sure for diagnosis as the medical practitioners suffer due to the huge workload to save each patient.

The organization of the paper is as follows: Section 2 discusses the related works. Section 3 describes the proposed FractalCovNet architecture for segmentation of chest CT-scan images and classification of the chest X-Ray images. Section 4 describes the various datasets used, evaluation metrics along with the implementation details. Section 5 discusses the results obtained using the proposed approach and other pre-trained CNN models. Section 6 discusses the advantages of proposed model over other models available in the literature. Section 7 presents the conclusion.

The process of automating the identification of COVID-19 patient has started in the early March 2020. But there were lesser number of test samples available for training the models. Some of the initial models developed for automatic detection of COVID-19 was designed using custom CNN models. And others were developed using the pre-trained CNN models like VGG-16 [10] , ResNet [11] , InceptionRes-netV2 [12] , etc. The models developed for automatic detection of COVID-19 can be classified into two types: Models developed using CT-scan images and models developed using X-Ray images. The processing of X-Ray images helps only in detection whether the patient has COVID-19 or not. Whereas the processing of CT-scan images can identify both whether the patient has COVID-19 or not, and also determine the severity of the infection.

There are many segmentation approaches proposed for segmentation of medical images [13] , [14] , [15] . The U-Net [13] is successfully used in many medical image segmentation approaches. The adaptive context selection approach in [14] uses an context selection method in U-Net for polyp segmentation. It adds a context detection block to process the features from down sampling block before giving as an input to the up sampling block. The approach in [15] proposes a parallel reverse attention network (PraNet) for segmentation of polyp. The parallel reverse attention block provides attention at multiple stages of up sampling to increases the accuracy of segmentation. In this work, we propose a variation of U-Net which uses fractal blocks in the U-Net for segmentation of chest CT-scan images. The features from the down sampling block is passes through an intermediate up sampling and down sampling blocks before provided as an input to the final up sampling block.

The approach in [16] is one of the early COVID-19 classification model developed using SqueezeNet to differentiate COVID-19 affected and non affected CT-scan images. The SqueezeNet consists of blocks called as "Fire Module" which is a simple bypass connection. All the features from parallel fire modules are concatenated using global average pooling layer. Dual-branch Combination Network (DCN) was developed in [17] to classify the COVID-19 and other diseases infected patients. A lesion attention module was developed to isolate the infected region from the CT-scan images.

COVID-19 pneumonia lesion segmentation network (COPLE-Net) [18] was developed for noise robust lesion region segmentation. It also uses a noise robust DICE loss for segmentation, which helps to overcome the problem of limited dataset images available for training the model. Inf-Net [19] was developed to segment the COVID-19 lung infection automatically from the chest CT-scan images. Inf-Net consist of parallel partial decoder, global map, implicit and explicit edge attention model, and semi-supervised segmentation framework for segmenting the lung infected region of CT-scan images.

A multi-tasking model [20] was proposed for classification and segmentation of the CT-scan images. The automatic classification as well as segmentation tool identify lung region affected by COVID-19 from the CT-scan images. Their proposed work uses a common encoder to represent the disentangled feature which support segmenting and classifying the lesion regions. In [21] automatic segmentation for COVID-19 was implemented using CT-scan images. Here the large scene small object issue of medical images is resolved by using the symmetric property of lungs and tissues, which yield higher accuracy of segmenting the COVID-19 affected regions.

The Multi-Scale Discriminative Network (MSD-Net) [22] is designed to perform multi-class segmentation of the CT-scan images. MSD-Net uses a Pyramid Convolutional Block (PCB) which has varying size of kernels. It also has a Channel Attention Block (CAB) used to focus on the area to be segmented. A weakly supervised deep learning [23] framework is developed to localize the lesion region of lung CT-scan images using pre-trained U-Net model. The segmented images is fed as an input to 3D deep neural network DeepCovNet for classification. The approach proposed in [24] uses Residual block with U-Net for segmentation of CTscan images. Here the ground truth labels for multiple class in segmentation is obtained from radiologists.

A deep learning based model is proposed for classification of CT-scan images in [25] using GoogleNet Incep-tionV3 by applying transfer learning. The approach pro-posed in [8] segment the region of infection using 3D-CNN model. The segmented region is given as input to the pretrained ResNet-18 model for feature extraction which is used for classification. Here the pre-trained CNN models are used only for classification, but we propose a model for segmentation of lesion region from the CT scan images.

Another approach in [26] uses UNet++ architecture proposed in [27] for medical image segmentation. The UNet++ architecture is similar to the U-Net [13] , but here the encoder and decoder are connected through nested, dense skip pathways. The UNet++ is proved to be efficient for various medical image segmentation tasks. Another AI based system is proposed for diagnosis of COVID-19 using UNet++ [28] . Here the segmented image along with help of medical practitioners is used to diagnosis the CT-scan of COVID-19 patients. The interpretation of the results by medical purpose does not provide an automated system for diagnosis of COVID-19. We propose a novel architecture that uses fractal blocks in U-Net architecture. This results in better segmentation of the lesion region compared to basic U-Net based models.

The Deep COVID proposed in [29] uses various deep learning model by applying transfer learning to detect COVID-19 from X-Ray images. They have trained various deep learning models like ResNet (18 and 50), SqueezeNet and DenseNet-121. The approach in [30] and [31] uses pretrained Xception [32] model for classification of COVID-19 chest X-Ray images by applying transfer learning. Another approach proposed in [33] uses the various pre-trained CNN model for feature extraction. A correlation based feature selection is applied on the extracted features and a Bayesnet classifier is developed for classification. The proposed approach uses FractalCovNet pre-trained on chest CT-scan images, this provides better results as the model is pre-trained on medical images rather than use of ImageNet dataset.

Computer Aided Diagnosis (CAD) [34] was developed to detect the infection occurred by coronavirus using pretrained VGG-16 model. The diaphragm regions are removed from the X-Ray image. Then two different pre-processing methods like histogram equalization and bilateral low pass filter are used to generate a pseudo color image. Here removing diaphragm region improves the performance of the model. The approach in [35] proposed a Bayesian-CNN for classification of X-Ray images with human intervention. This model is dependent on human experts for accurate recognition of COVID-19.

In [36] a sequential CNN model is proposed for detecting COVID-19 using chest X-ray images. The proposed model performs better than the other pre-trained CNN models for chest X-Ray classification. In an another approach COVID-GAN [37] is designed for generating synthetic chest X-ray (CXR) images. Initially, they generated CXR image from Auxiliary Classifier Generative Adversarial Network (AC-GAN) model. The model produces high accuracy of 95% for classification, but it also includes the generated synthetic images. A custom COVID-Net [38] model is developed for classification of chest X-Ray images of COVID-19 patients.

The model is first pre-trained using ImageNet dataset before fine tuning it for COVID-19 chest X-Ray classification.

An Artificial Intelligence (AI) based model is proposed in [39] for diagnosis of COVID-19 patients. Here the attention region of the deep learning model is used to visualise the effective representation of CT-scan images obtained from COVID-19 patients. The AI system acts as an integrated system with human expertise, which again requires time from the medical practitioners. Another approach in [40] proposes Fractal model for feature extraction from CTscan and X-Ray images for segmentation and classification. The fractal model extracts the features and uses Eigen analysis for feature selection and dimensionality reduction. These features are then provided to the CNN model for segmentation and classification.

Though there are many approaches proposed for detection of COVID-19 from the chest X-Ray and CT-scan images. Many approaches use CNN models pre-trained using ImageNet dataset for detection of COVID-19. The ImageNet dataset comprises of RGB images that belongs to wide variety of images. The fine tuning of these pre-trained model with larger number of parameters using the CT or X-Ray images reduces the performance of the model. This is due to the limited number of CT-scan and X-Ray images available from COVID-19 patients. Thus, we propose a custom Frac-talCovNet architecture for classification and segmentation. We have designed the FractalCovNet using U-Net along with the fractal blocks in the architecture for classification of chest X-Ray images and segmentation of lesion in the CT-scan images.

In addition to the proposed FractalCovNet model we have implemented the following models for segmentation of CT-scan images. The models are trained using transfer learning and the results are compared with the proposed FractalCovNet model for segmentation of CT-scan images.

This is a simple custom convolutional network built for the segmentation of the CT-scan images obtained from COVID-19 patients. The model is built using conv blocks and pooling layer to extract the features. At the end of each stage, a lambda layer is added with the linear activation. The linear activated layer is used to transform the result for pixel wise classification of the images. This model has disadvantages due to the bias generated while training the models.

The Segnet model consists of an encoder and decoder. The encoder consists of a set of convolution blocks with batch normalization layer used for feature extraction. The continuous blocks of convolution and batch normalisation extracts the normalized feature map. The decoder consists a set of convolution blocks and batch normalization in a decreasing order. The resultant feature set is send to output layer to accurately detect the spots of infection. It compares the pixel wise values to detect the lesions on the CT-scan images.

U-Net [13] is a Fully Convolutional Network (FCN) architecture proposed for segmentation of medical images. The U-Net model uses symmetric architecture at the encoder and decoder. The features from the encoder are concatenated to the decoder using skip connections instead of a sum. It consist of three parts: the contracting/down sampling path, Bottleneck layer, the expanding/up sampling path. The contracting path aims to extract the context from the input image to perform segmentation.

The ResUNet [24] model is constructed using residual blocks [11] at the U-Net architecture. The residual blocks helps in easier and fast computation of features by skipping certain convolution blocks. The ResUNet consists of identity blocks and convolution blocks. The identity block in the ResUNet helps to maintain the image dimension and helps in skipping part of the ResUNet architecture.

The DenseUNet architecture proposed in [41] is divided into two sections: down sampling and up sampling similar to U-Net. But at each step of up sampling and down sampling a dense block is used to process the input features. Then the resultant feature map is concatenated and given as input to the next stage after pooling. The dense block at each step further increases the depth of the model which helps in better segmentation of the input images.

The following pre-trained models are trained for classification of chest X-Ray images using transfer learning. We have compared the results obtained by the proposed Fractal-CovNet architecture with the results obtained by the following pre-trained CNN models.

The VGG [10] is a pre-trained Convolutional Neural Network developed by Visual Geometry Group for solving the ImageNet challenge. The model is trained on the ImageNet challenge (ILSVRC) for classifying the images that belongs to 1000 class. Here two architectures were proposed: VGG-16 which consists of 16 layers and VGG-19 which consists of 19 layers. Both the model consists of a stack of convolution and pooling layers. It was shown that the VGG-16 model provides better classification accuracy. Hence in our work we use the VGG-16 model by extracting features from the final fully connected layers to obtain a feature vector of size 4096.

Increasing the number of layers in the deep learning models have increased the performance of the models designed for computer vision task. But, as we increase the number of layers, many difficulties arise while training the model like overfitting, vanishing and exploding gradient problem, etc. To overcome these problems Residual Networks was proposed to solve the ImageNet challenge in [11] . Different ResNet architectures were proposed using 18, 34, 50, 101 and 152 layers. The ResNet-152 model provided best classification results for the ImageNet challenge. In this work, we have chosen the ResNet-50 model due to lesser amount of data available for training.

The InceptionResnetV2 [12] proposes a deep learning architecture which uses a Inception module. The inception module has a number of parallel convolutions, the output obtained from the convolutions are concatenated and given as input to the next convolution module. There is a filter expansion layer after every Inception block in the model which scales up the dimensionality of the filter bank. The Incep-tionResnetV2 model consists of both the Inception module and the residual blocks.

Xception [32] model is a deep convolutional neural network architecture that uses depth-wise separable convolutions followed by point-wise convolutions. The Xception (Extreme Inception) is similar to the Inception module but the Inception modules are replaced by the depth-wise separable convolutions. Every separable convolution layer in the architecture is followed by batch normalization layer.

We propose a novel FractalCovNet architecture for segmentation of CT-scan images and classification of X-Ray images. It enables automatic detection of the COVID-19 patients which will help in early treatment of the COVID-19 patients. The FractalCovNet consists of U-Net architecture with fractal blocks. The architecture of the proposed Frac-talCovNet is shown in Figure 2 . The architecture shows the various expanding and contracting layers of the model. The FractalCovNet is first trained for segmentation of CT-scan images. This model is then fine tuned for classification of X-Ray images using transfer learning.

Fractal network was first introduced in [9] for image classification task. It was shown that the very deep neural networks perform well in classification task, irrespective of the usage of residual block. Hence we choose a variant of the basic fractal network for our segmentation task. U-Net [13] was proposed for segmentation of medical images which was found to be successful in various medical image segmentation tasks. It is also used in various image reconstruction tasks. In this work, we propose a variant of U-Net model with fractal blocks for segmentation of lesion regions from CT-scan images of the COVID-19 patients. A U-Net usually has two path of CNN layers for feature extraction called as expanding path and contracting path. The contracting path down samples the input image into smaller dimensions. While the expanding path up samples the output from the previous layer. The U-Net uses feature concatenation from the contracting path to the expanding path at each level of the model which helps in reconstruction of the images. The contracting path reduces the feature dimension size till the bottleneck layer of the model. The features of the bottleneck layer are used by the expanding path to construct the segmented output images. As the proposed FractalCovNet has two paths for expansion and another two paths for contraction, the architecture has the double U-shape as shown in Figure 2 . The features from the first contracting path flows to the final expanding path through the fractal block.

Let be the input image for the FractalCovNet model. The input image is passed through a set of convolution layer 1 to 5 in the first contracting path of the model as shown in Figure 2 . Then the output is passed through the bottleneck layer 6. The output of the bottleneck layer is passed through the expanding path from 7 to 9. Again the output of the first expanding path is passed through the second bottleneck layer 1. The output of the bottleneck layer 1 is passed through the contracting path 2 to 4. 5 is the third bottleneck layer of the model. The layers from 6 to 9 forms the final expanding layer of the FractalCov-Net architecture. The final expanding path generates the segmented output image for the given input image . Features from the first contracting layers is passed through two set of convolutions before reaching the final expanding path.

The expanding paths enables precise localization combined with extraction of contextual information. The U-Net model combines the location information from the down sampling path with the contextual information obtained from the up sampling path. The localisation and context in-formation is obtained from the contracting and expanding path of the model. In this work, each layer in the first contracting path of the model is concatenated with features of the expanding path. These features are processed by a convolutional block and then concatenated with the features obtained from an another down sampling path of the second U-Net. For example we can see that the features from 5 is concatenated from up sampling features from 6. These features are processed by convolution block at 7 and provided as input to 8 and also 4. The features at 4 again undergoes the same process and given as input to 6. This acts like a fractal block, which processes the features at various levels of convolution, thus enables the model to accurately determine the segmentation regions.

After the final convolution and pooling layer we have applied sigmoid activation function to classify the pixel values and to detect the localization of lesion in the image pixel. The output array with dark region denotes 0's and white region represent the lesion region denoted by 1's. The model is trained using cross entropy error calculated for each pixel in the segmented output image. The proposed FractalCov-Net model yields lower MAE and higher F-measure value compared to the other deep learning models proposed in the literature.

The FractalCovNet architecture proposed in section 3.1 is adapted for classification of chest X-Ray images of COVID-19 patients. We apply transfer learning on the Frac-talCovNet architecture. Figure 3 shows the architecture for the classification of the chest X-Ray images using the pretrained FractalCovNet model used for segmentation. The input chest X-Ray image is passed through the FractalCovNet and an another set of convolution and pooling layers for extracting the features. Let be the input chest X-Ray images 

where ⋅ is the concatenation of the two vectors. We use categorical cross entropy loss function for training the model. The categorical cross-entropy loss function is defined as

where is the number of class and is the ground truth label and̂ is the predicted probability. As the FractalCov-Net is trained using slices of CT-scan images, it is not suitable for extracting features of X-Ray images. Hence we use transfer learning on the FractalCovNet for fine tuning the model for X-Ray images. The layers of the FractalCovNet is trainable during the training of classification model. The final fully connected layer produces the final label whether the input chest X-Ray image is diagnosed with COVID-19 or not.

The CT-scan image requires manual annotation of the slices for the CT-scan by radiologist for creating ground truth mask. There are only limited number of datasets available publicly for use. In this work, we use CT-scan image dataset available in [42] . It consists of initially 100 slices of CTscan images with masks. Later, another 373 CT-scan images was added with masks. Thus we have totally 473 CT-scan images with ground truth mask for lesion region. As we were interested only in segmentation of lesion region we chose only the CT-scan images available with ground truth masks.

Cohen et al. proposed a collection of X-Ray and CTscan images [43] on 25th march 2020 which comprised of 100 chest X-Ray images of COVID-19 patients and 22 chest X-Ray images of other disease. Later the number of images in the dataset was improved in [44] which consists of totally 422 images which comprises of chest X-ray / CTscan images of patients with different diseases like Acute Respiratory Distress Syndrome (ARDS), COVID-19, Middle East Respiratory Syndrome (MERS), pneumonia, Severe Acute Respiratory Syndrome (SARS) and several diseases related to lungs. Where the number of images with COVID-19 is maximum of 340 images. In addition to the above data we have used normal chest X-ray images from Kaggle repository [45] called "Chest X-Ray Images (Pneumonia)". It consists of totally 1583 normal chest X-Ray images and 4273 pneumonia images. We combined this data with Cohen dataset [44] for classification of COVID-19 from others.

This dataset is obtained from the Kaggle website [46] . It consists of totally 5914 normal/pneumonia chest X-Ray images and it also consists of 164 chest X-Ray images from COVID-19 patients (6078 images in total) obtained from [47] .

In this work, we use the accuracy, precision, recall, and F-measure as the evaluation metrics to evaluate the performance of classification of chest X-Ray images. We have used F-measure / Dice coefficient, Precision, Recall and Mean Absolute Error (MAE) for evaluating the segmentation results of CT-scan images.

Accuracy measures the number of instances correctly predicted over the total number of instances as in equation 3.

Precision calculates the ratio of number of sample correctly predicted as positive to the total number of samples predicted as positive as in equation 4.

Recall calculates the ratio of number of sample correctly predicted as positive to the total number of samples available in that particular class as in equation 5.

F-measure is the weighted average of both the Precision and Recall as om equation 6.

Where TP, FP, TN and FN represent the number of True Positive, False Positive, True Negative and False Negative, respectively. Given a test dataset, TP is the proportion of positive (COVID-19) that are correctly labeled as COVID-19 by the model; FP is the proportion of negative (OTHERS) that are mislabeled as positive ; TN is the proportion of negative (OTHERS) that are correctly labeled as normal and FN is the proportion of positive (COVID-19) that are mislabeled as negative (normal) by the model.

The Mean Absolute Error measures the correctness of the proposed FractalCovNet for segmentation by comparing the ground truth masks with the segmented output images. The MAE is calculated as

where ( , ) is the value of the segmented image at position , . ( , ) is the value of the ground truth mask at position , . Here and are the dimension of the images.

We have implemented the proposed FractalCovNet model for classification and segmentation using Keras with Tensorflow backend. All experiments were performed on Google Colaboratory and Kaggle Kernels. We have used random initialization weights for all the models. The model is trained using Adam optimizer with a learning rate of 1e-4. The batch size and number of epochs were experimentally set to 16 and 100 respectively for all experiments.

For classification the ResNet, DenseNet, InceptionRes-NetV3, and Xception models are used with transfer learning. These models were pre-trained with ImageNet dataset. The optimizer used for training the model is Adam optimizer with a learning rate of 1e-4. The batch size and number of epochs were experimentally set to 8 and 50 for all experiments. The dataset used was randomly split into two independent datasets with 60%, 20% and 20% for training, testing, and validation respectively.

In this section we discuss the results obtained for classification of X-Ray images and segmentation of lesion region from the CT-scan images. We have analysed the results obtained for classification and segmentation by plotting confusion matrix and ROC curve. Table 1 shows the accuracy, precision, recall and Fmeasure obtained for the proposed FractalCovNet and the other pre-trained CNN models for classification of chest X-Ray images from Dataset 1. Here ACC represents accuracy, PRE represents precision, REC represents Recall and F1 represents F-measure. Table 2 shows the results for chest X-Ray Dataset 2. Table 3 shows the result obtained by training Figure 4 shows the confusion matrix for the models trained using the proposed FractalCovNet and the pretrained CNN models like VGG-16, ResNet-50, Inception-ResnetV2, Xception and DenseNet-201 using only the test set of the images. Figure 4a and 4b shows the confusion matrix of InceptionResnetV2 and Xception, there is a larger number of false positive but no false negative. Figure 4c , 4d and 4e shows the confusion matrices of DenseNet-201, VGG-16 and ResNet-50 respectively. It is seen that there are few number of false negatives compared to the false positives. Figure 4f shows the confusion matrix obtained for the proposed FractalCovNet. It is seen that there is only one false negative compared to the false positive, which helps in containment of the COVID-19 spread. Figure 5a and 5b shows the Precision-Recall curve and ROC curve plotted for chest X-Ray images from Dataset 1 for FractalCovNet and other pre-trained CNN models like ResNet, InceptionResnetV2, VGG, Xception and DenseNet. It is seen that the precision of the proposed FractalCovNet is better than the other pre-trained CNN models. Figure 5c and 5d shows the plot of training accuracy and loss for Dataset 1.

The plot shows that the training loss and accuracy of Frac-talCovNet is better compared to ResNet model. Figure 6a shows the precision curve plot for the proposed FractalCovNet compared to the other pre-trained models like ResNet, InceptionResnetV2, VGG, Xception and DenseNet using chest X-Ray images from Dataset 2. It is seen that the proposed FractalCovNet provides better precision and recall compared to the other pre-trained CNN models. Figure  6b shows plot of ROC curves for chest X-Ray images from Dataset 2. Figure 6c and 6d shows the plot of accuracy and loss for chest X-Ray images from Dataset 2. It is seen that the proposed FractalCovNet model converges faster than the other pre-trained models used for classification. Table 4 shows the results obtained for segmentation using various segmentation models. It shows the MAE, precision and recall scores obtained for the segmentation. It Figure 7a and 7b shows the accuracy and the loss values generated while training the segmentation models for segmentation of lesion region from CT-scan images. It is seen that the proposed FractalCovNet model converges faster compared to the other models. Figure 7c and 7d shows the F-Measure and Precision-Recall curve plotted for segmentation using the various models. The F-Measure is calculated for segmentation by comparing the segmented results generated by models with the ground truth images. The segmented images shows that the proposed FractalCovNet model performs better segmentation than the other models in the literature. The FCN is very poor at segmentation whereas the U-Net, DenseUNet, and ResNetUNet provides better segmentation compared to the FCN model.

The segmentation results for the sample images are shown in figure 8 . The figures shows 6 sample CT-scan images with various levels of infection. It is seen that the proposed FractalCovNet provides better segmentation results and the segmentation is more similar to the ground truth mask region. The ResNetUNet model provides better results compared to the basic FCN and SegNet models. The ResNetUNet model provides better segmented image but not as good compared to the segmented image provided by the FractalCovNet model. Table 5 shows the results of the various approaches proposed for COVID-19 CT-scan image segmentation and classification.

Few approaches has been propose to use the pre-trained CNN models for classification of CT-scan images. The approach in [25] uses pre-trained GoogLeNet and the approach in [48] uses pre-trained ResNet50 for classification of CTscan images. In this work, we have proposed a model for only segmenting the lesion region from CT-scan images. Another AI based approach is proposed in [39] uses CNN for diagnosis of COVID-19 patients, it requires intervention by medical experts. But in our approach we propose an automated FractalCovNet architecture for segmentation of lesion region from CT-scan images of COVID-19 patients.

The approach proposed in [26] and [28] uses UNet++ architecture for segmentation and diagnosis of CT-scan images of COVID-19 patients. The UNet++ [26] uses only around 100 images for training and testing, it achieves accuracy of 96% per patient. The approach in [28] uses UNet+CNN with AI assisted model for diagnosis of COVID-19 patients using CT-scan images. In the proposed approach, we perform segmentation of lesion region which produces a F-measure The COVID-CT-Seg segmentation model in [21] uses data augmentation and performs 3D segmentation to achieve dice coefficient of 78.3%. But in our proposed approach we have used 2D-segmentation without any augmentation. Thus the proposed approach performs well compared to the approach used in [21] . The Inf-Net [19] uses reverse attention model with parallel partial decoder. It uses semisupervised algorithm for segmentation of the lesion to produce MAE error of 0.64. Without the semi-supervised training algorithm it produces an MAE of only 0.082, which is less than our proposed approach. The weakly supervised approach DeCovNet [23] achieves hit rate of 65.7% for segmentation. But the proposed approach provides F-measure of 74.5% which is better than the DeCovNet model.

The approach in [8] uses segmentation and provides segmentation region for classification, it produces an accuracy of 86.7% for the classification model. The approach in [40] produces segmentation accuracy of 70% for CT-scan images.

Our approach produces precision socre of 80.4% for pixelwise segmentation thus it will be able to diagnose COVID-19 with greater accuracy than the other existing approaches. The multi-tasking approach in MSD [22] produces dice coefficient of 74.22% but the proposed approach provides multitasking and produces dice coefficient of 74.5% and precision of 80.4%. Thus the proposed approach provides better segmentation results for segmentation of CT-scan images compared to the other state-of-the-art approaches. Table 6 shows the results for the various approaches proposed for COVID-19 X-Ray image classification. There are many approaches proposed for classification chest X-Ray of COVID-19 patients images using basic machine learning and deep learning approaches.

The approaches in [49] , [52] , [35] and [55] uses pretrained ResNet model for classification of chest X-Ray images of COVID-19 and non COVID-19 patients. The approach in [52] uses transfer learning of ResNet50 model and approach in [55] uses ResNet-101 for classification of chest X-Ray images. The approach in [35] uses Bayesian model with ResNet50 model for classification. The approach in [33] uses several pre-trained CNN models and uses feature selection for choosing the features and performs classification. The approach in [31] and [30] uses Xception architecture and for classification of chest X-Ray images. The approach in [34] and [53] uses VGG-16 with transfer learning for classification of chest X-Ray images. It is seen that the use of ResNet pre-trained model provides better accuracy when compared to the other approaches. But the proposed approach applies transfer learning on the Fractal-CovNet architecture for classification of the X-Ray images. Table 6 shows that the proposed approach provides better results compared to the models that uses pre-trained CNN using transfer learning for classification of X-Ray images. The OptiDCNN [56] uses Biogeography-Based optimisation for training the classification model, it produces an accuracy of 99.16%. But our proposed model uses Adam optimizer for training the model and provides an accuracy of 99.8% which proves the effectiveness of the proposed FractalCovNet architecture.

The lightweight architecture proposed in [50] uses spatial pyramid pooling along with convolution layers for developing a classification model. It provides an accuracy of 94.6% which is lower compared to our proposed approach. Another lightweight module [51] uses capsule module for classification of chest X-Ray images and produces an accuracy of 95.7%. A custom CNN model is developed for classification of X-Ray images in [36] , it produces an accuracy of 97.4%. The light weight models trade-off accuracy for the model complexity. But we need more accurate model for diagnosis of the COVID-19 disease for the containment of the disease. The proposed model provides higher accuracy than the other state-of-the-art approaches. Hence the proposed Table 6 Comparison of various approaches to COVID-19 chest X-Ray image classification approach will help in timely diagnosis of the disease.

The Fractal based network [40] produces an accuracy of 93.2% for classification of X-Ray images into COVID or non-COVID patients. But the proposed approach using Fractal blocks provides an accuracy of 99.8% for classification of chest X-Ray images. The approach in [57] uses Darknet-53 model for classification of chest X-Ray images. But it also uses the expertise of medical representative for the classification model, which is very difficult due to the pandemic. The proposed approach provides better results without intervention of human experts. Thus the proposed approach provides better results than the other state-of-theart approaches for classification of chest X-Ray images.

In this paper, we have proposed a novel FractalCovNet for classification and segmentation of chest X-Ray images and CT-scan images respectively. The classification of chest X-Ray images will help in faster diagnosis of the disease which in turn helps in early diagnosis and containment of the COVID-19 disease in the patients. The segmentation of the chest CT-scan images helps in identifying the percentage of infection which will help to provide timely medical support for the patients. The FractalCovNet is model adapted from the U-Net architecture for segmentation. The U-Net based segmentation have provided promising results of segmentation of CT-scan images of COVID-19 patients. The FractalCovNet proposed in this work is used for both classification and segmentation. The proposed approach provides an accuracy of 99% for classification which is better than the other state-of-the-art approaches. The segmentation model provides MAE of 0.064 and F-measure of 74.5% which is better than the other existing approaches proposed for segmentation of CT-scan images.

COVID-19 or others Accuracy-97

COVID-19, pneumonia, bacterial and other virus Accuracy-98.93% COVID-19:250 pneumonia

FractalCovNet COVID-19 or others Accuracy-99

A new coronavirus associated with human respiratory disease in China

SARS-CoV-2: an Emerging Coronavirus that Causes a Global Threat

The COVID-19 vaccine development landscape

SARS-CoV-2 and SARS-CoV Spike-RBD Structure and Receptor Binding Comparison and Potential Implications on Neutralizing Antibody and Vaccine Development

COVID-19 Variants Database: A repository for Human SARS-CoV-2 Polymorphism Data

Coronavirus biology and replication: implications for SARS-CoV-2

SARS-CoV-2 genomic variations associated with mortality rate of COVID-19

A Deep Learning System to Screen Novel Coronavirus Disease

FractalNet: Ultra-Deep Neural Networks without Residuals

Very Deep Convolutional Networks for Large-Scale Image Recognition

Deep Residual Learning for Image Recognition

Going deeper with convolutions

Convolutional Networks for Biomedical Image Segmentation

Adaptive Context Selection for Polyp Segmentation

PraNet: Parallel Reverse Attention Network for Polyp Segmentation

A light CNN for detecting COVID-19 from CT scans of the chest

Dual-branch combination network (DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images

A Noise-Robust Framework for Automatic Segmentation

Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images

Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation

Accurate and Machine-Agnostic Segmentation and Quantification Method for CT-Based COVID-19 Diagnosis

MSD-Net: Multi-Scale Discriminative Network for COVID-19 Lung Infection Segmentation on CT

A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization From Chest CT

Residual Attention U-Net for Automated Multi-Class Segmentation of COVID-19 Chest CT Images

A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19)

Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: a prospective study

UNet++: A Nested U-Net Architecture for Medical Image Segmentation

AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical AI system

Deep-COVID: Predicting COVID-19 from chest X-Ray images using deep transfer learning

Automated Deep Transfer Learning-Based Approach for Detection of COVID-19 Infection in Chest X-Rays

CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest X-Ray images

Xception: Deep Learning with Depthwise Separable Convolutions

Computer-aided detection of COVID-19 from X-Ray images using multi-CNN and Bayesnet classifier

Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-Ray images with preprocessing algorithms

Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection

Automatic Detection of COVID-19 from Chest X-Ray Images with Convolutional Neural Networks

CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved COVID-19 Detection

COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images

Development and evaluation of an artificial intelligence system for COVID-19 diagnosis

Diagnosis and detection of infected tissue of COVID-19 patients based on lung X-Ray image using convolutional neural network approaches

Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network. Quantitative imaging in medicine and surgery

COVID-19 CT Segmentation Dataset

COVID-19 Image Data Collection

COVID-19 Image Data Collection: Prospective Predictions Are the Future

Chest X-Ray Images (Pneumonia)

COVID19 classifier dataset

Labeled Optical Coherence Tomography(OCT) and Chest X-Ray Images for Classification

Deep learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) with CT images

ral Pneumonia Screening on Chest X-Ray Images Using Confidence-Aware Anomaly Detection. 2020

A Lightweight Deep Learning Model for COVID-19 Detection

COVID-CAPS: A capsule network-based framework for identification of COVID-19 cases from X-Ray images

Automatic Detection of Coronavirus Disease (COVID-19) Using X-Ray Images and Deep Convolutional Neural Networks

Application of deep learning for fast detection of COVID-19 in X-Rays using nCOVnet

Automated detection of COVID-19 cases using deep neural networks with X-Ray images

A deep learning approach to detect COVID-19 coronavirus with X-Ray images

Evolving Deep Learning Convolutional Neural Networks for Early COVID-19 Detection in Chest X-Ray Images

A deep learning-based COVID-19 automatic diagnostic framework using chest X-Ray images

Task Results Dataset CAAD [49] COVID- 19