key: cord-0931671-mukoor9t
authors: Shyni, Dr. H Mary; E, Dr. Chitra
title: A COMPARATIVE STUDY OF X-RAY AND CT IMAGES IN COVID-19 DETECTION USING IMAGE PROCESSING AND DEEP LEARNING TECHNIQUES
date: 2022-03-07
journal: Comput Methods Programs Biomed Update
DOI: 10.1016/j.cmpbup.2022.100054
sha: b5119250579f49cfeea6d5191560dc46e51f7d5d
doc_id: 931671
cord_uid: mukoor9t

The deadly coronavirus has not just devastated the lives of millions but has put the entire healthcare system under tremendous pressure. Early diagnosis of COVID-19 plays a significant role in isolating the positive cases and preventing the further spread of the disease. The medical images along with deep learning models provided faster and more accurate results in the detection of COVID-19. This article extensively reviews the recent deep learning techniques for COVID-19 diagnosis. The research articles discussed reveal that Convolutional Neural Network (CNN) is the most popular deep learning algorithm in detecting COVID-19 from medical images. An overview of the necessity of pre-processing the medical images, transfer learning and data augmentation techniques to deal with data scarcity problems, use of pre-trained models to save time and the role of medical images in the automatic detection of COVID-19 are summarized. This article also provides a sensible outlook for the young researchers to develop highly effective CNN models coupled with medical images in the early detection of the disease.

The seventh coronavirus Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) that infected humans emerged in Wuhan, Hubei province, China in early December 2019 and the outbreak spread rapidly across the world [1] . Because of its effect on health, psychological well-being and the economy, Coronavirus 2019 (COVID-19) has been affirmed as a worldwide public health catastrophe. Sore throat, dry cough, tiredness, fever, difficulty in breathing, loss of smell and disappearance of taste are the major symptoms of COVID-19 [2] . The incubation period may vary between 2 to 14 days from person to person and the disease is likely to be transmitted through contact and breathing droplets [3] . The hasty spread of the mortal virus has plunked down the entire healthcare system under immense pressure.

RT-PCR (Reverse Transcription -Polymerase Chain Reaction) is the initial laboratory testing procedure for COVID-19 diagnosis. Coronavirus contains only RNA (Ribonucleic acid) which needs to be converted to DNA (Deoxyribonucleic acid) for amplification which is done by RT-PCR for virus detection. Apart from its advantages, it is time-consuming which may lead to further spread of the disease from the infected person and the deep nasal swabs are troublesome. Early diagnosis of the disease plays a pivotal role in isolating the positive cases in advance and preventing community spread [4] . Since the lung region is the primarily infected area by the virus, medical imaging modalities like X-ray and Computed Tomography (CT) are generally considered in examining the severity of the infection [5, 6] . X-ray imaging techniques are often employed in the diagnosis of COVID-19 due to their wide availability, quick processing time and low cost. But CT imaging techniques are preferred as it carries detailed information of the infected region [7] . However, even for experienced radiologists, predicting the infection from medical images has become a challenging task because of the lack of advanced knowledge about the disease. The medical images along with the deep learning algorithms have become a valuable choice that yields faster and more accurate results in the diagnosis of COVID-19 [8, 9] . Pattern recognition is the primary function of the Convolutional Neural Network (CNN), which is a deep neural network and hence utilized to detect COVID-19 from medical images.

The objective of this article is to compile the workflow of the recent research works related to the automated detection of COVID-19 from medical images using deep learning techniques. 85 research articles have been chosen from journals with high impact-factor to frame this paper. The motivation of this article is to provide a comparison of the remarkable characteristics of the recent deep learning methods using X-ray and CT imaging modalities.

The comparison made for models with and without data augmentation shows that the models performed better when the datasets are augmented. It also provides a sensible outlook for the young researchers to develop highly effective CNN models coupled with medical images in the early detection of the disease.

The rest of this paper is structured as: Section 2 describes the pre-processing techniques to generate high-quality medical images. Section 3 presents the dataset split and the strategies to overcome the data scarcity problem. Section 4 discusses the often used pretrained models for COVID-19 detection. Section 5 illustrates the architecture and functions of each layer in the Convolutional Neural Network. Section 6 presents the role of medical images in COVID-19 early detection. Section 7 discusses the binary and multiclass classification of medical images. Finally, Section 8 concludes the article and future research suggestions are provided for the young researchers. 6 

Medical Imaging is the process of creating images of the interior parts for diagnosing various abnormalities in a human body. X-ray and CT imaging are the two often utilized medical imaging modalities for detecting COVID-19. But due to the low intensity and contrast in images, the borders and edges of the images are not clear which may lead to a false diagnosis of the disease. So there is a strong need to pre-process the medical images to extract the essential information and remove the irrelevant data to increase the accuracy of the model [10] . Medical image processing deals with the application of an algorithm on a digitized image to enhance the image quality of the raw medical data for further analysis.

Various pre-processing techniques are used in medical imaging applications to improve the visual information of the input image. Image resizing, image segmentation and image enhancement are the usually performed pre-processing techniques in X-rays and CT scans in COVID-19 diagnosis.

It is necessary to standardize the dataset as it is acquired from multiple centers and scanners which may vary in size. All the images in the dataset are generalized to a fixed dimension using the image resizing technique for better classification performance of the CNN model [11] .

Image Segmentation is an essential image processing technique to increase the prediction quality and reliability of the model. Segmentation focuses on the Region of Interest (ROI) and for COVID-19 detection the ROI is the lung region. It reduces the computational complexity by separating the lung region from other background information in the medical images [12] . Figure (1) shows samples for image segmentation. 

Image Enhancement is essential for enhancing the visual perception quality of the medical images for disease diagnosis. Histogram equalization is an enhancement technique that distributes the intensity level over the pixels of the image. In some cases, the information carried by the white pixels is washed out due to the high contrast in the white region [13] .

Adaptive Histogram Equalization (AHE) distributes the intensity values only on the small regions of the image. It may result in over-amplification of noise within the homogeneous regions [14] . CLAHE (Contrast Limited Adaptive Histogram Equalization) limits the over enhancement of noise caused by AHE. It enhances the image by fixing a maximum contrast limit beyond which the contrast cannot be improved that prevents over-amplification of noise [15] . Alaa S. Al-Waisy et.al have used CLAHE to enhance the image contrast and improve the visibility of the borders of the chest X-ray images [16] . Samples for image enhancement are shown in figure (2). 8 

Deep learning requires a large amount of data for the model to be trained efficiently and accurately. The data available in the dataset are split into three sets: i) Training dataset ii) Validation dataset iii) Test dataset. The training dataset is used during the learning process to train the model to perform tasks. The validation dataset is used to evaluate, fine-tune model hyperparameters during the training process and facilitates in optimizing model selection.

The test dataset is used to assess the model once it is completely trained using the training and the validation dataset [17] . ongoing and new pandemic, the available datasets are insufficient and imbalanced to train the model effectively [18] . To deal with the data scarcity problem two strategies are often used: i) Transfer Learning ii) Data Augmentation

Transfer Learning is an approach where a neural network model trained for a particular task is reused for a model on another task. For the new task, only the layers that are very close to the output units are retrained. The pre-trained model has to be trained with a sufficient amount of data because it gains knowledge about feature extraction of the image which is going to be transferred to another model [19] . The main application of transfer learning is the classification of medical images for emerging diseases due to the limited availability of samples [20] . Transfer learning has the benefit that the training time of the model decreases and is computationally less expensive as only a few layers are retrained.

Since the models are already trained, it does not require a vast amount of data. Figure (4) illustrates the concept of transfer learning. Here the knowledge gained by model 1, which is trained with a large amount of data is transferred to model 2 to perform a related task when the amount of data available to train model 2 is limited. 

Data augmentation is another strategy that overcomes the data scarcity problem. It In [23] the CNN used for the detection of COVID-19 was trained by the artificial data generated by COVIDGAN (COVID Generative Adversarial Network) which increased the to 89.08% using traditional augmentation and further increased to 95.66% using smart augmentation [26] . In [27] to perform teeth segmentation and classification, a dataset was created by applying image augmentation technique to the dental radiographs which increased the accuracy of the AlexNet model from 88.31% to 98.88%. Table ( 1) shows the increase in accuracy of the models after data augmentation.

Augmentation is applied only to the training dataset and not to the testing dataset [28] .

The training images are either position augmented or color augmented. Figure 

It is the type of augmentation where the pixel position of the image changes. The most used position augmentation techniques are scaling, flipping, rotation and cropping.

It is the type of augmentation where the color of an image is modified by changing the pixel values of the image. Colour augmentation can be performed by varying the brightness, contrast, saturation or hoe of an image. 14 

Adding noise to an image also helps in incrementing the dataset. Training the neural network with Gaussian noise performs model regularization which reduces overfitting [29] .

Pretrained models are the models that are trained with a huge amount of datasets for a specific task. Due to the rapid spread and limited availability of COVID-19 samples, training the CNN models from scratch is a difficult task. Hence pre-trained models are used in most of the COVID-19 detection architectures which saves time and produces higher accuracy.

Pretrained models often used in COVID-19 diagnosis architectures are briefly discussed:

Visual Geometry Group (VGG) in the year 2015. The depth of the architecture has been increased by increasing the number of convolutional layers and smaller kernels of size 3×3

were also used to improve the performance. VGG 16 and VGG 19 are the two versions of VGG architecture trained using the ImageNet dataset [30] .

Residual Neural Network is a 50-layered network that was trained using the ImageNet dataset developed by Kaiming He et al. Although stacking more convolutional and pooling layers produces better results further addition of these layers above a certain threshold level decreases the performance of the model. To overcome this problem, shortcut connections were added which randomly drops one or more layers without adding extra parameters and computational complexity to the model [31] .

This architecture was proposed by Gao Huang et al to ensure maximum flow of information between each layer in the network. It is an improved version of ResNet where each layer in the network is connected in a feed-forward fashion with every other layer.

DenseNet concatenates the output feature maps of a layer with the incoming feature maps [32] .

In the year 2014, Christian Szegedgy designed Inception Net also known as GoogleNet to deepen and widen the network. Inception modules were introduced which allows the use of multiple kernel sizes at the same level. It is a 22-layered network with limited computational resources [33] . Inception V2 and Inception V3 are improved versions of GoogleNet by adding kernel factorization and batch normalization even with relatively low computational cost with no compromise in quality [34] .

In this architecture proposed by Francois Chollet, the inception modules in the InceptionNet architecture were replaced by depth-wise separable convolutions. The depthwise separable convolution layers are linearly stacked with residual connections for easy modification of the network. Xception Net has almost the same number of parameters as Inception V3 [35] .

This model that also uses depth-wise separable convolutions was developed by Andrew G Howard et al. Two hyperparameters were introduced which allows considering low latency and small-sized models for mobile applications [36] . MobileNet V2 is an improved version of MobileNet where inverted residual and linear bottlenecks were introduced to decrease the number of parameters and memory consumption [37] . 16 

A branch of machine learning that is trained on a large amount of data from which it predicts the output for a given input is called deep learning. Neural networks are a set of algorithms designed to recognize patterns. The Convolutional Neural Network (CNN) is a type of Deep Neural Network (DNN) that is frequently employed in a variety of applications.

CNNs are primarily used for visual document analysis and to solve different pattern recognition tasks [38, 39] . The use of CNN has been rapidly increased in the medical field and has produced successful results in medical image classification [40] . Different CNN models have been proposed by researchers to detect abnormalities from medical images such as detection of lung nodules [41] , prediction of heart disease [42] , classification of dental diseases [43] , detection of skin diseases [44] , prediction of breast cancer [45] and many other diseases. Recently it has been found that CNN plays a crucial role in detecting COVID-19

from medical imaging modalities such as X-ray and Computed Tomography (CT). CNN

consists of multiple layers that are responsible for extracting distinguished features from the given image which are transferred to the classification stage.

CNN architecture is stacked with three primary layers namely i) Convolutional layer ii) Pooling layer and iii) Fully Connected layer. The basic CNN architecture along with the layers is shown in figure (7) . The responsibility of Convolutional and pooling layers are feature extraction and the fully connected layer is for classification purposes. Two more parameters dropout layer and activation function are also defined apart from these layers. 

This is the basic layer responsible for extracting various features from the input image pattern. The convolutional layer has a set of kernels that are nothing but filters. The kernel is a matrix, which is basically smaller than the input data that slides over the input from left to right and top to bottom and performs a dot function with the input data. The resultant is the generation of a feature map. The feature maps of the later layers are built by combining the feature maps of the earlier layers [46] .

The pooling layer is the second layer which follows the convolutional layer. This layer performs downsampling and thereby reduces the number of parameters and computations. Max pooling is the most commonly used pooling operation that produces the maximum element from the feature map.

The last layer in the architecture is the fully connected layer (FC layer). Its dimension is equal to the number of output classes. The input from the preceding stages is fed into the fully connected layer, which then classifies the images.

It may lead to overfitting when all the features are connected to the FC layer. A dropout layer is used before the output layers which randomly discards a few neurons from the neural network which results in the size reduction of the model [47] .

Activation Functions get the output from the previous layer and convert it to a form suitable for the next layer to consider that as its input. They can be used at any part of the network. Without activation functions, the model would not be able to learn the complex patterns from the data. It adds nonlinearity to the network. Rectified Linear Unit (ReLu), softmax, sigmoid and tanh are some of the commonly used activation units and ReLu is most widely used in deep learning models [48, 49] .

X-ray and CT are two common medical imaging modalities that are used to diagnose and analyze the severity of an infection. Each medical imaging modality has its advantages and limitations.

X-ray imaging is the most often utilized medical imaging tool for the diagnosis of COVID-19 due to its extensive availability [50, 51] . It can be processed by simple procedures and thus reduces the imaging time which minimizes the possibility of spreading the virus [52] . It is economical when compared with other medical imaging modalities. It is noninvasive and produces a low radiation dose when compared to a CT scan. Despite its advantages, X-rays are less sensitive which may lead to a false prediction of the disease with early and mild symptoms [53] .

On the other hand, CT scans are highly sensitive and contain detailed information about the affected region and thus provide accurate results. CT imaging plays a major role in the diagnosis of lung abnormalities [54] . It is more reliable which helps in the early diagnosis of COVID-19 [55] . But CT screening limits its usage due to high cost, higher radiation dose and resource constraints. Matias Cam Arellano and Oscar E Ramos [56] used the DenseNet121 pre-trained CNN model whose last layer alone is retrained for detecting COVID-19 from chest radiographs using open databases. As the model is already trained for detecting different lung diseases, the network provided distinctive features with an accuracy of 94.7%. In [57] COVID-19 and pneumonia are detected from chest X-ray images using a three-step process.

The Conditional Generative Adversarial Network (C-GAN) is used at the first step to segment the lung region from the CXR images. In the second step, the feature extraction 32 CT imaging serves as a valuable tool in COVID-19 early detection. It is preferred as it provides a three-dimensional view of the lung which contains detailed information of the affected region. Figure (9 The following research gaps are identified from the related works:

 Existing models are trained with limited data samples.

 Limited number of models are proposed for multiclass classification.

 Limited number of studies have used ultrasound as the imaging modality. 40  No complete real time end-to-end systems are available using deep learning methods. Ioannis D Apostopoulos et al [84] employed MobileNet to detect abnormalities from chest X-ray images in three distinct ways. The MobileNet trained from scratch performed 43 better than the other two ways for seven class and binary classification. This method achieved 99.18% accuracy for binary classification and 87.66% accuracy for multiclass classification.

In [85] an artificial neural network based on capsule networks named convolutional capsnet was proposed to detect COVID-19 from CXR images using a fewer number of layers. Using this method, the accuracy obtained is 97.24% for binary classification and 84.22% for multiclass classification. Table (4) shows the difference in accuracy obtained by the discussed models for binary and multiclass classification. From the table, it can be observed that the models performed better for binary classification than multiclass classification.

There exists a shortage of RT-PCR kits due to the rapid rise in COVID-19 cases. The medical images coupled with deep learning techniques are very helpful to provide faster and more accurate results during the rapid spread of COVID-19. Deep Learning models produce better accuracy when trained with larger datasets. In this article, different image processing techniques have been discussed which enhances the quality of the medical images. The accuracy of the deep learning model can be increased by using high-quality medical images.

To enhance the image and predict the disease accurately there is a strong need to pre-process the medical images. As there are no sufficient datasets publicly available to train the model most of the proposed works have used transfer learning and data augmentation strategies to overcome the data scarcity problem. A comparison has been made for a few models with and without data augmentation whose results show that the models performed better when the datasets are augmented. Most of the COVID-19 diagnosis architectures used pre-trained models because of the rapid spread of the disease to save time. VGG, ResNet 50, DeepNet, Inception V3, Xception and MobileNet are the pre-trained models that are often employed in COVID-19 detection. The CNN architecture which is widely used in image classification tasks is discussed. 44 The role of the two medical imaging techniques X-rays and CT in the detection of COVID-19 are described briefly. Though X-ray imaging is simple, less expensive and widely available, CT imaging is highly sensitive in predicting the severity of the disease. Due to the wide availability of the X-ray image dataset in comparison with the CT image dataset, most of the researchers have utilized chest X-ray images for the detection of COVID-19. A comparison has been made for the state-of-the-art methods which could guide the young researchers to find future direction. Models proposed for binary and multiclass classification are studied and observed that the models produced better accuracy for binary classification than multiclass classification. Accuracy, Specificity, Precision, Recall, F1-score, ROC curve 

The proximal origin of SARS-CoV-2

A review of coronavirus disease-2019 (COVID-19)

The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: estimation and application

The Need of Early Detection of Positive COVID-19 Patients in the Community by Viral Tests (eg RT-PCR Tests) and Antibody Tests (Serological Tests) to Stop the Spread

The Role of Imaging in the Detection and Management of COVID-19: A Review

Realizing an effective COVID-19 diagnosis system based on machine learning and IOT in smart hospital environment

COVID-19 detection in CT images with deep learning: A votingbased scheme and cross-datasets analysis

Weakly supervised deep learning for covid-19 infection detection and classification from ct images

A Two-Dimensional Sparse Matrix Profile DenseNet for COVID-19 Diagnosis Using Chest CT Images

Image Enhancement Based Medical Image Analysis

Efficient Medical Image Segmentation Of COVID-19 Chest CT Images Based on Deep Learning Techniques

Impact of lung segmentation on the diagnosis and explanation of COVID-19 in chest X-ray images

A Combined Approach Using Image Processing and Deep Learning to Detect Pneumonia from Chest X-Ray Image

Adaptive histogram equalization and its variations

Improved Method for Enhancing Dark Images based on CLAHE and Morphological Reconstruction

COVID-DeepNet: hybrid multimodal deep learning system for improving COVID-19 pneumonia detection in chest X-ray images

Convolutional neural networks: an overview and application in radiology

COVID-19 detection from scarce chest x-ray image data using few-shot deep learning approach

A comprehensive investigation of machine learning feature extraction and classification methods for automated diagnosis of COVID-19 based on X-ray images

Deep-covid: Predicting covid-19 from chest x-ray images using deep transfer learning

Application of deep learning techniques for detection of COVID-19 cases using chest X-ray images: A comprehensive study

Deep rank-based average pooling network for COVID-19 recognition

Covidgan: data augmentation using auxiliary classifier gan for improved covid-19 detection

Can AI help in screening viral and COVID-19 pneumonia?

Deep convolutional neural networks and data augmentation for environmental sound classification

Smart augmentation learning an optimal data augmentation strategy

Dataset Augmentation for Machine Learning Applications of Dental Radiography

Covid-resnet: A deep learning framework for screening of covid19 from radiographs

Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack

Very deep convolutional networks for largescale image recognition

Deep residual learning for image recognition

Densely connected convolutional networks

Going deeper with convolutions

Rethinking the inception architecture for computer vision

Xception: Deep learning with depthwise separable convolutions

Mobilenets: Efficient convolutional neural networks for mobile vision applications

Mobilenetv2: Inverted residuals and linear bottlenecks

Best practices for convolutional neural networks applied to visual document analysis

An introduction to convolutional neural networks

Overview of deep learning in medical imaging

Computerized scheme for automated detection of lung nodules in low-dose computed tomography images for lung cancer screening1

Prediction of heart disease using CNN

Classification of dental diseases using CNN and transfer learning

Skin diseases prediction using deep learning framework

Deep learning to improve breast cancer detection on screening mammography

Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks

Deep neural network to detect COVID-19: one architecture for both CT Scans and Chest X-rays

Suitable CNN weight initialization and activation function for Javanese vowels classification

The influence of the activation function in a convolution neural network model of facial expression recognition

High discordance of chest x-ray and CT for detection of pulmonary opacities in ED patients: implications for diagnosing pneumonia

COVID-XNet: A custom deep learning system to diagnose and locate COVID-19 in chest X-ray images

Current limitations to identify COVID-19 using artificial intelligence with chest X-ray imaging

Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images

Computer analysis of computed tomography scans of the lung: a survey

Diagnosis of COVID-19 using CT scan images and deep learning techniques

Deep Learning Model to Identify COVID-19 Cases from Chest Radiographs

A deep learning based approach for automatic detection of COVID-19 cases using chest X-ray images

Automatic detection of COVID-19 from chest X-ray images with convolutional neural networks

A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images

COVID-CheXNet: hybrid deep learning framework for identifying COVID-19 virus in chest X-rays images

Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Deep learning covid-19 features on cxr using limited training data sets

Artificial Intelligence applied to chest X-Ray images for the automatic detection of COVID-19. A thoughtful evaluation approach

Using X-ray images and deep learning for automated detection of coronavirus disease

Advanced Meta-Heuristics, Convolutional Neural Networks, and Feature Selectors for Efficient COVID-19 X-Ray Chest Image Classification

Detecting SARS-CoV-2 From Chest X-Ray Using Artificial Intelligence

Automatic diagnosis of COVID-19 and pneumonia using FBD method

An uncertainty-aware transfer learning-based framework for covid-19 diagnosis

MIDCAN: A multiple input deep convolutional attention network for Covid-19 diagnosis based on chest CT and chest X-ray

CVDNet: A novel deep learning architecture for detection of coronavirus (Covid-19) from chest x-ray images

COVID-AL: The diagnosis of COVID-19 with deep active learning

Covidnet-ct: A tailored deep convolutional neural network design for detection of covid-19 cases from chest ct images

CovTANet: A Hybrid Tri-Level Attention-Based Network for Lesion Segmentation, Diagnosis, and Severity Prediction of COVID-19 Chest CT Scans

Prior-attention residual learning for more discriminative COVID-19 screening in CT images

A weakly-supervised framework for COVID-19 classification and lesion localization from chest CT

Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks

Transfer learning for establishment of recognition of COVID-19 on CT imaging using small-sized training datasets

Deep learning assisted COVID-19 detection using full CTscans

A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19)

A novel approach of CT images feature analysis and prediction to screen for corona virus disease (COVID-19)

Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks

Automated detection of COVID-19 cases using deep neural networks with X-ray images

CovXNet: A multi-dilation convolutional neural network for automatic COVID-19 and other pneumonia detection from chest X-ray images with transferable multi-receptive feature optimization

Extracting possibly representative COVID-19 biomarkers from X-ray images with deep learning approach and image data related to pulmonary diseases

Convolutional capsnet: A novel artificial neural network approach to detect COVID-19 disease from X-ray images using capsule networks