key: cord-1009664-b907i9ek
authors: Chauhan, Tavishee; Palivela, Hemant; Tiwari, Sarveshmani
title: Optimization and Fine-Tuning of DenseNet model for classification of Covid-19 cases in Medical Imaging
date: 2021-06-10
journal: nan
DOI: 10.1016/j.jjimei.2021.100020
sha: 2d6995f81fd463845bdb715ecabf7791a757eeb1
doc_id: 1009664
cord_uid: b907i9ek

It’s been more than a year that the entire world is fighting against Covid-19 pandemic. Starting from the Wuhan city in China, COVID-19 has conquered the entire world with its rapid progression. But seeking the importance towards the human situation, it has become essential to build such an automated model to diagnose COVID-19 within less computational time easily. As the disease has spread, there is not enough data to implement an accurate COVID-19 predicting model. But technology is a boon, which makes it possible. Effective techniques based on medical imaging using artificial intelligence have approached to assist humans in needful time. It has become very essential to detect COVID-19 in humans at an early stage to prevent it from becoming more infectious. The neural networks have shown promising results in medical imaging. In this research, a deep learning-based approach is used for image classification to detect COVID-19 using chest X-ray images (CXR). A CNN classifier have been used to classify the normal-healthy images from the COVID-19 images, using transfer learning. The concept of early stopping is used to enhance the accuracy of the proposed DenseNet model. The results of the system have been evaluated using accuracy, precision, recall and F1-score metrics. An automated comparative analysis among multiple optimizers, LR Scheduler and Loss Function is performed to get the highest accuracy suitable for the proposed system. The Adamax optimizer with Cross Entropy loss function and StepLR scheduler have outperformed with 98.45% accuracy for normal-healthy CXR images and 98.32% accuracy for COVID-19 images.

Technology is that stroke of luck which humans did not even contemplate could even transpire and shape the human living. Digital transformation has brought the world too close to connect over worldwide. Technology has secured the life of humans through its victory over diseases like cancer and Ebola. It was December 31 2019, when Wuhan city in China notified the World Health Organization about the rapid spread of disease pneumonia. On January 30, 2020, the World Health Organization declared the outbreak of disease a Public Health catastrophe of the worldwide concern. And on March 11, 2020, the disease was declared a pandemic. This was worldwide identified as SARS-CoV-2. The virus went rolling out from China to all over the world. The possible identical element common between SARS and COVID-19 was the origination of virus from specific species called bats. As COVID-19 was zoonotic, it was likely that humans who handled animals could get affected by the transmission of coronavirus Kumar et al. (2020) ; Rothan and Byrareddy (2020) . It is the global urgency wherein people are endeavoring to live in peace and safe. This pandemic has enforced people to live in their own cocoon. Human interaction is over an end, by then technology made it possible to connect to people anywhere around the globe. Pandemic has put forward an opportunity for learning technology to people. Pandemic has affected the humankind of almost all age groups divergently Xiang et al. (2021) . Since COVID-19 can proliferate through direct transfer of viruses or direct communication, social distancing has accompanied people to stay protected. Also, wearing masks has been proved a healthy habit to prevent from COVID-19 Liao et al. (2021) . The coro-navirus eventually started spreading across Europe and the United States, reaching an average of more than 10,000 new cases by the month of February 2020 Gupta et al. (2020) ; Schuchat et al. (2020) . China being an epicenter of COVID-19, Asia-Pacific sector is at high risk of disease transmission Sen-Crowe et al. (2020) . Within certain times, there have been multiple variants of coronavirus including SARS-CoV-2, B.1.1.7, B.1.3.5.1, P.1, B.1.427, B.1.429, black fungus. After struggling to battle with the known variant SARS-CoV-2, there emerged multiple variants that came up dreadfully. The very first was the Variant of Interest (VOI), then came up the Variant of Concern (VOC), and next was hazardous, known as Variant of High Consequence (VOHC). Each of these three categories comprised additional breeds of variants that mutate to produce the new variants of themselves. There are five kinds of VOC which include B.1.1.7 variant, B.1.351 identified in the United States during December 2020. P.1 was the first variant found in the United States, and was examined in brazil travelers around January 2021 Duong (2021) . In India, the vaccine against the global outbreak COVID-19, began on January 16th, 2021 Abu-Raddad et al. (2021) . Besides multiple variants, the vaccine could resist the new variants that are less harmful and on the other hand would make it more infectious and difficult to prevent against COVID-19 Sah et al. (2021) . It has been becoming a heart wrenching journey for the planet, while at the present time also there has been uplift in the cases of COVID-19. Day by day the journey has been seeming to grow lengthy. Figure 1 shows the statistical graph of spread of the COVID-19 disease worldwide, from January 21st 2020 to May 15th 2021. It shows the gradual rise in the number of new COVID-19 cases, with the planet losing an abundant number of lives in April. This suffices to let know the catastrophic urge and need to construct a model that aids Having to deal with the pandemic situation, recent technologies like machine learning, natural language processing, deep learning and neural networks have gained importance because of their satisfactory results. These technologies have helped in treating COVID-19, through the lung Xray images, which is the most ordinary suspect of having coronavirus disease. COVID-19 rules over the lungs of people. With the increasing progression of COVID-19, it became essential for people to learn about its symptoms. The chief symptoms included insufficient breath, drowsiness, fever, dry cough, headache, drop of taste and smell Gaythorpe et al. (2020) ; Chang et al. (2020) . Naturally, it then became challenging to treat people with the specific treatment so that recovery rate gets more sped up. Various techniques have been getting involved treating coronavirus patients in the shortest possible hour. Reverse-Transcription Polymerase Chain Reaction (RT-PCR) is found to be the technique for diagnosis of COVID-19 Lan et al. (2020) . But insufficient cellular content has resulted in false negatives of PCR performance, even when the radiological images have stated positive results Wu et al. (2020) . It's possible that the lungs could become inflamed, making breathing difficult. This can cause pneumonia, which is a disease that could lead to the next stage causing respiratory problem where possibility of being COVID-19 positive is the highest. According to the researchers, CT-scan is a subtle method for detecting COVID-19 pneumonia and can be used as a diagnostic appliance with RT-PCR Lee et al. (2020) . This is the reason chest x-ray (CXR) and CT-scan play a vital role in diagnosing COVID-19 Mukherjee et al. (2021) . So, chest x-ray and CT-scan have a significant character during the entire process of detecting COVID-19 disease Blain et al. (2021) ; Hira et al. (2021) .

Presently, the world needs a helping hand that could rescue the life of living beings from the current state of devastation. In consequence, the foremost deed is to assemble such a technique that could precisely detect COVID-19 during the early stage, in the shortest possible time. There have been many machine learning models that do detect COVID-19 automatically but shortfalls in time check, or even in accurate diagnosis of COVID-19 Sun et al. (2021); Müller et al. (2021) ; Muhammad et al. (2021) ; Rasheed et al. (2021) . As the planet scuffles with COVID-19, every ounce of technical creativity and imagination is deployed to combat this pandemic and bring COVID-19 to an end. Machine learning has proved its innovative approach of dealing with the sudden crisis of COVID-19. Various machine learning methods such as clustering, regression, classification, transfer learning, reinforcement learning, natural language processing have helped in medical imaging and for the treatment of COVID19 Kushwaha et al. (2020) . Also, decision tree-based techniques have been tried to predict COVID-19 diagnosis on the basis of symptoms Zoabi et al. (2021) . Along with machine learning, artificial intelligence has also made efforts into diagnosis of COVID-19 disease. Artificial Intelligence have been performing a key role in contributing to the crisis of COVID-19 Ahuja et al. (2020) . In this paper Jamshidi et al. (2020) , artificial intelligence has been used to combat against COVID-19 disease, along with deep learning approaches. Also, various techniques including Long-Short Term Memory (LSTM), Generative Adversarial Networks (GANs) are used. Over these Extreme Learning Machines (ELM) were also used. But Artificial Intelligence possesses some limitations over COVID-19, and careful balance between data privacy and public health, as well as robust human-AI interaction to control these limitations Naudé (2020) .

The rest elements of the paper are assembled as follows. For the next Section 2, the insights have been acquired from the reviews of related work in regard with diagnosing of COVID-19. Section 3, provides a glimpse of details on the dataset as well as the major steps involved in building this model. Section 4, exhibits about the proposed model architecture and introduces the mechanism of how DenseNet works. The performance evaluation of the model is done in Section 5, where model is evaluated on the basis of accuracy, precision, F1-score and recall metric based on the comparative analysis of multiple optimizers and various loss functions. We also present an inference technique. Ultimately, Section 6, talks about the conclusions.

Chest X-ray has become the requirement of current times. Chest X-ray is used to monitor the patient's lung condition including progression of disease or even any wound relating to accident. Considering the times of COVID-19, chest X-ray have shown promising results in contrast to CT-scan images Sverzellati et al. (2021) . Furthermore, because of the domain's rapid expansion, researchers are becoming increasingly oblivious of progress across diverse approaches, and therefore awareness across algorithms is dwindling. As a result, the literature on bio-inspired computing is biased with algorithms including neural networks, ant colony optimization, particle swarm and genetic algorithms. The authors have then analysed various algorithms relating to bioinspired domain and addressed the algorithms so that it will be easy to choose the best fit algorithm for respective study Kar (2016) . Big-data is present almost in every domain. However, the authors of this paper bring back the importance of use of the information system, rather than the current use of technologies like text mining, neural networks for manipulation of data Kar and Dwivedi (2020) . In this paper, a neural network based method of learning tree is used which enhances picture information storage efficiency Han et al. (2020) . Here, the goal of authors is to present the notion of image word of mouth (IWOM) and a thorough framework for it, which defines UGI as visual articulations of service experiences that lead to consumer assessments of service brand image Bakri et al. (2020) . The authors of this paper have said about the importance of generative adversarial networks and its various applications in the field of image segmentation. Applications of GAN that were found includes medicine, pandemics, face detection, traffic control, texture transfer, image processing and 3-D object generation Aggarwal et al. (2021) . The state-of-art computation and deep learning have explored a number of opportunities in dealing radiography images to achieve diagnostic analysis. Since, 1980s the rapid growth of convolutional neural networks and its contributions towards the healthcare sector is boundless. Convolutional Neural Networks (CNN) have been working dynamically to benefit in the diagnosis of COVID-19, gradient-weighted class activation mapping having been holding up with CNN to subjugate over the detection of COVID-19. Apart from time constraints, convolutional neural networks (CNN) are acting as reassurance to diagnose COVID-19 using chest x-ray images, with no false negatives Mukherjee et al. (2021) . The key benefit of CNN is that it automatically identifies essential features with no human intervention.

Knowing the current situation, and regardless of COVID-19 diagnosis, it is essential to detect COVID-19 in limited time such that COVID-19 positive patient can be recovered from further lung infection. Considering the character of chest X-ray and medical imaging techniques, image classification and image segmentation have a major part. A convolution deep and wide network (CDWN) was used to extract relevant features and segment the lung area more precisely based on automatic lungs segmentation Agnes et al. (2018) . In this paper Tan et al. (2021) , SRGAN+VGG model is proposed, where visual geometry group network (VGG16) has been used which is deep convolutional neural network to identify the COVID-19 positive and negative result from the chest image and to reconstruct those chest images to high quality, a deep learning method called Super Resolution Generative Adversarial Network (SRGAN) is used. In Abbas et al. (2021) , for the purpose of classification of chest X-ray images of COVID-19 disease, a deep convolutional neural network (CNN) known as Decompose, Transfer, and Compose (DeTraC) has been used. DeTraC can deal with any inconsistencies in the image dataset by investigating its class borders using a class breakdown methodology. Multiple stateof-art pre-trained models can be used for image classification including VGG16, ResNet50 (residual neural network), Inceptionv3, EfficientNet Zebin and Rezvy (2021) , where transfer learning pipeline have been implemented for classification of COVID-19 chest X-ray images from a healthy chest X-ray image and the one affected with pneumonia.

To enhance the findings of COVID-19 disease using our proposed deep learning model, DenseNet is used for this research. DenseNet as the name suggests the denser connections higher will be the accuracy. For our Deep Learning Model, the DenseNet-121 architecture have been used as the foundation and used pre-trained weights as the concept of Transfer Learning. Dense connections are essentially feed-forward networks, which is DenseNet's main advantage over other networks Zhang et al. (2021) . The central concept of DenseNet is the feature reuse, which results in extremely compact versions. They intensify feature propagation and also stimulate feature regeneration, and turn down the number of parameters which has improved our model's accuracy faster. DenseNet makes the connectivity of each layer much easier by simply connecting every layer directly with each layer. DenseNets make use of the network's capacity by reusing features. Also torchvision is used to build and train our deep learning model from scratch. DenseNet as the name suggests the denser connections higher will be the accuracy. Also a deep learning library PyTorch is used and torchvision which is a pre-trained data learning model which has a maximum of control across overfitting and it also enhances the optimization of results from the very first.

For diagnosis of COVID-19, the chest X-ray images have been gathered from two different origins. The COVID-19 chest X-ray data set was magnified by Joseph Paul Cohen Cohen et al. (2020) using pictures from a variety of public domain sources. This is a constantly expanding dataset, and presently the dataset contains information about 950 patients of which 584 patients have been diagnosed with COVID-19 positive. Figure 2 shows the image distribution as a percentage of the diagnosis, with COVID-19 findings in 53.3 percent of the images whereas 46.7 percent of normal-healthy X-ray images. And the normal-healthy Chest X-ray images were developed by Paul Mooney Mooney (2020) inspired in context with an article Kermany et al. (2018) . In this dataset there are in all 1341 normal -healthy images. Amongst those, 300 random images have been selected. Only a few images for normal-healthy chest X-ray images are used because learning with an unbalanced dataset will lead to a skewed prediction model that favors classes with multiple samples. The dataset is assembled in two classes, one with label normal-healthy chest X-ray and second class labelled as COVID-19 chest X-ray. Considering the gender information count for diagnosing COVID-19, there are in all 346 patients who are male, whereas there are 175 patients who are female who have been detected to get affected by the disease COVID-19. Considering the age of patients, there have been 88 patients in a range of 20-40 years of age who have been diagnosed with disease COVID-19 positive. Whereas, the highest count of 175 patients was received yielding from the age group between 41-61. While 172 patients from age group Fine-Tuning of DenseNet model for classification ofCovid-19 cases in Medical Imaging Taking a glance of view at figure 3, have you noticed how close they seem to be? Truly fair, every so often even for a medical specialist, the X-ray images could be arduous to diagnose. As a result, our proposed model can assist them. For implementing the model, PyTorch is used. PyTorch is a Deep Learning library that performs Tensor Computations, which are a key component of Deep Learning algorithms. The Facebook AI Research Lab created it and has been using it. It has been a library that has attracted many researchers and has created many State-of-the-Art Algorithms in all areas of Deep Learning, involving Natural Language Processing and Computer Vision.

The most essential purpose of using PyTorch is to implement the model for image classification purposes for the diagnosing of COVID-19 chest X-ray. By putting up an image classification model could grip up a lot of worries for doctors, specially the one in X-ray diagnosis.

To identify the lung image scenario and its affected infection, two kinds of views have been selected which are Postero-anterior (PA) and Antero-posterior (AP). The Posteroanterior (PA) view is the chest X-ray which is covered from posterior to anterior side of the patient. While Antero-posterior is the chest-Xray taken from anterior to posterior coverage of the patient. Before image labelling, the images need to be scaled which is the requirement for data augmentation. As various transformations are to be applied on the image, it is necessary to scale it before normalizing. The COVID-19 data is small whereas the chest X-ray images or normal patients is readily available. Having such difference in the amount of dataset is called to be the skewed datasets, which need to be developed by various methods such as K-Fold Cross Validation. While staging throughout the data all the images were taken in correspondence to proportions to keep away from the over-fitted results.

It is the next stride after loading the dataset for COVID-19 positive and normal -healthy images. The synthesis of new data using existing data and a few minor manipulations and image processing is known as data transforms or augmentation. Augmentation will aid in the model's generalisation and will prevent overfitting to the training results. Image augmentation allows to add more data to the current dataset without having to spend hours manually doing so. These tasks can be performed using torchvision from Py-Torch. Torchvision provides tools for data transformation, data handling and for pre-defined deep learning state-of-art (SOTA) models.

Example image rotation, shifting the image (geometric transformation that maps the location of every object in the image to a new location in the last image), image flipping, image noising are few image augmentation techniques. As only a small amount of data is used for training and validation, Image Transforms have been used to create some extra data. All pre-trained models need input images that have been normalised in the same way. To pre-process the file, we create a transformation sequence where the datasets are read into PILImage (Python imaging format) by Torchvision. And ToTensor transforms a PIL Image with a range of [0, 255] with a shape of (C x H x W) to FloatTensor with a range of [0, 1]. During the normalization of image, every input gets converted in the range of 0 to 1 using the formula below, with 0.5 as the standard deviation:

where is equal to standard deviation which is equal to 0.5. C is the mini-batch of the channel, H is the height and W is the width. H and W are expected to be of minimum 224. Mean and standard deviation has been used for normalization with mean range of [0.485, 0.456, 0.406] and standard deviation of [0.229, 0.224, 0.225] . The normalization is performed on the ImageNet dataset. ImageNet is one of the most well-known datasets for training high-end neural networks. It contains over 1.2 million images divided into 10,000 categories. This dataset is typically loaded into a high-end hardware device because a CPU alone cannot support datasets this large.

To avoid the skewed datasets, the dataset is split into the 80 is to 20 ratios for training and validating the model with the distinguished datasets. This takes the images from each folder named after the class name and automatically labels them. And DataLoader loads those labelled images and tracks of the Train Data (Image) and Label (Class Name). This assembles our dataset in two classes, one with label normal -healthy chest X-ray and second class labelled as COVID-19 chest X-ray. Once the classes are ready, data is loaded in either CUDA (GPU) or CPU heading towards model defining. Torchvision also includes a sub package of definition of models for various functions including image classification, object detection, semantic segmentation and many more. In regard to image classification, the DenseNet model is one of the pre-trained deep learning model architectures included in torchvision.

For our Deep Learning Model, the DenseNet-121 Architecture is used as the foundation. Also transfer learning have been used for DenseNet framework to enhance the performance of model Montalbo (2021) ; Maghdid et al. (2021) . DenseNets on contrary to popular belief, need fewer parameters than conventional CNNs because they do not require to learn unessential feature maps. Some ResNet disparities have shown that number of several layers have not been enhancing the framework of ResNet and so they can be removed. ResNets have numerous parameters since every layer has its own weights to understand whereas DenseNets layers are narrow and they introduce new feature-maps in negligible quantity. Another issue with it was the difficulty in training them because of the previously described data and gradients. Since each layer can connect to the gradients through the loss function and the actual image, then DenseNets solve this problem.

The key difference with ResNet is that instead of incorporating function maps, they can concatenate. The central concept of DenseNet is the feature reuse, which results in extremely compact versions. As a result, it needs fewer parameters than other CNNs since no feature-maps are replicated. When CNNs go deeper, they run into issues. The reason behind this is the mapping of data from the inner layer to the outer layer (as well as the gradient in the opposite direction) gets so long that it can disappear before entering any further side. DenseNet makes this connectivity much easier by simply connecting every layer directly with each layer. DenseNets make use of the network's capacity by reusing features. As conveyed in, DenseNet is a form of convolutional neural network and a typical DenseNet architecture uses dense blocks to link all layers (with corresponding feature-map sizes) directly to each other, resulting in dense connections between layers. DenseNet as the name suggests the denser connections higher will be the accuracy. Each layer in DenseNet receives additional input across each previous layer and sends its feature-maps to the successive layers. Each layer receives collective understanding from the layers above it, that is the concept of concatenation which is used. To maximise computation recycling between the classifiers, integrating multiple classifiers into an ideal and deep convolutional neural network and interconnects them with dense connectivity for efficient image classification Huang et al. (2017a) . Research has shown that convolutional networks with small connections between layers and those near to the output can be significantly deeper, and will be much more accurate to train Huang et al. (2017b) . On most them, DenseNet achieves substantial advancements over the stateof-the-art while consuming minimal memory and processing to enhance its effectiveness.

In correspondence with this research, our aim for the classification of COVID-19 chest X-ray images into COVID-19 positive and normal-healthy image. A CNN classifier is created to diagnose COVID-19 disease using chest X-ray images. Also a deep learning library PyTorch and torchvision are used, which is a pre-trained data learning model which has a maximum of control across overfitting and it also enhances the optimization of results from the very first. As shown in Figure 4 the block diagram of DenseNet defines a five-layer dense block having a growth rate of k=4. The number 121 in DenseNet-121 shows the total number of layers in the neural network. A typical composition of DenseNet-121 is the combination of various layers. It includes five convolution and pooling layers, three transition layers (6, 12, 24), one classification layer (16), and two Dense-Blocks (1 x 1 and 3 x 3 convs). The DenseNet stimulates feature reuse and reduces the parameters which enhances the accuracy of model for diagnosis of COVID-19 using chest X-ray images.

Following the composite function operation, the result of the preceding layer becomes an input of the second layer. The composite process is composed of a non-linear activation layer, pooling layer, batch normalization and convolution layer. A detailed process of diagnosing COVID-19 has been mentioned in the following sections.

Transfer Learning is a robust deep learning method in whichthe model parameters are used, that have already been trained on a larger dataset (ImageNet Dataset comprises 1000 Classes). Also use of pre-trained weights is made for transfer learning, to learn and achieve higher accuracy in our model faster. For example, it is transferring a learned man's knowledge to another. During transfer learning process, all the layers are trained, rather than freezing the CNN layer and training only the Fully connected layers. One way to increase the performance of model is to train only the CNN layers either completely or partially, as there is need to deal with the chest X-ray images. This lets the weights to get tuned from the pre-trained dataset features to the specialized featured of COVID-19 chest X-ray image dataset. And backward function has been used for backpropagation. There is no need to rack up the gradients from previous loss, and that is why zero-grad optimizer is used to clear all the old gradients from the last stage.

In Kingma and Ba (2014) , as a function extractor, various pre-trained networks including VGG16, ResNet50, and EfficientNetB0 were used. Since all of these supporting techniques were pre-trained on the massive ImageNet dataset, they have gained low-level features for appropriate representation which can be shared across different computer vision problems for the transfer of information and ultimately for new images it will set out as a feature extractor. The pretrained model was used for extraction of certain features from these images that rely on transfer learning concepts. The new images will have various classes other than the original dataset.

As shown in the Figure 5 below, the transfer learning process for diagnosis of COVID-19 chest X-ray images has been demonstrated. On an already pre-trained dataset of images called ImageNet, the COVID-19 images are being added, with the pre-trained weights and learned weights, the knowledge transfer gets enabled and behaves as a feature extractor for the new set of images.

Transfer learning also plays a major role in diagnosing COVID-19 disease. The Figure 6 . below shows the working of our proposed mode classification of COVID-19 chest Xray images.

Early Stopping is a technique for selecting the model that performs best under given conditions. There is a point where For this model, the model needs to be chosen with the lowest validation loss to achieve high accurate results. There is no any need of a false positive or false negative because mostly the medical systems are going to diagnose a human. At the time of training, there is no need to update the weights of pre-trained network. So, the newly created classifier is trained to obtain the weights in accordance with new combinations. The aim of fine tuning is to learn the new features and work with the newly added COVID-19 data on top of the pre-trained dataset.

Besides predicting the class label of image, the function of image visualization will also display the image with the predicted class label. Once this process is finished, following is the output that is achieved as shown in Figure 7 above.

The ideal concept of DenseNet is the feed-forward networks which is the major benefit of DenseNet over other networks. There are multiple mesmerizing advantages of DenseNet. Firstly, DenseNet diminishes the vanishing-gradient problem. Secondly, they intensify feature propagation and also stimulate feature regeneration, and turn down the num-Fine-Tuning of DenseNet model for classification ofCovid-19 cases in Medical Imaging Figure 8 illustrates the function of DenseNet workflow that includes combination of Dense Blocks and Transition Layers for classification of given input chest X-ray image. An image is given as an input to the DenseNet, and passes through several dense blocks where each layer's feature maps remain same in the block except for the number of filters that varies within the blocks. After a dense block, it passes to the transition layer. The transition layer performs two operations, namely the convolution and pooling. The transition layer carries out the down sampling operations outside the dense blocks. To perform feature concatenation, there is need to confirm that the feature maps are of the similar size in the dense block. To bring down the number of input featuremaps and thus improve overall computational performance, 1 x 1 bottleneck convolution layer can be added prior to every 3 x 3 convolution. The transition layers present in DenseNet architecture include a layer of batch normalization (BN), 1 x A detailed working to activities in the dense block is shown in Figure 9 . The dense block present in DenseNet architecture includes a layer of batch normalization (BN), a rectified linear unit (ReLU) activation, accompanied along 3 x 3 convolution (convs). A global average pooling layer follows the final dense block, which is then fed into a Softmax classifier. Considering that L layers are present in DenseNet, there will be following direct connections in correspondence to L layers: L(L+1)/2. As shown in Figure 9 . for non-linear transformations , the layer receives the following input:

where [ 0 , 1 , ...., −1 ] pertains to the concatenation of feature-maps from layer 0 to l-1. Each layer uses the feature maps of all previous layers as inputs, and each subsequent layer uses its own feature maps as inputs. Thus, for layer l, and collection of layers on the top of depth dimension H, as illustrated in Figure 9 the DenseNet shows the output as follows:

All the inputs of H in Eq. (2), get concatenated into an ideal tensor at the time of execution. In DenseNet architecture, feature-map sizes can be changed through convolution and pooling.

Batch normalisation is a technique for standardising network inputs that can apply to either the activations of a previous layer or the inputs themselves. When constructing transfer learning and convolutional neural networks, the rectified linear activation is the default activation that takes place. The DenseNet diverges into Dense blocks, each of which has a unique set of filters but the same dimensions. Downsampling is used by Transition Layer to apply batch normalisation. Average pooling includes determining the average for each section of the feature map. This means that each of the function map's 2 x 2 squares is then down-sampled to the square's average value.

The model was trained for 10 epochs. Also the Cross Entropy loss function have been implemented and an Adamax optimizer to upgrade the weights and the learning rate is chosen as 1e-3, having batch size of 8. The developed deep learning model comprises of 6955906 parameters. The training of proposed model has been done on Windows machine using an AMD Threadripper processor of 64 cores, 128 threads and 256 MB cache. The graphic card used was RTX3080 and RAM of about 128GB.

The model must now recognize the images that the model has not noticed. It should be able to flawlessly distinguish the image having COVID-19 class and the ones with normal class. As the model is trained to produce minimum validation loss, so it must identify the appropriate image class among the precise set of images that are provide as input. Also, the model is trained to achieve a good accuracy and this procedure is known as Model Evaluation. The dataset used belongs to the COVID-19 Radiography database which was collected and published by Tawsifur Rahm. As the dataset have been assembled in two classes, one with label normal-healthy chest X-ray and second class labelled as COVID-19 chest X-ray. So, for evaluation we have taken 100 images from each class for testing from the above-mentioned repository. The database contains numerous amounts of images among which we settled to choose 100 images from the individual classes that is COVID-19 and normal -healthy. The model is evaluated using accuracy, precision, F1-score and recall. Each of the metric is identified for both the classes of out proposed model that is COVID-19 and normal-healthy. Additionally, the use of multiple optimizers is done, to evaluate our proposed model. And subsequently we have obtained accuracy, precision, F1-score and recall metrics for each of the corresponding optimizers. While trying to analyse the highest accuracy given by respective optimizer, there have been an attempted to enhance the accuracy of model on the basis of various loss functions as well as LR Scheduler. A comparative analysis among the optimizers and various loss functions is effectively performed in order to examine the chief optimizer and loss function and achieve a higher accuracy. Comparative analysis is done among optimizers that include Adamax, Adam, AdamW and stochastic gradient descent (SGD). While various loss functions consist of Cross Entropy, NLLLoss, and MultiMarginLoss and different LR Schedulers include StepLR and MultiStepLR.

Based on deep learning and CNN framework, various optimizers have been compared to reach out to the highest accuracy of our proposed model. The optimizers to be differentiated include Adamax, Adam, AdamW and Stochastic Gradient Descent (SGD). Adam is the blend of Stochastic Gradient Decent and RMSProp with momentum. Adam computes the learning rate of each parameter separately. Basically, Adamax is obtained from Adam optimizer and it calculates gradient form the first two moments of it. Since few models have embeddings which is why Adamax is more useful as compared to other optimizers Kingma and Ba (2014) .

where 'm' stands for the moment of that certain variable, 'A' is any variable and 'E' is expected value of n variable.

where stands for updating of norms and examines the extensiveness of norm In Vani and Rao (2019) , comparative analysis of multiple optimizers has been done and the results showed that among Adadelta, Adam, Adamax, Adagrad, RMSEProp, Nadam and SGD; Adamax outperformed with 99.58% accuracy. The proposed deep learning and CNN based DenseNet model have successfully achieved 98% of accuracy with the use of Adamax optimizer, Cross Entropy loss function and namely the StepLR LR Scheduler.

There are different Parameters to evaluate the performance of the model like Training Accuracy, Training loss, Validation Accuracy and Validation Loss. Accuracy is a most important for evaluating the classification models. Accuracy is fraction of number of predictions and total number of predictions. Now coming on to the training accuracy it is clear that the accuracy of a model is based on the examples it was trained on. The plot describes the accuracy on every epoch. Here for every epoch the accuracy was recorded and then a final plot was constructed. So, the Plot tells us that as the number of epochs increases the accuracy of the model also increases. So, training accuracy should be high. Here the highest training accuracy is 98.94%. In our case Training Accuracy is important because it is necessary to identify a greater number of correct Covid-19 cases. Here Training Accuracy shows us that model is learning all the parameters in a very right way and there will be less misclassification as shown in Figure 10 .

Validation Accuracy is also called as testing accuracy. It is the accuracy that needs to be calculated on the dataset Fine-Tuning of DenseNet model for classification ofCovid-19 cases in Medical Imaging on which the model has not been trained. We only provide that set of which the model has not seen yet. It is used for finding how much generalized our model is. Validation accuracy should be less than or equal to the training accuracy. If there is a significance difference between validation accuracy and training accuracy then it can be said that our model is overfitting. The plot says the validation accuracy on every epoch and there the result is slightly greater than the training accuracy. Here the validation is accuracy is important to identify the test data in a correct way as both the classifications are important. It basically tells us how the model will be classifying the covid-19 and normal cases for the real chest X-rays. So, model should be able to classify a greater number of correct cases here Figure 11 .

Training loss is the error occurred on the training data. Loss is the figure which indicate how bad the model is. If a model is predicting perfectly then it can be said that the loss is zero. Otherwise, we can say that loss is greater. The main aim of training the model is to find the set of weights which can give us low loss. Here for every loss occurred the weights get updated to minimize the loss. So, it is clear that training loss should be low as possible. Because the less the loss is the more perfect is our model. In this there is a try to see the loss of model on every epoch and similarly the number of epochs is getting increased the loss is decreasing. In some cases, it has risen up. So, training loss here represents for every iteration how much well the model is actually learning and which features to be included so that for next iteration model will commit less mistakes and can distinguish the covid-19 and normal cases in a better way. Less number of loss represents that model is very much accurate and there are less errors for classification of covid and normal cases as shown in Figure 12 .

Validation loss is almost similar to the training loss but the difference is that it is the loss which is calculated on the validation set. Here the weight doesn't get updated while training. Validation loss should be also similar to the training loss. If the validation loss is greater than the training loss then the model is overfitting. And if the validation loss is less than training then it is underfitting. Validation loss should also be as low as possible and a small overfitting can be accepted. So here the plot says the validation loss on every epoch and parallely the validation loss is getting decreased as the number of epochs increases as shown in Figure 13 . Validation loss will tell that model is having how much error while classifying the covid-19 and normal cases for test data. If the validation loss is less, it means the model is having less errors for classifying the covid and normal cases for the test data. The following Tab. 1. gives a precise view about comparative analysis to automate the highest accuracy among multiple optimizers, loss functions, LR Schedulers and the class label to detect COVID-19 chest X-ray that is either normal-healthy or COVID19

The COVID-19 literature has dramatically risen to over thousands of scholarly articles in just a minimal time. Numerous researchers have spent their valuable time in designing an approach to diagnose covid-19, to rescue the life of humans at earliest. Our approach provides an aerial perspective to understand the spread and the treatment of covid-19 by using deep neural networks. 

The determined finding can help researchers with relevant study. This research includes advantages that contribute in early detection of covid-19. Our theoretical contributions includes multiple concepts. Firstly, a deep learning based densenet model is built from scratch to enhance our model. Secondly, fine-tuning is performed based on pre-trained networks and re-training them to learn about the new features involved in the dataset. Also, a concept of early stopping have been used to improve the accuracy of our proposed model.

With the constant growing of covid-19 disease and its research, there is need to to assess articles in a methodical and effective manner. This study will help researchers and educators get a quantitative study of the literature. Journals will also get a better understanding of the most commonly discussed diseases, and also a better understanding of covid-19 can be made which could help the policymakers and funding organisations to avail about the upcoming plans. Our approach towards assisting human in detection of covid-19, uses a deep learning methodology. The accelerated growth of covid-19 have left no other solution other than technology that could gradually help in aiding covid-19. Rather than individually identifying the best optimizer, loss function, or LR Schedulers for our proposed model; a comparative analysis have been provided to automate the selection of best hyperparamaeters. An automated analysis of optimizer is done to assist throughout the evaluation of the model. Among optimizer including Adamax, Adam, AdamW and SGD; Adamax outperforms from others. Also various loss functions are analyzed including cross entropy, NLLLoss and multimargin loss where cross entropy function enhanced the accuracy of the model. Along with optimizer and loss functions, LR Scheduler also have been analyzed to achieve the best possible scheduler. The stepLR scheduler proved in achieving higher accuracy than MultistepLR.

In this research, a model have been proposed using a combination of deep learning and convolutional neural networks (CNN) with the use of DenseNet architecture to diagnose COVID-19 using the Chest X-ray dataset. Automated comparison among multiple optimizers, loss functions and LR Schedulers have been done to achieve the highest accuracy for classification of COVID-19 chest X-ray images. Among multiple optimizers, Adamax optimizer having Cross Entropy loss function and Step LR Scheduler has outperformed showing an accuracy of 98.45% for normal-healthy class and 98.32% for COVID-19 class. The model was trained at 10 epochs. Additionally the concept of Early Learning is applied which enhances the performance of the system. Also transfer learning is introduced in DenseNet model using Py-Torch and torchvision from scratch. Using Early Stopping, various optimizers, loss functions and LR Scheduler have been automated with their values to achieve the highest acuuracy of the proposed model. In future, if data such as inference code is obtained, then it is possible to integrate the model into web app or any mobile devices, for better ease of application use.

Hemant Palivela :Investigation, Conceptualization, Methodology Sarveshmani Tiwari: Software, Data curation, Tavishee Chauhan Writing -original draft.

The author declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Classification of covid-19 in chest x-ray images using detrac deep convolutional neural network

Effectiveness of the bnt162b2 covid-19 vaccine against the b. 1.1. 7 and b. 1.351 variants

Generative adversarial network: An overview of theory and applications

Automatic lung segmentation in low-dose chest ct scans using convolutional deep and wide network (cdwn)

Artificial intelligence and covid-19: A multidisciplinary approach

Decoding service brand image through user-generated images

Determination of disease severity in covid-19 patients using deep learning in chest x-ray images

Risk factors for disease progression in covid-19 patients

Covid-19 image data collection: Prospective predictions are the future

What's important to know about the new covid-19 variants?

Report 8: Symptom progression of covid-19

Effect of weather on covid-19 spread in the us: A prediction model for india in 2020

Named data networking with neural networks for intelligent image processing information systems

An automatic approach based on cnn architecture to detect covid-19 disease from chest x-ray images

Multi-scale dense networks for resource efficient image classification

Densely connected convolutional networks

Artificial intelligence and covid-19: deep learning approaches for diagnosis and treatment

Bio inspired computing-a review of algorithms and scope of applications

Theory building with big data-driven research-moving away from the "what" towards the "why

Identifying medical diagnoses and treatable diseases by image-based deep learning

Adam: A method for stochastic optimization

Sars-cov-2 (covid-19): zoonotic origin and susceptibility of domestic and wild animals

Significant applications of machine learning for covid-19 pandemic

Positive rt-pcr test results in patients recovered from covid-19

Covid-19 pneumonia: what has ct taught us?

A technical review of face mask wearing in preventing respiratory covid-19 transmission. Current Opinion in Colloid & Interface Science

Diagnosing covid-19 pneumonia from x-ray and ct images using deep learning and transfer learning algorithms

Diagnosing covid-19 chest x-rays with a lightweight truncated densenet with partial layer freezing and feature fusion

Chest x-ray images (pneumonia)

Supervised machine learning models for prediction of covid-19 infection using epidemiology dataset. SN computer science

Shallow convolutional neural network for covid-19 outbreak screening using chest x-rays

Convolutional neural networks for semantic segmentation as a tool for multiclass face analysis in thermal infrared

Artificial intelligence vs covid-19: limitations, constraints and pitfalls

A machine learning-based framework for diagnosis of covid-19 from chest x-ray images

The epidemiology and pathogenesis of coronavirus disease (covid-19) outbreak

Accelerated vaccine rollout is imperative to mitigate highly transmissible covid-19 variants

Public health response to the Fine-Tuning of DenseNet model for classification ofCovid-19 cases in Medical Imaging initiation and spread of pandemic covid-19 in the united states

Predicting covid-19 disease progression and patient outcomes based on temporal deep learning

Chest x-ray or ct for covid-19 pneumonia? comparative study in a simulated triage setting

Classification of covid-19 pneumonia from chest ct images based on reconstructed super-resolution images and vgg neural network

An experimental approach towards the performance assessment of various optimizers on convolutional neural network

Radiological findings and clinical characteristics of pregnant women with covid-19 pneumonia

Modern senicide in the face of a pandemic: An examination of public discourse and sentiment about older adults and covid-19 using machine learning

Covid-19 detection and disease progression visualization: Deep learning on chest x-rays for classification and coarse localization

Resnet or densenet? introducing dense shortcuts to resnet

Machine learningbased prediction of covid-19 diagnosis based on symptoms. npj digital medicine

☒ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.☒The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: