key: cord-254124-4p1ah2h3 authors: Khan, Asif Iqbal; Shah, Junaid Latief; Bhat, Mohammad Mudasir title: CoroNet: A Deep Neural Network for Detection and Diagnosis of COVID-19 from Chest X-ray Images date: 2020-06-05 journal: Comput Methods Programs Biomed DOI: 10.1016/j.cmpb.2020.105581 sha: doc_id: 254124 cord_uid: 4p1ah2h3 BACKGROUND AND OBJECTIVE: The novel Coronavirus also called COVID-19 originated in Wuhan, China in December 2019 and has now spread across the world. It has so far infected around 1.8 million people and claimed approximately 114698 lives overall. As the number of cases are rapidly increasing, most of the countries are facing shortage of testing kits and resources. The limited quantity of testing kits and increasing number of daily cases encouraged us to come up with a Deep Learning model that can aid radiologists and clinicians in detecting COVID-19 cases using chest X-rays. METHODS: In this study, we propose CoroNet, a Deep Convolutional Neural Network model to automatically detect COVID-19 infection from chest X-ray images. The proposed model is based on Xception architecture pre-trained on ImageNet dataset and trained end-to-end on a dataset prepared by collecting COVID-19 and other chest pneumonia X-ray images from two different publically available databases. RESULTS: CoroNet has been trained and tested on the prepared dataset and the experimental results show that our proposed model achieved an overall accuracy of 89.6%, and more importantly the precision and recall rate for COVID-19 cases are 93% and 98.2% for 4-class cases (COVID vs Pneumonia bacterial vs pneumonia viral vs normal). For 3-class classification (COVID vs Pneumonia vs normal), the proposed model produced a classification accuracy of 95%. The preliminary results of this study look promising which can be further improved as more training data becomes available. CONCLUSION: CoroNet achieved promising results on a small prepared dataset which indicates that given more data, the proposed model can achieve better results with minimum pre-processing of data. Overall, the proposed model substantially advances the current radiology based methodology and during COVID-19 pandemic, it can be very helpful tool for clinical practitioners and radiologists to aid them in diagnosis, quantification and follow-up of COVID-19 cases. deaths respectively [2] . Once infected, a COVID-19 patient may develop various symptoms and signs of infection which include fever, cough and respiratory illness (like flu). In severe cases, the infection may cause pneumonia, difficulty breathing, multi-organ failure and death [2, 3] . Due to the rapid and increasing growth rate of the COVID-19 cases, the health system of many advanced countries has come to the point of collapse. They are now facing shortage of ventilators and testing kits. Many countries have declared total lockdown and asked its population to stay indoors and strictly avoid gatherings. A critical and important step in fighting COVID-19 is effective screening of infected patients, such that positive patients can be isolated and treated. Currently, the main screening method used for detecting COVID-19 is real-time reverse transcription polymerase chain reaction (rRT-PCR) [4, 5] . The test is done on respiratory samples of the patient and the results can be available within few hours to 2 days. An alternate method to PCR screening method can be based on chest radiography images. Various research articles published in Radiology journal [6, 7] indicate that that chest scans might be useful in detecting COVID-19. Researchers found that the lungs of patients with COVID-19 symptoms have some visual marks like ground-glass opacities-hazy darkened spots that can differentiate COVID-19 infected patients from non COVID-19 infected ones [8, 9] . The researchers believe that chest radiology based system can be an effective tool in detection, quantification and follow-up of COVID-19 cases. A chest radiology image based detection system can have many advantages over conventional method. It can be fast, analyze multiple cases simultaneously, have greater availability and more importantly, such system can be very useful in hospitals with no or limited number of testing kits and resources. Moreover, given the importance of radiography in modern health care system, radiology imaging systems are available in every hospital, thus making radiography based approach more convenient and easily available. Today, researchers from all around the world, from various different fields are working day and night to fight this pandemic. Many researchers have published series of preprint papers demonstrating approaches for COVID-19 detection from chest radiography images [10, 11] . These approaches have achieved promising results on a small dataset but by no means are production ready solutions. These approaches still need rigorous testing and improvement before putting them in use. Subsequently, a large number of researchers and data scientists are working together to build highly accurate and reliable deep learning based approaches for detection and management of COVID-19 disease. Researchers are focusing on deep learning techniques to detect any specific features from chest radiography images of COVID-19 patients. In recent past, deep learning has been very successful in various visual tasks which include medical image analysis as well. Deep learning has revolutionized automatic disease diagnosis and management by accurately analyzing, identifying, classifying patterns in medical images. The reason behind such success is that deep learning techniques do not rely on manual handcrafted features but these algorithms learn features automatically from data itself [12] . In the past, deep learning has had success in disease classification using chest radiography image. ChexNet [13] is a deep neural network model that detects Pneumonia from chest X-ray image. ChexNet achieved exceptional results exceeding average radiologist performance. Another similar approach called ChestNet [14] is a deep neural network model designed to diagnose thorax diseases on chest radiography images. The success of AI based techniques in automatic diagnosis in the medical field and rapid rise in COVID-10 cases have necessitated the need of AI based automatic detection and diagnosis system. Recently, many researchers have used radiology images for COVD-19 detection. A deep learning model for COVID-19 detection (COVID-Net) proposed by Wang and Wong [10] obtained 83.5% accuracy in classifying COVID-19, normal, pneumonia-bacterial and pneumonia-viral classes. Hemdan et al. [15] used various deep learning models to diagnose COVID-19 from cheat X-ray images and proposed a COVIDX-Net model comprising seven CNN models. Apostolopoulos and Mpesiana [16] trained different pre-trained deep learning models on a dataset comprising of 224 confirmed COVID-19 images and achieved 98.75% and 93.48% accuracy for two and three classes, respectively. Narin et al. [11] trained ResNet50 model using chest X-ray images and achieved a 98% COVID-19 detection accuracy for two classes. However, the performance for multi class classification is not known. Sethy and Behera [17] Deep learning is all about data which serves as fuel in these learning models. Since COVID-19 is a new disease, there is no appropriate sized dataset available that can be used for this study. Therefore, we had to create a dataset by collecting chest X-ray images from two different publically available image databases. COVID-19 X-ray images are available at an open source Github repository by Joseph et al [19] . The authors have compiled the radiology images from various authentic sources (Radiological Society of North America (RSNA), Radiopaedia etc) of COVID-19 cases for research purpose and most of the studies on COVID-19 use images from this source. The repository contains an open database of COVID-19 cases with chest X-ray or CT images and is being updated regularly. At the time of writing this paper, the database contained around 290 COVID-19 chest radiography images. Pneumonia bacterial, Pneumonia viral and normal chest X-ray images were obtained from Kaggle repository -Chest X-Ray Images (Pneumonia)‖ [20] . The dataset consists of 1203 normal, 660 bacterial Pneumonia and 931 viral Pneumonia cases. We collected a total of 1300 images from these two sources. We then resized all the images to the dimension of 224 x 224 pixels with a resolution of 72 dpi. Table I below shows the summary of the prepared dataset. Figure 1 below shows some samples of chest X-ray images from the prepared dataset. In order to overcome the unbalanced data problem, we used resampling technique called random under-sampling which involves randomly deleting examples from the majority class until the dataset becomes balanced. We used only 310 normal, 330 pneumonia-bacterial and 327 Pneumonia-viral X-ray images randomly from this chest X-ray pneumonia database. In this section, we will discuss the work methodology for the proposed technique, model architecture, implementation and training. The work methodology is also illustrated in Figure 2 . Convolutional Neural Network also known as CNN is a deep learning technique that consists of multiple layers stacked together which uses local connections known as local receptive field and weight-sharing for better performance and efficiency. The deep architecture helps these networks learn many different and complex features which a simple neural network cannot learn. Convolutional neural networks are powering core of computer vision that has many applications which include self-driving cars, robotics, and treatments for the visually impaired. The main concept of CNN is to obtain local features from input (usually an image) at higher layers and combine them into more complex features at the lower layers [21] [22] . A typical Convolutional Neural Network architecture consists of the following layers: Convolution layer is the core building block of a Convolutional Neural Network which uses convolution operation (represented by *) in place of general matrix multiplication. Its parameters consist of a set of learnable filters also known as kernels. The main task of the convolutional layer is to detect features found within local regions of the input image that are common throughout the dataset and mapping their appearance to a feature map. The convolution operation is given as Where I is the input matrix (image), K is the 2D filter of size m x n and F represents the output 2D feature map. Here the input I is convolved with the filter K and produces the feature map F. This convolution operation is denoted by I*K. The output of each convolutional layer is fed to an activation function to introduce nonlinearity. There are number of activation functions available but the one which is recognized for deep learning is Rectified Linear Unit (ReLU). ReLU simply computes the activation by thresholding the input at zero. In other words, ReLU outputs 0 if the input is less than 0, and raw output otherwise. It is mathematically given as: In CNN, the sequence of convolution layer is followed by an optional pooling or down sampling layer to reduce the spatial size of the input and thus reducing the number of In fully connected layer each neuron from previous layer is connected to every neuron in the next layer and every value contributes in predicting how strongly a value matches a particular class. The output of last fully connected layer is then forwarded to an activation function which outputs the class scores. Softmax and Support Vector Machines (SVM) are the two main classifiers used in CNN. Softmax function which computes the probability distribution of the n output classes is given as Where x is the input vector and Z is the output vector. The sum of all outputs (Z) equals to 1. The proposed model CoroNet uses Softmax, to predict the class to which the input X-ray image belongs to. All the layers discussed above are stacked up to make a full CNN architecture. In addition to these main layers mentioned above, CNN may include optional layers like batch normalization layer to improve the training time and dropout layer to address the overfitting issue. CoroNet is a CNN architecture tailored for detection of COVID-19 infection from chest X-ray images. It is based on Xception CNN architecture [23] . Table II . To initialize the model parameters, we used Transfer Learning to overcome the problem of overfitting as the training data was not sufficient. We The multi-class classification result of the proposed model was recorded for each Fold and then average numbers were calculated. The performance of the proposed CoroNet on each fold is presented in the form of Confusion matrix (CM) in Figure 5 . Overall Accuracy, precision, recall and F-measure computed for each fold by formulae given below are summarized in Table III . Figure 6 (a). After combining the two non-COVID-19 pneumonia infections, the overall accuracy of CoroNet increased from 89.5% to 94.59%. Finally, the confusion matrix for the binary classification problem in detecting COVID-19 positive is presented in Figure 6 (b). In addition, precision, recall, F-measure, and accuracy results for all three classification tasks are given in Table V . To check the robustness, we tested our proposed model on another dataset prepared by Ozturk et al. [18] . The dataset-2 contains around 500 normal, 500 pneumonia and 157 COVID-19 chest X-ray images. This dataset contains same COVID-19 X-ray images as in our prepared dataset, however normal and pneumonia X-ray images were collected from ChestX-ray database provided by Wang et al. [26] . After slight modification and fine-tuning, our proposed model achieved an overall accuracy of 90%, The results are illustrated in Table VI and corresponding confusion matrix is given in Figure 7 . Table VI shows accuracy, precision, recall and F-measure of CoroNet on Dataset-2. In this study, we proposed a deep model based on Xception architecture to detect COVID-19 cases from chest X-ray images. The proposed model was tested on two datasets and performed exceptionally well on both of them. Our model achieved an accuracy of 89.5%, 94.59% and 99% Pneumonia and 500 normal chest X-ray images. The promising and encouraging results of deep learning models in detection of COVID-19 from radiography images indicate that deep learning has a greater role to play in fighting this pandemic in near future. Some limitation of this study can be overcome with more in depth analysis which is possible once more patient data (both symptomatic and asymptomatic patients) becomes available. The authors have no conflict of interest to disclose. Overall Accuracy 90.21% Coronavirus: covid-19 has killed more people than SARS and MERS combined, despite lower case fatality rate COVID-19 symptoms Detection of SARS-CoV-2 in different types of clinical specimens Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest Radiography Images Automatic Detection of Coronavirus Disease (COVID-19) Using Xray Images and Deep Convolutional Neural Networks Deep learning in medical image analysis. Annual review of biomedical engineering Radiologist-level pneumonia detection on chest X-rays with deep learning A deep neural network for classification of thoracic diseases on chest radiography Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in X-ray images Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine Detection of coronavirus disease (covid-19) based on deep features. Preprints Automated detection of COVID-19 cases using deep neural networks with X-ray images COVID-19 image data collection Chest X-ray images (pneumonia) Deep learning for content-based image retrieval: A comprehensive study Advances in Deep Learning Xception: Deep learning with depthwise separable convolutions Rethinking the inception architecture for computer vision Chestx-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases