key: cord-0963875-792pgs06 authors: Pandit, Mohammad Khalid; Banday, Shoaib Amin; Naaz, Roohie; Chishti, Mohammad Ahsan title: Automatic detection of COVID-19 from chest radiographs using deep learning date: 2020-11-11 journal: Radiography (Lond) DOI: 10.1016/j.radi.2020.10.018 sha: f30c4043aefc2292b4f41ff6c1f80cc2d8929096 doc_id: 963875 cord_uid: 792pgs06 Introduction The breakdown of a deadly infectious disease caused by a newly discovered coronavirus (named SARS n-CoV2) back in December 2019 has shown no respite to slow or stop in general. This contagious disease has spread across different lengths and breadths of the globe, taking a death toll to nearly 700k by the start of August 2020. The number is well expected to rise even more significantly. In the absence of a thoroughly tested and approved vaccine, the onus primarily lies on obliging to standard operating procedures and timely detection and isolation of the infected persons. The detection of SARS n-CoV2 has been one of the core concerns during the fight against this pandemic. To keep up with the scale of the outbreak, testing needs to be scaled at par with it. With the conventional PCR testing, most of the countries have struggled to minimize the gap between the scale of outbreak and scale of testing. Method One way of expediting the scale of testing is to shift to a rigorous computational model driven by deep neural networks, as proposed here in this paper. The proposed model is a non-contact process of determining whether a subject is infected or not and is achieved by using chest radiographs; one of the most widely used imaging technique for clinical diagnosis due to fast imaging and low cost. The dataset used in this work contains 1428 chest radiographs with confirmed COVID-19 positive, common bacterial pneumonia, and healthy cases (no infection). We explored the pre-trained VGG-16 model for classification tasks in this. Transfer learning with fine-tuning was used in this study to train the network on relatively small chest radiographs effectively. Results Initial experiments showed that the model achieved promising results and can be significantly used to expedite COVID-19 detection. The experimentation showed an accuracy of 96% and 92.5% in two and three output class cases, respectively. Implication for practice Its simplicity drives the proposed deep neural network model, the capability to work on small image dataset, the non-contact method with acceptable accuracy is a potential alternative for rapid COVID-19 testing that can be adapted by the medical fraternity considering the criticality of the time along with the magnitudes of the outbreak Conclusion We believe that this study could be used as an initial screening, which can help healthcare professionals to treat the COVID patients by timely detecting better and screening the presence of disease. The brisk spread of novel coronavirus disease (COVID-19) throughout the world has put an unprecedented load on healthcare systems around the globe. It is a highly infectious disease caused due to SARS n-CoV2 (severe acute respiratory syndrome novel coronavirus-2). The disease originated in December 2019 and has affected more than 200 countries worldwide (1). It is declared a pandemic by the world health organization (WHO) (2) . The disease has a mortality rate of 2%, which is due to massive respiratory failure and alveolar damage (3) . The current form of testing, i.e., viral nucleic acid detection using real-time polymerase chain reaction (RT-PCR), is the acceptable form of COVID diagnosis. However, in many countries, especially developing nations where testing kits are not adequate and widespread testing has not started, early, automatic, and cost-effective diagnosis can be crucial for timely monitoring of the spread of disease (4) (5) . Non-contact automated diagnosis systems can prove to be an essential tool in containing the virus spread (even in healthcare professionals) with the timely referral of patients to care facilities and quarantine. Currently, in India, the cost of the RT-PCR test is very high and is out of budget for the majority of the population (6) . Financial constraints arising from diagnostic test cost is a significant concern for patients in major developing nations (7) . Healthcare systems across the globe have evolved in multiple domains to enhance the detection and diagnosis rates with the central aim of being minimally invasive. This underlines the fact that invasive procedures for the detection and diagnosis of diseases should be avoided to a great extent and wherever possible. The inclusion of medical imaging techniques like ultrasound, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Functional Magnetic Resonance Imaging (fMRI), etc. has dramatically changed the way that diseases are detected, diagnosed, examined, or analyzed. Since each of these imaging modalities has a unique underlying physics of operation, each imaging modality is different. Hence, every imaging modality cannot be used for every anatomical site. Among these imaging modalities, the best that suits the study of lungs and its health conditions is the radiograph. The details from a radiograph can aid a radiologist or other relevant healthcare professionals to evaluate lungs to diagnose conditions like a persistent cough, breathlessness, etc. Radiographs can also be utilized to diagnose conditions like emphysema, pneumonia, and cancer. In addition to this, generating radiographs from a subject is relatively straightforward in practice. This makes radiographs suitable for detection and diagnosis in emergency situations. With the advent of these medical imaging modalities, researchers and other professionals have continuously attempted to develop computer-aided systems that act a second opinion to the healthcare experts. There is a plethora of work in literature wherein computer-aided detection and diagnosis systems have been developed. These systems have been developed for numerous applications like the detection of brain tumors, thyroid nodules, ground-glass opacity, Alzheimer's disease, etc. With the help of publicly available datasets of chest radiographs (X-ray images) of COVID-19 patients and healthy cases, the study of automatic COVID detection became possible through the use of radiographs, which demonstrate COVID positive patients, other bacterial pneumonia patients, and healthy patients (8) . Chest radiographs are the universally used imaging technique for diagnosis, and almost all the healthcare facilities, even in remote (underdeveloped) areas, have radiographic imaging as a basic diagnosis system. CT imaging can also be used for COVID-19 detection (9)(10)(11), but due to the non-availability of CT scanners in small healthcare facilities and its time-consuming nature, it prohibits the timely detection and screening of COVID patients. Also, real-time chest radiographic imaging can help to study the progression of the disease, which in turn can help to better screen the patients at different stages of disease (sec-4). Figure 1 demonstrates the radiographs of COVID-19 positive, bacterial pneumonia, and healthy cases. In view of the above-mentioned advantages, a deep learning-based model was developed that can be used to automate COVID-19 detection and screening with high accuracy and sensitivity. This could reduce the number of RT-PCR tests required as only those patients can be sent for viral nucleic tests, which test positive with this model. In recent years deep learning models have been very successful in object detection and classification (12)(13) (14) . In medical image analysis and classification, these models have started to prove very useful and are of great help to doctors, especially radiologists, to detect patterns in medical J o u r n a l P r e -p r o o f images (15)(16)(17)(18)Computer-aided diagnosis (CAD) systems employed with deep learning techniques help professionals to make clinical decisions. Deep learning architectures, especially convolutional neural networks (CNNs), help in automatic feature detection in images (19) . The repeated process learns rich and discriminative features of linear and non-linear transformations at every layer of the CNN model (20) . The network starts with more straightforward features and learns more abstract and discriminative features deeper into the network. This study aimed to utilize state of the art deep learning techniques for automatic COVID-19 detection on chest radiographs to assist in testing and screening of COVID-19 patients. Transfer learning is the strategy by which we can use the knowledge gained while solving some problems and applying it to some different but related issues. Usually, the dataset involved in the new problem is small in order to train the CNN from scratch. Transfer learning involves initially training the deep neural network for a specific task using a large-scale dataset like ImageNet. To obtain useful features extracted from the network, it is usually believed that the dataset must have at least 5000 to 6000 instances per class (21), i.e., data availability is the most important factor for initial training for successfully extracting the significant features. After successfully initial training of the CNN, the network is set to process the new data and extract features from it based on knowledge gained from the initial training. Transfer learning in deep neural networks can be done in two ways. The first method involves feature extraction using transfer learning, where the original CNN model is treated as a feature extractor, and a new classifier is trained on top of that (22) . In this method, the pre-trained model retains its model architecture as well as its learned parameters. The learned features from this model are given to the new classifier learned for the specific task at hand. The second method involves network modification to pre-trained models to obtain better results. Usually, some blocks in these models are replaced with new fine-tuned ones according to the specific task at hand. Mostly fully connected (FC) layers in the original pre-trained model are replaced with a new FC head whose weights are initialized randomly. In order to preserve the rich discriminating filters learned by convolutional (Conv) layers, the convolutional layers are frozen, i.e., backpropagation is not allowed through these layers but only up to FC layers, as their weights are random. This method allows FC layers to start to learn the patterns from highly discriminative and feature-rich convolutional layers. After FC layers have started to learn the patterns of a new dataset, the whole network is allowed to train (unfreeze) with very small learning in order to achieve sufficient accuracy on the new task. In this study, we use the second strategy for training the VGG-16 network for COVID-19 detection. A pre-trained VGG-16 model was used for a classification task (23). VGG-16 is a 16-layer convolutional neural network which consists of 13 convolutional layers and 3 fully connected layers. It also contains 5 max-pooling layers. In two experimental settings, the last dense layer of the network is changed to two classes (COVID and non-COVID) and three class (COVID, non-COVID pneumonia, and normal) output settings. It takes an input image of size 224 X 244 X 3 images and produces a feature vector of 1 X 1 X 4096 at the first dense layer. We freeze the convolutional layers and use a new FC head containing three fully connecting layers with 512, 64, and 2 neurons in two-class cases while 512, 64, In this work, we used the following datasets: 1. We used an open-source database containing chest X-rays of COVID and non-COVID patients (including different diseases like SARS, Streptococcus, etc.) (24). 2. Kaggle chest X-ray competition dataset. (25) A dataset consisting of 1,428 radiographs is created by a data augmentation technique that generates more samples by rotation at five different angles, translation, and flipping (up/down, right/left). We allocated the dataset randomly into training and test sets with 70% from each class (COVID, non-COVID pneumonia, and healthy) as the training set and the remaining 30% as the test set. Regarding the classification, specific metrics are recorded as follows: The main goal of the study is to identify and detect COVID-19 cases; therefore, in three-class cases (COVID, non-COVID pneumonia, and normal (healthy)). TP indicates the correctly classified COVID-19 cases, TN indicates the cases belonging to non-COVID pneumonia or healthy, which are classified as COVID-19. FP shows the cases that actually belong to non-COVID pneumonia or normal but classified as COVID-19, while as FN indicates cases belonging to COVID-19 but classified as normal or non-COVID pneumonia. We use gradient class activation maps (Grad-CAM) (26) The study is aimed at designing the deep neural network-based model to automatically detect the COVID-19 using chest radiographs without explicit feature engineering. Figure 7 shows the timeline of the radiographs of a 50-year-old COVID-19 positive patient for a week (27) . In saliency maps, the highlighted regions represent the regions of interest in the radiograph. This can be a useful tool for doctors to elevate the efficiency of diagnosis based on saliency maps. The frequently observed characteristics are observed in COVID-19 radiographs (28): 1. Ground glass opacities. 2. Broncho vascular thickening. 3. Air space consolidation. J o u r n a l P r e -p r o o f 29) the already saturated hospital systems. The availability of chest radiographs for COVID-19 detection is the limitation of this study as only limited number of samples are present in the datasets. In this study, we utilize the dataset of 1428 chest radiographs, of which 224 radiographs are REFERENCES 1. Coronavirus cause: Origin and how it spreads Rolling updates of COVID-19 Pathological findings of COVID-19 associated with acute respiratory distress syndrome Development of a laboratory-safe and low-cost detection protocol for SARS-CoV-2 of the Coronavirus Disease 2019 (COVID-19) Overcoming the bottleneck to widespread testing: A rapid review of nucleic acid testing approaches for COVID-19 detection Rs 2,400 Per Covid Test Incurs Loss, Hampers Testing Developing countries face diagnostic challenges as the COVID-19 pandemic surges COVID-19 image data collection Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classification Rich feature hierarchies for accurate object detection and semantic segmentation Imagenet classification with deep convolutional neural networks. InAdvances in neural information processing systems Very deep convolutional networks for large-scale image recognition A survey on deep learning in medical image analysis Deep learning of feature representation with multiple instance learning for medical image analysis Manifold learning of brain MRIs by deep learning Radiologist level pneumonia detection on chest x rays with deep learning Pneumonia Detection Using CNN based Feature Extraction Deep Neural Networks for the Internet of Things Deep Learning for Computer Vision What makes ImageNet good for transfer learning A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition COVID-19 image data collection COVID-19 X-rays Kaggle dataset Visual explanations from deep networks via gradient-based localization. InProceedings of the IEEE international conference on computer vision 2017 COVID-19 pneumonia -evolution over a week Automated detection of COVID-19 cases using deep neural networks with X-ray images Detection of coronavirus disease (covid-19) based on deep features. Preprints Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. medRxiv A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19),. medRxiv A Deep Learning System to Screen Novel Coronavirus Disease Deep learning-based detection for COVID-19 from chest CT using weak label. medRxiv Covid-19: automatic detection from X-ray images utilizing transfer learning with convolutional neural networks COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from