key: cord-0782816-vst2v5tw authors: Asif, S.; Wenhui, Y. title: Automatic Detection of COVID-19 Using X-ray Images with Deep Convolutional Neural Networks and Machine Learning date: 2020-05-06 journal: nan DOI: 10.1101/2020.05.01.20088211 sha: 03499759c304ea827ecb34f6de18ad01bda688b7 doc_id: 782816 cord_uid: vst2v5tw The COVID-19 pandemic continues to have a devastating effect on the health and well-being of the global population. A vital step in the combat towards COVID-19 is a successful screening of contaminated patients, with one of the key screening approaches being radiological imaging using chest radiography. This study aimed to automatically detect COVID-19 pneumonia patients using digital chest x-ray images while maximizing the accuracy in detection using deep convolutional neural networks (DCNN). The dataset consists of 864 COVID-19, 1345 viral pneumonia and 1341 normal chest x-ray images. In this study, DCNN based model Inception V3 with transfer learning have been proposed for the detection of coronavirus pneumonia infected patients using chest X-ray radiographs and achieved more than 96% accuracy. The results demonstrate that transfer learning proved to be effective, showed robust performance and easily deployable approach for COVID-19 detection. continuous clinical preliminaries assessing potential medicines. About 3,401,002 infected cases are confirmed in more than 210 countries until 1 st May 2020, where 239,602 deaths, 1,081,599 recovered, 2,028,446 mild and 51,355 critical cases were found [3, 4] . It has been expressed that so as to battle with the spreading of COVID-19 sickness compelling screening of patients and prompt clinical reaction for the contaminated patients is a crying need. The highest quality level screening strategy utilized for testing the COVID-19 patients is the Reverse Transcription Polymerase Chain Response (RT-PCR) test on respiratory examples [5] . This procedure is the most generally utilized strategy for testing for COVID-19 identification however is a manual, confused, relentless and time-consuming process with a positivity rate of only 63 % [5] . The other diagnosis tools of COVID-19 can be clinical symptoms investigation, epidemiological history and positive radiographic images (computed tomography (CT)/Chest radiograph (CXR)) as well as positive pathogenic testing. The clinical attributes of serious COVID-19 contamination are that of bronchopneumonia causing fever, hack, dyspnea, and respiratory failure with acute respiratory distress syndrome (ARDS) [6] [7] [8] [9] . Promptly accessible and radiological imaging is another major symptomatic instrument for COVID-19. Most of COVID-19 cases have comparable highlights on radiographic pictures including reciprocal, multi-focal, ground-glass opacities with a fringe or back dissemination, primarily in the lower projections, in the early stage and pulmonary consolidation in the late stage [9] [10] [11] [12] [13] [14] [15] . Although typical CXR images may help early screening of suspected cases, the pictures of different viral cases of pneumonia are comparative and they overlap with other infectious and inflammatory lung diseases. Therefore, it is hard for radiologists to recognize COVID-19 from other viral pneumonia. The side effects of COVID-19 being like viral pneumonia can at times lead to wrong determination in the present circumstance, where medical clinics are over-burden and working nonstop. Therefore, incorrect diagnosis can prompt a non-COVID viral Pneumonia being falsely marked as exceptionally suspicious of having COVID-19 and in this manner deferring in treatment with resulting costs, exertion and danger of presentation to positive COVID-19 patients. Currently many biomedical complications (e.g., brain tumor detection, breast cancer detection, etc.) are using Artificial Intelligence (AI) based solutions [16] [17] [18] [19] . Deep learning techniques can reveal image features, which are not apparent in the original images. In particular, Convolutional Neural Network (CNN) has been demonstrated amazingly helpful in include extraction and learning and therefore widely adopted by the research community [20] . CNN was utilized to improve picture quality in low-light pictures from a high-speed video endoscopy [21] and was additionally applied to distinguish the idea of aspiratory knobs through CT pictures, the conclusion of pediatric pneumonia by means of chest X-ray pictures, robotized marking of polyps during colonoscopy recordings, cryptoscopic picture acknowledgment extraction from recordings [22] [23] [24] [25] . Machine learning techniques on chest X-Rays are getting popularity as they can be easily utilized with low-cost imaging techniques and there is an abundance of data available for training different machine-learning models. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. . The test of COVID-19 is currently a difficult task because of inaccessibility of diagnosis system everywhere, which is causing panic. Because of the limited availability of COVID-19 testing kits, we have to depend on different determination measures. Since COVID-19 assaults the epithelial cells that line our respiratory tract, we can utilize X-rays to investigate the strength of a patient's lungs. The medical practitioner frequently uses X-ray images to analyze pneumonia, lung inflammation, abscesses, and enlarged lymph nodes. And almost in all hospitals have X-ray imaging machines, it could be possible to use Xray's to test for COVID-19 without the dedicated test kits. Again, a drawback is that X-ray examination requires a radiology master and takes huge time, which is valuable when people are sick around the world. Therefore, developing an automated analysis system is essential to save medical professionals valuable time. Recently presented a DCNN, called COVID-Net for the detection of COVID-19 cases from the chest X-ray images with an accuracy of 83.5%. Ayrton [36] used a small dataset of 339 images for training and testing using ResNet50 based deep transfer learning technique and reported the validation accuracy of 96.2%. In this study, we have developed an automatic detection of COVID-19 using a DCNN based Inception V3 model and Chest X-ray images. This paper proposes advanced deep learning approach to predict the COVID-19. The proposed work is implemented with TensorFlow and Inception V3 pre-trained models that was trained to classify normal, viral and COVID-19 pneumonia images and tested on Chest X-ray images and obtained more than 96% accuracy. In order to control the spread of COVID-19, a large number of suspected cases need to be screened for proper isolation and treatment. Pathogenic research facility testing is the indicative best quality level however it is tedious with noteworthy bogus negative outcomes. Quick and precise analytic strategies are desperately expected to battle the sickness. In light of COVID-19 radiographical changes in X-ray pictures, we meant to build a deep learning method that could extract COVID-19's graphical features so as to give a clinical analysis in front of the pathogenic test, thus saving critical time for disease control. In this paper, (DCNN) [37], a machine learning classification technique is used to classify the Chest X-ray images. As accuracy is the most significant factor in this issue, by taking a more prominent number of pictures for training the network and by increasing the number of iterations, the DCNN accuracy can be improved. Tensor Flow is a large-scale machine learning system developed by Google [38] and Inception V3 is Google's CNN architecture [39] . Here, the DCNN algorithm is executed with Tensor Flow and Inception V3. DCNN typically perform better with a larger dataset than a smaller one. Transfer learning can be beneficial in those applications of CNN where the dataset is not large. The idea of transfer All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. Transfer learning is a machine learning technique [39] which is based on the concept of reusability Transfer learning is often used with CNN in the way that all layers are kept except the last one, which is trained for the specific problem. This technique can be particularly useful for medical applications since it does not require as much training data, which can be hard to get in medical situations. In the analysis of medical data, one of the biggest difficulties faced by researchers is the limited number of available datasets. Deep learning models often need a lot of data. Labeling this data by experts is both costly and time consuming. The biggest advantage of using transfer learning method is that it allows the training of data with fewer datasets and requires less calculation costs. With the transfer learning method, which is widely used in the field of deep learning, the information gained by the pre-trained model on a large dataset is transferred to the model to be trained. Fig. 2 defines the Inception V3 model which performs convolution, pooling, softmax and fully connected procedures. Here a pre-trained neural network established for one task can be utilized as the initial point of another task. The Inception-v3 architecture comprises two fragments: 1) Use the feature extraction section of the convolutional neural network. All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20088211 doi: medRxiv preprint 2) Classification section utilizing fully-connected and softmax layers. It is an artificial neural network there are more than three layers, shown in Fig. 3 . It has single input, single output and many invisible layers [38] . To use transfer learning for classifying chest X-ray images, we used the TensorFlow library [38] to load the Inception V3 model on our local machine, retrain it on the chest X-ray dataset and then classify new images to be one of the three categories normal, viral pneumonia and COVID-19. It is a deep learning framework established by Google that can control all neurons (nodes) in the system and has a library appropriate for image processing. Neural network weights can be changed to improve performance. In this study, we built DCNN based InceptionV3 model for the classification of COVID-19 Chest X-ray images to normal, viral pneumonia and COVID-19 classes. In addition, we applied All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20088211 doi: medRxiv preprint transfer learning technique that was realized by using ImageNet data to overcome the insufficient data and training time. The schematic representation of conventional CNN including InceptionV3 model for the prediction of COVID-19 patients, viral pneumonia and normal were depicted in Fig. 4 . Chest X-ray images are taken as input, Inception V3 is applied, convolution, pooling, softmax, and fully connected processes are performed. Upon completing these tasks, they are classified according to different training modules and eventually classified as normal, viral pneumonia and COVID-19 classes. Inception V3 is one of the states of art architectures in image classification challenge. The best network for medical image analyses seems to be the Inception V3 architecture and it preforms better than even the more recent architectures. So, we selected Inception V3 model that is implemented using TensorFlow and hence the retraining is done with TensorFlow. The steps for classification using the proposed work are as follows: Step 1: Start Step 2: Create list of images // start training the model Step 3: Provide a directory for storing the bottleneck value of each image Step 4: Provide inference to the images // to create bottleneck values Step 5: Create a folder for all images of bottleneck values Step 6: Generate bottleneck values for each individual image Step 7: Create new softmax layers and fully connected layers // end of training Step 8: Test new image // input chest x-ray image to get the result Step 9: Finish. nt ng re n, ks, al, ral est ms is All rights reserved. No reuse allowed without permission. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. . In this study, we had presented a novel method that could screen COVID-19 fully automatically by DCNN. Chest X-ray images have been used for the prediction of COVID-19 infected patients. Popular pre-trained model Inception V3 has been trained and after training, the model was experienced on chest X-ray images of COVID-19, normal and viral pneumonia that are not used in the training phase. We have obtained the best performance as an accuracy of more than 96%. The experimental work is done by connecting the docker to the virtual box. The neural network is trained to create bottleneck values. As shown in Fig. 5. (A) , create a bottleneck for individual image and save it to a folder. After all the bottlenecks have been created, the last few layers of the model are trained. We have set the training steps to 4000, which is sufficient to generate sufficient outcomes. There are a series of accuracy, cross-entropy, and verification accuracy as shown in Fig. 5. (B) . Training accuracy indicates a correctly labeled share of the current training image. Cross-entropy indicates how finely the training is performed on the cross-entropy loss function, and validation accuracy is related to training accuracy. After 4000 iterations, the final test accuracy is 96.9%. was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. To provide more human-interpretable explanations, we conducted several experiments on the chest X-ray images to evaluate the classification performance of the network investigated, let's consider the following examples. Example 1: the CXR image is classified to contain a confirmed COVID-19 case with a probability of 99.59%, the true class is COVID-19, as shown in Fig. 6. (A) . Example 2: the CXR image is classified to contain a confirmed COVID-19 case with a probability of 98.30%, the true class is COVID-19, as shown in Fig. 6. (B) . shows the results when the sample test image was taken. Since COVID-19 has a higher score compared to normal and viral pneumonia, the test image is classified as COVID-19. It can be concluded that the proposed technique can classify COVID-19 X-ray images very reliably. Early prediction of COVID-19 patients is important to avoid spreading the disease to different people. In this study, we proposed a deep transfer learning-based approach the use of chest X-ray images obtained from COVID-19 patients, normal and viral pneumonia for automatic detection of COVID-19 pneumonia. The proposed classification model for the detection of COVID-19 achieved more than 96% accuracy. In the light of our findings, it's far believed that it's going to help medical doctors to make decisions in scientific practice due to the high overall performance. In order to come across COVID-19 at an early stage, this study gives insight on how deep transfer learning methods can be used. COVID-19 has already become a danger to the world's healthcare system and thousands of people have already died. Deaths were initiated by way of respiration failure, which ends up in the failure of other organs. Since a big range of sufferers was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint (which this version posted May 6, 2020. . attending out-door or emergency, doctor's time is limited and computer-aided-analysis can save lives via early screening and proper-care. Inception V3 model exhibits an excellent performance in classifying COVID-19 pneumonia by effectively training itself from a comparatively lower collection of images. We believe that this computer-aided diagnostic tool can significantly improve the speed and accuracy of diagnosing cases with COVID-19. This could be highly useful in a pandemic, where the burden of disease and the need for preventive measures do not match the availability of resources. WHO Director-General opening remarks at the media briefing on COVID-19 -11 Global COVID-19 report Coronavirus COVID-19 Global Cases by the Detection of SARS-CoV-2 in Different Types of Clinical Specimens Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia Clinical features of patients infected with 2019 novel coronavirus in Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Molecular diagnosis of a novel coronavirus (2019-nCoV) causing an outbreak of pneumonia Recent advances in the detection of respiratory virus infection in humans CT imaging features of 2019 novel coronavirus (2019-nCoV) Radiology perspective of coronavirus disease 2019 (COVID-19): lessons from severe acute respiratory syndrome and Middle East respiratory syndrome Wearable real-time heart attack detection and warning system to reduce road accidents Real-Time Smart-Digital Stethoscope System for Heart Diseases Monitoring How far have we come? Artificial intelligence for chest radiograph interpretation Imagenet classification with deep convolutional neural networks Low-light image enhancement of high-speed endoscopic videos using a convolutional neural network Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses Identifying medical diagnoses and treatable diseases by image-based deep learning Application of artificial neural networks for automated analysis of cystoscopic images: a review of the current status and future prospects Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy Sohaib Asif were involved in the study conception, collected the datasets, interpreted the data and performed experiments; Sohaib Asif designed the system architecture and data infrastructure, training, testing setup and statistical study; Sohaib Asif wrote the manuscript and revised the draft critically and Yi Wenhui is the corresponding author, contributing to the conception and design of this research. The authors of this manuscript no conflict of interest and relationships with any companies, whose products or services may be related to the subject matter of the article. Datasets used in the experiments were obtained from J. C. Monteral. (2020). COVID-Chest Xray Database (https://github.com/ieee8023/covid-chestxray-dataset), Italian Society of Medical and Interventional Radiology (https://www.sirm.org/category/senza-categoria/covid-19/) and COVID- 19 Radiography Database (https://www.kaggle.com/tawsifurrahman/covid19radiography-database). All data needed to evaluate the conclusions in the paper are present in the paper. Additional data related to this paper may be requested from the authors.