key: cord-0542690-xfmonhns authors: Lokwani, Rohit; Gaikwad, Ashrika; Kulkarni, Viraj; Pant, Aniruddha; Kharat, Amit title: Automated Detection of COVID-19 from CT Scans Using Convolutional Neural Networks date: 2020-06-23 journal: nan DOI: nan sha: 60458c46b670c5d2c818046defb94bcfd9a4dafc doc_id: 542690 cord_uid: xfmonhns COVID-19 is an infectious disease that causes respiratory problems similar to those caused by SARS-CoV (2003). Currently, swab samples are being used for its diagnosis. The most common testing method used is the RT-PCR method, which has high specificity but variable sensitivity. AI-based detection has the capability to overcome this drawback. In this paper, we propose a prospective method wherein we use chest CT scans to diagnose the patients for COVID-19 pneumonia. We use a set of open-source images, available as individual CT slices, and full CT scans from a private Indian Hospital to train our model. We build a 2D segmentation model using the U-Net architecture, which gives the output by marking out the region of infection. Our model achieves a sensitivity of 96.428% (95% CI: 88%-100%) and a specificity of 88.39% (95% CI: 82%-94%). Additionally, we derive a logic for converting our slice-level predictions to scan-level, which helps us reduce the false positives. Coronaviruses are a large family of RNA viruses which are usually known to cause respiratory tract illnesses like the common cold. They appear crown-like due to their spiked surface and are categorized into 4 major groups: alpha, beta, gamma, and delta. Most coronaviruses affect animals and can be transmitted between animals and humans [1] . COVID-19 is the latest addition to the list of animal-to-human transmissions, preceded by SARS and MERS. COVID-19 is an infectious disease which has affected more than 6.8 million people in the world as of June 8, 2020. The most common clinical manifestations include fever (83% of patients), cough (82% of patients), and shortness of breath (31% of patients) [2] . The hallmarks of COVID-19 include bilateral distribution of minute patchy shadows and ground-glass opacity in the nascent stages. The progression of this disease is marked by the spread of these opacities and infiltrates to both the lungs [3] . The World Health Organization has published several testing protocols for detecting the disease [4] . The most commonly used reference test for the diagnosis of COVID-19 is the real-time reverse transcription-polymerase chain reaction (RT-PCR) [5] . Reverse Transcription Polymerase Chain Reaction (RT-PCR) tests are the key approach used for diagnosing COVID-19. However, they have a few limitations; their shortcomings include the complex process used for specimen collection, the amount of time required for the analysis, and variability in the accuracy of the tests [6] . Apart from this, a major hurdle in controlling the spread of the disease is the accuracy and shortage of testing kits [7] . Hence, computer-based detection assisted by an expert in the loop with minimal infrastructure is proposed as an alternative to testing kits and vaccines. Computer-aided detection has helped in detecting, localizing, and segmenting out a varied set of diseases using medical imaging analysis. In particular, machine learning is being used for medical imaging analysis by developing deeplearning systems that extract the spatio-temporal representative features from an image, analyze them, and decide the diagnostic outcomes [3] . The most common, economical, and easy-to-use medical imaging and diagnostic technique is chest radiography or chest X-rays. This technique plays an important role in the diagnosis of lung diseases. Expert radiologists use chest X-ray images (CXRs) to detect pathologies like pneumonia, tuberculosis, atelectasis, infiltrates, and early lung cancer [8] . But, detecting COVID-19 using CXRs is challenging due to the less evident visual features in CXRs caused by the overlapping of ribs and soft tissues and low contrast [9] . The limited availability of annotated images adds to the difficulty. The PCR-test is very specific but has a lower sensitivity of 65-95%, which means that the test can be negative even when the patient is infected [10] [11] . These shortcomings can be resolved by using chest CT scans, a cross-sectional imaging modality with high accuracy and speed, instead of CXRs. A recent study of the coronavirus infection on the cruise ship "Diamond Princess" showed evidence of the lung parenchymal pattern (classic for COVID-19) on CT studies of the chest in 54% of the asymptomatic cases [12] . Most of the recent literature reported that COVID-19positive patients had characteristic features highly evident in the CT scan images [13] . These features included different degrees of ground-glass opacities with or without crazypaving sign, multifocal organizing pneumonia, and architectural distortion in a peripheral distribution [11] . COVID-19 eventually develops into chronic pneumonia, and thus the visual symptoms it has are much similar to bacterial and viral pneumonia. In CT scans, the ground-glass opacities are more similar to consolidation [3] . Studies have proven that chest CT has a higher sensitivity for the diagnosis of COVID-19 as compared with RT-PCR tests taken from swab samples [11] . To curb human-to-human transmission and isolate the affected from the healthy, it is essential to detect the presence of COVID-19 at an early stage. This is where CT assists in the detection of minor infections [14] . In this paper, we propose a prospective method using neural networks wherein our model helps in identifying the pixels showing COVID-19 infected regions in a CT scan and helps in marking a patient under consideration as either COVID-19-positive or negative [15] . In the past few years, deep learning has evolved as a technique with its capabilities extending from classification and object detection to segmentation in medical image analysis. Some studies showed better results than expert radiologists. Rajpurkar et al. [16] proposed and presented a DenseNET-121 model for pneumonia detection which performed binary classification on CXRs using CNNs. Qin et al. [8] proposed pneumonia and pulmonary edema classification by extracting texture features. Parveen et al. [17] used an FCM clustering algorithm to detect pneumonia, where they showed that the lung area of the chest was low in black or dark gray when it became infected with pneumonia. Recently, there have been many developments in detecting COVID-19 from CXRs and CTs. Xu et al. [18] proposed a 3-D deep learning model that categorized CTs as either COVID-19 pneumonia-positive or viral pneumonia-positive. They trained a location-attention classification model and used the predicted probabilities to give a prediction calculated by a Bayesian function. Chen et al. [19] built a model using UNet++ [20] , a powerful architecture for medical image segmentation, and used a 3-consecutive slice and quadrant based post-processing approach to mark a scan as positive or negative. This post-processing approach helped them reduce the number of false positives. Several studies have addressed diagnosis as a binary classification problem, i.e. healthy vs. COVID-19-positive [6] . For example, Wang et al. [3] used a modified Inception neural network architecture and attained an accuracy of 79.3%. Szegedy et al. [21] trained on the cropped regions of interest identified by radiologists and distinguished the healthy patients from COVID-19-positive patients. Several other approaches used a 3-category classification approach, differentiating healthy patients from pneumonia and COVID-19. Xu et al. [18] used classical ResNet architectures, adding fully-connected layers at the end, and took the classification approach to solve the problem. He et al. [22] used ResNets for feature extraction, and Song et al. [23] used the Feature Pyramid Networks [24] , which are the backbone in U-Nets, for learning fine-grained features in the images. Shan et al. [25] developed a deep learning system that automatically quantified infection regions of interest (ROIs) and their volumetric ratios with respect to the lung. Li et al. [26] put forth a 3D deep learning model, referred to as ConvNet, wherein they combined the 2D local and 3D global features using a max-pooling operation and predicted the class using the probability score from the softmax activation. Jianpeng et al. [9] proposed a deep-learning architecture to differentiate COVID-19 from non-COVID-19 cases from CXRs. Their model is composed of three components: a backbone network, a classification head, and an anomaly detection head. The backbone network extracts the high-level features and feeds it to the rest of the heads. Gurujit et al. [27] identified an intrinsic COVID-19 genomic signature and used it together with a machine learningbased alignment-free approach for an ultra-fast, scalable, and highly accurate classification of whole COVID-19 genomes. We used COVID-19-positive and non-COVID data from GitHub [28] and consolidation and healthy CT scans from a private Indian hospital. The data obtained contained 275 CT scans labeled as COVID-19-positive. The ground truth in these images was decided on the basis of their RT-PCR test results. These CT images had different sizes from 143 patient cases [7] . In total, the data contained 5212 slices and was split into training, validation, and test sets. The prevalence of positive cases in each of the sets was kept at 20%. As the available open-source data had varied resolutions, we decided to fix our input size to 512x512 pixels. The original images were in the unsigned int8 format, in the range of [0, 255]. We converted these images to floating-point 16, in the range of [0, 1]. The output masks were in the binary form [0, 1] at pixel-level, where 1s indicated the region of interest. Table 1 shows the detailed distribution of data. Training 657 2628 3285 Validation 120 477 597 Test 266 1064 1330 Table 1 : Slice-level Dataset splits The CT slices were annotated, classified, and marked positive by a group of trained expert radiologists. The positive CT slices had typical findings including bilateral pulmonary parenchymal ground-glass and consolidative pulmonary opacities, sometimes with a rounded morphology and a peripheral lung distribution [29] . Ground-glass opacification was defined as hazy increased lung attenuation with preservation of bronchial and vascular margins, and consolidation was defined as opacification with obscuration of margins of vessels and airway walls [30] . Notably, lung cavitation, discrete pulmonary nodules, pleural effusions, and lymphadenopathy were marked as negative. Our radiologists used an open source tool called VIA [31] for annotating the images, as it supports various shapes, including ellipses, circles, and polygons, for marking the ROI. figure 1 shows an example of the annotation. In this section, we give a brief overview of our training and the inference algorithms. We used U-Net [32] for medical image segmentation, which uses the concept of deconvolution [33] . U-Nets are built on the architecture of fully convolutional networks. The most important property of U-Net is the shortcut connections between the layers of equal resolution in the encoder path and the decoder path. These connections provide essential highresolution features to the deconvolution layers [15] . Here, we used Xception [34] as the encoder for U-Net. We used transfer learning by fine-tuning a network pretrained on CXRs for the same problem but a different task [35] . Transfer learning is proven to give better performance when the tasks of source and target network are more similar, and yet even transferring the weights of far and distant tasks has been proved to be better than random initialization [36] . Here, we have tried to solve the problem of distinguishing COVID-19 cases from non-COVID-19 by using weights from our COVID-19 vs Healthy model, as pre-trained weights for this model already gave a sensitivity of 0.9 with a specificity of 0.8. Initially, we built a CT model for consolidation vs healthy and later fine-tuned our model for COVID-19 vs non-COVID-19. In the training stage, we use binary cross entropy as the loss function and the standard Adaptive Adam Optimizer with a batch size of 4. We set the max epochs to 50 and set the learning rate to 10 −4 , which is decayed on the plateau after patience of 4 epochs. We resize each training image to a fixed size of 512 × 512 pixels. To alleviate the overfitting of our model on the training data from a particular source, we try to include data from varied sources. One of the drawbacks of having a 2D CT model is that the inference tends to be slow. Our model has a sensitivity of 0.964, hence we plan to use specific slices for inference. We tested our model using varied sets of data from different sources. We initially evaluated the model on our test set, consisting of 1330 images, in which COVID-19-positive samples had a prevalence of 20%. Our model gave a sensitivity of 0.963 (95% CI: 0.94-0.98) and a specificity of 0.936 (95% CI: 0.92-0.95). The dice coefficient on positive samples was Apart from this, we evaluated the model on a total of 140 scans with a prevalence of 20% for positive cases. These scans were tested on data from three sources. One source contained Italian and Chinese scans, while the remaining came from two separate private Indian hospitals. After passing these images through our model, we sorted the slices as per the position of the slice in the CT scan. We observed a pattern wherein the consecutive slices had the same predictions which is expected from a radiology perspective. figure 4 provides an example of the predictions for a positive CT scan. Here we see the expected pattern of consecutive slices, predicted as positive by the model. Hence, we convert the slice-level prediction to scan-level prediction using the logic that if 15 consecutive slices in a scan are marked as positive, then we mark the scan as positive. The diagnosis of COVID-19 using CXRs and CT scans has gained significance since the ubiquitous spread of this disease. Given the predominance of ground-glass opacities, chest CT scans usually tend to show the region of infection more clearly than CXRs [1] . Our current implementation is a 2D model built at slice-level. Since a CT study could have the number of slices running into thousands, this 2D model certainly adds to the time complexity of processing the whole scan. Although we are satisfied with the performance our model currently shows on the data from diverse distributions, given the time complexity, deploying the model in production is a challenge. In the future, we plan to implement a 3D model that will take the whole CT scan as input and give out masks for the infected areas. The primary challenge with this approach will be the requirement of a lot of annotated data to give a good performance. Additionally, we propose a model that differentiates between COVID-19 and chronic and viral pneumonia and address the challenges associated with it, like fine-grained, accurate annotations and large amounts of data for all the specified categories. In conclusion, chest CT has proved to have a higher sensitivity than RT-PCR tests [11] . Our analysis suggests that chest CT can be considered for COVID-19 screening and evaluation, especially in epidemic situations where the spread is uncontrollable, and diagnosis needs to be done with celerity. Chest imaging appearance of covid-19 infection Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study A deep learning algorithm using ct images to screen for corona virus disease A precision medicine approach to managing 2019 novel coronavirus pneumonia A novel coronavirus disease (covid-19) outbreak as a pandemic crisis Mapping the landscape of artificial intelligence applications against covid-19 Covid-ctdataset: a ct scan dataset about covid-19 Computer-aided detection in chest radiography based on artificial intelligence: a survey Covid-19 screening on chest x-ray images using deep learning based anomaly detection Sensitivity of chest ct for covid-19: comparison to rt-pcr Correlation of chest ct and rt-pcr testing in coronavirus disease 2019 (covid-19) in china: a report of 1014 cases Chest ct findings in cases from the cruise ship Chest ct for typical 2019-ncov pneumonia: relationship to negative rt-pcr testing Lung pattern classification for interstitial lung diseases using a deep convolutional neural network Deep learning techniques for medical image segmentation: Achievements and challenges Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning Detection of pneumonia in chest x-ray images Deep learning system to screen coronavirus disease 2019 pneumonia. arxiv 2020 Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: a prospective study Unet++: A nested u-net architecture for medical image segmentation Going deeper with convolutions Deep residual learning for image recognition Deep learning enables accurate diagnosis of novel coronavirus (covid-19) with ct images Feature pyramid networks for object detection Lung infection quantification of covid-19 in ct images with deep learning Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct Machine learning using intrinsic genomic signatures for rapid classification of novel pathogens: Covid-19 case study Covid-ctdataset: a ct scan dataset about covid-19 Ct imaging features of 2019 novel coronavirus (2019-ncov) Fleischner society: glossary of terms for thoracic imaging The vgg image annotator (via) U-net: Convolutional networks for biomedical image segmentation Visualizing and understanding convolutional networks Xception: Deep learning with depthwise separable convolutions Deep convolutional neural networks for computeraided detection: Cnn architectures, dataset characteristics and transfer learning How transferable are features in deep neural networks?