key: cord-288030-69e8cmy2 authors: Ardakani, Ali Abbasian; Kanafi, Alireza Rajabzadeh; Acharya, U. Rajendra; Khadem, Nazanin; Mohammadi, Afshin title: Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks date: 2020-04-30 journal: Comput Biol Med DOI: 10.1016/j.compbiomed.2020.103795 sha: doc_id: 288030 cord_uid: 69e8cmy2 Fast diagnostic methods can control and prevent the spread of pandemic diseases like coronavirus disease 2019 (COVID-19) and assist physicians to better manage patients in high workload conditions. Although a laboratory test is the current routine diagnostic tool, it is time-consuming, imposing a high cost and requiring a well-equipped laboratory for analysis. Computed tomography (CT) has thus far become a fast method to diagnose patients with COVID-19. However, the performance of radiologists in diagnosis of COVID-19 was moderate. Accordingly, additional investigations are needed to improve the performance in diagnosing COVID-19. In this study is suggested a rapid and valid method for COVID-19 diagnosis using an artificial intelligence technique based. 1020 CT slices from 108 patients with laboratory proven COVID-19 (the COVID-19 group) and 86 patients with other atypical and viral pneumonia diseases (the non-COVID-19 group) were included. Ten well-known convolutional neural networks were used to distinguish infection of COVID-19 from non-COVID-19 groups: AlexNet, VGG-16, VGG-19, SqueezeNet, GoogleNet, MobileNet-V2, ResNet-18, ResNet-50, ResNet-101, and Xception. Among all networks, the best performance was achieved by ResNet-101 and Xception. ResNet-101 could distinguish COVID-19 from non-COVID-19 cases with an AUC of 0.994 (sensitivity, 100%; specificity, 99.02%; accuracy, 99.51%). Xception achieved an AUC of 0.994 (sensitivity, 98.04%; specificity, 100%; accuracy, 99.02%). However, the performance of the radiologist was moderate with an AUC of 0.873 (sensitivity, 89.21%; specificity, 83.33%; accuracy, 86.27%). ResNet-101 can be considered as a high sensitivity model to characterize and diagnose COVID-19 infections, and can be used as an adjuvant tool in radiology departments. In December 2019, a new coronavirus disease, termed coronavirus disease 2019 (COVID- 19) , was reported in Wuhan, China [1] . The most common symptoms of COVID-19 are fever, dyspnea, cough, myalgia, and headache [2] . At the time of the study, no specific drug or treatment was available. According to the WHO, all COVID-19 diagnoses must be confirmed by molecular assays, such as the reverse-transcription polymerase chain reaction (RT-PCR) [3] . Besides RT-PCR and, medical imaging, computed tomography (CT) has become a vital method to assist in the diagnosis and management of patients with COVID-19. The prominent chest CT findings of COVID-19 are ground-glass opacity (GGO), multifocal patchy consolidation, and a 'crazy-paving' pattern with a peripheral distribution [4] [5] [6] . The critical role of radiologists is to provide early diagnosis and treatment to distinguish the COVID-19 infection from other conditions, which may have similar findings at CT [4, 5] . For example, GGO is a common finding among other atypical and viral pneumonia diseases such as influenza, severe acute respiratory syndrome (SARS), and Middle East respiratory syndrome (MERS) [5, 7] . However, the specificity and sensitivity of radiologists in diagnosis of COVID-19 were high and moderate, respectively [8] . Hence, further investigations are needed to help and improve the radiologist's performance. In addition, congestion of patients, as well as a high workload of radiologists, which can increase fatigue, affect diagnostic performance [9, 10] . Although for the first time, Lodwick used the term computer-aided diagnosis (CAD) in the literature in 1966 [11] , serious attempts on CAD began in the 1980s [12] . Recently, CAD systems have been an inseparable part of a routine clinical practice that assist radiologists in the process of diagnosis. CAD systems have advantages over radiologists. They are reproducible and detect the subtle changes that cannot be observed by the visual inspection [13] . The CAD systems have been widely used to assist radiologists in detecting lung abnormalities. We have included a brief literature review and report the highlights of computerized methods in lung disease management. The system proposed by Than et al. using Riesz and Gabor transforms obtained an accuracy of 98.73% in detecting lung disease. The accuracy increased to 99.53% with the combination of Riesz, Gabor, fractal dimension, grey level co-occurrence matrix, and grey level run-length matrix based features [14] . Gu et al. indicated that a deep convolutional neural network (CNN) could detect lung nodules with a competition performance metric of 0.7967. They proposed a density-based spatial clustering of applications with noise (DBSCAN) algorithm to improve the detection sensitivity of the network [15] . Also, the CAD system based on a 3D skeletonization feature proposed by Zhang et al. can assist radiologists to differentiate lung nodules from interferential vessels [16] . Transforming an image to the time-frequency domain by the optimal fractional S-transform (OFrST) method can provide meaningful information and features, which can be used to diagnose diseases. This method was applied on lung CT images by Sun et al. to detect and differentiate nodules from the vessels. They employed the Teager-Kaiser energy (TKE) in the time-frequency domain to obtain the energy distribution and characterize lung nodules with 97.87% sensitivity [17] . Sun et al. concluded that the CNN represented higher performance than deep belief network (DBN) and stacked denoising autoencoder (SDAE) in diagnosing malignant lung nodules with an area under the curve (AUC) of 0.899 [18] . Furthermore, CAD systems are a useful tool to monitor lung function after transplantation. In this regard, Barbosa et al. compared the quantitative CT (qCT), pulmonary function testing (PFT), and semi-quantitative image scores (SQS) metrics and found that pulmonary function testing and qCT metrics demonstrated the highest accuracy for monitoring bi-and unilateral lung transplantation with an AUC of 0.771 and 0.817, respectively [19] . The potential of CAD systems can go even further and help radiologists to detect lung abnormalities by computerized analysis of pulmonary sounds [20] and flow and gas concentration of breath [21] . In this study, we propose a CAD system based on deep learning to classify COVID-19 infection versus other atypical and viral pneumonia diseases. We hypothesized that the deep learning method would help radiologists in diagnosing infection related to COVID-19. We used ten well-known pre-trained convolutional neural networks (CNN) to diagnose infections related to the COVID-19. During the COVID-19 pandemic, the entirety of patients representing flu-like symptoms with an initial diagnosis of the novel coronavirus, regardless of demographic values such as age and sex, were included in the study. Prior to enrolment, chest high-Resolution CT (HRCT) examination was obtained from all patients during the acute phase of the disease. The inclusion criterion was the confirmation of diagnosis of COVID-19 through RT-PCR performed on nasopharyngeal swabs samples. Patients with concurrent pulmonary infections, as confirmed by laboratory tests and negative RT-PCR, were excluded. In addition, patients with CT imaging suggestive of chronic lung diseases and subsequent pulmonary involvement were excluded. Imaging studies were performed between 3-6 days from the onset of flu-like symptoms. We retrospectively analyzed the HRCT images of patients from September 2019 to December 2019 with other causes of atypical and viral pneumonia as adenoviral or H1N1 flu from the PACS of our university hospital. HRCT images of all subjects were acquired from a 16-MDCT scanner (Alexion, Toshiba Medical System, Japan) using the high-resolution protocol as follows: patient in the supine position with the arms above the head; 1-to 2-mm slice thickness in increments of up to 10 mm from the lung apices through the hemidiaphragm, at deep inspiration; tube voltage, 120 kVp; tube current-time, 50-100 mAs; pitch, 0.8-1.5, and matrix size, 512×512 pixel. For further analysis, parenchymal window settings were set to a window level and a window width of -600 HU and 1650 HU, respectively. All CT examinations were performed without the use of contrast agent, and images were reconstructed in the transverse plane using a high spatial resolution algorithm. All CT images of patients were converted to the gray-scale, and then reviewed by a radiologist with more than 15 years of experience in thoracic imaging. In this study, ten well-known pre-trained CNN were used to distinguish infection of COVID-19 from non-COVID-19 group: 1-AlexNet, 2-VGG-16, 3-VGG-19, 4-SqueezeNet, 5-GoogleNet, 6-MobileNet-V2, 7-ResNet-18, 8-ResNet-50, 9-ResNet-101, and 10-Xception ( Fig. 3 ). AlexNet is a type of feedforward CNN with 8-layer deep. It contains five convolution layers (conv1 to conv5) and three fully-connected layers (fc6 to fc8) [22] . It was proposed by Alex Krizhevsky and trained on 1 million images to classify images into 1000 different classes (Fig. 3a) . VGG-16 is a combination of five convolutional blocks (13 convolutional layers) and tree fully-connected layers (fc6 to fc8) [23] . It was trained on a million images of 1000 classes (Fig. 3b ). VGG-19 uses 19 layers, including five convolutional blocks (16 convolutional layers) and tree fully-connected layers (fc6-8) [23] . Compared to VGG-16, the VGG-19 is a deeper CNN architecture with more layers (Fig. 3c ). SqueezeNet is a compact CNN with up to 18 learnable layers deep to classify images into 1000 different classes [24] . The network starts with a stand-alone convolution layer followed by eight fire modules and ends with a final convolution layer. A scheme of one sample module is illustrated in Fig 3d. The GoogleNet is a deep model, which is trained on either the ImageNet or Places365 datasets. In this study, we used the GoogleNet trained on the ImageNet dataset [25] . The network is 22-layers deep, starts with three convolution layers, followed by 9 inception blocks, and ends with a fully-connected layer. The inception block as a core part of the GoogleNet is shown in Fig. 3e . MobileNet-V2 is a light-weight CNN with 53 layers deep (52 convolution and one fully connected layers) [26] . The primary part architecture of the network is based on inverted residual and linear bottlenecks. The network starts with three convolution layers, followed by 16 inverted residual and linear bottleneck blocks and ends with one convolution layer and one fully-connected layer. The scheme of the inverted residual block, which is the main part of the network and is illustrated in Fig 3f. ResNet is a type of deep network based on residual learning. This kind of learning can facilitate the training of networks by considering the layer inputs as a reference [27] . All types of ResNet, ResNet-18, ResNet-50, and ResNet-101, are versions of ResNet that have their specific residual block. ResNet-18 with 22-layers are deep, starts with a convolution layer followed by 8 residual blocks and ends with a fully-connected layer. The main components of the residual blocks of ResNet-18 are shown in Fig. 3g . ResNet-50 is similar to ResNet-18 but has different residual blocks scheme and different number (16) of residual blocks that contain in the network (Fig. 3h) . The ResNet-50 contains 50 layers and same is true for ResNet-101. Therefore, it is 101 layers deep with 33 residual blocks (Fig. 3i) . The last network, Xception, is CNN based on depthwise separable convolution layers [28] . It starts with two convolution layers, followed by depthwise separable convolution layers, four convolution layers, and a fully-connected layer. The main component of depthwise separable convolution layers are shown in Fig. 3j . We used a transfer-learning to optimize the CNNs to the datasets. In this regard, the input layer of the CNNs were replaced with a new one, which is consistent with the size of the infection patches (i.e., 60×60×1). In addition, the dimension of the last fully connected layer of all networks was set to the number of the classes, i.e., two ( Table 1) . All networks were trained as follows: optimizer, SGDM; initial learning rate, 0.01; validation frequency, 5. The dataset was shuffled at every epoch, and the training process stopped if the training process did not change significantly. For all networks, the dataset was divided into 80% and 20% for training and validating sets, respectively. The same training and validation datasets were selected for all networks to facilitate the performance comparison of networks. The Kolmogorov-Smirnov test was used to check the normality of all quantitative data. In addition, the age and sex distributions among COVID-19 and non-COVID-19 groups was evaluate by the two-tailed independent samples t-test and chi-square test, respectively. A Pvalue smaller than 0.05 was considered as statistically significant. In order to compare performance of the CNNs and the radiologist, five performance indices were calculated as follows: In this study, the positive and negative cases were assigned to COVID-19 and non-COVID- In this study, 108 patients with laboratory-proven COVID-19 pneumonia (COVID-19 group) including 60 males and 48 females with mean age of 50.22±10.85 (mean age ± SD) were studied. Eighty-six patients with atypical pneumonia diseases including 51 male and 35 female with mean age of 61.45±15.04 were enrolled as controls (nonCOVID-19 group). No significant differences was seen between COVID-19 and non-COVID-19 groups in the term of sex distribution (P>0.05). However, the mean age of COVID-19 group was significantly lower than the non-COVID-19 group (P<0.001, Table 2 ). 1020 image patches including 510 COVID-19 and 510 non-COVID-19 were extracted from CT slices by a specialist radiologist. The dataset was divided into 816 (with 50%-50% distribution), and 102 (with 50%-50% distribution) for training and validating process. In this study, ten well-known CNNs were used to provide a comprehensive view of the role ResNet-101 has an advantage over ResNet-50, the network they used, due to its higher AUC. Unlike us, they fed the whole lung region to their network to differentiate CAP form COVID-19 patients. In their approach, some redundant data such as interferential vessels can be misdiagnosed as pathology [16, 31] . Hence, we used ROI based analysis to increase the performance of the network. In addition, they differentiated COVID-19 from communityacquired pneumonia (CAP) diseases. Differentiating CAP from other atypical and viral pneumonia diseases like COVID-19 is not critical in clinics and CT images. The most critical and important issue for radiologists is differentiating COVID-19 and other atypical and viral pneumonia diseases, which are the same in CT imagery and have similar symptoms [32] . Contrary to their study, we include two groups with similar indications, COVID-19, and other atypical and viral pneumonia diseases, to obviate this problem and help radiologists in diagnostic tasks. Few patients with COVID-19 may have initial negative RT-PCR results [33] . Hence, to eliminate any confounding factors, the non-COVID-19 patients who referred to the department before the COVID-19 outbreak were included to ensure that no patients with COVID-19 exist in the non-COVID-19 group. In contrast, they included both patients with COVID-19 and CAP during the same period, which could introduce bias and affect reliability. Moreover, a clinical method with highest sensitivity and earliest diagnostic ability is not yet confirmed. Currently, two primary modalities, RT-PCR and CT, are used to diagnose COVID-19. In this regard, many studies have utilized CT and RT-PCR to evaluate their diagnostic performances; however, the findings must yet be validated. Bernheim et al. [34] and Yang et al. [35] reported that RT-PCR diagnosed COVID-19 earlier than CT, while in some cases, the CT findings could be normal even ten days after symptoms onset. In contrast, Xie et al. [36] , Ai et al. [37] and Huang et al. [38] found some cases with positive CT findings and an initial negative RT-PCR screening result, which later became positive upon repeating swab tests. On the other hand, in the pandemic phase of COVID-19, requests for CT Exam has substantially increased. The increased workload can affect the diagnostic performance of radiologists [9, 10] . The results of our study indicate that deep learning can characterize the pattern of abnormality effectively, and may help physicians in managing workflow and diagnosing infection related to the COVID-19. The limitations of this study are given below. First, performance of the proposed CAD system was not compared with radiologists. So, future studies should plan to compare the CAD system with radiologists. Second, few patients with COVID-19 may have negative RT-PCR results [39] . Therefore, these patients may have been incorrectly excluded from the present study. RT-PCR is a current standard method for diagnosing of COVID-19. This molecular assay has limitations including time consumption, high cost, shortage of the kit, and requires well-equipped laboratories. These limitations are overwhelming, especially in middle and low-income countries, and may not be effective in reducing the risk of spreading the disease. Furthermore, since health systems are often weaker in lower and middle income countries, they require technical and financial support to better manage and diminish the risk of COVID-19 efficiently [40] . CT as a first-line imaging modality is fast, widely available, and does not waste materials. In this study, we proposed a CAD system based on CT images to improve diagnostic performance, which can implement in any radiology department and also in deprived areas via telecommunication and analyzing the images remotely. Accordingly, the proposed system could reduce radiologist workload burden and could play a role as an auxiliary tool with regard to decision-making as to whether an infection is COVID-19 related. In conclusion, a CAD approach based on CT images with promising potential was proposed to distinguish infection of COVID-19 from other atypical and viral pneumonia diseases. This study showed that the ResNet-101 can be considered as a promising model to characterize and diagnose COVID-19 infections. This model does not impose substantial cost, and can be used as an adjuvant method during CT imaging in radiology departments. Fig. 3 . An overview of the ten pre-trained networks architecture used in this study. In each network, the convolution layers with the same color are in the same size. The straight arrows represent the direction of flow and computation. All convolution layers of the AlexNet, VGG-16 and VGG-19 networks are depicted. However, for the other networks, convolution layers of the most repetitive (core) part are depicted. A Novel Coronavirus from Patients with Pneumonia in China Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China, The Lancet Novel Coronavirus. (2019-nCoV) technical guidance: laboratory guidance Essentials for Radiologists on COVID-19: An Update-Radiology Scientific Expert Panel Coronavirus Disease 2019 (COVID-19): A Perspective from China CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV) Radiographic and CT Features of Viral Pneumonia Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT Monitoring of fatigue in radiologists during prolonged image interpretation using fNIRS Fatigue in radiology: a fertile area for future research Computer-aided diagnosis in radiology. A research plan Computer-Aided Diagnosis in Medical Imaging: Historical Review, Current Status and Future Potential Texture analysis of medical images Lung disease stratification using amalgamation of Riesz and Gabor transforms in machine learning framework Automatic lung nodule detection using a 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs 3D skeletonization feature based computer-aided detection system for pulmonary nodules in CT datasets Spectral analysis for pulmonary nodule detection using the optimal fractional S-Transform Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis Multivariate modeling using quantitative CT metrics may improve accuracy of diagnosis of bronchiolitis obliterans syndrome after lung transplantation Analysis of pulmonary sounds for the diagnosis of interstitial lung diseases secondary to rheumatoid arthritis Novel approach to computerized breath detection in lung function diagnostics Imagenet classification with deep convolutional neural networks Very deep convolutional networks for large-scale image recognition SqueezeNet: AlexNetlevel accuracy with 50x fewer parameters and< 0.5 MB model size Going deeper with convolutions Mobilenetv2: Inverted residuals and linear bottlenecks, Proceedings of the IEEE conference on computer vision and pattern recognition Deep residual learning for image recognition Xception: Deep learning with depthwise separable convolutions Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology Automatic nodule detection for lung cancer in CT images: A review The atypical pneumonias: clinical diagnosis and importance Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT? Relationship to Duration of Infection Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease (COVID-19):A multi-center study in Wenzhou city Chest CT for Typical 2019-nCoV Pneumonia: Relationship to Negative RT-PCR Testing Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases Use of Chest CT in Combination with Negative RT-PCR Assay for the 2019 Novel Coronavirus but High Clinical Suspicion Fact sheet for healthcare providers: CDC -2019-nCoV Real-Time RT-PCR Diagnostic Panel COVID-19: towards controlling of a pandemic None.