key: cord-0704405-e9id36xq
authors: İn, Erdal; Geçkil, Ayşegül A.; Kavuran, Gürkan; Şahin, Mahmut; Berber, Nurcan K.; Kuluöztürk, Mutlu
title: Using artificial intelligence to improve the diagnostic efficiency of pulmonologists in differentiating COVID‐19 pneumonia from community‐acquired pneumonia
date: 2022-05-02
journal: J Med Virol
DOI: 10.1002/jmv.27777
sha: 194b1e7f34659f5cf2d8a0e4baa11099901941f8
doc_id: 704405
cord_uid: e9id36xq

Coronavirus disease 2019 (COVID‐19) has quickly turned into a global health problem. Computed tomography (CT) findings of COVID‐19 pneumonia and community‐acquired pneumonia (CAP) may be similar. Artificial intelligence (AI) is a popular topic among medical imaging techniques and has caused significant developments in diagnostic techniques. This retrospective study aims to analyze the contribution of AI to the diagnostic performance of pulmonologists in distinguishing COVID‐19 pneumonia from CAP using CT scans. A deep learning‐based AI model was created to be utilized in the detection of COVID‐19, which extracted visual data from volumetric CT scans. The final data set covered a total of 2496 scans (887 patients), which included 1428 (57.2%) from the COVID‐19 group and 1068 (42.8%) from the CAP group. CT slices were classified into training, validation, and test datasets in an 8:1:1. The independent test data set was analyzed by comparing the performance of four pulmonologists in differentiating COVID‐19 pneumonia both with and without the help of the AI. The accuracy, sensitivity, and specificity values of the proposed AI model for determining COVID‐19 in the independent test data set were 93.2%, 85.8%, and 99.3%, respectively, with the area under the receiver operating characteristic curve of 0.984. With the assistance of the AI, the pulmonologists accomplished a higher mean accuracy (88.9% vs. 79.9%, p < 0.001), sensitivity (79.1% vs. 70%, p < 0.001), and specificity (96.5% vs. 87.5%, p < 0.001). AI support significantly increases the diagnostic efficiency of pulmonologists in the diagnosis of COVID‐19 via CT. Studies in the future should focus on real‐time applications of AI to fight the COVID‐19 infection.

The pandemic caused by the coronavirus disease 2019 (COVID- 19) resulted in severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a new type of human coronavirus, has become the focus of worldwide attention. Since its first report in Wuhan, China, at the end of December 2019, COVID-19 has spread aggressively around the world, significantly affecting people's health and daily life. 1, 2 As of March 24, 2022, a total of over 477 million cases of COVID-19 were recorded and the worldwide death rate was more than 6.1 million. 3 Community-acquired pneumonia (CAP) covers the infection of the pulmonary parenchyma, which is acquired outside of the healthcare setting. CAP is also one of the primary causes of morbidity and mortality worldwide. Although bacterial infections are responsible for the majority of CAP, viral infections are also common. 4, 5 In terms of the clinical symptoms, CAP and COVID-19-associated pneumonia share common characteristics. 6 The gold standard method for diagnosing COVID-19 is reverse transcriptase-polymerase chain reaction (RT-PCR), which aims to reveal the RNA of the virus in respiratory samples such as bronchial aspirates or nasopharyngeal swabs.

However, in cases where the amount of viral genome is not sufficient or the correct time window of viral replication is missed, this test may lead to false-negative results. In addition, the RT-PCR is a timeconsuming process and there may be shortages of assay kits, especially in periods when the infection is very common. [7] [8] [9] Equipment for computed tomography (CT) is rather common around the world and readily available in many hospitals. Also, the screening process is relatively simple and fast, which can enable suspected patients to be quickly screened for COVID-19. Thorax CT has been shown to have a higher sensitivity than RT-PCR samples in the diagnosis of COVID-19.

Thus, thorax CT becomes a major factor in the early detection and treatment processes of COVID-19 pneumonia. [10] [11] [12] The concept of artificial intelligence (AI) is a popular topic in medical imaging. It certainly revolutionized the present diagnostic systems, especially those involved in imaging. Furthermore, the advancements in deep learning methods, specifically in the utilization of convolutional neural networks (CNNs), have enabled notable performance developments compared to the standard machine learning techniques. Currently, the utilization of AI in thoracic imaging facilitates diagnostic practices, such as the evolution of pulmonary nodules, detection of interstitial lung diseases, and diagnoses of tuberculosis and pneumonia. 13 Recently, certain studies investigated the efficiency of AI in the diagnostic processes for COVID-19 pneumonia and reported high diagnostic outputs in the related applications. [14] [15] [16] [17] It has also been shown that an AI-based quantitative CT analysis can be an objective tool in demonstrating the severity of the disease. 18 In two recent studies on thorax CT images, it was found that AI support increased the radiologist's performance in differentiating COVID-19 pneumonia and contributed to the diagnostic process. 19, 20 However, COVID-19 is mainly a respiratory disease, and patients often present to the hospital with pulmonary symptoms. The contribution of AI to pulmonologists in the diagnosis of COVID-19 has not been analyzed so far. To detect and properly manage all cases of COVID-19 pneumonia, it is vital to create test methods to distinguish the disease from the other causes of pneumonia detected on CT. In this context, AI applications can make a significant contribution to pulmonologists in the diagnosis of COVID-19.

This retrospective study aims to evaluate the effectiveness of AI application in differentiating COVID-19 pneumonia from other pneumonia using thorax CT images and to analyze the contribution of this system to the diagnostic performance of pulmonologists.

Patients who applied to the pandemic or chest diseases outpatient clinics of our hospital between the specified dates and met the study criteria were included in this retrospective study. Patients older than 18 were included in the study and no gender difference was regarded between the patients. Patients were analyzed retrospectively using the hospital electronic record system. The current study was conducted in a tertiary university hospital. The hospital serves as the primary referral center in the region for COVID-19 patients. The diagram containing the design from the study is given in Figure 1 .

All the scans in the study were obtained via a 16-slice multidetector scanner (Philips Medical Systems), covering the following parameters: 120 kV, 250 mA, reconstruction matrix of 512 × 512, slice thickness of 0.625 mm, and high spatial resolution algorithm. The axial CT images were obtained craniocaudally at full inspiration with the patient in the supine position and covered the body parts from the thoracic inlet to the diaphragm. All images were viewed on lung setting (width, 1500 HU; level, −700 HU).

Thorax CT images of the study were evaluated independently by four pulmonologists, who had thoracic imaging experiences for at least 8 years. In thorax CT scans, to exclude the extrapulmonary sites, the lung was manually segmented. Then, the whole data set was subjected to preliminary processing by adjusting the width of the CT window and the level of the lung window. Furthermore, the lesion sections in COVID-19 or CAP patients were labeled manually and utilized as references for the training of the deep neural network of the AI.

The design and assessment of the proposed decision support architecture for detecting COVID-19 pneumonia include the follow- The second session was held at least one day after the first session.

The order of the CT images in the test set was changed in the second session.

In this study, a confusion matrix is used to visualize the performance of the proposed method and pulmonologists for the statistical classification problem. Additionally, we evaluated the models' accuracy and robustness using metrics such as sensitivity, specificity, precision, F1 score, and Matthew Correlation Coefficient (MCC). 3 | RESULTS

Total of 553 COVID-19 and 334 CAP patients were enrolled in the study. The mean age of COVID-19 patients was 66.3 ± 14.9 years, while the mean age of CAP patients was 67.9 ± 16.8 years. There was no statistically significant difference in age and gender between the two groups (p > 0.05 for both) ( Table 1 ). Table 2 .

The model with the selected features from the fc7 layer achieved a total accuracy of 93.2% for the test data set. For the rest of the total performance metrics, the sensitivity, specificity, precision, F1 The detailed classification results of the pulmonologist for the test set with and without AI assistance are given in Table 3 

In conclusion, in this study, the results revelated that AI support 

The authors do not have any financial and personal relationships with other people or organizations that could inappropriately influence (bias) their work. The authors have no conflicts of interest to declare.

The data that support the findings of this study are available from the corresponding author on reasonable request.

İn http://orcid.org/0000-0002-8807-5853

Ayşegül A. Geçkil https://orcid.org/0000-0003-0348-3194

Nurcan K. Berber https://orcid.org/0000-0001-8634-2543

Mutlu Kuluöztürk http://orcid.org/0000-0003-2749-9166

Coronaviridae Study Group of the International Committee on Taxonomy of Viruses. The species severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2

Understanding of COVID-19 based on current evidence

Johns Hopkins Coronavirus Resource Center

Update in adult community-acquired pneumonia: key points from the new American Thoracic Society/ Infectious Diseases Society of America 2019 guideline

Diagnosis and treatment of adults with community-acquired pneumonia

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Combination of RT-qPCR testing and clinical features for diagnosis of COVID-19 facilitates management of SARS-CoV-2 outbreak

SARS-CoV-2 viral load in upper respiratory specimens of infected patients

The incubation period of coronavirus disease 2019 (CoVID-19) from publicly reported confirmed cases: Estimation and application

Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Sensitivity of chest CT for COVID-19: comparison to RT-PCR

COVID-19): a perspective from China

Artificial intelligence applications for thoracic imaging

Using artificial intelligence to detect COVID-19 and community-acquired pneumonia based on pulmonary CT: evaluation of the diagnostic accuracy

Development and evaluation of an AI system for COVID-19 diagnosis

A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19)

A fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis

AI-based quantitative CT analysis of temporal changes according to disease severity in COVID-19 pneumonia

Artificial intelligence augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other origin at chest CT

Using artificial intelligence to assist radiologists in distinguishing COVID-19 from other pulmonary infections

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

Comparison of clinical characteristics between coronavirus disease 2019 pneumonia and community-acquired pneumonia

Chest CT for typical coronavirus disease 2019 (COVID-19) pneumonia: relationship to negative RT-PCR testing

Use of chest CT in combination with negative RT-PCR assay for the 2019 novel coronavirus but high clinical suspicion

Diagnostic yield of computed tomography for the identification of coronavirus disease 2019 using repeated reverse transcriptase polymerase chain reaction testing or confirmed true-negative state as reference standard: systematic review and meta-analysis

Using artificial intelligence to improve the diagnostic efficiency of pulmonologists in differentiating COVID-19 pneumonia from community-acquired pneumonia