key: cord-0824976-g4d6mmnw
authors: Alzubaidi, Mahmood; Zubaydi, Haider Dhia; Bin-Salem, Ali; Abd-Alrazaq, Alaa A; Ahmed, Arfan; Househ, Mowafa
title: Role of Deep Learning in Early Detection of COVID-19: Scoping Review
date: 2021-07-30
journal: Comput Methods Programs Biomed Update
DOI: 10.1016/j.cmpbup.2021.100025
sha: d1467c0f192041e505eacfd13d90b3dc69fe85c9
doc_id: 824976
cord_uid: g4d6mmnw

Background: Since the onset of the COVID-19 pandemic the world witnessed disruption on an unprecedented scale affecting our daily lives including but not limited to healthcare, business, education, and transportation. Deep Learning (DL) is a branch of Artificial intelligence (AI) applications, the recent growth of DL includes features that could be helpful in fighting the COVID-19 pandemic.. Utilizing such features could support public health efforts. Objective : Investigate the literature available in the use of DL technology to support dealing with the COVID-19 crisis. We summarize the literature that uses DL features to analyze datasets for the purpose of a quick COVID-19 detection. Methods : This review follows PRISMA Extension for Scoping Reviews (PRISMA-ScR). We have scanned the most two commonly used databases (IEEE, ACM). Search terms were identified based on the target intervention (DL) and the target population (COVID-19). Two authors independently handled study selection and one author assigned for data extraction. A narrative approach is used to synthesize the extracted data. Results : We retrieved 53 studies and after passing through PRISMA excluding criteria, only 17 studies are considered in this review. All studies used deep learning for detection of COVID-19 cases in early stage based on different diagnostic modalities. Convolutional Neural Network (CNN) and Transfer Learning (TL) were the most commonly used techniques. Conclusion : The included studies showed that DL techniques has significant impact on early detection of COVID-19 with high accuracy rate. However, most of the proposed methods are still in development and not tested in a clinical setting. Further investigation and collaboration is required from the research community and healthcare professionals in order to develop and standardize guidelines for use of DL in the healthcare domain.

The COVID-19 pandemic was first detected in December 2019 (Kong et al., 2020) . By October 2020 more than 1.4 million deaths by COVID-19 were reported (Sheng, 2020) . Due to the sheer number of lives taken by the COVID-19 virus or Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), this pandemic was pronounced as a global pandemic by the World Health Organization (WHO) . Symptoms reported for those infected with COVID-19 include fever, dry cough, fatigue, and losing sense of taste and smell; many victims were referred to intensive care units for immediate mechanical ventilation (Carlos del Rio and Preeti N. Malani, MD, 2020) . Emergency countermeasures were required in order to mitigate the harms of this pandemic. Therefore, extensive change in public health system were required which included but not limited to diagnostics, clinical treatment, surveillance, and research (P.B. and H., 2020; Yang and Wang, 2020) .

Utilizing current technologies in the fight against COVID 19 can enhance the public health system capability (Ting et al., 2020) . Mobile awareness apps that broadcast notifications regarding the spreading of COVID-19 and reported cases, wearable device for real-time tracking of infected cases can all help in this fight. Furthermore, unprocessed data can be analysed via Deep Learning (DL) to provide significant information in a relatively short period with little effort (Shaw et al., 2019) , augmenting public health capability against COVID-19. DL applications have slowly advanced since 1986, but post 2010 the DL field saw rapid growth due to the availability of high-performance tools such as graphics processing unit (GPU) and a massive amount of unstructured data (van Hartskamp et al., 2019) .

DL is based on new structured algorithms allowing the development of intelligent machines and methods with less human interaction, some examples of DL mechanisms include smart reply regarding queries, plotting data in a meaningful way, or provide a highly accurate prediction for any matter (Alimadadi et al., 2020) . Designing a sharp taxonomy for DL categorization is challenging as it relies on different aspects such as type of pursued object, learning from experience, exploring structured and unstructured data, and extracting required information and reasons from knowledge bases (Shaw et al., 2019) .

Since the onset of COVID-19, many technology companies, governments, and institutes initiated urgent announcement for researchers to adopt AI applications to assist in COVID-19 mitigation (Alimadadi et al., 2020) . From a researchers point view, AI can contribute for fighting COVID-19 at certain levels:

demographic level e.g (prediction future forecasting infection), patient level e.g (diagnose COVID-19 in early stage) (Bullock et al., 2020) .

As the scope of this review is limited to the use of DL against COVID-19, the reader may refer to comprehensive surveys about AI field (Mondal and Mondal, 2020) and lectures (Alom et al., 2019; Glassner, 2019) .

Huge amount of dataset can be manipulated and processed using AI features, in particularly, patient data can go through many stages such as analysing, segmentation, augmentation, scaling, normalization, sampling, aggregating, and sifting, in order to obtain accurate prediction that assists the healthcare ecosystem as well the rest of stakeholders in the public health. Recently, the number of DL studies have increased aiming to address and/or propose a solution for the COVID-19 pandemic (Bullock et al., 2020) . However, most of these studies conducted during the COVID-19 pandemic are dispersed. We summarize the involvement of DL technologies in resolving challenges that related to COVID-19, an appropriate summarization will help new researchers to understand the present role of DL in the fight against COVID-19 opening new opportunities for researchers to continue with future work whilst building upon what has already been reported within the research community. Fig. 1 illustrates the most of DL techniques and dataset that have been used against . Previous studies report on AI techniques that have been used to mitigate the COVID-19, (Abd-Alrazaq et al., 2020; Alimadadi et al., 2020; Bullock et al., 2020; Santosh, 2020; Shi et al., 2021; Vaishya et al., 2020) . The reported approaches are conducted in the form of systematic reviews or surveys (Chamola et al., 2020; Pham et al., 2020; Ulhaq et al., n.d.; Zheng et al., 2020) whilst focusing on the different applications of AI such as patient diagnosis, epidemiological monitoring, drug, and vaccine discovery. Nevertheless, a massive number of research papers are constantly published and overwhelming the electronic databases. Therefore, it is necessary to carry out an updated review that focuses on DL and its use in the COVID-19 pandemic. The aim of this review is to identify and illustrate the role of DL technology during the COVID-19 pandemic as illustrated in Fig. 1 ; Deep Learning; 1) Convolutional Neural Network (CNN): 2) Transfer Learning. The outcome can be used as a guidance in the healthcare sector for developers who consider the utilization of DL to improve the public health capability as a quick response to COVID-19.

In order to ensure the transparency and reliability of this study, this scoping review is conducted following the guidelines of PRISMA Extension for Scoping Reviews (PRISMA-ScR) (Tricco et al., 2018) . PRISMA-SCR is the most popular and comprehensive guidelines for scoping reviews, and it is highly recommended by Cochrane and the Joanna Briggs Institute (JBI) (Munn et al., 2018) .The protocol outline in this review are detailed in the following sections.

In this review, the research queries were conducted between the 10th and 13th of October 2020. The online database for this search as follow: IEEE Xplore and ACM Digital library. The search is mainly focused on the computer science database and due to the limitation of this research we excluded medical databases.

Specified search terms were used to distinguish between related and unrelated studies that available on the targeted databases. these terms were chosen based on the target intervention "deep learning, artificial intelligence and deep learning" and the target disease "Coronavirus and COVID-19". total retrieved studies in (Appendix A).

In this review, the main focus on the DL-based approaches or technology that used Chest X-Ray radiography (CXR), Ultrasonography (ULS), and Computed Tomography Scan (CT) for the purpose of detecting the COVID-19 in the early stage. In addition, studies with an overview such as scoping or systemic reviews were excluded. Furthermore, to assure the novelty of this review, we consider studies that have been published between (May-September 2020) which are written in English. The collected studies were limited to; 1) peer reviewed articles: 2) conference proceedings. However, the following publication types were excluded; 1) reviews: 2) conference abstracts: 3) proposals: 4) pre-printed studies.

Appendix B explains how the data extraction form is organized. the extracted data from selected studies includes the following characteristics: 1) DL models:

2) datasets used for the training and testing the model: 3) validation and evaluation of DL models. After we built the structured data, the narrative approach was used to synthesize it. in particular, we divided and described the DL model in the included studies based on the images used (e.g. XRAY, CT, ULS), DL branch (e.g CNN and transfer learning), described the used dataset in term of source (e.g public and private datasets). furthermore, validation and evaluation methods were described to determine the efficiency of each model. Excel sheet is used to manage data synthesis.

53 studies were retrieved via the search within the selected database as shown in Fig. 2 . Out of these studies no duplicate studies were found, we screened the titles and the abstracts for 53 studies, 23 studies were excluded during this step for reasons mentioned in Fig. 2 . In last step during full-texts screening for 30 studies we have excluded 13 studies due to irrelevant and different study design.

Finally, 17 studies were included in this review. 

In the included studies, 12 are peer-reviewed journals and 5 conference articles Table. 1 In addition, a third (n=5) of the studies are published in May 2020, and the rest of the studies are published in June, July, August, and September 2020.

The included studies were conducted in 10 different countries, even though, a third of the studies were published in China as shown in Table 1 . 

Deep learning is the current trend in dealing with medical images. It is intended to assist radiologists in giving a more precise diagnosis by giving a quantitative analysis of worrisome lesions and allowing for a faster clinical workflow. Deep learning has already demonstrated performance in recognition and computer vision tasks that may overcome humans' abilities (Kim et al., 2019) . The architecture of deep learning algorithm is more complex compared to the traditional algorithm (machine learning). DL architecture is composed of 3 main stages; the first step is done through pre-process and enhancement for each input. The second step is used to extract the features of that input. The third step is related to the classification process for each input based on different classifiers. As shown in Fig. 3 , deep learning model minimizes human intervention, processes a complex data that might be challenging in machine learning and produce accurate results in a short time .

Learning for detection COVID-19 because they are the backbone methods for dealing with medical image in deep learning (Kim et al., 2019) FIGURE 3. Comparison Deep Learning and Machine Learning.

Convolutional neural networks (CNNs) is an artificial neural networks that consist of several layers and each layer has multiple neurons that operate similarly like human brain neurons. CNN proofed its efficiency in the medical image classification [26] . In addition, CNN is the backbone for all proposed models that used for detection COVID-19 via three imaging modes chest X-Ray radiography (CXR), Ultrasonography (ULS), and Computed Tomography Scan (CT) scan. Fig. 4 illustrate the basic concept of how the dataset is passed into CNN to train the model and used for COVID-19 prediction. (Phankokkruad, 2020) . 

Transfer learning is a deep learning method that can utilize the gained knowledge from the previous training and apply it on the new training set as shown in Fig. 5 In this review Transfer learning is developed based on the CNN model and they are called pre-trained model (Phankokkruad, 2020) . 

In the collected studies the performance of each model is evaluated using confusion matrix. Hence, folds-cross validation is included in some studies for validation purpose (Hussain et al., 2020; Oh et al., 2020) . The assessment measurements for model is used to ensure the efficiency for recognizing COVID-19. 

As shown in Table. 3. In all studies DL approaches are aimed to detect COVID 19 in the early stage based on three indicators, 10 studies used X-Ray radiography (CXR) (Abdani et al., 2020; Babukarthik et al., 2020; Makris et al., 2020; Oh et al., 2020; Phankokkruad, 2020; Rajaraman et al., 2020; Sethi et al., 2020; Waheed et al., 2020) , 5 studies used Computed Tomography Scan (CT) (Han et al., 2020; Hu et al., 2020; Li et al., 2020; , and 2 studies used Ultrasonography (ULS) (Horry et al., 2020; Roy et al., 2020) . In addition, 5 of the studies build their model from scratch using CNN (Abdani et al., 2020; Babukarthik et al., 2020; Han et al., 2020; Roy et al., 2020; while other 12 studies used pre-trained model to build their model (Horry et al., 2020; Hu et al., 2020; Li et al., 2020; Makris et al., 2020; Oh et al., 2020; Phankokkruad, 2020; Rajaraman et al., 2020; Sethi et al., 2020; Waheed et al., 2020; . Sethi et al., 2020; Waheed et al., 2020; , and in one study no segmentation either augmentation is applied (Abdani et al., 2020) . moreover, only 5 studies adopted folds-cross validation (Abdani et al., 2020; Han et al., 2020; Hu et al., 2020; Roy et al., 2020; and 13 studies used F1-score as evaluation metrics (Babukarthik et al., 2020; Han et al., 2020; Horry et al., 2020; Li et al., 2020; Makris et al., 2020; Oh et al., 2020; Rajaraman et al., 2020; Roy et al., 2020; Sethi et al., 2020; Waheed et al., 2020; . For better visualization for the infection area in the lung the visual explanation technique Gradient Weighted Class Activation Mapping (Grad-CAM) is adopted in five studies (Oh et al., 2020; Rajaraman et al., 2020; Roy et al., 2020; . 

All proposed studies are summarized in Table. Table. 5, most of studies (Abdani et al., 2020; Babukarthik et al., 2020; Makris et al., 2020; Oh et al., 2020; Phankokkruad, 2020 dataset (Cohen et al., 2020) meets their requirement to develop a COVID-19 detection model. However, there is a trade-off between using public dataset and private dataset, for example by using public dataset Images are randomly collected without consideration for the end researchers, but the result can be evaluated and future work can be carried out by other researcher. On other hand, by using private dataset images can be collected carefully based on the researcher requirement and many images can be taken for one patient with less noise and blurrily. However, the work that implemented on private dataset cannot be evaluated and the future work is limited. (Oh et al., 2020) USNLM Dataset: National Library of Medicine Data Distribution. (Oh et al., 2020) Corona Hack: Chest X-Ray-Dataset (Kaggle). (Oh et al., 2020) IEEE COVID-19 Image Data Collection (GitHub). (Abdani et al., 2020; Babukarthik et al., 2020; Makris et al., 2020; Oh et al., 2020; Phankokkruad, 2020; Sethi et al., 2020; Waheed et al., 2020) COVID-19 Radiography Database (Kaggle). (Waheed et al., 2020) COVID-19 Chest X-ray (GitHub). (Waheed et al., 2020) RSNA CXR DATASET (Kaggle). (Rajaraman et al., 2020) TWITTER COVID-19 CXR DATASET (Twitter). (Rajaraman et al., 2020) CheXpert Chest X-ray Dataset. (Babukarthik et al., 2020) COVID-19 Database Italian Society of Radiology. (Abdani et al., 2020) Chest X-Ray Images Pneumonia (Kaggle). (Abdani et al., 2020; Makris et al., 2020) CT Scan

The Cancer Imaging Archive (TCIA) Public Access. (Hu et al., 2020) Local hospital Union Hospital, Tongji Medical College (X. SARS-CoV-2 (Kaggle). (Z. Designated COVID-19 hospitals in Shandong. (Han et al., 2020) COVID-CT (GitHub). (Phankokkruad, 2020) 10 medical centres China. ULS POCOVID (GitHub). (Horry et al., 2020) 5 Local Italian hospital COVID-19 Lung Ultrasound Database (ICLUS-DB). (Roy et al., 2020) 4. DISCUSSION

The use of DL was investigated in this scoping review and its usage against the COVID-19 virus. Only 17 publications that met our predefined inclusion criteria were reported via the targeted database libraries (ACM and IEEE). This is not extraordinary due to (a)In such a pandemic, most of the studies are 

In future, DL Technologies must be integrated with public education, current DL proposed approaches are somewhat treated with caution due to the lack of understanding on how DL function at the most profound level. May ethical points are raised and require further clarification before the acceptance of DL approaches is likely to see an upsurge. Furthermore, most of DL studies that detect COVID-19 are not described consistently which makes the comparison between studies more challenging. In this review, we found out that only 70% of these studies disclose how the training-testing dataset is split, 30% implement validation method while other 70% did not mention how the validation is conducted, more than 50% of the studies did not provide their work for public sharing, 30% of the studies are missing significant evaluation metrics. The scientific community and developers need to standardize a protocol in an attempt to minimize the huge volume of studies for COVID-19 that can be confusing to interested researchers and provide robust studies by following Criteria: 1. Collect proper dataset from different medical centers including many images for each patient. 2. Dataset pre-processed phase should be improved in term of the used model such as using FC-DenseNet103 for segmentation instead of UNet, also in the reviewed studies data augmentation is either excluded completely or not significant in size. 3. In the case of COVID-19 pandemic, researchers should provide light-weight models to be used by developers and researchers in countries that have constraint resources. 4.

However, some studies have successfully show using two type of modes (X-RAY, CT scan) and this requires further examination. 5. All evaluation metrics should be used for the aim of COVID-19 detection to come up with a solid prototype that can detect different type of diseases based on images. We found that, Recurrent Neural Network (RNN) and Reinforcement learning is not used in the field against COVID-19 and rather considered as new direction for future work.

This study reviews DL techniques that used for detection COVID-19, without restriction on the characteristics, country, and study design. To the best of our knowledge this review is the first comprehensive study in the field of DL approaches and their application for COVID-19 detection. This scoping review can aid researchers to understand how DL was and is being used efficiently during COVID-19 pandemic. Comparing with other similar reviews (Ahir et al., 2020; Hussain et al., 2020; Jamshidi et al., 2020; Pham et al., 2020; Ulhaq et al., n.d.) this review is unique in its field as it describes and summarizes features of the identified DL models, datasets, evaluation, and validation. Furthermore, in comparison to previous reviews (Ahir et al., 2020; Hussain et al., 2020; Jamshidi et al., 2020; Pham et al., 2020; Ulhaq et al., n.d.) it follows the scientific of PRISMA-ScR (Tricco et al., 2018) . Finally, we limited the studies to the most popular computer science databases in order to determine the most relevant studies as possible.

We excluded proposals of DL techniques; as a result, we have likely excluded other applications of DL for COVID-19 detection. Research was conducted only on two digital libraries (ACM and IEEE) so we could not highlight all potential DL studies. Due to the search query that did not include special terms that related to each technique such as CNN, image classification, and transfer learning. Thus, it is possible that we dropped some studies that used previous terms in their abstract or title instead of the terms that we used (DL, AI, machine learning, and deep learning). This number of studies was identified using only 2 databases, which are the most popular computer science databases, further, we restricted our search to a specific period (May-September). To address this limitation, the findings in this review are based on the results that are provided in each study, the reliability of the given information in the studies may affect the findings of this review.

In this review, 17 studies on DL against COVID-19 are provided to form the scoping review, published date and country are included to clarify how this pandemic is tackled by various entities, with a pre-knowledge that many of the proposed mechanisms, are not clinically implemented. The used approaches are described based on medical diagnosis (early detection of COVID-19), the exciting works are summarized and represented including deep learning methods. we have noticed that most of medical diagnosis for image classification are handled via CNN and transfer learning. This review covered all models and algorithm that used and discussed the validation and ovulation process. We provided a specific section to cover the dataset that used in most of the studies, including public and private datasets. However, due to the huge number of studies that daily updated to the online database, this review can be further extended to cover other research direction such as treatment and vaccines discovery, and prediction of patient outcomes. o This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue.

o The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript o The following authors have affiliations with organizations with direct or indirect financial interest in the subject matter discussed in the manuscript:

Artificial intelligence in the fight against COVID-19: Scoping review

A Lightweight Deep Learning Model for COVID-19 Detection

The impact of Artificial Intelligence, Blockchain, Big Data and evolving technologies in Coronavirus Disease-2019 (COVID-19) curtailment

Artificial intelligence and machine learning to fight covid-19

A state-ofthe-art survey on deep learning theory and architectures

Prediction of covid-19 using genetic deep learning convolutional neural network (GDCNN)

Mapping the landscape of artificial intelligence applications against COVID-19

COVID-19-New Insights on a Rapidly Changing Epidemic

A Comprehensive Review of the COVID-19 Pandemic and the Role of IoT, Drones, AI, Blockchain, and 5G in Managing its Impact

SARS-CoV-2: virus dynamics and host response

COVID-19 Image Data Collection: Prospective Predictions Are the Future

Deep learning: A crash course

Association for Computing Machinery

Accurate Screening of COVID-19 Using Attention-Based Deep 3D

Multiple Instance Learning

COVID-19 Detection through Transfer Learning Using Multimodal Imaging Data

Weakly Supervised Deep Learning for COVID-19 Infection Detection and Classification from CT Images

AI Techniques for COVID-19

Artificial Intelligence and COVID-19: Deep Learning Approaches for Diagnosis and Treatment

Deep learning in medical imaging

SARS-CoV-2 detection in patients with influenza-like illness

Efficient and Effective Training of COVID-19 Classification Networks with Self-Supervised Dual-Track Learning to Rank

COVID-19 detection from chest X-ray images using deep learning and convolutional neural networks

Artificial Intelligence: State of the Art

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets

COVID-19 -Looking beyond Tomorrow for Health Care and Society

Artificial Intelligence (AI) and Big Data for Coronavirus (COVID-19) Pandemic: A Survey on the State-of-the-Arts

COVID-19 Pneumonia detection in chest X-ray images using transfer learning of convolutional neural networks

Development of a clinical decision support system for the early detection of COVID-19 using deep learning based on chest radiographic images

Early detection of COVID19 by deep learning transfer Model for populations in isolated rural areas

Detection in Chest X-Rays

Deep Learning for Classification and Localization of COVID-19 Markers in Point-of-Care Lung Ultrasound

AI-Driven Tools for Coronavirus Outbreak: Need of Active Learning and Cross-Population Train/Test Models on Multitudinal/Multimodal Data

Deep Learning based Diagnosis Recommendation for COVID-19 using Chest X-Rays Images

Artificial Intelligence and the Implementation Challenge

Coronavirus disease 2019 (covid-19)

Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19

Digital technology and COVID-19

PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation

Artificial Intelligence (AI) applications for COVID-19 pandemic

Artificial Intelligence in Clinical Health Care Applications: Viewpoint. Interact

CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection

A Weakly-Supervised Framework for COVID-19 Classification and Lesion Localization from Chest CT

Contrastive Cross-Site Learning with Redesigned Net for COVID-19 CT Classification

COVID-19: a new challenge for human beings

Predicting COVID-19 in China Using Hybrid AI Model

No fund to be declare

None declared

AuthorThe first author of the study.

The year in which the study was submitted.

The country where the study was published. Publication typeThe paper type (i.e., peer-reviewed, conference or preprint).

Detection modality What type of medical images are used (e.g., XRAY, CT, and ULS )? DL branchesThe branches/areas of that were used (e.g., CNN, Transfer learning). AI models/ algorithms The specific AI models or algorithms that were used (e.g., VGG).

Data sources Source of data that were used for the development and validation of AI models/ algorithms (e.g., public dataset, private dataset).

The total number of data that were used for the development and validation of AI models/ algorithms.

How the dataset was split/used to develop and test the proposed models/ algorithms (e.g., Train-test split, K-fold cross-validation, External validation). Proportion of training set Percentage of the training set of the total dataset.

Percentage of the test set of the total dataset. Evaluation metrics Any evaluation method that are used to check the performance of the model .(e.g., accuracy, precision, F1 score, recall and Kappa).

Type of used visualization method