key: cord-1043134-c4c9sn3e
authors: Singh, Mukul; Bansal, Shrey; Ahuja, Sakshi; Dubey, Rahul Kumar; Panigrahi, Bijaya Ketan; Dey, Nilanjan
title: Transfer learning–based ensemble support vector machine model for automated COVID-19 detection using lung computerized tomography scan data
date: 2021-03-18
journal: Med Biol Eng Comput
DOI: 10.1007/s11517-020-02299-2
sha: aad486945f37b76de7da0c1bd5d7cf547fd52a44
doc_id: 1043134
cord_uid: c4c9sn3e

The novel discovered disease coronavirus popularly known as COVID-19 is caused due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and declared a pandemic by the World Health Organization (WHO). An early-stage detection of COVID-19 is crucial for the containment of the pandemic it has caused. In this study, a transfer learning–based COVID-19 screening technique is proposed. The motivation of this study is to design an automated system that can assist medical staff especially in areas where trained staff are outnumbered. The study investigates the potential of transfer learning–based models for automatically diagnosing diseases like COVID-19 to assist the medical force, especially in times of an outbreak. In the proposed work, a deep learning model, i.e., truncated VGG16 (Visual Geometry Group from Oxford) is implemented to screen COVID-19 CT scans. The VGG16 architecture is fine-tuned and used to extract features from CT scan images. Further principal component analysis (PCA) is used for feature selection. For the final classification, four different classifiers, namely deep convolutional neural network (DCNN), extreme learning machine (ELM), online sequential ELM, and bagging ensemble with support vector machine (SVM) are compared. The best performing classifier bagging ensemble with SVM within 385 ms achieved an accuracy of 95.7%, the precision of 95.8%, area under curve (AUC) of 0.958, and an F1 score of 95.3% on 208 test images. The results obtained on diverse datasets prove the superiority and robustness of the proposed work. A pre-processing technique has also been proposed for radiological data. The study further compares pre-trained CNN architectures and classification models against the proposed technique. [Image: see text]

A chest infection is a kind of infection that affects the proper functioning of the lungs (both larger and smaller airways) [1] . The severity of a lung infection depends on several factors like causes of lung infection (virus or bacteria) and the overall health of the infected person. The most common lung infections are pneumonia, chronic obstructive pulmonary disease (COPD), asthma, bronchitis, and lung cancer. Coronavirus disease popularly known as COVID-19 is a kind of lung infection disease. It is caused due to the novel discovered virus know as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Coronaviruses are required to tackle this outbreak. The need for a faster screening technique to control the COVID outbreak has also been studied by the authors in [7, 8] . Researchers have found that radiological analysis like X-rays and chest CT (computed tomography) scans have high accuracy in COVID-19 diagnosis and can be an effective tool for large scale screening. A high correlation between RT-PCR and radiological results for COVID-19 diagnosis is established in [9] . Also, COVID-19 infection is identified through ground-glass opacity patches (GGO) in radiographic scans of patients. This encouraged the development of a faster and cheaper COVID-19 screening mechanism using a radiological approach. Also, deep learning is playing a critical role in medical image analysis which motivates its use in screening of COVID-19. The growing role of deep learning has also been analyzed by the study of AIbased COVID classification techniques [10] where authors find that deep-learning-based techniques can provide very promising results for COVID classification. The details of the techniques available in the literature for COVID-19 diagnosis is put forth in Table 1 .

From the detailed analysis of the state of the art of COVID-19 diagnosis field, it can be inferred that chest radiography (X-rays and CT scan) is the best alternative for COVID-19 detection in comparison to the RT-PCR test kits [31] . However, CT scan modality seems to be most efficient in comparison to chest X-ray due to the following reasons: (a) CT scan gives a detailed 3-dimensional view of the diagnosed organ whereas X-rays give a 2-D view, (b) the CT scan does not overlap the organ, whereas in X-rays ribs overlap the lungs and heart. Due to the high precision of a CT-scan based screening system, a deep learning-based 3 step model is proposed which consists of a transferlearning-based feature extractor, a feature selector, and a feature classifier. In the proposed work, a truncated VGG16 architecture is proposed for extracting features. The last two blocks of the truncated architecture are fine-tuned with differential learning rates. PCA is applied to the features extracted by CNN. For the classification task, four different classifier models are compared.

The study addresses various issues with the current COVID-19 datasets and proposes various techniques to overcome these. Transfer learning capabilities of various models have been demonstrated and compared. Furthermore, various techniques like truncation and differential learning rates are proposed to increase robustness. The effect of various feature selection techniques has also been studied. Finally, bagging SVM is chosen for classification after a comparative study of popular classifiers. The proposed model within 385 ms achieved an accuracy of 95.7%, the precision of 95.8%, area under curve (AUC) of 0.958, and an F1 score of 95.3% on the 208 test images. The results obtained on diverse datasets prove the superiority and robustness of the proposed work.

The rest of the paper is organized as follows: Section 2 illustrates the proposed methodology; Section 2.7 put-forth the details of different classifiers. Section 3 gives the details of results and discussion. Then Section 5 concludes the proposed work.

The chest CT scans of COVID-19 patients contain patches of ground glass opacity (GGO); thus, a multi-dimensional feature extractor is required for screening [32] . In the proposed work, the VGG16 architecture is fine-tuned and used to extract features from lung CT scan images. Since the size of the COVID-19 dataset is very small, a truncated version of the VGG16 architecture is used. PCA is used to reduce the dimensionality of the features obtained from truncated VGG-16. The final classification is performed using four different classifiers. The self-explanatory block diagram of the proposed methodology for COVID-19 classification is shown in Fig. 1. 

In the proposed work, the dataset is collected from three different sources to ensure the robustness of the model. The brief details of datasets used are: The final split of the data is summarized below:

-Training : 432 images (204 COVID and 228 non-COVID before augmentation -Validation : 62 images (29 COVID and 33 non-COVID) -Test : 208 images (111 COVID and 97 non-COVID) Some images in these datasets had markings and other non-removable artifacts and hence had to be dropped. The model is trained on 5323 chest Xray images (COVID19 positive cases-115, pneumonia-3867, and normal cases -1341) [14] .

[20] Domain extension transfer learning (DETL) with gradient class activation map (Grad-CAM)

Fivefold cross-validation accuracy of 90.13% and test set accuracy of 95.3% are obtained on the proposed X-ray dataset.

A total of 305 COVID-19 X-Ray images were used in the study. [14] [21] ResNet, Inception, and GoogleNet

The classification of COVID-19 positive cases based on X-ray modality is done. The approach achieved 98% of accuracy with VGG19, 95% with Resnet50, and 96% with InceptionV3.

A dataset of nearly 100 subject, among them 50 x-ray images subjects were tested positive with COVID-19 and 50 x-ray images of normal subjects. [14, 15] [22] ResNet18, ResNet50, SqueezeNet,

and DenseNet-121

The model achieved a specificity of 90% and sensitivity of 96.5% on testing data of 3000 chest X-rays (COVID and non-COVID patients).

The model is trained on 5000 chest x-rays dataset [14] [23] Joint classification and segmentation (JCS)

Classification is done with a specificity of 93% and a sensitivity of 95%. A dice score of 78.3% is obtained for the segmentation task.

JCS system is implemented on 400 COVID-19 patients (144,167 images) and A study of 73 patients with 205 COVID positive images. [30] positive, 169 of them have age information and 137 of them have gender information. The images are from multiple sources including hospital donations, pre-prints, and reports released by the ISMIR [15] . The exact patient and image details can be found at [14, 15, 24 ].

As the input images are of different sizes, thus all the input images are resized to 112 × 112 × 3 to maintain the uniformity.

CT scans have artifacts like beam hardening, noise, and scatter, which reduce the accuracy of the model. To overcome this, first, a median filter has been applied. Median filtering is a widely used nonlinear method used to remove noise from images while preserving edges. The median filter operates by replacing each pixel value with the median value of neighboring pixels. 5 × 5 is a popular median filter kernel choice for biomedical images and has hence been chosen [33] . Finally, morphological close transformation is applied to the image. A morphological close operation is a dilation operation followed by an erosion operation. It removes holes and any remaining salt and pepper noise from the images. It has been shown to be highly effective on binary and gray-scale images [34] .

Since the images are of different scales and have labelings and markings around corners, an adaptive ROI selector has been applied to the images. The filter first centers and straightens the image, then applies an elliptical mask on the images such that the non-lung parts get cropped out. The ellipse is fitted to the image using the abrupt pixel-value changes as we move close to the rib-cage bones (black to white color change). Furthermore, all the masked images are manually checked to ensure that no image is over or under-cropped. Such images are manually fixed. Figure 2 shows the pictorial representation of the various pre-processing module used in the study.

Data augm'entation allows the model to learn a more diverse set of features and also increases the size of the dataset thereby preventing the model from overfitting. Each training image is augmented by a random affine transformation, random flip, and random changes in hue, brightness, and saturation of the image. The random affine transformation consists of shearing and rotation. The details of image augmentation parameters include (a) rotation-within range of 0 to 30 • , (b) shearing-0.2, (c) zooming-0.2, and (d) changing the brightness level-within range of 0.75 to 1.5. The augmentation parameters were chosen based on the study of the effectiveness of image augmentation techniques on deep networks [35] . The parameters chosen have also been used in [36] where authors have achieved great results with these settings on a CT scan classification problem similar to this.

The training data after augmentation is (a) 612 of COVID-19 images and (b) 684 of non-COVID-19 images. Table 3 shows the result of the comparative study considering the popular CNN architectures. The accuracy reported is on the test set by adding a classification layer to the model, and it can be seen that VGG16 outperforms other deeper architectures. This is an interesting observation that deeper models are performing poorer on the current COVID-19 datasets. This is probably due to the size and quality of the currently available datasets. Due to the better performance of VGG16, it is the model of choice in this study. Similar observations have been made in [37] where authors use VGG16 to classify COVID-19 from a multimodal input.

Initially, the VGG model is trained on the ImageNet database with over 14 million images [38] . Instead of using large receptive fields, VGG16 uses very small receptive fields (3 × 3 with a stride of 1). VGG16 incorporates 1 × 1 convolution layers to make the decision function more non-linear without changing the receptive fields. Since the COVID-19 dataset is much smaller with only 591 training images (before augmentation), the high complexity of the feature set will be difficult to generalize. To prevent this, a truncated VGG16 architecture is proposed which limits the complexity of the features. The first four convolution blocks of the VGG16 architecture are used for the proposed truncated architecture as shown in Fig. 3 . The truncation of the architecture reduces model complexity and number of trainable parameters, which eventually helps in reducing overfitting. This technique has also been used in Inception-net based COVID classification in [39] where the authors truncated the architecture to reduce overfitting. The truncation layer is determined by evaluating performance on the validation set with different points of truncation as detailed out in Table 4 .

Training a neural network from scratch requires huge amounts of data. As the COVID-19 dataset available is significantly smaller, transfer learning is applied to extract an accurate and concise feature set from the training data. This is a popular technique and has also achieved great results in [11, 13, 17, 19] .

In the proposed methodology, a representation learningbased approach is used. A pre-trained VGG-16 model is fine-tuned and its intermediate outputs act as a representation of raw data. This representation serves as features for the classifier module. The first four blocks of the VGG16 architecture pre-trained on ImageNet weights are used for this purpose [13, 17] . Since the Image net set is non-overlapping to the problem, the last 8 layers, i.e., the third and fourth convolution blocks are fine-tuned on the augmented CT scan training data [40] . While training these, it is desired that the fourth block adapts more to the data compared to the third block. The third block carries relatively fewer complex features that do not need to change much. Hence, a higher learning rate has been used for the fourth convolutional block compared to the third convolutional block while fine-tuning [11] . The extracted features are displayed as a color map as shown in Fig. 4 . Figure 5 shows the confusion matrices of the proposed model with and without fine-tuning of the VGG16 based feature extractor. The feature extractor module reduces the dimension of the data to 25,000 features per image for an image size of 112 × 112 × 3 pixels. However, with only 591 training examples (before augmentation), the model would still overfit the features. To prevent this, feature selection and dimensionality reduction of data are performed.

Principal component analysis (PCA), autoencoders, and variance-based selectors are the most popular feature selectors for image data. [41] finds PCA to perform significantly better as a feature selector on biomedical data. PCA finds the eigenvectors of a covariance matrix with the highest eigenvalues and then uses those to project the data into a new subspace of equal or fewer dimensions. Autoencoders compress the input to a lower dimension. Variance-based methods select the features which have the highest variance over the data. PCA, autoencoder, and variance-based selector have been used to reduce the dimensionality of the feature set, and then their accuracies on the validation set are compared after classification with an SVM. Applying PCA with 95% variance representation yields 359 components. Since 95% variance is a standard value for variance-based reductions, the autoencoder and variance-based selectors were also configured to retain 95% variance of the original feature set. The results of the analysis are tabulated in Table 5 . For the proposed model, PCA gives the highest accuracy because it represents the low-dimensional sample and synchronized variables. Furthermore, the extracted features from the training set are used to train the classification module to screen COVID-19 CT scans. The better performance of PCA as a feature selector has also been emphasized in [42] .

For the classification task, the required features are extracted using the truncated VGG16 model and selected using PCA. In machine-learning, no single algorithm is suitable for all problems. Thus, for achieving the highest performance, 4 different classification models are evaluated. Various classification techniques used in the proposed work are as follows: (a) deep CNN, (b) bagging ensemble with SVM, (c) extreme learning machine (ELM), and (d) online sequential ELM (OS-ELM).

CNN can successfully capture the spatial and temporal dependencies in an image through the application of relevant filters. The architecture performs a better fitting to the image dataset due to the reduction in the number of parameters involved and the re-usability of weights [43] .

Since VGG is itself a CNN architecture, for the deep CNN model, a fully connected layer of size 1024 is added to the truncated VGG architecture followed by a softmax layer for classification. This gives us the most direct classification model where the feature extraction and classification are in the same CNN architecture. The deep CNN utilizes the fine-tuned weights and uses it to directly predict the output. Similar model has been used by authors in [11] where 

ELMs are single-hidden layer feedforward neural networks (SLFNs) that randomly choose hidden nodes and analytically determines the output weights of SLFNs through the generalized inverse operation of the hidden layer output matrices. The implementation of ELM is as described in [44] . The number of hidden nodes in the model is experimentally determined with the best-suited gamma (width multiplier for RBF distance). L2-normalized RBF activation function has been used. Experimentation has also been conducted with varying numbers of neurons in the hidden layer. Using validation set accuracy, it was observed that the highest accuracy was observed at 1000 hidden nodes. The performance of ELM is comparable to [45] where authors have used ELM to classify COVID-19 chest C-rays and have achieved an F1 score of 0.95.

OS-ELM can learn data chunk by chunk with varying chunk size and provides faster sequential learning. The implementation of the model is the same as that described in [46] . It uses the idea of ELMs with a sequential determination of the output weights through the recursive least-squares (RLS) algorithm. OS-ELM consists of two phases, namely an initialization phase and a sequential learning phase. In the initialization phase, a base extreme learning machine model is trained using a small chunk of initial training data. For classification using OS-ELM, SLFN is implemented with a sigmoid activation function with 2500 hidden layers. As the model has very few hyperparameters, they have been optimized using a gridsearch optimization.

To improve the limited performance of the SVM (accuracy of 93.4%) due to the high complexity of time and space, the SVM ensemble with bagging is used. A single classifier may have a high test-error, but many small classifiers can produce a low test error and increase robustness because diversity compensates for error. For classification using the bagging SVM, the dataset is randomly divided into 10 parts. The individual classifiers are trained independently with the bootstrap technique and aggregated to make a joint decision by the deterministic averaging process. The proposed classifier model with "RBF" kernel and tuned hyperparameters is used as the base estimator. Bagging ensemble with SVM achieves the highest accuracy of 95.7% on the testing data. Due to the high accuracy of the bagging with the SVM model, it is the proposed classification method for COVID-19 screening. SVM also achieved exceptional results in biomedical studies such as [47, 48] .

Confusion matrices for different classifiers are shown in Fig. 8 . The classifiers are evaluated on the test set with 111 Variance-based selector 87.3 COVID-19 images and 97 non-COVID images. The features for the model are extracted using the truncated VGG16 model and selected using PCA. The screening performance of the model is assessed using generalized performance parameters derived from the confusion matrix. Table 6 put forth the generalized performance parameters, namely, true positive (TP), false positive (FP), true negative (TN), false negative (FN), AUC, accuracy (ACC), precision (PRE), sensitivity (S1), specificity (S2), negative predictive value (NPV), and F1 score (F1).

The proposed methodology is implemented on python software, run on a CPU. The system requirements are an Intel Core i7 processor with a 4 GB graphic card, a 64-bit operating system at 1.80 GHz, and 16 GB RAM. Figure 6 shows the convergence graph of training and validation accuracy of the transfer learning-based CNN model for the bagging ensemble classifier with SVM. Figure 5 shows the confusion matrices of the proposed architecture with and without fine-tuning of the VGG16 based feature extractor. The confusion matrices are obtained by evaluating the models on the test set with bagging SVM as the classifier.

K-fold cross-validation divides the training set into ksubsets and trains k-models such that one of the subsets is left out while training the model. The accuracy of these k models is then averaged. The benefit of k-fold is that the entire dataset is used for both training and validation, and each sample is used for validation exactly once. The 10fold cross-validation curve obtained for the bagging SVM is shown in Fig. 7 . This is a 10-fold cross-validation curve 

Deep learning models are often fooled with noise perturbations in the image. Such perturbations or attacks lead to miss classification of images. To defend the model against such noise attacks, a defense module has been designed.

To remove noise from an image before prediction, three image denoiser have been applied namely total variation, Gaussian filter, and wavelet denoising. The prediction of all three denoised images is passed to an ensemble which finally classifies the image as shown in Fig. 8 . On evaluating this module with the test set after adding random noise, the model gave an accuracy of 82.34%.

In the proposed work, the best performing model achieves an accuracy of 95.67% along with a precision of 96.83%. The area under the ROC curve (AUC) obtained is 95.8, as shown in Fig. 9 . The proposed method aims to reduce the false-negative rate as much as possible since falsepositive cases can potentially be identified in subsequent tests, but false-negative cases might not have that chance.

The proposed model has a false negative rate of 4.33%, 

Binary classification of total 746 chest CT images COVID-19 and non-COVID-19 [15] Accuracy -87.6%, AUC-0.961, and precision -84.3%.

Resnet50 and VGG16 [13] Total 102 X-ray images of COVID-19positive and -pneumonia patients [24] , [49] , [50] , [14] Overall accuracy achieved is 89.2%.

3. AI methods (JCS and DenseNet169) [15] Binary classification of CT scan data into COVID (349 images) and non-COVID (463 CT images) [ [14] , [24] , and [15] .

Prediction time is 385ms, Accuracy -95.7%, Precision -95.8%, AUC -0.958, and F1 score -95.3%.

which is significantly lower than other COVID-19 CT scan screening models. The model proposed in this study achieves a very high accuracy of 95.67% on the testing data with a very low prediction time of 358 ms. This proves that deep learning-based approaches could be used to effectively and accurately screen COVID-19 at very large scales. Table 7 puts forth the comparative analysis of the proposed methodology with other existing techniques.

A deep learning-based truncated VGG16 model is proposed in this study to screen COVID-19 patients using chest CT scans. The VGG16 architecture is fine-tuned and used to extract features from CT scan images. An interesting observation has been that pre-trained models are able to learn features very effectively with fine-tuning. The study demonstrates that VGG16 outperforms other models on biomedical image feature extraction. Another important observation has been that truncation improves the model's performance on the limited dataset. The performance also improves on applying dimensionality reduction techniques, indicating a high correlation among features that need to be removed to boost the performance of the classifier. It was experimentally found that PCA performs much better than auto-encoders for biomedical image features, as has been verified by several other studies. Finally, a comparison of various popular classification techniques shows us that a bagging ensemble of SVM gives the best results and outperforms direct CNN classification, ELM, and OS-ELM. Our study addresses the challenges associated with the limited and poor quality of COVID-19 radiology data. The study has proposed techniques like transfer learning, finetuning, model-truncation, image augmentation, and featurereduction to overcome these. This should be helpful for practitioners aiming to use these datasets for their research and development. Furthermore, the importance of various pre-processing techniques has also been highlighted. While comparing different pre-trained models, it was found that both VGG16 and VGG19 gave great results within the constraints of the small dataset. While deeper networks were found to struggle, they will perform better when larger datasets are available. For currently available datasets, we conclude that VGG16 with appropriate truncation and finetuning gives the best feature maps. These features give good results when classified directly but the results are found to improve significantly when a feature selector like PCA is applied before classification.

With the quality of available COVID-19 data increasing, clinically verified, and trusted deep learning models may be developed for fast diagnosis of COVID-19. The superior performance of the deep models may result in AI-based diagnosis for various diseases especially in times of outbreak where rapid screening and early identification are crucial for effective containment. As the quality of available data increases, deep models can be very effective for diagnosis. Future works on the study would focus on making the model interpretable and subjective in classification with a severity score which may be useful in the case of screening. Furthermore, a region of infection marking may assist the medical staff to treat the infection.

Conflict of interest The authors declare that they have no conflict of interest.

Ethical approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

WebMD Lung Diseases Overview

Role of biological data mining and machine learning techniques in detecting and diagnosing the novel coronavirus (covid-19): a systematic review

Real-time rt-pcr in covid-19 detection: issues affecting the results

Correlation of chest ct and rt-pcr testing for coronavirus disease 2019 (covid-19) in china: A report of 1014 cases

Helping doctors hasten covid-19 treatment: towards a rescue framework for the transfusion of best convalescent plasma to the most critical patients based on biological requirements via ml and novel mcdm methods

Multi-biological laboratory examination framework for the prioritization of patients with covid-19 based on integrated ahp and group vikor methods

Chest ct manifestations of new coronavirus disease 2019(covid-19): a pictorial review

Systematic review of artificial intelligence techniques in the detection and classification of covid-19 medical images in terms of evaluation and benchmarking: taxonomy analysis, challenges, future solutions and methodological aspects

Deep transfer learning based classification model for covid-19 disease. IRBM, 2020

Can ai help in screening viral and covid-19 pneumonia

Finding covid-19 from chest x-rays using deep learning on a small dataset

Covid-19 image data collection

Covid-ct-dataset: a ct scan dataset about covid-19

Covid19xraynet: a two-step transfer learning model for the covid-19 detecting problem based on a limited number of chest x-ray images

Towards an effective and efficient deep learning model for covid-19 patterns detection in x-ray images

A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images

Covidaid: Covid-19 detection using chest x-ray

Deep learning for screening covid-19 using chest x-ray images

Detection of covid-19 from chest x-ray images using artificial intelligence: an early review

Deepcovid: predicting covid-19 from chest x-ray images using deep transfer learning

Jcs: an explainable covid-19 diagnosis system by joint classification and segmentation

Covid-19 patients lungs x ray images 10000

Covidpen: a novel covid-19 detection model using chest x-rays and ct scans

Radiologist-level covid-19 detection using ct scans with detail-oriented capsule networks

Automated deep transfer learning-based approach for detection of covid-19 infection in chest x-rays

Classification of covid-19 patients from chest ct images using multiobjective differential evolution-based convolutional neural networks. European journal of clinical microbiology & infectious diseases : official publication of the

Ct manifestations of coronavirus disease-2019: a retrospective analysis of 73 cases by disease severity

Coronavirus detection and analysis on chest ct with deep learning

Essentials for radiologists on covid-19: an update-radiology scientific expert panel

Robustness of median filter for suppression of salt and pepper noise (spn) and random valued impulse noise (rvin)

Bakar ZA Noise removal and enhancement of binary images using morphological operations

A survey on image data augmentation for deep learning

Covid-19 detection in ct images with deep learning: a voting-based scheme and cross-datasets analysis

Covid-19 detection through transfer learning using multimodal imaging data

Very deep convolutional networks for large-scale image recognition

Truncated inception net: Covid-19 outbreak screening using chest x-rays

Covid-19 detection through transfer learning using multimodal imaging data

Comparing pca to information gain as a feature selection method for influenza-a classification

Echad: embedding-based change detection from multivariate time series in smart grids

Deep residual learning for image recognition

Extreme learning machine: theory and applications

Cov-elm classifier: an extreme learning machine based identification of covid-19 using chest x-ray images

A fast and accurate online sequential learning algorithm for feedforward networks

Exploiting transfer learning for the reconstruction of the human gene regulatory network

Covid-19 chest ct scans Radiopedia (2020) https://radiopaedia. org/search?utf8=%E2%9C%93&q=covid&scope=all&lang=us

Last accessed date 7th

NIH (2020) Nih chest x-ray dataset

Sample-efficient deep learning for covid-19 diagnosis based on ct scans