key: cord-1011144-u4kkbq2t
authors: Santosh, KC; Das, Nibaran; Ghosh, Swarnendu
title: COVID-19: prediction, screening, and decision-making
date: 2021-10-01
journal: Deep Learning Models for Medical Imaging
DOI: 10.1016/b978-0-12-823504-1.00015-5
sha: 1150111d0c5d15905abe0f84149cee00d71c4981
doc_id: 1011144
cord_uid: u4kkbq2t

In this chapter, we mainly focus on the use of AI-driven tools for COVID-19 predictive modeling, screening, and decision-making. We first discuss prediction models, their merits, and pitfalls. We then review deep learning models for COVID-19 detection and/or screening (with experiments) by taking different dataset sizes into account, which is followed by a conclusive study on how big data is big. The chapter provides a journey of deep neural networks for lung abnormality screening, where we consider COVID-19 as a particular case.

to exploit crucial and unprecedented uncertainties/factors, such as hospital settings and test rate, changes in demography, population density (including immunocompromised people), and poverty. With a high rise in deaths caused due to nCoV, immunocompromised persons (e.g., lung cancer) are at high risk. No prediction models consider immunocompromised population in terms of death cases (including recovery). Predictions can be short-term and long-term, and they rely on the aforementioned factors [4, 10] . Such continuous and unprecedented factors lead us to designing complex models, rather than just relying on stochastic and/or discrete ones that are driven by randomly generated parameters. In the literature, prediction models are limited to data visualization, and they are hardly extended to simulating the data, so trends can be visualized.

To amplify/visualize COVID-19 outbreak, it requires data visualization tools. Data visualization can help estimate the trend. of the prediction models are limited to data visualization. As an example, data simulations always help better understand the particular event(s). However, it must be limited to education/training. In simple words, simulations help us build up our intuition about how diseases work in a way that words and even static charts cannot. For visualization, distribution can be of great help (see Fig. 5 .3).

Predictive analytical results hit media a lot even though tools are limited to education and training. Note that, more often, due to unprecedented nature of the situation and many uncertainties related to diseases, inaccurate information was predicted. As an example, on March 31, 2020, the White House projected 100 K to 240 K Coronavirus deaths in the next two weeks. 1 Later, on April 8, 2020, we had another media statement 2 "not every model agrees: America's most influential coronavirus model just revised its estimates downward" as previous prediction was too far from actual values (84,575 death cases in the U.S., dated May 14, 2020). Media did not intentionally broadcast/announce inaccurate information; instead, the estimated values were based on prediction models. Not to be confused, authors are not aimed at blaming neither media nor prediction models.

Artificial and augmented intelligence (A2I) play crucial roles in understanding data by using multiple different tools/techniques. They include data analytics, machine learning, and pattern recognition, where anomaly detection is a primary element [1, 9] . Predictive modeling requires exploiting comprehensive data. Missing one or two features/factors can deviate predictive values from actual ones. More often, discrete models rely on their input parameters and are application dependent. In case of continuous data (e.g., , where there exist unavoidable uncertainties, these models behave differently. As a result, these models provide incoherent results. The primary reason behind this is lack of understanding about the particular events, that is, data sentiments and additional unavoidable uncertainties/factors, such as hospital settings/capacity, number of tests on a daily basis, demographics, and population (density) and their vulnerability in that particular region. In particular, we observed that the higher the population density, the higher the spread rate, and New York City is an example. This brings an idea that the same exact models with exact input parameters cannot be repli-cated to other regions. Also, immunocompromised (plus old) people need to be considered in their models; Italy is an example.

In the literature, we found three different model types for COVID-19 predictions: a) SEIR/SIR models, b) agent-based models, and c) curve-fitting models [10] . Categorically, inspired by [10] , let us briefly discuss them.

Medical Research Council (MRC) Centre for Global Infectious Disease Analysis used a nonpharmaceutical intervention (NPI) model, which employed SEIR approach. In a similar fashion, Columbia University used SEIR model and forecasted number of severe cases, hospitalizations, critical care, ICU use, and deaths under different social distancing scenarios for 3-week and 6-week periods starting from April 2, 2020. 3 University of Pennsylvania used CHIME, COVID-19 Hospital Impact Model, and predicted for the next three months. 4 

A group of research centers and universities, Fogarty International Center, Fred Hutchison Cancer Center, Northeastern University, University of Florida, and more employed the agent-based COVID-19 prediction model. 5 They forecasted based on two different scenarios: a) no mitigation and b) stay-at-home. Compared to actual data, their range can be considered even though the range is really wide.

Curve-fitting model can be described by considering Fig. 5 .4, where polynomial regression models are studied. As COVID-19 predictions are complex by nature, linear regression (described by the linear modelŷ = w 1 x + w o ) does not fit, and therefore no estimation is possible. In such a case, considering higher-order models would be a better fit, and that would bring an idea of higher-order polynomial regression (ŷ = w k x k + · · · + w 2 x 2 + w 1 x 1 + w o ). In the figure, we address a better fit by tuning a parameter k by taking an error rate for each degree of freedom (DoF) into account. In other words, we fix the value of k when we find the best fit. In Fig. 5 .4, k = 6 is the best fit for this data, and we conclude the regression model. Let us summarize the data: 

On the whole, curve-fitting model is brittle. It does not predict the long-term behavior. In Fig. 5 .4, we predict possible confirmed cases for the next five days. Los Alamos National Laboratory (LANL) and the Institute for Health Metrics and Evaluation (IHME) employed a curve-fitting technique. 6,7 LANL's best guess was for California state as of April 08, 2020, 4,082 deaths (compared to 2,974 actual deaths, dated May 14, 2020). IHME's predictions varied over time.

In the literature, models are transparent enough in terms of how they were built. However, their predictions were far from actual values. Not to be confused, as input parameters vary, their models forecasted different results. Besides, no models integrate factors, such as social distancing and/or 100% lockdown. Other than aforementioned three different models, the authors used machine learning and/or deep learning models that are built on statistics and probabilities.

In machine learning, we call such models "garbage-in garbage-out", 8, 9 as they predict values far from what they are. Stochastic models require fairly large amount of data to tune/stabilize their randomly generated parameters. Unlike the data-independent or discrete model, we are now required to employ mathematically proved data-driven models that have luxury to dynamically tune parameters over time.

It is a time to revisit how complex a model can be if we consider unprecedented events/factors (including social factors). As scientists, we do not like to limit to win over others in terms of validation; we rather focus on developing a prediction tool that is scalable and generalizable for any upcoming infectious disease outbreaks. Within the scope, it is a time to see whether deep neural networks 10 can be realized with thousands of parameters. Studying all data analytical tools is limited to education and training [5, 6] . In case we consider using data science and 6 Confirmed and Forecasted Cased Data Model: https://covid-19.bsvgateway.org. Los Alamos National Laboratory (June 20, 2020, last accessed). 7 The Institute for Health Metrics and Evaluation (IHME) COVID-19 Model: https://covid19.healthdata.org/ united-states-of-america (June 20, 2020, last accessed). 8 deep learning models, their numbers of hyperparameters could potentially supersede the size of input data. In such a case, the model technically works for hyperparameters, not for input data. This is an alarming event for data science and machine learning scientists.

To sum up, the existing literature includes heavily parameterized computationally expensive tools. They neither consider unavoidable social factors nor include immunocompromised population density in that particular region. Besides, impact of immunocompromised persons (e.g., lung cancer) on mortality and/or recovery rates in COVID-19 era was not revisited [4, 11] . This brings an idea of nested statistical models, where the parameters must be data-driven, and the social factors are considered as weights. The parameters are statistically adjusted based on social factors, and therefore they are dynamic in nature. Then such datadriven parameters (in terms of weights) are integrated with another statistical model for a better prediction. This, in long-term, could benefit to predict any possible infectious disease outbreaks.

Collecting large amount of data is not trivial, and we have to wait for a long time. Most of the reported AI-driven tools are limited to proof-of-concept models for coronavirus case. AI experts state that limited data may skew results away from the severity of coronavirus outbreak. The Wall Street Journal 11 reported that coronavirus reveals limits of AI health tools: some diagnostic-app makers hold off updating their tools, highlighting the shortage of data on the new coronavirus and the limitations of health services billed as AI when faced with novel, fast-spreading illnesses (Parmy Olson, February 29, 2020). In a nutshell, social medias, newspapers, and health reports, we note that conventional AI-driven tools for real-world cases (with less data) may not provide optimal performance. Unlike other healthcare issues, for COVID-19, to detect COVID-19, AI-driven tools are expected to have AL-based cross-population train/test models that employ multitudinal and multimodal data [1] . In Fig. 5 .5, we provide a better understanding of AL (in dotted red circle) with deep learning (DL) for all possible data types. In AL, expert's feedback is used in parallel with the decisions from each data type. Since DL are data dependent, separate DLs are used for different data types. The final decision is made based on multitudinal and multimodal data. For a quick understanding, two different image data are shown in Figs. 5.6 and 5.7 with clinical manifestations (for COVID-19). 

Following previous research papers, let us summarize deep neural networks (DNNs). As mentioned earlier, among radiological imaging data, Chest X-rays (CXRs) are of great use in observing COVID-19 manifestations. For mass screening, using CXRs, a computationally efficient AI-driven tool must detect COVID-19-positive cases from non-COVID ones (including healthy cases as well). 

Motivated by the fact that X-ray imaging systems are more prevalent and cheaper than CT scan systems, we proposed a deep learning-based CNN model, which we call Truncated Inception Net (see Fig. 5 .8). Our aim is to detect COVID-19 positive cases from non-COVID and/or healthy cases using chest X-rays.

To validate our proposal, we employed six different types of datasets by taking the following CXRs into account: COVID-19 positive, pneumonia positive, tuberculosis positive, and healthy cases [12] [13] [14] (see Table 5 .1). For better understanding, activation maps are shown in Fig. 5 .9.

The model achieved an accuracy of 99.96% (AUC of 1.0) in classifying COVID-19 positive cases from combined pneumonia and healthy cases. Similarly, an accuracy of 99.92% (AUC of 0.99) in classifying COVID-19 positive cases from combined pneumonia, tuberculosis, and healthy CXRs was reported. We proved the viability of using the proposed Truncated Inception Net as a screening tool. For more information, we refer to [7] . 

In [2] , we proposed a light-weight CNN-tailored shallow architecture that can automatically detect COVID-19-positive cases using CXRs. We aimed no false negatives in our experiments. The shallow CNN-tailored architecture was designed with fewer parameters as compared to other deep learning models. For this, we refer the readers to Fig. 5.10 and Table 5 .2.

The shallow CNN-tailored architecture was validated using 321 COVID-19-positive CXRs.

In addition to COVID-19-positive cases, another set of non-COVID-19 5856 cases (publicly available, source: Kaggle) was taken into account, consisting of normal, viral, and bacterial pneumonia cases. For a better visual understanding, feature maps for COVID-19 and pneumonia cases are shown in Fig. 5 .11. Using 5-fold cross-validation, we achieved the highest possible accuracy of 99.69%, sensitivity of 1.0, where AUC was 0.9995. The results were taken for a comparison with other existing deep learning models. For a comparison, the same exact set of experimental datasets was applied to other popular DL architectures, such as Mo-bileNet [15] , InceptionV3 [16] , and ResNet50 [17] . The results are provided in Table 5 .3. 

For COVID-19 screening purpose, multiple image modalities would provide higher confidence in decision-making. As chest CT and X-rays provide consistent COVID-19 manifestations [18, 19] , both can be considered. They can help predict, screen, and diagnose COVID-19 positive cases.

Within this scope, imaging with chest CT and X-ray images is widely used in mass triage situations. In the literature, AI-driven tools are limited to one data type, either CT scan or CXR, to detect COVID-19 positive cases. Integrating multiple data types could possibly provide more information in detecting anomaly patterns due to COVID-19. A CNN-tailored DNN that can collectively train/test both chest CT and X-rays is shown in Fig. 5 .12, and its corresponding parameters are provided in Table 5 .4.

Using a data collection (see Table 5 .5), such a DNN architecture achieved an overall accuracy of 96.28% (AUC = 0.9808). Performance scores (using a complete dataset CXRs + CT scans) are provided in Table 5 .6. The uniqueness behind this work is that no existing models worked on two different modalities, chest CT and X-ray images. It opens a new window for machine learning scientists that two modalities can be used in one DNN.

In what hollows, we elaborate on the use of image data for COVID-19 screening, where the focusing point is their performance in accordance with the dataset size. As mentioned earlier, we consider both image modalities, chest CT and X-ray images. For a thorough study, we refer to [23] .

For COVID-19, we elaborate on the use of chest CT imaging methods based on the performance by taking dataset size into account (see Table 5 .7). Farid et al. [24] devised a CNN-based approach to classify COVID-19 and SARS images (51 each class). Using 10-fold cross validation, they reported an accuracy of 94.11%. Tables 5.7 and 5.8) have used different feature extractors, decision-making processes, and experimental setups. More importantly, for COVID-19, their dataset sizes are varied over time, and so the sources are. For a fair analysis, let us not discuss on their methodologies and/or techniqut rather focus on the dataset size. We then elaborate on the strength of machine learning and deep learning algorithms by taking the following factors into account, such as fitting, transfer learning in the era of deep learning, and data augmentation. For easy understanding, we organize research papers, in both Tables 5.7 and 5.8, in accordance with the dataset size. In machine learning, we state that the bigger the data, the better the performance. This is true as we are looking at collecting all possible COVID-19 manifestations, rather than just increasing number of images. We have not observed better results from bigger datasets. If so, then how big data is big? Machine learning tools require to learn all possible manifestations related to particular diseases (COVID-19 in our case) not just the size of the dataset. However, the dataset size opens the possibility of having new cases (i.e., manifestations), which is always not the case. Underfitting and overfitting situations are not explicitly discussed/analyzed in all these aforementioned COVID-19 screening tools (see Tables 5.7 and 5.8). They rather engaged in producing better performance scores by tuning (hyper)parameters, due to which biased results are possible. In general, transfer learning works relatively greatly in computer vision. It may not work as expected for COVID-19 screening when we consider both image data types, chest CT and X-rays.

COVID-19 dataset (Kaggle): 260 images COVID-19 +ve (130)

dataset: 2, 905 images COVID-19 (219) + pneumonia (1, 345) + normal (1, 341) 98.97, 99

NIH Chest X-ray: 6, 523 images COVID-19 (250) + pulmonary (2, 753) + normal (3, 520) 97, −

Dataset (Kaggle): 1, 251 images COVID-19 (284) + bac (330) + viral (327) + normal (310) 89

A chest (pneumonia) and a COVID-19 dataset: 1, 508 images COVID-19 +ve (504) + pneumonia (504) + normal (500) 99

Ai-driven tools for coronavirus outbreak: need of active learning and cross-population train/test models on multitudinal/multimodal data

Shallow convolutional neural network for Covid-19 outbreak screening using chest X-rays

Deep neural network to detect COVID-19: one architecture for both CT scans and chest X-rays

COVID-19 prediction models and unexploited data

Intelligent Systems and Methods to Combat Covid-19

Covid-19: Prediction, Decision-Making, and Its Impacts

Truncated inception net: Covid-19 outbreak screening using chest X-rays

Deep learning and medical diagnosis

The role of augmented intelligence (AI) in detecting and preventing the spread of novel coronavirus

Covid-19 models: can they tell us what we want to know?

Revisited COVID-19 mortality and recovery rates: are we missing recovery time period?

Covid-19 image data collection

Kaggle chest X-ray images (pneumonia) dataset

Tuberculosis chest x-ray image data sets

An enhanced hybrid mobilenet

Rethinking the inception architecture for computer vision

Extremely large minibatch SGD: training ResNet-50 on Ima-geNet in 15 minutes, CoRR

Covid-19 pneumonia manifestations at the admission on chest ultrasound, radiographs, and CT: single-center study and comprehensive radiologic literature review

Lung imaging: how to get better look inside the lung

Covid-19 imaging tools: how big data is big?

A novel approach of ct images feature analysis and prediction to screen for corona virus disease (Covid-19)

Classification of Covid-19 coronavirus, pneumonia and healthy lungs in ct scans using qdeformed entropy and deep learning features

A deep transfer learning model with classical data augmentation and CGAN to detect Covid-19 from chest ct radiography digital images

Afshin Mohammadi, Application of deep learning technique to manage Covid-19 in routine clinical practice using ct images: results of 10 convolutional neural networks

Explainable Covid-19 detection using chest CT scans and deep learning

A deep learning approach to characterize 2019 coronavirus disease (Covid-19) pneumonia in chest CT images

Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography

Covid-2019 detection using X-ray images and artificial intelligence hybrid models

X-ray image based Covid-19 detection using pre-trained deep learning models, engrXiv

A modified deep convolutional neural network for detecting Covid-19 and pneumonia from chest X-ray images based on the concatenation of Xception and ResNet50V2

Kemal Polat, A novel medical diagnosis model for Covid-19 infection detection based on deep features and Bayesian optimization

Explainable deep learning for pulmonary disease and coronavirus Covid-19 detection from X-rays

Coronet: a deep neural network for detection and diagnosis of Covid-19 from chest X-ray images

Isabel de la Torre Díez, Automated medical diagnosis of Covid-19 through efficient net convolutional neural network