key: cord-0556562-5zxkd2jy
authors: Schwab, Patrick; Schutte, August DuMont; Dietz, Benedikt; Bauer, Stefan
title: predCOVID-19: A Systematic Study of Clinical Predictive Models for Coronavirus Disease 2019
date: 2020-05-17
journal: nan
DOI: nan
sha: 1eed2b953b4f2fa64c2779b98f6e7b8cb92278de
doc_id: 556562
cord_uid: 5zxkd2jy

Coronavirus Disease 2019 (COVID-19) is a rapidly emerging respiratory disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Due to the rapid human-to-human transmission of SARS-CoV-2, many healthcare systems are at risk of exceeding their healthcare capacities, in particular in terms of SARS-CoV-2 tests, hospital and intensive care unit (ICU) beds and mechanical ventilators. Predictive algorithms could potentially ease the strain on healthcare systems by identifying those who are most likely to receive a positive SARS-CoV-2 test, be hospitalised or admitted to the ICU. Here, we study clinical predictive models that estimate, using machine learning and based on routinely collected clinical data, which patients are likely to receive a positive SARS-CoV-2 test, require hospitalisation or intensive care. To evaluate the predictive performance of our models, we perform a retrospective evaluation on clinical and blood analysis data from a cohort of 5644 patients. Our experimental results indicate that our predictive models identify (i) patients that test positive for SARS-CoV-2 a priori at a sensitivity of 75% (95% CI: 67%, 81%) and a specificity of 49% (95% CI: 46%, 51%), (ii) SARS-CoV-2 positive patients that require hospitalisation with 0.92 AUC (95% CI: 0.81, 0.98), and (iii) SARS-CoV-2 positive patients that require critical care with 0.98 AUC (95% CI: 0.95, 1.00). In addition, we determine which clinical features are predictive to what degree for each of the aforementioned clinical tasks. Our results indicate that predictive models trained on routinely collected clinical data could be used to predict clinical pathways for COVID-19, and therefore help inform care and prioritise resources.

C ORONAVIRUS Disease 2019 (COVID- 19) was first discovered in December 2019 in China, and has since rapidly spread to over 200 countries [1] . The COVID-19 pandemic challenges healthcare systems worldwide as a high peak capacity for testing and hospitalisation is necessary to diagnose and treat affected patients, particularly if the spread of SARS-CoV-2 is not mitigated. To avoid exceeding the available healthcare capacities, many countries have adopted social distancing policies, imposed travel restrictions, and postponed non-essential care and surgeries in order to reduce peak demand on their healthcare systems [2] , [3] , [4] . We study the use of predictive models (light purple) to estimate whether patients are likely (i) to be SARS-CoV-2 positive, and whether SARS-CoV-2 positive patients are likely (ii) to be admitted to the hospital and (iii) to require critical care based on clinical, demographic and blood analysis data. Accurate clinical predictive models stratify patients according to individual risk, and, in this manner, help prioritise healthcare resources, such as testing, hospital and critical care capacity.

The adoption of clinical predictive models that accurately predict who is likely to require testing, hospitalisation and intensive care from routinely collected clinical data could potentially further reduce peak demand by ensuring resources are prioritised to those individuals with the highest risk ( Figure  1 ). For example, a clinical predictive model that accurately identifies patients that are likely to test positive for SARS-CoV-2 a priori could help prioritise limited SARS-CoV-2 testing capacity. However, developing accurate clinical prediction models for SARS-CoV-2 is difficult as relationships between clinical data, hospitalisation, and intensive care unit (ICU) admission have not yet been established conclusively due to the recent emergence of SARS-CoV-2.

In this systematic study, we develop and evaluate clinical predictive models that use routinely collected clinical data to identify (i) patients that are likely to receive a positive SARS-CoV-2 test, (ii) SARS-CoV-2 positive patients that are likely to require hospitalisation, and (iii) SARS-CoV-2 positive patients that are likely to require intensive care. Using the developed predictive models, we additionally determine which clinical features are most predictive for each of the aforementioned clinical tasks. Our results indicate that predictive models could be used to predict clinical pathways for COVID-19 patients. Such predictive models may be of significant utility for healthcare systems as preserving healthcare capacity has been linked to successfully combating SARS-CoV-2 [5] , [6] . This work contains the following contributions:

• We develop and systematically study predictive models for estimating the likelihoods of (i) a positive SARS-CoV-2 test in patients presenting at hospitals, (ii) hospital admission in SARS-CoV-2 positive patients, and (iii) critical care admission in SARS-CoV-2 positive patients. • We validate the performance of the developed clinical predictive models in a retrospective evaluation using realworld data from a cohort of 5644 patients. • We determine and quantify the predictive power of routinely-collected clinical, demographic, and blood analysis data for the aforementioned clinical prediction tasks.

A substantial body of work is dedicated to the study, validation and implementation of predictive models for clinical tasks. Clinical predictive models have, for example, been used to predict risk of septic shock [7] , [8] , risk of heart failure [9] , readmission following heart failure [10] , [11] , [12] , false alarms in critical care [13] , risk scores [14] , outcomes [15] and mortality in pneumonia [16] , [17] , and mortality risk in critical care [18] , [19] , [20] . Predicting clinical outcomes for individual patients is difficult because a large number of confounding factors may influence patient outcomes, and collecting and accounting for these factors in an unbiased way remains an open challenge in clinical practice [21] . Systematic studies, such as the one presented in this work, enable medical practitioners to better understand, assess and potentially overcome these issues by systematically evaluating generalisation ability, expected predictive performance, and influential predictors of various clinical predictive models. Beyond the need for systematic evaluation, missingness [22] , [23] , [24] , [25] , noise [26] , [27] , multivariate input data [13] , [28] , [29] , [30] , and the need for interpretability [31] , [32] , [33] , [34] have been highlighted as particularly important considerations in healthcare settings. In this work, we build on recent methodological advances to develop and systematically study clinical predictive models that may aid in prioritising healthcare resources [35] for COVID-19, and thereby help prevent a potential overextension of healthcare system capacity.

Several clinical predictive models have recently been proposed for COVID-19, for example, for predicting potential COVID-19 diagnoses using data from emergency care admission exams [36] and chest imaging data [37] , [38] , [39] , [40] , [41] , [42] , for predicting COVID-19 related mortality from clinical risk factors [43] , [44] , and for predicting which patients will develop acute respiratory distress syndrome (ARDS) from patients' clinical characteristics [45] . [46] presented a review of epidemiology and clinical features associated with COVID-19, and [47] a critical review that assessed limitations and risk of bias in diagnostic and prognostic models for COVID-19. In addition, [48] performed a cohort study for clinical and laboratory predictors of COVID-19 related inhospital mortality that identified baseline neutrophil count, age Fig. 2 : The presented multistage machine-learning pipeline consists of preprocessing (light purple) the input data x, developing multiple candidate models using the given dataset (orange), selecting the best candidate model for evaluation (blue), and evaluating the selected best model's outputsŷ.

and several other clinical features as top predictors of mortality. Beyond prediction, [49] have argued for the responsible use of data in tackling the challenges posed by SARS-CoV-2.

Owing to the recent emergence of SARS-CoV-2, there currently exists, to the best of our knowledge, no prior systematic study on clinical predictive models that predict likelihood of a positive SARS-CoV-2 test, hospital and intensive care unit admission from clinical, demographic and blood analysis data that accounts for the missingness that is characteristic for the clinical setting. We additionally assess the influence of various clinical, demographic, and blood analysis measurements on the predictions of the developed clinical predictive models.

1) Problem Setting: In the given setting, we are given 106 routine clinical, laboratory and demographic measurements, or features, x i ∈ x for presenting patients. Features may be discrete or continuous, and some features may be missing as not all tests are necessarily performed on all patients. The clinical predictive tasks consist of utilising the routine clinical features x i to predict, for a newly presenting patient, (i) the likelihoodŷ SARS-CoV-2 of receiving a positive SARS-CoV-2 test result, (ii) the likelihoodŷ admission of requiring hospital admission, and the (iii) likelihoodŷ ICU of requiring intensive care. In addition, we are given a development dataset consisting of N patients, their corresponding observed routine clinical features x i , SARS-CoV-2 test results y SARS-CoV-2 ∈ {0, 1}, hospital admissions y admission ∈ {0, 1}, and ICU admissions y ICU ∈ {0, 1}, where 1 indicates the presence of an outcome. Using this development dataset, our goal is to derive clinical predictive modelsf SARS-CoV-2 ,f admission andf ICU for the respective before-mentioned tasks in order to inform care and help prioritise scarce healthcare resources.

2) Methodology: To derive the clinical predictive modelŝ f SARS-CoV-2 ,f admission andf ICU from the given development dataset, we set up a systematic model development, validation, and evaluation pipeline (Fig. 2) . To evaluate the generalisation ability of the developed clinical predictive models and to rule out overfitting to patients in the evaluation cohort, the development data is initially split into independent and stratified training, validation, and test folds without any patient overlap. Concretely, the multistage pipeline consists of (i) preprocessing, (ii) model development, (iii) model selection, and (iv) model evaluation stages. For preprocessing and model development, only the training fold is used, and only the validation and test folds of the development data are used for model selection and model evaluation, respectively. We outline the pipeline stages in detail in the following paragraphs.

3) Preprocessing: In the preprocessing stage, we first drop all input features that are missing for more than 99.8% of all training set patients to ensure we have a minimal amount of data for each feature. This removes a total of 9 features from the original 106 routine clinical, laboratory and demographic features. We then transform all discrete features for each patient into their one-hot encoded representation with one out of p indicator variables set to 1 to indicate the discrete value for this patient, and all others set to 0 with p being the number of unique values for the discrete feature. We defined those features as discrete that have fewer than 6 unique values across all patients in the training fold. For discrete features, missing features were counted as a separate category in the one hot representation. Next, we standardised all continuous features to have zero mean and unit standard deviation across the training fold data. Lastly, we performed multiple imputation by chained equations (MICE) to impute all missing values of every continuous feature from the respective other features in an iterative fashion [50] . We additionally added a missing indicator that indicates 1 if the feature was imputed by MICE and 0 if it was originally present in order to preserve missingness information in the data after imputation. After the preprocessing stage, continuous input features are standardised and fully imputed, and discrete input features are one-hot encoded. All preprocessing operations are derived only from the training fold, and naïvely applied without adjustment to validation and test folds in order to avoid information leakage. 4) Model Development: In the model development stage, we train candidate clinical predictive modelsf SARS-CoV-2 ,f admission andf ICU using supervised learning on the training fold of the preprocessed data. To derive the models from the preprocessed training fold data, we optimise various types of predictive models, and perform a hyperparameter search with m runs for each of them. The model development process yields m candidate models with different hyperparameter choices and predictive performances for each model category.

5) Model Selection: In order to select the best model amongst the set of candidate models, we evaluate their predictive performance against the held-out validation fold that had not been used for model development. We choose the top candidate model by ranking all models by their evaluated predictive performance. The model selection stage using the independent validation fold enables us to optimise hyperparameters without utilising test fold data. 6) Model Evaluation: In the model evaluation stage, we evaluate the selected best clinical predictive model against the held-out test fold that had not been used neither for training nor model selection in order to estimate the expected generalisation error of the models on previously unseen data.

Using this approach, every selected best model from the model selection stage is evaluated exactly once against the test fold.

Using the presented standardised model development, selection and evaluation pipeline, we compare various types of clinical predictive models in the same test setting, with exactly the same amount of hyperparameter optimisation and input features against the same test fold. This process enables us to systematically study the expected generalisation ability, predictive performance and influential features of clinical predictive models for predicting SARS-CoV-2 test results, hospital admission for SARS-CoV-2 positive patients, and ICU admission for SARS-CoV-2 positive patients.

We conducted retrospective experiments to evaluate the predictive performance of a number of clinical predictive models on each of the presented clinical prediction tasks using the standardised development, validation and evaluation pipeline. Concretely, our experiments aimed to answer the following questions:

1 What is the expected predictive performance of the various clinical predictive models in predicting (i) SARS-CoV-2 test results for presenting patients, (ii) hospital admission for SARS-CoV-2 positive patients, and (iii) ICU admission for SARS-CoV-2 positive patients? 2 Which clinical, demographic and blood analysis features were most important for the respective best encountered predictive models for each clinical prediction task? The following subsections describe the conducted experimental evaluation in detail.

We used anonymised data from a cohort of 5644 patients seen at the Hospital Israelita Albert Einstein in São Paulo, Brazil in the early months of 2020 1 . Over the data collection time frame, the rate of SARS-CoV-2 positive patients at the hospital was around 10% of which around 6.5% and 2.5% required hospitalisation and critical care, respectively (Table I) .

Notably, younger patients were underrepresented in the SARS-CoV-2 positive group relative to the general patient population which may have been caused by the reportedly more severe disease progression in older patients [51] . Information on patient sex was not included in our dataset. We randomly split the entire available patient cohort into training (50%), validation (20%) and test folds (30%) within strata of patient age, SARS-CoV-2 test result, hospital admission status, and ICU admission status. After stratification, the three folds were approximately balanced across the stratification dimensions.

Using the presented systematic evaluation methodology, we trained five different model types: Logistic Regression (LR), Neural Network (NN), Random Forest (RF), Support Vector Machine (SVM), and Gradient Boosting (XGB) [52] . The NN was a multi-layer perceptron (MLP) consisting of L hidden layers with N hidden units each followed by a non-linear activation function (ReLU [53] , SELU [54] , or ELU [55] ) and batch normalisation [56] , and was trained using the Adam optimiser [57] for up to 300 epochs with an early stopping patience of 12 epochs on the validation set loss.

We followed an unbiased, systematic approach to hyperparameter selection and optimisation. For each type of clinical predictive model, we performed a maximum of 30 hyperparameter optimisation runs with hyperparameters chosen from predefined ranges (Table II) . The performance of each hyperparameter optimisation run was evaluated against the validation cohort. After computing the validation set performance, we selected the best candidate predictive model across the 30 hyperparameter optimisation runs by area under the receiver operator curve for further evaluation against the test set.

D. Metrics 1) Predictive Performance: To assess the predictive performance of each of the developed clinical predictive models, we evaluated their performance in terms of area under the receiver operator curve (AUC), area under the precision recall curve (AUPR), sensitivity, specificity, and specificity at greater than 95% sensitivity (Spec.@95%Sens.) on the held-out test set cohorts for each task (Table I) . After model development and hyperparameter optimisation, we evaluated each model type exactly once against the test set to calculate the final performance metrics. Operating thresholds for each model were the operating points on the receiver operator characteristic curve closest to the top left coordinate as calculated for the validation cohort. We chose a variety of complementary evaluation metrics in order to give a comprehensive picture of the expected performance of each clinical predictive model on the evaluated tasks. For each of the performance metrics, we additionally computed 95% confidence intervals (CIs) using bootstrap resampling with 100 bootstrap samples on the test set cohort in order to quantify the uncertainty of our analysis results. We also assessed whether differences between clinical predictive models were statistically significant at significance level α = 0.05 using pairwise t-tests with the respective best models for each task as measured by AUC.

2) Importance of Test Types: To quantify the importance of specific clinical, demographic and blood analysis features on each of the predicted outcomes, we utilised causal explanation (CXPlain) models [34] . CXPlain provides standardised relative feature importance attributions for any predictive model by computing the marginal contribution of each input feature towards the predictive performance of a model [58] , and is therefore particularly well-suited for assessing feature importance in our diverse set of models. We used the test fold's ground truth labels to compute the exact marginal contribution of each input feature without any estimation uncertainty.

In terms of predictive performance (Table III) , we found that the overall best identified models by AUC were XGB for predicting SARS-CoV-2 test results, RF for predicting hospital admissions for SARS-CoV-2 positive patients, and SVM for predicting ICU admission for SARS-CoV-2 positive patients . Notably, we found that predicting positive SARS-CoV-2 results from routinely collected clinical measurements was a considerably more difficult task for clinical predictive models than predicting hospitalisation and ICU admission. Nonetheless, the best encountered clinical predictive model for predicting SARS-CoV-2 test results (XGB) achieved a respectable sensitivity of 75% (95% CI: 67%, 81%) and specificity of 49% (95% CI: 46%, 51%). After fixing the operating threshold of the model to meet a sensitivity level of at least 95% (Spec.@95% Sens.), the best XGB model for predicting SARS-CoV-2 test results would achieve a specificity of 23% (95% CI: 7%, 32%). We additionally found that the differences in predictive performance between the best XGB model for predicting SARS-CoV-2 test results and the other predictive models was significant at a pre-specified significance level of α = 0.05 (ttest) for all but the AUPR metric, where NN achieved a significantly better AUPR of 0.22 and the difference to SVM was not significant at the pre-specified significance level. On the task of predicting hospital admissions for SARS-CoV-2 positive patients, the best encountered RF model achieved a sensitivity of 55% (95% CI: 19%, 85%), a high specificity of 96% (95% CI: 92%, 98%), and a specificity at a fixed sensitivity of at least 95% (Spec.@95% Sens.) of 34% (95% CI: 29%, 97%).

Owing to the lower sample size due to the smaller cohort of SARS-CoV-2 positive patients, the performance results for predicting hospital admission generally had wider uncertainty bounds but were nonetheless significantly better for RF than the other predictive models at the pre-specified significance level of α = 0.05 (t-test) for most performance metrics with the exception of AUC where XGB achieved an AUC of 0.91 and AUPR where LR achieved an AUPR of 0.44. On the task of predicting ICU admission for SARS-CoV-2 positive patients, SVM had a sensitivity of 80% (95% CI: 36%, 100%), a specificity of 96% (95% CI: 92%, 98%), and a specificity at a fixed sensitivity of at least 95% (Spec.@95% Sens.) of 95% (95% CI: 91%, 100%). Due to the small percentage of around 3% of SARS-CoV-2 positive patients that were admitted to the ICU (Table I) , uncertainty bounds were wider than for the models predicting hospital admissions, and the results of the best encountered SVM were found to be not significantly better than LR and RF in terms of AUC, LR and NN in terms of sensitivity, and NN in terms of Spec.@95% Sens. at the pre-specified significance level of α = 0.05 (t-test).

In terms of feature importance, we found that importance scores were distributed highly unequally, relatively uniform and highly uniform for the best models encountered for predicting SARS-CoV-2 test results, for predicting hospital admissions for SARS-CoV-2 positive patients, and for predicting ICU admission, respectively (Figure 4 ). Most notably, we found that 71.7% of the importance for the best XGB model for predicting SARS-CoV-2 test results was assigned to the missing indicator corresponding to the Arterial Lactic Acid measurement, i.e. much of the marginal predictive performance gain of the XGB model was attributed to whether or not the Arterial Lactic Acid test had been ordered. Beyond Arterial Lactic Acid being missing, age, leukocyte count, platelet count, and creatinine were implied to be associated with a positive SARS-CoV-2 test result by the best encountered predictive model, which further substantiates recent independent reports of those factors being potentially associated with SARS-CoV-2 [59] , [60] , [61] , [62] , [48] . Similarly to the best encountered XGB model for predicting SARS-CoV-2 test results, the top encountered predictive models for hospital admission and ICU admission for SARS-CoV-2 positive patients assigned a considerable degree of importance to missingness patterns associated with a number of measurements. A possible explanation for missingness appearing as a top predictor across the different tasks is that decisions whether or not to order a certain test to be performed for a given patient were influenced by patient characteristics that were not captured in the set of clinical measurements that were available to the predictive models. A controlled setting with standardised testing guidelines would be required to determine which confounding factors are behind the predictive power of the missingness patterns that have been implied to be associated with COVID-19 by the predictive models. Beyond missingness patterns, top predictors for predicting hospital admission were lactate dehydrogenase [63] , gammaglutamyltransferase, which through abnormal liver function has been reported to be implicated in COVID-19 severity [64] , and HCO 3 [65] . For predicting ICU admission in SARS-CoV-2 positive patients, pCO 2 and pH [48] were top predictors. Blood pH, and in particular respiratory alkalosis, has been reported to be associated with severe COVID-19 [66] .

We presented a systematic study of predictive models that predict SARS-CoV-2 test results, hospital admission for SARS-CoV-2 positive patients, and ICU admission for SARS-CoV-2 positive patients using routinely collected clinical measurements. Models that predict SARS-CoV-2 test results could help prioritise scarce testing capacity by identifying those individuals that are more likely to receive a positive result. Similarly, predictive models that predict which SARS-CoV-2 positive patients would be most likely to require hospital and critical care beds could help better utilise existing hospital capacity by prioritising those patients that have the highest risk of deterioration. Facilitating the efficient utilisation of scarce healthcare resources is particularly important in dealing with SARS-CoV-2 as its rapid transmission significantly increases demand for healthcare services worldwide. The main limitation of the presented study is that its experimental evaluation was based on data collected from a single study site, and its results may therefore not generalise to settings with significantly different patient populations, admission criteria, patterns of missingness, and testing guidelines. In addition, we did not have access to mortality data for the analysed cohort, and we were therefore not able to correlate our predicted individual risk scores with patient mortality, which is another related prediction task that may be of clinical importance. Future studies should include a broader set of clinical measurements and outcomes, cohorts from multiple distinct geographical sites and under varying patterns of missingness in order to determine the robustness of the clinical predictive models to these confounding factors. Finally, we believe that the inclusion of data from other modalities, such as genomic profiling and medical imaging, and data on co-morbidities, symptoms and treatment histories could potentially further improve predictive performance of clinical predictive models across the presented prediction tasks.

We presented a systematic study in which we developed and evaluated clinical predictive models for COVID-19 that estimate (i) the likelihood of a positive SARS-CoV-2 test in patients presenting at hospitals, (ii) the likelihood of hospital admission and (iii) intensive care unit admission in SARS-CoV-2 positive patients. We evaluated our developed clinical predictive models in a retrospective evaluation using a cohort of 5644 hospital patients seen in São Paulo, Brazil. In addition, we determined the clinical, demographic and blood analysis measurements that were most important for accurately predicting SARS-CoV-2 status, hospital admissions, and ICU admissions. Our experimental results indicate that clinical predictive models may in the future potentially be used to inform care and help prioritise scarce healthcare resources by assigning personalised risk scores for individual patients using routinely collected clinical, demographic and blood analysis data. Furthermore, our findings on the importance of routine clinical measurements towards predicting clinical pathways for patients increase our understanding of the interrelations of individual risk profiles and outcomes in SARS-CoV-2. Based on our study's results, we conclude that healthcare systems should explore the use of predictive models that assess individual COVID-19 risk in order to improve healthcare resource prioritisation and inform patient care.

The anonymised data used in this manuscript were generously contributed by patients at Hospital Israelita Albert Einstein in São Paulo, Brazil, and are freely available at https://www.kaggle.com/einsteindata4u/covid19.

Coronavirus disease (COVID-19) Pandemic

The effect of travel restrictions on the spread of the 2019 novel coronavirus

Update: Public health response to the coronavirus disease 2019 outbreak -United States

A conceptual model for the coronavirus disease 2019 (COVID-19) outbreak in Wuhan, China with individual reaction and governmental action

Response to COVID-19 in Taiwan: Big data analytics, new technology, and proactive testing

Interrupting transmission of COVID-19: Lessons from containment efforts in Singapore

A targeted real-time early warning score (TREWScore) for septic shock

Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning

Prediction modeling using EHR data: Challenges, strategies, and a comparison of machine learning approaches

Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: Comparison of machine learning and other statistical approaches

Predictive modeling of hospital readmission rates using electronic medical recordwide machine learning: A case-study using Mount Sinai heart failure cohort

A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: A retrospective analysis of electronic medical records data

Not to cry wolf: Distantly supervised multitask learning in critical care

Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission

Patient-specific models for predicting the outcomes of patients with community acquired pneumonia

An evaluation of machine-learning methods for predicting pneumonia mortality

Using data-driven rules to predict mortality in severe community acquired pneumonia

Predicting hospital mortality for patients in the intensive care unit: A comparison of artificial neural networks with logistic regression models

Unfolding physiological state: Mortality modelling in intensive care units

Reproducibility in critical care: A mortality prediction case study

Predicting the futureâȂŤbig data, machine learning, and clinical medicine

Multiple imputation versus data enhancement for dealing with missing data in observational health care outcome analyses

Strategies for handling missing data in electronic health record derived data

Directly modeling missing data in sequences with RNNs: Improved classification of clinical time series

Recurrent neural networks for multivariate time series with missing values

Computational phenotype discovery using unsupervised feature learning over noisy, sparse, and irregular clinical data

Beat by beat: Classifying cardiac arrhythmias with recurrent neural networks

PhoneMD: Learning to diagnose Parkinson's disease from smartphone data

A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data

A Deep Learning Approach to Diagnosing Multiple Sclerosis from Smartphone Data

Retain: An interpretable predictive model for healthcare using reverse time attention mechanism

Towards a rigorous science of interpretable machine learning

Right for the right reasons: Training differentiable models by constraining their explanations

CXPlain: Causal Explanations for Model Interpretation under Uncertainty

Machine learning and prediction in medicineâȂŤbeyond the peak of inflated expectations

COVID-19 diagnosis prediction in emergency care patients: A machine learning approach

Covid-net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images

Automatic detection of coronavirus disease (COVID-19) using x-ray images and deep convolutional neural networks

Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT

Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: A first experience from Lombardy

A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19)

Deep learning system to screen coronavirus disease 2019 pneumonia

A Machine Learning Model Reveals Older Age and Delayed Hospitalization as Predictors of Mortality in Patients with COVID-19

Prediction of criticality in patients with severe COVID-19 infection using three clinical features: a machine learning-based prognostic model with clinical data in Wuhan

Towards an artificial intelligence framework for data-driven prediction of coronavirus clinical severity

Epidemiology and Clinical Features of COVID-19: A Review of Current Literature

Prediction models for diagnosis and prognosis of COVID-19 infection: Systematic review and critical appraisal

Clinical and Laboratory Predictors of In-Hospital Mortality in 305 Patients with COVID-19: A Cohort Study in Wuhan, China

On the responsible use of digital data to tackle the COVID-19 pandemic

Multiple imputation using chained equations: Issues and guidance for practice

COVID-19 and Italy: what next

Xgboost: A scalable tree boosting system

Rectified linear units improve restricted Boltzmann machines

Selfnormalizing neural networks

Fast and accurate deep network learning by exponential linear units (ELUs)

Batch normalization: Accelerating deep network training by reducing internal covariate shift

Adam: A method for stochastic optimization

Granger-causal attentive mixtures of experts: Learning important features with neural networks

Severe outcomes among patients with coronavirus disease 2019 (COVID-19) -United States

The epidemiology and pathogenesis of coronavirus disease (COVID-19) outbreak

Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: A metaanalysis

Kidney disease is associated with in-hospital death of patients with COVID-19

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study

Clinical features of COVID-19-related liver damage

Respiratory Support in Novel Coronavirus Disease (COVID-19) Patients, with a Focus on Resource-Limited Settings

COVID-19 in Iran, a comprehensive investigation from exposure to treatment outcomes