key: cord-0304651-5vlowyb7
authors: Zucco, A. G.; Agius, R.; Svanberg, R.; Moestrup, K. S.; Marandi, R. Z.; MacPherson, C. R.; Lundgren, J.; Ostrowski, S. R.; Niemann, C. U.
title: Personalized survival probabilities for SARS-CoV-2 positive patients by explainable machine learning
date: 2021-10-29
journal: nan
DOI: 10.1101/2021.10.28.21265598
sha: 7c3612bd1e6e8137e3b59c6379535509b09a9db5
doc_id: 304651
cord_uid: 5vlowyb7

Interpretable risk assessment of SARS-CoV-2 positive patients can aid clinicians to implement precision medicine. Here we trained a machine learning model to predict mortality within 12 weeks of a first positive SARS-CoV-2 test. By leveraging data on 33,928 confirmed SARS-CoV-2 cases in eastern Denmark, we considered 2,723 variables extracted from electronic health records (EHR) including demographics, diagnoses, medications, laboratory test results and vital parameters. A discrete-time framework for survival modelling enabled us to predict personalized survival curves and explain individual risk factors. Performances of weighted concordance index 0.95 and precision-recall area under the curve 0.71 were measured on the test set. Age, sex, number of medications, previous hospitalizations and lymphocyte counts were identified as top mortality risk factors. Our explainable survival model developed on EHR data also revealed temporal dynamics of the 22 selected risk factors. Upon further validation, this model may allow direct reporting of personalized survival probabilities in routine care.

Coronavirus disease 2019 caused by infection with Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has by October 2021 claimed almost 5 million lives since its outbreak in late 2019 1 . Infected individuals present a variety of symptoms, ranging from asymptomatic to lifethreatening diseases 2 . Although the majority of cases experience mild to moderate disease approximately 15% of confirmed SARS-CoV-2 positive cases are estimated to develop severe disease 3 . Progression to severe disease seems to occur within 1-2 weeks from symptom onset, and is characterized by clinical signs of pneumonia with dyspnea, increased respiratory rate, and decreased blood oxygen saturation requiring supplemental oxygen [3] [4] [5] [6] [7] . Development of critical illness is driven by systemic inflammation, leading to acute respiratory distress syndrome (ARDS), respiratory failure, septic shock, multi-organ failure, and/or disseminated coagulopathy 4, 5, 8 . The majority of these patients require mechanical ventilation, and mortality for patients admitted to an Intensive Care Unit (ICU) is reported to be 32-50% 3, [8] [9] [10] . Despite the current vaccination program, both people already vaccinated and patients not being vaccinated continue to develop critical COVID-19 disease 11 . Thus, the pandemic still poses a great burden on health care systems worldwide, locally approaching the limit of capacity due to high patient burden and challenging clinical management.

Several factors associated with increased risk of severe disease course have been established including old age, male gender, and lifestyle factors such as smoking and obesity 12, 13 . Comorbidities including hypertension, type 2 diabetes, renal disease, as well as pre-existing conditions of immune dysfunction and cancer, are also associated with a higher risk of severe disease and COVID-19 related death 12, [14] [15] [16] . Among hospitalized patients, risk factors for severe disease or death include low lymphocyte counts, elevated inflammatory markers and elevated kidney and liver parameters indicating organ dysfunction 6 . However, many of these factors likely reflect an ongoing progression of COVID-19. Thus, identification of high-risk patients at or prior to hospital admission is warranted to facilitate personalized interventions.

Multiple COVID-19 prognostic models have been built on reduced sets of predictive features from demographics, patient history, physical examination, and laboratory results 17 processed by traditional statistical frameworks or machine learning (ML) algorithms. A systematic review of 50 prognostic models has concluded that overall such models have been poorly reported and are at a high risk of bias 18 . While great efforts have been put into providing prognostic models based on data collected from health systems, traditional modelling approaches solely based on domain knowledge may fail.

This represents a risk of missing novel markers and insights about the disease that could come from data-driven models in a hypothesis-free manner 19 , which have been reported to outperform models based on curated variables from domain experts 20 .

Furthermore, ML models facilitate clinical insights 21 when coupled with methods for model explainability such as SHapley Additive exPlanations (SHAP) values 22 . Model explainability has been developed mainly in the context of regression and binary classification, but in clinical research where censored observations are common, explainable time-to-event modelling is required to avoid selection bias 23, 24 . Multiple ML algorithms have been developed for time-to-event modelling, either by building on top of existing models such as Cox proportional hazards or by defining new loss functions that model time as continuous 25 . Here we used an alternative approach that considered time in discrete intervals and performed binary classification at such time intervals 26 . This allowed us to implement gradient boosting decision trees for binary classification to predict personalized survival probabilities 27 and allow explainability at the individual patient level using SHAP values 22 including temporal dynamics of risk factors over the course of the disease. This approach not only allows to predict personalized survival probabilities and risk factors for SARS-CoV-2 positive patients but also provides a framework for precision medicine that can be applied to other diseases based on routine electronic health records.

Based on centralized EHR and SARS-CoV-2 test results from test centers in eastern Denmark, we identified 33,938 patients who had at least one SARS-CoV-2 RT-PCR positive test from 963,265 individuals who had a test performed between 17th of March 2020 and 2 nd of March 2021 (Fig. 1) . In this cohort, 5,077 patients were hospitalized, of whom 502 were admitted to the ICU (Supplementary is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint Fig. 1) . Overall, 1,803 (5.34%) deaths occurred among all individuals with a positive SARS-CoV-2 RT-PCR test, of whom 141 died later than 12 weeks from the first positive test (FPT) hence considered as alive for this analysis. Right-censoring was only observed for patients tested after the 8 th of December 2020 with less than 12 weeks of follow-up available while deaths that occurred the same day of FPT were not considered for training. For the initial model, demographics, laboratory test results, hospitalizations, vital parameters, diagnoses, medicines (ordered and administered) and summary features were included. Feature encoding resulted in 2,723 features (Supplementary Table   2 ) which after feature selection were reduced to 23 features. A summary of the cohort based on the final feature set can be found in Table 1 . This cohort represents an updated subset of individuals residing in Denmark characterized in a previous publication 28 .

To predict the risk of death within 12 weeks from FPT, we trained gradient boosting decision trees considering time as discrete in a time-to-event framework. Performance was measured on 20% of the data (test set) unblinded only for performance assessment. The weighted concordance index (Cindex) for predicting risk of death for all 12 weeks with 95% confidence intervals (CI) was 0.946 (0.941-0.950). Binary metrics were calculated for each predicted week by excluding censored individuals ( Fig. 2) . At week 12, the precision-recall area under the curve (PR-AUC) and Mathew correlation coefficient (MCC) with 95% CI were 0.686 (0.651-0.720) and 0.580 (0.562-0.597) respectively. The sensitivity was 99.3% and the specificity was 86.4%. The performance for subgroups of patients displayed some differences. In patients tested outside the hospital (Fig 2b) , the C-index was 0.955 (0.950-0.960), the PR-AUC and MCC were 0.675 (0.632-0.719) and 0.585 (0.562-0.605) respectively. 98.9% sensitivity and 89.9% specificity were measured in this group. For patients previously admitted to the hospital at the time of test (Fig. 2c) , the C-Index was 0.809 (0.787-0.829), the PR-AUC and MCC were 0.705 (0.640-0.760) and 0.357 (0.325-0.387) respectively. The sensitivity was 100% and the specificity 31.0% indicating a higher number of false positives when using a 0.5 probability threshold for this group (Supplementary Table 1 ).

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Individual survival distributions were predicted for patients in the test set. The median of the predicted cumulative death probabilities by survival status reflected the discriminative performance of the individual survival predictions (Fig. 3a) . Deceased patients exhibited a risk of mortality that increased for the first month after FPT. Patients who died 2 months after the FPT exhibited a higher instant risk of death at these later periods than those patients who died earlier (Fig 3b) . Our survival modelling approach is also able to approximate the time of death within the 12-week time window, as highlighted by the predicted discrete (Fig 3c) and cumulative death probabilities (Fig 3d) for three individual patients. Early death was observed as a steep increase in death probability in the first weeks while late death was observed as a gradual increase in cumulative death probability (Fig 3c) . Our modelling approach also considered censored patients for which death probabilities were predicted for all periods even after censoring (Fig 3c-d) 

Feature selection for the final model was data-driven using 5-fold cross-validation on the training set.

From the original set of 2,723 features generated from routine EHR data (Supplementary Table 2 Table 1 ). We also identified the week during the pandemic in which the FPT was taken as having an impact on the risk of death. Furthermore, the risk of death was higher within the first four weeks after FPT as encoded by the week from the prediction feature. The model also allowed us to explore the temporal dynamics of individual risk factors across the predictive 12-weeks window (Fig 4b) . Features such as age, ordered loop diuretics, and admission at the time of FPT had a higher impact on the risk of dying early, while BMI, diagnosis of Alzheimer's . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint disease, and ordered B-vitamin contributed more to late risk. Thus, identification of such time dependency for features at the individual patient level further reveals different risk factors acting on different time-horizons for the predicted risk of individual patients (Fig 4c-d) .

Partial dependency plots (PDP) showed that the model learned non-linear contributions to the risk of mortality. We found that age contributes to the risk of death over 60 years of age (Fig 5a) . BMI seemed to explain a higher risk of mortality in patients with BMI lower than 30 (Fig 5b) , and males presented a higher risk of mortality than females. (Fig 5c) . A higher risk of death was also seen for patients with low lymphocyte count (Fig 5d) . As expected, patients with more hospitalizations and longer cumulative admission days prior to FPT exhibited a higher risk of death (Fig 5e-f) . Similarly, the previously mentioned contribution of being admitted in the hospital at the time of the FPT to the risk of death was observed (Fig 5g) . We found that the number of ordered medicines was a better predictor of death than the number of diagnoses, showing non-linear patterns where patients with less than five ordered medications in the last year showed up to 10% less risk of death whereas some patients with more than 20 ordered medications had up to 40% higher risk of death (Fig. 5h ).

To unravel interactions between risk factors, we explored the interdependence of the selected features by their SHAP interactions values (Fig. 6 ). The interaction map for patients who died within 4 weeks from FPT revealed that the week of prediction feature and age interacted with several other features including previous hospital admissions and prescriptions of several drugs for at least 80% of patients (Fig. 6a) . Thus, the information provided by these specific variables combined seems of particular importance for predicting early death (< 4 weeks). For patients who died after 8 weeks post-FPT, different interaction clusters emerged in which age, number of ordered medicines, BMI, and vitamin supplements like B-vitamins and calcium with vitamin D interacted in more than 70% of the patients. Also, lymphocyte count and admission at the time of FPT interacted with the number of medications in at least 60% of the patients (Fig. 6b) . is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint

We here developed an explainable Machine Learning model for predicting the risk of death within the first 12 weeks from a positive SARS-CoV-2 PCR test. By implementing a discrete-time modelling approach we computed personalized survival probabilities, explained individual risk factors and achieved high discriminative performance in terms of C-index (0.946 CI 95%: 0.941-0.950) and PR-AUC (0.686 CI 95%: 0.651-0.720). Compared to traditional approaches we could model non-linear effects, learn interactions and explain temporal dynamics of risk factors without compromising discriminative performance.

During the COVID-19 pandemic, attempts have been made to provide prognostic models by implementing diverse modelling approaches. This has resulted in publications using statistical and Machine Learning (ML) approaches to predict the diagnosis or prognosis of COVID-19 related outcomes. Meta-analyses have indicated that the majority of published models suffer from a risk of bias due to overfitting, small sample sizes, poor cohort definition or not considering censored patients 18, 29 . To overcome some of these previous limitations, we used electronic health records (EHR) from eastern Denmark, identifying 33,938 patients who had at least one positive SARS-CoV-2 RT-PCR test. To enable ML algorithms, clinical data need to be encoded into features that can be computed. Multiple approaches have been suggested for encoding EHR into computationally meaningful representations 30, 31 . We opted for a simple feature engineering approach by considering the latest values or counts in clinically relevant time windows prior to FPT depending on the type of variable. Additionally, instead of characterizing patients' relevant history using a limited set of preselected variables, the set of 22 features in the final model were derived using a data-driven approach from an initial set of 2,723 features that encoded available demographics, laboratory test results, hospitalizations, vital parameters, diagnoses and medicines. This approach enabled us to reduce model complexity to a smaller feature set while avoiding potential bias introduced by pre-selecting variables. While EHR are more representative of patient populations in terms of real-world data (RWD) 32 , some challenges arise when processing EHR for clinical research. Data collected from routine care may present inconsistencies 33 that cannot be appropriately curated for in such big data . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint sets, especially for information regarding clinical interventions or hospitalization status. We thus selected SARS-CoV-2 positive status and mortality for patient selection and outcome, respectively, based on robustness to bias from clinical management. Characteristics of these variables have been previously defined in a Danish nationwide cohort 28 from 20 th of February 2020 until 19 th of May 2020 in alignment with our subset of patients in eastern Denmark.

More importantly, handling time in ML is not only relevant for encoding features but also for the modelling framework to use. When handling longitudinal data, time is usually fixed for a specific period and ML algorithms for binary classification are applied. To do so, patients for which the event of interest was not observed before they were lost to follow-up (censored) are excluded, resulting in underestimation of predicted risks 23, 24 . This has been the predominant modelling approach in COVID-19 18,34 related outcomes. Cox models 35 are the most common statistical model for time-to-event considering censoring, but multiple ML algorithms allowing for censoring have been proposed 25 .

Models such as regularized Cox models or Random Survival Forests have been successfully implemented for EHR 36 and COVID-19 37 data. These models are based on underlying assumptions such as proportional hazards in the case of Cox based models 35 and handle time as continuous. An alternative is to consider time as discrete 26, 35 which has demonstrated performance as good or better than continuous-time models 38, 39 with the advantage of accounting for censoring while enabling the implementation of existing ML algorithms such as gradient boosting decision trees 40 . In this way, we overcame the limitations of Cox based models, by training ML models that learned complex interactions and non-linear effects from the data. Moreover, because no proportionality of hazards was assumed, our model could predict personalized survival probabilities 27 for each patient given their specific context, further facilitating a precision medicine approach 41 .

To understand model predictions, ML explainability, or explainable artificial intelligence (xAI), is particularly powerful to enable scientific insights by leveraging the ability of ML models to learn complexity transcending traditional assumptions 21 . In some cases, seemingly paradoxical effects have been unraveled when modelling clinical data 42 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint their impact on the model 43 . These methods have been successfully applied in clinical research for various diseases 20, 44 , but in the case of COVID-19 45 most of these are limited to scenarios of binary classification that ignored censoring. As an alternative approach, we provide explanations of the model predictions based on SHAP values 46 that not only decompose the predicted survival probability for each patient in terms of the features' contributions but also reflect temporal dynamics of such contributions in the context of time-to-event modelling. Local explanations as provided in our study are critical for precision medicine by indicating patient-specific risk factors, but also raise epistemological challenges on how to extrapolate from local to global explanations 47, 48 . We employed traditional summary statistics to shed some light on common risk factors, but such reduction of complexity may imply a reduction of granularity of factors that are not relevant at the population level but critical for specific patients. Importantly, the features selected as good predictors do not necessarily imply causality 21, 52 and different sets of features have been demonstrated to be equally predictive in terms of performance in some cases 49 .

In line with previous studies, we here identified high age 15 and sex (male) 50 as important risk factors in COVID-19. As the importance of age increased significantly for age over 60 years, while capturing high age as a risk factor in itself, our model may further reflect other age-related factors such as an increased prevalence of comorbidities, which is supported by the interaction plots. BMI and obesity have previously been reported as risk factors for severe COVID-19 13 and severe obesity as a risk factor for COVID-related mortality, especially for younger patients 51 , who are likely candidates for ICU care and treatment with mechanical ventilation, resulting in improved survival. In contrast, we identified an increased risk of death for patients with BMI below 30. This could reflect several other risk factors associated with low BMI, such as elderly, frail, patients with comorbidity. This is supported by the interaction plots demonstrating an interaction between BMI and the number of ordered medicines in early deaths, and interactions with the number of diagnoses, cumulative days in hospital prior to FPT, and several specific medications for late deaths. Lymphocytopenia was also identified as a predictor of high mortality in line with previous findings 52 . This may be a proxy for immune dysfunction, due to prior or ongoing therapy, malignancy or comorbidity, as well as a severe ongoing COVID-19 disease itself. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint As expected, an increased risk of death was observed in patients with an increased number of medications and diagnosis codes, likely representing comorbidities, in line with previous studies 53 .

We found that the number of ordered medicines was a better predictor of death than the number of diagnoses, emphasizing the need to capture disease burden based on actual medication in addition to coded diagnoses. This highlights the need to further explore feature encoding of clinical variables 30 , to more accurately represent clinical concepts such as comorbidities. We also observed that hospital encounters for medical examination with known or unknown causes correlated with a lower risk of death. This may indicate in-patient management of COVID-19 early in the pandemic or reflect increased monitoring of patients with anticipated increased risk of COVID-19, thereby enabling earlier interventions. Similarly, including the pandemic week in which a patient had their FPT as a feature revealed that patients early rather than later in the pandemic, had a higher risk of dying. As our data covered both the first and second pandemic wave in Denmark, this finding likely reflects that our model captured improvements in the clinical management of patients throughout the pandemic 54 .

The implemented discrete-time modelling approach required encoding the week from FPT as a feature, revealing explanations of temporal dynamics through SHAP values. When interpreting this feature, a higher risk of death in the first four weeks was observed, probably capturing the risk due to active infection during that period 55 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint predicting early death. Thus, uncovering the interdependency of features important for early vs late death also indicated time dependency of risk factors.

We developed a data-driven machine learning model to identify SARS-CoV-2 positive patients with a high risk of death within 12-week from the first positive test. The discrete-time modelling approach implemented not only allowed us to train survival models with high performance but also enabled model explainability through SHAP values. By learning temporal dynamics and interactions between clinical features, the model was able to identify personalized risk factors and high-risk patients for early interventions while improving the understanding of the disease. At the same time, we demonstrate that leveraging electronic health records with explainable ML models provide a framework for the implementation of precision medicine in routine care which can be adapted to other diseases.

The study is approved by the Danish Regional Ethical Committee is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint 

To perform time-to-event modelling we considered a discrete-time modelling approach 26 to predict 12week mortality since a first SARS-CoV-2 positive test. Described by Cox as an approximation to his proposed proportional hazards assumption for continuous-time modelling 35 , discretizing time in intervals allowed us, to perform binary classification at each time interval. By doing this, we trained models that accounted for right-censored observations, hence reducing the risk of selection bias 23 , is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint and estimated conditional probabilities of death given the features that could be computed and explained efficiently without stringent assumptions. Data was generated from EHR on 2 nd of March 2021, hence right-censoring was observed for patients that had a positive test from 8 th of December 2020 (12-weeks before data generation) and did not die. The survival status of these patients could not be ascertained in such a period hence they were only considered for the follow-up period available. (2)

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint 

We trained gradient boosting decision tree models (LightGBM 40 ) using cross-entropy as the objective function for optimization. To do this, the full dataset was split into training (60%), validation (20%), and test (20%) sets each one with the same distribution of deaths. Cross-validation (CV) was performed in two steps. First, the training set was divided into 5 subsets and the subsample rate (0.7), learning rate (0.05), number of iterations (50) and positive class weight (100) were adjusted using 5fold cross-validation while the rest of the parameters were set to default (Supplementary table 3) .

Once suitable parameters were found, feature selection was performed based on the validation set.

Second, the training set and validation set were combined and split into 5 folds to re-train and generate a final ensemble of 5 models trained on 80% of the data. The performance reported was assessed . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint by averaging the predictions of the ensemble on the test set (20%), which was not used for model development.

Based on the predicted cumulative probabilities of death, time-to-event performance was measured by the concordance index (C-index) based on the inverse probability of censoring weights 57 across all weeks. Performance was further assessed at each week by excluding right-censored cases when calculating binary metrics and measured in terms of precision-recall area under the curve (PR-AUC), Mathew Correlation Coefficient (MCC) 58 , sensitivity and specificity. A threshold of 0.5 was used to turn predicted probabilities into binary classes. Confidence intervals (95% CI) for the performance metrics were calculated by bootstrapping with resampling for 1000 iterations.

Data wrangling was performed using R 59 and the tidyverse library 60 . Feature engineering was performed in Python using the pandas 61 and numpy 62 libraries. Gradient boosting decision trees were trained and implemented using LightGBM 40 assessing model performance using the implementations in scikit-Learn 63 and scikit-survival 64 . Summary statistics were generated using tableone 65 .

Data can be requested through the corresponding author, however, due to data protection regulations, data cannot be made publicly available, but the authors will assist external researchers in accessing the data on a collaborative basis upon request. The trained models and code to run predictions are publicly available on Github under a GNU Affero General Public License v3.0 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint

The study was supported by a COVID-19 grant from the Ministry of Higher Education and Science (0238-00006B) and the Danish National Research Foundation (DNRF126). The Capital Region of Denmark, Center for Economy, provided data extracts from the EHR system. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; Table 1 . Summary statistics of the cohort based on the final feature set.

Values up to the day of the first positive SARS-CoV-2 test used for training and prediction were considered. Continuous variables were summarized by the median and interquartile ranges (Q1, Q3).

Diagnoses and medicines with their ICD-10 and ATC codes in parentheses respectively were summarized as the number of patients with at least one code assigned. Only body mass index and absolute lymphocyte counts reported missing values for 17,823 and 32,803 patients respectively.

Patients that had a positive test from the 8 th of December 2020 (12-weeks before data generation) and did not die before the 2 nd of March 2021 were censored. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted October 29, 2021. ; https://doi.org/10.1101/2021.10.28.21265598 doi: medRxiv preprint

Comparison of Clinical Characteristics of Patients with Asymptomatic vs Symptomatic Coronavirus Disease

Characteristics of and Important Lessons from the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72314 Cases from the Chinese Center for Disease Control and Prevention

Clinical Characteristics of Coronavirus Disease 2019 in China

Clinical features of patients infected with 2019 novel coronavirus in Wuhan

Clinical and immunological features of severe and moderate coronavirus disease 2019

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Baseline Characteristics and Outcomes of 1591 Patients Infected with SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy

Characteristics of Hospitalized Adults With COVID-19 in an Integrated Health Care System in California

Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: Prospective observational cohort study

BNT162b2 vaccine breakthrough: clinical characteristics of 152 fully vaccinated hospitalized COVID-19 patients in Israel

The effect of smoking on COVID -19 severity : A systematic review and meta -analysis

Obesity Is a Risk Factor for Greater COVID-19 Severity

Effects of cancer on patients with COVID-19: a systematic review and metaanalysis of 63,019 participants

Risk factors for severe and critically ill COVID-19 patients: A review

Risk Factors Associated with Acute Respiratory Distress Syndrome and Death in Patients with Coronavirus Disease

Prognostic factors for severity and mortality in patients infected with COVID-19: A systematic review

Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal

A hypothesis is a liability

Machine learning can identify newly diagnosed patients with CLL at high risk of infection

Explainable Machine Learning for Scientific Insights and Discoveries

A Unified Approach to Interpreting Model Predictions

Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar

Adapting machine learning techniques to censored time-to-event health record data: A general-purpose approach using inverse probability of censoring weighting

Machine Learning for Survival Analysis: A Survey

Modeling Discrete Time-to-Event Data

Effective Ways to Build and Evaluate Individual Survival Distributions

Characteristics and predictors of hospitalization and death in the first 11 122 cases with a positive RT-PCR test for SARS-CoV-2 in Denmark: a nationwide cohort

Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans

Deep representation learning of electronic health records to unlock patient stratification at scale. npj Digit

Transformer for Electronic Health Records

Can real-world data really replace randomised clinical trials?

Mining electronic health records: towards better research applications and clinical care

Developing and validating COVID-19 adverse outcome risk prediction models from a bi-national European cohort of 5594 patients

Regression Models and Life-Tables

Machine learning models in electronic health records can outperform conventional survival models for predicting patient mortality in coronary artery disease

Early triage of critically ill COVID-19 patients using deep learning

Continuous and Discrete-Time Survival Prediction with Neural Networks

Empirical Comparison of Continuous and Discretetime Representations for Survival Prediction

LightGBM: A Highly Efficient Gradient Boosting Decision Tree

Classification, Ontology, and Precision Medicine

Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

Explaining by Removing: A Unified Framework for Model Explanation

Explainable artificial intelligence model to predict acute critical illness from electronic health records

An interpretable mortality prediction model for COVID-19 patients

From local explanations to global understanding with explainable AI for trees

Interpretable machine learning: definitions, methods, and applications

Explanation in artificial intelligence: Insights from the social sciences

General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models

Risk factors for disease severity, unimprovement, and mortality in COVID-19 patients in Wuhan, China

Severe Obesity as an Independent Risk Factor for COVID-19 Mortality in Hospitalized Patients Younger than 50

A data-driven approach to identify risk profiles and protective drugs in COVID-19

Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis

Improved Survival Among Hospitalized Patients With Coronavirus Disease 2019 (COVID-19) Treated With Remdesivir and Dexamethasone. A Nationwide Population-Based Cohort Study

How does SARS-CoV-2 cause COVID-19?

Consistent Individualized Feature Attribution for Tree Ensembles

On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

R: A Language and Environment for Statistical Computing. (R Foundation for Statistical Computing

Welcome to the tidyverse

Array programming with NumPy

Scikit-learn: Machine Learning in Python

scikit-survival: A Library for Time-to-Event Analysis Built on Top of scikit-learn

tableone: An open source Python package for producing summary statistics for research papers