key: cord-1003526-zqnnsfpx
authors: Molani, S.; Hernandez, P. V.; Roper, R. T.; Duvvuri, V. R.; Baumgartner, A. M.; Goldman, J. D.; Ertekin-Taner, N.; Funk, C. C.; Price, N. D.; Rappaport, N.; Hadlock, J. J.
title: Risk factors for severe COVID-19 differ by age: a retrospective study of hospitalized adults
date: 2022-02-04
journal: nan
DOI: 10.1101/2022.02.02.22270287
sha: 3671155d1827c70134c953539ad72aabaf1dc3df
doc_id: 1003526
cord_uid: zqnnsfpx

Background: Risk stratification for hospitalized adults with COVID-19 is essential to inform decisions for individual patients and allocation of potentially scarce resources. So far, risk models for severe COVID outcomes have included age but have not been optimized to best serve the needs of either older or younger adults. Additionally, existing risk models have been limited to either small sample sizes, or modeling mortality over an entire hospital admission. Further, previous models were developed on data from early in the pandemic, before improvements in COVID-19 treatment, the SARS-CoV-2 delta variant, and vaccination. There remains a need for early, accurate identification of patients who may need invasive mechanical ventilation (IMV) or die, considering multiple time horizons. Methods: This retrospective study analyzed data from 6,906 hospitalized adults with COVID-19 from a community health system with 51 hospitals and 1085 clinics across five states in the western United States. Risk models were developed to predict mechanical ventilation illness or death across one to 56 days of hospitalization, using clinical data collected available within the first hour after either admission with COVID-19 or a first positive SARS-CoV-2 test. The relative importance of predictive risk factors features for all models was determined using Shapley additive explanations. Findings: The percentage of patients who required mechanical ventilation or died within seven days of admission to the hospital due to COVID-19 was 10.82%. For the seven-day interval, models for age [≥] 18 and < 50 years reached AUROC 0.80 (95% CI: 0.70-0.89) and models for age [≥] 50 years reached AUROC 0.83 (95% CI: 0.79-0.88). Models revealed differences in the statistical significance and relative predictive value of risk factors between older and younger patients, including age, BMI, vital signs, and laboratory results. In addition, sex and chronic comorbidities had lower predictive value than vital signs and laboratory results. Interpretation: For hospitalized adults, baseline data that is readily available within one hour after hospital admission or a first positive inpatient SARS-CoV-2 test can predict critical illness within one day, and up to 56 days later. Further, the relative importance of risk factors differs between older and younger patients.

The number of global confirmed cases with severe acute respiratory syndrome Coronavirus 2 (SARS-CoV-2) infection has surpassed 257 million as of December 10, 2021, with over 5 million reported deaths. 1 Although the majority of patients infected by SARS-CoV-2 present with mild symptoms, studies reported that 20% get hospitalized and 5% of patients with Coronavirus disease 2019 (COVID-19) become critically ill. 2, 3 From early on of the pandemic, both age and chronic comorbidities have been reported as a significant risk factor for poor outcomes 4 , and evidence supports increased risk with hypertension, diabetes, chronic obstructive pulmonary disease, chronic renal disease, and cardiovascular conditions. [4] [5] [6] Although young patients have a lower prevalence of comorbidities than aging patients, the relative risk of fatal outcome in young patients with hypertension, diabetes and cardiovascular diseases has been shown to be higher than in elderly patients. 7 In addition, some studies show patient population tends to be younger with the emergence of delta as the variant of concern in U.S. with regional proportions being greater than 99% as of November 2021. 8 Assessing risk for severe COVID-19 in specific age groups is complicated by both the heterogeneity of clinical presentation and age-related differences in the prevalence of chronic multimorbidities. A deeper understanding of risk factors for COVID-19 severity among different age subpopulations is needed, as well as practical, explainable risk stratification for bedside clinical decision support, research stewardship, and advancing our biomedical understanding of SARS-CoV-2.

Several studies have described successful development of machine learning models to predict COVID-19 outcomes in hospitalized patients. [9] [10] [11] [12] [13] [14] [15] [16] [17] Further, explainable models can also inform care decisions by showing which factors lead a specific individual patient to be at risk for severe outcomes, and can also help show which variables are most important at the population level, suggesting areas for further research investigation. 18 However, existing studies have several limitations; 1) most are based on small sample sizes from academic centers, 2) higher incidence of severe outcomes in hospitalized cohorts than are typically observed with current treatments, 3) reliance on laboratory tests that are not routinely administered to all patients, 4) lack of investigation of differences in risk factors between younger and older hospitalized patients, and 5) marginal model performance for either of age groups. 11 In this study, we report age-stratified machine-learning models to predict the severity of COVID-19 progression within one, seven, 14, 28, 56 days of the time of hospital admission. The models are developed from data for 6,906 patients from community hospitals across a large geographic area in the western United States over five months after the delta variant became predominant.

Evidence before this study

We searched google scholar, PubMed and its associated LitCovid repository for publications in English from database inception until November 15, 2021, using the terms "COVID-19", "critical illness", "mortality", "hospitalized", "risk score" and "prediction". The studies we identified generally focused on small cohorts or limited geographic regions and time periods with higher incidence of severe outcomes in hospitalized patients. Earlier papers with more inclusive cohorts were limited to predicting mortality, and we were unable to find papers which developed risk models for current COVID-19 standard of care and Delta variant predominance. None of the studies were optimized to model critical illness or mortality in different age groups or investigate differences between risk factors in each age-group. Lack of age-stratified models has also caused lower performance of general models for younger and older patients.

In this study, we analyzed the data from a large cohort across five states in western United States from June 31, 2021 to November 15, 2021, looking at hospitalized patients who were not already on invasive mechanical ventilation. We investigated risk factors for the need for invasive mechanical ventilation or death by developing linear and non-linear complex age-stratified models to accurately predict for time horizons of one to 56 days. Models were based on readily available data within one hour after either hospital admission with COVID-19 or a first positive inpatient SARS-CoV-2 test. In addition, we analyzed how risk factors differ by age.

We present a set of age-stratified COVID-19 critical illness and mortality prediction analyses, derived from large population of patients over a wide geographic region, in the context of current standard of care for COVID-19 and delta-variant predominance. The cohort and analytical methods that we used resulted in prediction models with strong performance, considering there is now lower severe COVID-19 incidence than early in the pandemic. Based on previous studies and linear statistical analyses on the population in this study, comorbidities such as diabetes, hypertension, cardiac disease, chronic lung and kidney disease, and immunosuppression, along with factors including higher age, being male or African American and Hispanic have been associated with higher mortality in patients with COVID-19. However, our study suggests those chronic historical factors and demographic features are less important than biomedical observations in the acute setting. After validation in other health system populations, our model could be implemented in clinical practice to enable better identification of patients with severe COVID-19 outcome.

This retrospective study analyzed data gathered from Providence St. Joseph Health (PSJH), a community health system with 51 hospitals and 1085 clinics across five states in the western United States: Alaska, California, Montana, Oregon, and Washington. Inclusion criteria was age ≥18 years and confirmation of COVID-19 by a positive PCRbased SARS-CoV-2 test result. This study was performed in compliance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule and was approved by the Institutional Review Board (IRB) at PSJH with Study Number STUDY2020000196 with waiver of consent. We follow STROBE reporting guidelines (Supplemental Table 5 ).

In this study, we hypothesized that age-stratified risk models for hospitalized patients with COVID-19 can accurately predict critical illness and mortality due to COVID-19 based on readily available patient data. Outcomes of patients were defined using the World Health Organization Ordinal Scale (WOS), proposed by the WHO R&D Blueprint group in their COVID-19 Therapeutic Trial Synopsis. 19 The WHO ordinal scale ranges from 0 (uninfected) to 8 (deceased) with gradations depending on hospitalization, supplemental oxygen, mechanical ventilation, and organ support (vasopressors, renal replacement therapy, and extracorporeal membrane oxygenation). See Supplemental Table 1 . In this study, we categorized WHO ordinal scores of 3-5 as the mild cases of COVID-19 and WHO ordinal scores of 6-8 as the critical illness and death within hospitalized patients. The objective is to develop machine learning models to predict critical illness and death with COVID-19 in hospitalized patients using easily available variables, including aggregated laboratory biomarkers and vital signs within one hour of either admission to the hospital or the first positive inpatient SARS-CoV-2 test. These predictive models are developed and tested on time horizons for one, seven, 14, 28, and 56 days from the confirmation of the infection and hospitalization. We compare the performance of machine learning models for 1) all-ages population, and 2) age-stratified subpopulations, to report the effect of age and compare the relative importance of risk factors between younger and aging adults.

The start time point of study is defined as June 31, 2021, after the delta became the predominant SARS-CoV-2 variant in the Western United States. Studied population included hospitalized individuals who received a positive test for COVID-19 between June 31, 2021 and November 15, 2021. This was confirmed by reverse-transcriptase polymerase chain reaction (RT-PCR) for the SARS-CoV-2 ribonucleic acid (RNA). Patients were excluded if they were already receiving mechanical ventilation at the time of admission to the hospital.

The factors analyzed for prediction of COVID-19 outcomes were demographic characteristics, medical history, vital signs, and laboratory biomarkers (n = 64). Medical conditions included known risk factors for poor COVID-19 outcomes reported in the literature 5 and conditions which are prevalent in aging patients ( Table 1 ). Comorbidities that . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint are usually chronic, such as hypertension, were included if they were active at the time of admission. Other comorbidities were included if they had been active any time within 2 years prior to admission, except for malignancy, which was included if active any time within the past 5 years. Note that active conditions mean health issues that affect the individual's current functioning and all health. For biomedical precision, we used the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT©) hierarchy (Supplemental Table 2 ), which can be mapped to ICD-10. Laboratory results and vital signs were included (both inpatient and outpatient) if they were collected between 24 hours before and one hour after either admission to the hospital or the first positive inpatient SARS-CoV-2 test ( Table 1 ). Note that, we used aggregated temporal longitudinal vital signs in our model as described in Lee, et.al. 20 Additionally, the risk factor list included patients need for supplemental oxygen mode, need for vasopressors, total number of comorbidities, and COVID-19 vaccination status.

Descriptive analyses are presented as frequencies and percentage for categorical variables, and as mean and standard deviation (std) for numerical variables. Fisher exact test was applied to compare distributions of categorical variables. The differences between distributions of numerical variables were calculated using Mann Whitney U-test. Results were considered statistically significant at a (2-tailed) p-value < 0.1. All statistical analyses were completed using PySpark version 2.4.5.

In data preprocessing for development of each risk model, we removed features with missing values greater than 20% (Supplemental Table 3 ). We used IterativeImputer from Scikit Learn version 0.24.0 for imputing missing data in numerical features. 21 Missing values for comorbidities were assumed to be absent from the patient's medical history and imputed with a constant number of 0. Outliers were detected by calculating the modified z-score based on median absolute deviation with a threshold of 3.5 and then these outliers were imputed by the median.

To build machine learning models, we randomly split the dataset into 80% training data and 20% testing data and analyzed each patient using multiple algorithms including logistic regression (LR), random forest classification (RF), Adaptive Boosting (AdaBoost), and Gradient Boosting Decision Tree (GBDT). The parameters for each model were optimized using a 10-fold cross-validation on the training set with the maximum scoring value for the area under receiver operating characteristic curve score (AUROC). We then balanced true and false positive rates by optimizing the probability threshold for each class.

To address collinearity between predictors, we compared the optimum performance of logistic regression using the least absolute shrinkage and selection operator (LASSO) feature selection method. For non-linear tree-based models all features were included. Performance of models was reported as the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), true positive rate (TPR), true negative rate (TNR), predictive positive value (PPV), and negative predictive value (NPV). We reported the 95% confidence interval for performance metrics of the models using Wilcoxon statistics, 22 and binomial interval 23 for the area under the ROC and precision-recall curves, respectively. All ML models were applied using Spark version 2.4.5, in the Python interface. We presented the interpretation of the model with the highest relative performance, gradient boosting, using the Shapley additive explanations (SHAP) algorithm, which uses cooperative game theory to calculate the marginal contribution of each feature, and examines the feature influence on model prediction. 24 Predictive models were reported following TRIPOD guidelines. 25

In the Providence St Joseph cohort (described in Methods above), 6,906 patients with positive tests for SARS-CoV-2 were analyzed (Supplemental Figure 1 ). Percent female was 44·25 and mean age was 59·90 years (SD ± 17·83 years), with a range 18 to 90+ years old. The distribution of relative frequency of hospitalizations by age is shown in Supplemental Figure 1 . We divided the patients into two age subgroups: younger (age ≥ 18 and < 50 years with 1,963 patients) and older (≥ 50 years with 4,943 patients). The reported variables for prognosis of COVID-19 critical illness . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint Page 5 Table 2, Supplemental Table 3, and Supplemental Table 4 . For patients with age ≥ 18 and < 50 years, the variables that had statistically significant correlation with critical illness and death in patients with COVID-19 were BMI, age, heart failure, and cardiomyopathy. For patients with age ≥ 50 years, the statistically significant variables were BMI, age, sex, dementia, and use of vasopressors within one hour of either admission to the hospital with COVID-19 or a first positive inpatient SARS-CoV-2 test. Vital signs values were aggregated from 24 hours before to one hour after and included (mean and standard deviation) for heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure DBP, respiratory rate (RR), blood oxygen saturation (SpO2), and body temperature. 

In this paper, we trained five ML models including LR, RF, GBDT, and AdaBoost for the all-age population (n=6,906), and two different age subpopulations (patients with age ≥ 18 and < 50 years with n=1,963 and patients with age ≥ 50 years with n=4,943) using the aggregated values of predictors. Class distribution for outcomes show that patients with critical illness and death accounted for 7·79% of the younger cohort with age ≥ 18 and < 50 years and 12·04% of the older cohort with age ≥ 50 years. This class imbalance was addressed by undersampling patients with mild severity from the training set. Results were reported on the complete test dataset, representing actual population distribution. Supplemental Table 5 represents the performance results for three sets of developed models for younger, older patients and all-age groups. These performance results were reported after adjusting the probability threshold to optimize models for clinical and research applications. Among four models for the younger population, GBDT had the highest true positive rate of 67·73%, true negative rate of 67·40%, and AUROC value of 0·80. For the older population, GBDT had a maximum true positive rate of 76·92%, true negative rate of 76·95% and AUROC of 0·83. Figure 1 represents the comparison between the AUROC value for four ML models based on the patient's age. Relative feature importance for the younger, older and generalized GBDT models was determined by Shapley additive explanations (SHAP), as shown in Figure 2 , and Supplemental Figure 3 , respectively. SHAP values were also used to assess the contribution of age on each model outcome (Supplemental Figure 3) . We used the distribution of importance for each variable to assess its contribution to model outcome. In the younger population, some variables for comorbidities added no predictive value, which resulted in them being automatically removed from the SHAP plot. outcomes in hospitalized patients A) Feature importance and the influence of higher and lower values of the risk factors on the patient with age ≥ 18 and < 50 years outcome, B) Feature importance and the influence of higher and lower values of the risk factors on the patient with age ≥ 50 years outcome. Note that the left side of this graph represents reduced risk of critical illness or death, and the right side of the graph represents the increased risk of critical illness and death outcome. Nominal classes are binary [0,1]. For sex, female is 0 (blue) and for race, White is 0 (blue).

Additionally, we used the GBDT to validate and assess the performance of the model for different time horizons. For the all-age group, gradient boosting showed an AUROC value of 0·80, 0·78, 0·77 and 0·77 for respectively, one, 14, . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint 28 and 56 day intervals after the confirmation of infection. Furthermore, we predicted the mortality of patients (WHO ordinal score of 8) using the GBDT model and the full set of aggregated risk factors. Note that to predict the mortality of patients with COVID-19, we also included the patients who were already receiving mechanical ventilation and additional organ support (WHO ordinal score of 6 and 7), see Supplemental Figure 1 . Therefore, the number of all age group patients for predicting mortality increased to 7,063. The results show the AUROC value of 0·85 for the general population and 0·76 and 0·80 for the younger and aging population, respectively.

In this study, we developed risk models to predict the outcomes of hospitalized adult patients with COVID-19, in the context of current COVID-19 standard of care and delta variant predominance. We used clinical data from within one hour of either admission to the hospital with COVID-19 or the first positive inpatient SARS-CoV-2 test result. Explainability analysis on the machine learning models showed that risk factors are different for older patients compared to younger patients. This is the first study that investigates age-stratified modeling for COVID-19 severity for hospitalized adults for early prediction across multiple time horizons. Data from 6,906 patients across five states was used to develop predictive models for COVID-19 critical illness and death in younger and older hospitalized adults within one, seven, 14, 28 and 56 days of positive infection test and hospitalization. The key findings are: 1) risk models perform well using readily available clinical data, 2) vital signs and laboratory results at the time of admission are more important for prediction than the presence of comorbidities, 3) age-stratified models show that the relative importance of risk factor differs between younger and older adults.

Since the beginning of the pandemic, standard of COVID-19 care has improved and delta has become the predominant variant. Further, risk models from earlier in the pandemic relied on labs that are not routinely used in many patients. 26 This was reflected by the high rate of missing values for tests required for in early risk scores, including INR, Ddimer, ferritin and procalcitonin (PCT). The models developed here are both performant and pragmatic.

Our statistical analysis revealed new insights on how variables that correlate significantly with critical illness and death in COVID-19 differ between younger and older age groups. For example, most comorbidities such as malignancy, cardiomyopathy and COPD have higher odds ratios for severe outcomes in younger patients than in older patients. Conversely, lower BUN/creatinine ratio and lower potassium are only statistically associated with critical illness and death in older patients.

We chose GBDT, a sequential ensemble approach, 27 as the model with the best relative performance to define the most predictive variables for COVID-19 outcomes. Non-linear models showed higher performance than linear models, suggesting better representation of complex interactions across multiple mechanisms of disease. Stratifying patients by age group revealed that, in general, vital signs and laboratory tests have a higher relative importance than comorbidities. Because age is such a significant risk factor, it can mask other important predictors. By removing the confounding effects of age, these models highlight new insights into risk factors for IMV and death.

Additionally, we investigated the effect of age on predictive models for younger and older COVID-19 patients. For patients with age ≥ 18 and < 50 years (Supplemental Figure 3C ), age has a relatively high and more consistent predictive effect on the performance of the model. Within patients younger than 50 years old, higher age had a negative effect on outcome. However, in patients with age ≥ 50 years (Supplemental Figure 3D) , age has less effect on the model performance. Patient stratification removed some of the confounding effect of age 28 in this group, better revealing the contribution of laboratory results, vital signs and comorbidities as predictors.

For the younger population, the patient's initial oxygen mode and aggregated vital signs demonstrate the highest predictive value for outcome severity. Other predictive factors include higher AST, higher creatinine, lower alkaline phosphatase, and lower calcium levels, higher age, and higher BMI. Laboratory results have higher importance for older patients than they do for younger patients. Features such as higher BUN, higher AST, lower HCO3, lower calcium, and some aggregated vital signs (respiratory rate, blood pressure and SpO2) are among the most predictive. Sex is not a strong predictive factor, despite it having an odds ratio of ~1·25 in both the older and younger population. BMI is another feature that supports the importance of analyzing age subgroups separately. It is statistically correlated to the severity of COVID-19 and is an important predictor for the younger population but shows no significant correlation in the older population (Supplemental Figure 3 ). This could be explained by higher BMI in younger . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint hospitalized patients compared to the older hospitalized patients with COVID-19. 29 Future investigation is needed to determine risks with being underweight or overweight, potentially with BMI-stratified models. Neither race or ethnicity had strong feature importance for prediction in the younger and older population. This shows that although chronic comorbidities (binary diagnostic labels), BMI, sex, race, ethnicity may have high odds ratios in a univariate analysis, these factors are much less important in the acute setting for predicting critical illness. Once hospitalized, biomedical observations are more predictive.

SHAP values also indicate the direction of variables' impact on outcomes. For example, higher serum creatinine levels, lower platelet counts, lower lymphocyte counts, and higher neutrophil count are all predictive of critical illness and death. [33] [34] [35] Lower calcium is associated with more severe COVID-19, as noted in previous studies, 30 and this analysis shows it has higher predictive value in older patients.

Hence, age stratification shows that risk factors for severe COVID-19 differ by age, in ways that cannot be determined in all-age models. This affirms the importance of analyzing each different age group separately, particularly for the older population who have the greater overall risk for poor outcomes.

Also, as expected, vaccination reduced the risk of severe outcomes in the older population. Vaccination status had relatively low importance, which may reflect the low number of hospitalized patients who had received vaccination during the observation window; only 8·10% of the younger hospitalized patients and 25·48% of older hospitalized patients had received at least one dose of a vaccine.

Early risk stratification in patients with COVID-19 is essential to inform decisions about what level of care a patient is likely to need. One of the main challenges of COVID-19 is the heterogeneity of presentation; therefore factors related to poor outcomes are not always evident at admission. 13 In this study, ML models using readily available variables (demographics, vital signs, common laboratory test and medical history) demonstrated strong performance for predicting the severity of COVID-19. Importantly, the population in this study included patients from 51 hospitals and 1081 clinics across five states, using data based on the current standard of care for COVID-19 and the delta variant. Five limitations of this retrospective study are: 1) reliance on EHR structured data which can miss medical conditions that not diagnosed, not recorded, or noted only in free text, 2) use of hospital reported race and ethnicity of patients 31 as opposed to direct per-patient measures of potential confounders (genetic information, disparities in healthcare, and individual lifetime history of beneficial and harmful exposures, 4) use of data from within a single healthcare system. Concerns regarding generalizability of this study are partially mitigated by the size and diversity of PSJH, which serves both urban and rural communities from California to Alaska. Future investigations will benefit from finer granularity of subdivisions by age, BMI, and more detailed variables on conditions and drugs that affect individual immune response.

We developed two age-stratified risk models for critical illness in hospitalized patients with COVID-19 and tested them on data from patients during times of improved standard of care treatment and delta variant predominance. For hospitalized adults, baseline data that is readily available within one hour after hospital admission or a first positive inpatient SARS-CoV-2 test can predict critical illness within one day, and up to 56 days later. The models for age ≥ 18 and < 50 years and the model for age ≥ 50 years were both more performant than all-age models. These agestratified models also revealed differences in the statistical significance and relative predictive value of risk factors between older and younger patients, including age, BMI, vital signs, and laboratory results. In addition, sex and chronic comorbidities had lower predictive value than vital signs and laboratory results. The results of this agestratified modeling approach provide advanced understanding of current risk factors for severe COVID-19 outcomes and can help inform care decisions and prioritize next steps for research.

JJH have been funded through Pfizer Inc. Grant No. 68114891 on work unrelated to this study. NET is also funded by NIH NIA R01 AG061796.

The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Nominal classes are binary [0,1]. For sex, female is 0 (blue) and for race, White is 0 (blue), B: Variable "Age" contribution in prediction process of model for patients' with all age groups, C: Variable "Age" contribution in prediction process of model for patients' age ≥ 18 and < 50 years, D: Variable "Age" contribution in prediction process of model for patients' age ≥ 50 years. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint Supplemental . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 4, 2022. ; https://doi.org/10.1101/2022.02.02.22270287 doi: medRxiv preprint

WHO Coronavirus Disease (COVID-19) Dashboard. Accessed

Pathophysiology of COVID-19: Mechanisms Underlying Disease Severity and Progression

Transmission, Diagnosis, and Treatment of Coronavirus Disease 2019 (COVID-19): A Review

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

Risk factors for severe and critically ill COVID-19 patients: A review

Aging & COVID-19 susceptibility, disease severity, and clinical outcomes: The role of entangled risk factors

Impact of cardiovascular disease and risk factors on fatal outcomes in patients with COVID-19 according to age: a systematic review and meta-analysis

Lifting of COVID-19 restrictions in the UK and the Delta variant

Development of a severity of disease score and classification model by machine learning for hospitalized COVID-19 patients

External validation of prognostic scores for COVID-19: a multicenter cohort study of patients hospitalized in Greater Paris University Hospitals

Accuracy of the Veterans Health Administration COVID-19 (VACO) Index for predicting short-term mortality among 1307 US academic medical centre inpatients and 427 224 US Medicare patients

Clinical features of COVID-19 mortality: development and validation of a clinical prediction model

Estimating risk of mechanical ventilation and in-hospital mortality among adult COVID-19 patients admitted to Mass General Brigham: The VICE and DICE scores

Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

A risk score based on baseline risk factors for predicting mortality in COVID-19 patients

COVID-IRS: A novel predictive score for risk of invasive mechanical ventilation in patients with COVID-19

Interpretable Machine Learning in Healthcare

Ambulatory Risk Models for the Long-Term Prevention of Sepsis: Retrospective Study

Scikit-learn: Machine learning in Python. the Journal of machine Learning research

The meaning and use of the area under a receiver operating characteristic (ROC) curve

Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals

Interpretation of Compound Activity Predictions from Complex Machine Learning Models Using Local Approximations and Shapley Values

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement

Procalcitonin is not an independent predictor of 30-day mortality, albeit predicts pneumonia severity in patients with pneumonia acquired outside the hospital

Stochastic gradient boosting

How to control confounding effects by statistical analysis

Is BMI Higher in Younger Patients with COVID-19? Association Between BMI and COVID-19 Hospitalization by

Hypocalcemia in COVID-19: Prevalence, clinical significance and therapeutic implications

Acknowledgements/Contributions: SM, PVH, JDG, and JJH conceptualized the study. SM, RTR, PVH, VRD and AMB were involved in the EHR data extraction, data cleaning, and codification. SM performed data analysis including statistical analysis, machine learning and data interpretation. JJH supervised implementation. Administrative and material support was provided by JJH and NDP. SM and PVH prepared the manuscript with critical revision of the manuscript for important intellectual content provided by JDG, NR, and JJH. All authors reviewed and approved the final version of the manuscript. We would also like to acknowledge SNOMED International for developing and maintaining SNOMED CT.