key: cord-0698022-8m784ugf
authors: Lam, Carson; Siefkas, Anna; Zelin, Nicole S.; Barnes, Gina; Dellinger, R. Phillip; Vincent, Jean-Louis; Braden, Gregory; Burdick, Hoyt; Hoffman, Jana; Calvert, Jacob; Mao, Qingqing; Das, Ritankar
title: Using Machine Learning as a Precision Medicine Approach for Remdesivir and Corticosteroids as COVID-19 Pharmacotherapies
date: 2021-03-29
journal: Clin Ther
DOI: 10.1016/j.clinthera.2021.03.016
sha: 1a196e1080c81485d668712178caae36f91442e6
doc_id: 698022
cord_uid: 8m784ugf

PURPOSE: : Coronavirus Disease 2019 (COVID-19) continues to be a global threat and remains a significant cause of hospitalizations. Recent clinical guidelines have supported the use of corticosteroids and remdesivir in the treatment of COVID-19. However, uncertainty remains about which patients are most likely to benefit from treatment with either drug; such knowledge is crucial for avoiding preventable side effects, minimizing costs, and effectively allocating resources. This study presents a machine learning system capable of identifying patients for whom treatment with corticosteroids or remdesivir is associated with improved survival time. METHODS: : Gradient boosted decision tree models to predict treatment benefit were trained and tested on data from patients hospitalized at 10 hospitals in the United States between December 18, 2019 and October 18, 2020. 893 patients were treated with remdesivir, and 1,471 were treated with corticosteroids. Models were evaluated for their ability to identify patients that exhibited longer survival times when treated with corticosteroids or remdesivir. Fine and Gray models for the proportional hazard were evaluated comparing treated and untreated patients in the full COVID-19 population, in patients receiving supplemental oxygen, and in patients identified by the algorithm. Inverse probability of treatment weights were used to adjust for confounding. Models for each treatment were trained and tested separately. FINDINGS: : .Adult patients (age ≥ 18) were included in this study, with men comprising slightly more than 50% of the sample. After adjusting for confounding, neither corticosteroids nor remdesivir were associated with increased survival time in the full hospitalized COVID-19 population or in the population receiving supplemental oxygen. However, in the populations identified by the algorithms, both corticosteroids and remdesivir were significantly associated with an increase in survival time, with hazard ratios of 0.56 (p = 0.04) and 0.40 (p = 0.04), respectively IMPLICATIONS: : Machine learning methods are capable of identifying hospitalized COVID-19 patients for whom treatment with corticosteroids or remdesivir is associated with an increase in survival time. These methods may help improve patient outcomes and allocate resources during the COVID-19 crisis.

Faced with the emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus and the novel disease it causes, coronavirus disease 2019 (COVID-2019), the scientific and medical community have raced to identify, test, and implement effective treatments. 1 As infection and mortality rates continue to climb within the United States, 2 these efforts remain critically important to national public health. Over 400 therapeutics are currently under investigation, with over 300 currently in clinical trials. 3 Potential treatments have been identified based on historical effectiveness against related diseases, theoretical activity at key points for SARS-COV-2 infections and COVID-19 disease progression, and in vitro antiviral activity. [4] [5] [6] Research to date supports the use of corticosteroids like dexamethasone and the antiviral remdesivir in specific subpopulations of COVID-19 patients.

Corticosteroids and other immunosuppressive medications were initially identified as candidate treatments based on a dysregulated systemic inflammatory response implicated in severe cases of 8 This phenomenon, sometimes referred to as a -cytokine storm,‖ is thought to provoke the most severe clinical manifestations of COVID-19, including the need for mechanical ventilation or other oxygen support,, thromboembolic events (as the proinflammatory and pro-coagulant stimulation pathways are interconnected, acute respiratory distress syndrome (ARDS), acute cardiac injury, and multi organ dysfunction. 8 Clinical trials have since demonstrated that corticosteroids may benefit patients with severe manifestations of COVID-19 that are suggestive of this inflammatory hyperactivation. 9 The RECOVERY Trial found that 28-day mortality among COVID-19 patients receiving mechanical ventilation or supplemental oxygen was 3% lower for patients receiving dexamethasone than among patients receiving the standard of care (p < 0.05); this mortality benefit was not observed among patients who did not require respiratory support. 10 Based on this and similar clinical research, clinical guidelines released by the National Institutes of Health (NIH) and Infectious Diseases Society of America (IDSA) currently recommend dexamethasone and related corticosteroids as treatments for patients with severe COVID-19, as classified by decreased oxygenation on room air or requirement for supplementary oxygen or mechanical ventilation. 11, 12 However, ambiguity remains about optimal dosing strategies, contraindications for treatment, and the effectiveness of corticosteroids for COVID-19 treatment in different subpopulations. 7, [13] [14] [15] Remdesivir, a nucleotide analogue which inhibits RNA-dependent RNA polymerase, was initially developed in 2009 for hepatitis C virus and respiratory syncytial virus and subsequently repurposed for the treatment of Ebola virus infection. 16, 17 Based on in vitro evidence of anti-viral activity against SARS-CoV-2 and other single-stranded RNA viruses, COVID-19 patients were treated with remdesivir under compassionate use authorizations and clinical trials were designed to test its efficacy. While observational research and clinical trials, including the recently completed Adaptive Covid-19 Treatment Trial (ACTT-1), demonstrated that remdesivir treatment may reduce recovery time, treatment has not been shown to consistently reduce mortality across all timepoints post drug administration in any subgroup of COVID-19 patients. 16-18 Improvements with remdesivir treatment have been most pronounced among patients with severe respiratory COVID-19, defined by the requirement for oxygen support. As a result, clinical guidelines in the US recommend remdesivir treatment only for patients within this category; remdesivir is not currently recommended for COVID-19 patients that do not require supplemental oxygen. 12,19 However, even among patients requiring oxygen support, variability in the response to remdesivir treatment has been noted. For example, while a recent metaanalyses reveal that patients treated with remdesivir receiving noninvasive oxygen support achieve higher recovery rates and lower mortality rates relative to patients on mechanical ventilation, 16 a preprint of an interim analysis of the World Health Organization (WHO) SOLIDARITY trial did not show a mortality benefit for remdesivir treatment among any patient group including those receiving respiratory support. 20 While the study's results must be interpreted in light of its methodological limitations, these findings have fueled ongoing debate on which COVID-19 patients would benefit from the use of remdesivir 21 and recently led the WHO to amend their own clinical guidelines to recommend against the use of remdesivir in hospitalized COVID-19 patients, due to perceived lack of consistent evidence of efficacy. 22 As with corticosteroids, uncertainty thus remains about the effectiveness of remdesivir across different patient populations, and the patient populations most appropriate for treatment. 11, 12 Precision medicine offers a potential avenue for addressing remaining questions about treatment efficacy as well as a means of identifying responsive populations for COVID-19 therapeutics. Precision medicine approaches have successfully improved patient outcomes in other clinical areas. [23] [24] [25] [26] Machine learning (ML) represents one means by which the likely effectiveness of specific treatments in a given individual may be predicted. 27 While ML has been applied to diverse tasks related to COVID-19, 28 within the context of therapeutics this technology has overwhelming been used to identify novel and repurposed drugs which may be effective in treating COVID-19, 28-34 rather than to identify which patients are most likely to experience a survival benefit from available treatments. To fill this gap, we present a pair of machine learning algorithms (MLAs) to encourage precision in the use of remdesivir or dexamethasone and related corticosteroids to treat COVID-19 patients using readily available data derived from electronic health records (EHRs).

Two MLAs were developed and trained to predict survival time with corticosteroid and remdesivir treatment, respectively. Algorithms were trained on a dataset of COVID-19 hospitalized patients admitted to nine U.S. hospitals ( Table 1) . Use of this deidentified data was approved by the Pearl Institutional Review Board (IRB) under IRB protocol 20-DASC-121, including a waiver for obtaining patient consent for data inclusion in the study.

For corticosteroids, all patients were required to have a length of stay longer than 4 hours and if treated, treated within the first 2 days. The corticosteroid algorithm was trained on data from patients admitted between December 18, 2019 and March 1, 2020; data on patients admitted from March 2, 2020 through October 18, 2020 were set aside into a hold out test set (826 of 1471 patients, 56% of the total corticosteroids data). For remdesivir, all patients were required to have a length of stay longer than 4 hours and if treated, treated within 7 days. Given the more recent approval and subsequent availability of remdesivir, the remdesivir algorithm was trained on data from patients admitted between March 1, 2020 and June 15, 2020; data on patients admitted from June 16, 2020 through October 18, 2020 were set aside into a hold out test set (185 of 893 patients, 21% of the total remdesivir data).

Data were extracted from the patient charts in the first four hours following hospital admission. Data used to generate predictions included age, sex, vital signs (temperature, respiratory rate, peripheral oxygen saturation, heart rate, systolic blood pressure, diastolic blood pressure) and laboratory results (blood pH, glucose, creatinine, blood urea nitrogen, bilirubin, red blood cell count, hemoglobin, hematocrit, white blood cell count, percent lymphocytes, percent neutrophils, platelet count), timing of COVID-19 diagnosis (early in hospitalization vs late in hospitalization or prior to hospitalization), need for oxygen support (via supplemental oxygen or mechanical ventilation), and pre-existing medical conditions or medical history (myocardial infarction, congestive heart failure, peripheral vascular disease, cardiovascular disease, chronic pulmonary disease, pneumonia, rheumatologic disease, renal disease, diabetes mellitus with or without complications, or cancer). These predictors were chosen to make use of a wide variety of commonly collected data present in the EHR, as well as to include relevant comorbid medical conditions.

Each machine learning algorithm architecture is a gradient boosted decision tree implemented using the XGBoost library in Python. 35 The XGB method iteratively trains collections of gradient-boosted decision trees to classify training data. Each step incorporates a new decision tree, which preferentially weights the correct classification of previously misclassified training examples. XGBoost progressively builds on the loss generated by weak decision tree base learners, learns quickly and effectively from large amounts of data, and learns even from missing features. The XGBoost method was chosen for this study due to its simplicity, high performance, and useful implementation features, which provides options for handling imbalanced classes and regularization. The XGBoost method combines results from various decision trees to generate prediction scores. Each tree has several branches. Each branch splits the patient population into successively smaller groups based on their individual feature values. For example, a branch might send a patient along one of two directions depending on whether a patient's creatinine is greater than or less than 1.2 mg/dl. If creatinine is missing, the model chooses the branching direction that, on average, results in the better prediction. Additionally, a single decision tree might contain multiple creatinine branching points, for example one that comes after a male branching point and one that comes after the female branching point. This would allow there to be two different cutoff values for creatinine conditioned on the gender of the patient. At the end of the decision tree, each patient encounter was represented in one -leaf‖ of the tree, with patients in each leaf predicted to have the same probability of mortality response to drug. Two models were trained independently, one where drug = remdesivir, one where drug = corticosteroid. This is what is referred to by the -Remdesivir model‖ and -corticosteroid model.‖

The task of predicting responsiveness to treatment is multifactorial and clinical improvement is dependent on several important factors unrelated to treatment type. However, it is still possible to design a target for the MLA for the purpose of training the MLA to extract any signal present in the clinical data that may improve the model's ability to predict the outcome of interest (treatment responsiveness). In the present study, this was done through a binary logistic objective that was used to predict positive class (improved disease when treated or worsening disease when not treated) and negative class (worsened disease when treated or improved disease when not treated) patients. For our purposes, improved disease was defined as last recorded oxygen saturation 95% or above, or survival (discharged alive). Worsening disease was defined as last recorded oxygen saturation below 95%, or mortality. Within the training dataset, 3-fold cross validation was used to select model hyperparameters. For both MLAs, final hyperparameters were: base score 0.5, learning rate 0.1, maximum depth 3 and regularization penalty 1.0. When trained in this manner, an area under the receiver operating characteristic (AUC) of 0.57 was found for Remdesivir and 0.65 for corticosteroids for prediction of positive and negative class patients. Unlike the standard use of AUC, which is to gauge performance of an MLA to diagnose disease and where AUC > 0.85 would be considered a reasonable decision maker, in this case AUC is used simply to gauge whether any signal at all (> 0.5) could be extracted that could assist in the final study objective, that is, prediction of survival benefit (i.e. increased survival time) with treatment. As a signal was found, we proceeded with model implementation and survival analysis.

For the development of each algorithm, we labeled patients as uniquely treated or untreated for the relevant pharmaceutical (corticosteroids or remdesivir). Patients were considered to be treated with corticosteroids if they received regular IV or PO treatment with dexamethasone, prednisone, prednisolone, methylprednisolone or hydrocortisone in the first 2 days following hospital admission. Patients were considered to be treated with remdesivir if they received treatment within the first 7 days following hospital admission. Patients receiving these drugs beyond the initial specified treatment windows were excluded from analysis.

The outcome of interest was survival time (measured in days). Algorithms were trained on the training set to identify patients for whom treatment was associated with an increase in survival time. For training purposes only, patients were labeled as having survived if they were discharged alive to any setting, and as not having survived if their discharge disposition was dead. To expand the number of patients included in the training set, those for whom mortality status could not be ascertained were included. For these patients, oxygen saturation was used as a proxy for mortality outcome. If the final recorded oxygen saturation prior to discharge was ≥95%, patients were labeled as having survived, while patients with a final recorded oxygen saturation below 95% were labeled as not having survived. This method was chosen because the proxy outcome is correlated with mortality in the appropriate direction. 36 Patients were only included in the test dataset if their mortality status could be ascertained. The discharge disposition (discharged alive vs in-hospital mortality) was ascertained for each patient, as was time to death for patients who experienced in-hospital mortality.

To control for confounding by indication, we extracted information from the patient EHR on several patient characteristics. These characteristics included demographics (age, sex, race, instituion at which the patient received care), vital signs (temperature, respiratory rate, peripheral oxygen saturation, heart rate, systolic blood pressure, diastolic blood pressure), laboratory results (white blood cell count, platelet count, glucose, blood pH, lactate, D-Dimer), comorbid diagnoses (cardiovascular disease, hypertension, long QT interval, chronic pulmonary disease, chronic obstructive pulmonary disease (COPD), penumonia, acute respiratory distress syndrome (ARDS), cancer, metastatic cancer, obesity, hypoglycemia, acute kidney injury (AKI), rheumatologica disease, diarrhea, sepsis), medications (insulin, beta-agonists, beta antagonsits, aldosterone receptor blockers, angiotension-convertining enzyme inhibitors, macrolide antibiotics, any antibiotics, statins, nonsteroidal anti-inflammatory drugs (NSAIDs) and hydroxychloroquine), location of COVID diagnosis (community or the hospital) and oxygen requirement (supplemental oxygen or mechanical ventilation). We note that more specific diagnoses are used to control for confounding, while more general diagnostic groups are used for model training purposes. Since some of these diagnoses were relatively rare in the data, relying on them for model purposes may have biased the model towards better performance on those for whom more granular data were available. However, to accurately control for confounding, we prioritized the use of specific diagnoses in the cases where they were available.

Each algorithm was applied to the hold-out test set of COVID-19 positive patients 4 hours after inpatient admission. All performance metrics reported herein are from the test dataset, which was not seen by the model during the training process. No performance metrics on the training dataset have been included.

For both algorithms (corticosteroids and remdesivir), performance was measured through survival analysis. Each algorithm was evaluated for its ability to identify patients for whom the relevant treatment was associated with an increase in survival time, as measured by a time-toevent analysis. Increase in survival time was measured through adjusted hazard ratios. Adjustment for confounding was appropriate due to the fact that sicker patients were generally more likely to receive treatment with either of the drugs for which algorithms were developed. Adjusted covariates varied by treatment, as described in detail below.

Survival analyses were performed in the full patient population, comparing the survival times of patients treated versus untreated with the relevant pharmaceutical, and in the population of patients receiving supplemental oxygen (a more critically ill population, and a population for whom corticosteroids and remdesivir are explicitly recommended per current clinical guidelines 11 ). The analyses were then repeated in the population of patients indicated by the algorithm.

To control for confounding, we constructed stabilized inverse probability of treatment weights (IPTW) separately for each treatment. IPTW were constructed using gradient boosted decision trees, as this method implicitly handles missing data prevalent in EHR information. This method also allowed for inclusion of a larger number of covariates than regression methods generally allow, enabling us to make use of all available patient data. All variables listed in the covariates section were used to construct the IPTW for each treatment; each participant was weighted by their IPTW in the time-to-event models. To mitigate the effects of any model misspecification in the IPTW, all adjustment covariates were also included in the final time-to-event models. The event of interest was time to in-hospital mortality; hospital discharge was therefore treated as a competing event under a Fine-Gray framework for competing risks. Fine-Gray survival models for the subdistribution hazard allow for a direct estimate of the cumulative incidence of inhospital mortality despite the presence of a competing event; this in turn allows for the computation of hazard ratios in the presence of competing events. 37 Analyses were performed and are presented separately for corticosteroids and remdesivir models. We examined associations between each treatment and mortality in unadjusted models (e.g. models containing neither adjustment covariates nor IPTW) and adjusted time-to-event models. For all analyses, the level of significance was set at alpha of 0.05.

In addition to assessing survival time, we evaluated the model inputs using SHapley Additive exPlanation (SHAP) values 38 to determine which features were most strongly associated with model predictions. SHAP is a method for quantifying the contribution of an individual feature when that feature interacts with several other features in determining the output. The method considers the model predictions with and without the individual feature, in the context of different combinations of other features and other branching orders of features.

In total, 826 patients were included in the corticosteroid algorithm test set, 525 of whom received supplemental oxygen and 616 of whom were indicated by the algorithm as suitable for treatment with corticosteroids ( Table 2 ). In the full population, 200 were treated with corticosteroids, while 174 of those receiving supplemental oxygen and 161 of those indicated by the algorithm received corticosteroids, respectively. Patients were more likely to be Hispanic in the general population than in the population recommended for corticosteroid treatment (p = 0.03). No other differences between the general population and the population for which the MLA recommended corticosteroid treatment were statistically significant (p > 0.1 for all other comparisons).

In the unadjusted time-to event analysis, corticosteroids were associated with a decrease in survival time in the general population, although the relationship was not statistically significant (hazard ratio 1.38, p = 0.13). After adjusting for confounding by indication, corticosteroids were not associated with survival in the general COVID-19 population ( Table 3) . Among patients requiring supplemental oxygen, the relationship remained statistically insignificant, although the point estimate supported a survival benefit (hazard ratio 0.731, p = 0.20). However, among the patients indicated by the MLA, corticosteroids were significantly associated with an increase in survival time (hazard ratio 0.56, p = 0.04). Adjusted survival curves for all three groups are presented in Figure 1 . These results support that the MLA can identify patients for whom corticosteroids are associated with a survival benefit. Most important model features for generating predictions include timing of COVID-19 diagnosis, systolic blood pressure, and red blood cell count.

In total, 185 patients were included in the remdesivir algorithm test set, 157 of whom received supplemental oxygen and 110 of whom were indicated by the algorithm as suitable for treatment with remdesivir (Table 4 ). In the full population, 60 were treated with remdesivir, while 57 of those receiving supplemental oxygen and 43 of those indicated by the algorithm received remdesivir, respectively. No differences between the general population and the population for which the MLA recommended remdesivir treatment were statistically significant (p > 0.2 for all comparisons).

The unadjusted time-to-event analysis found that remdesivir was significantly associated with a decrease in survival time (hazard ratio 2.52, p = 0.04). Adjusting for confounding attenuated the relationship, and remdesivir was not significantly associated with survival time in the general population after adjustment ( Table 5 ). The adjusted association in the group receiving supplemental oxygen was similarly insignificant. However, remdesivir was statistically significantly associated with an increase in survival among those indicated by the algorithm as suitable for remdesivir treatment (hazard ratio 0.40, p = 0.04). Adjusted survival curves for all three groups are presented in Figure 2 . As with remdesivir, these results support that the MLA can identify patients for whom corticosteroids are associated with improved survival outcomes. For remdesivir, the most important model features included use of supplemental oxygen, SpO 2 measure, and diastolic blood pressure.

In this study, we have shown that machine learning algorithms are capable of identifying a group of hospitalized COVID-19 patients for whom treatment with either corticosteroids or remdesivir were associated with a statistically significant survival benefit. These algorithms were able to do so relying only on routinely collected EHR information such as blood pressure, oxygen saturation, and common lab measurements. We found that these survival predictions were possible despite the relatively low AUC of the models for predicting mortality conditioned on treatment as a binary outcome. The AUC is likely low because treatment with remdesivir or corticosteroid is a less important contributor to the final patient outcome when compared to covariates such as age, severity of infection, or comorbidities. However, the AUC > 0.5 does indicate that mortality could be predicted with an effectiveness greater than random chance, and the results of the survival analysis support that these machine learning methods may help improve patient survival and allocate drug resources.

In our data, we found that neither treatment was significantly associated with survival time among the general COVID-19 inpatient population after adjustment for confounding. Perhaps more importantly, we found no association between treatment and survival time among patients receiving supplemental oxygen, despite recommendations for use in this subgroup. 12,19 his finding indicates that clinicians may be currently limited in their ability to identify COVID-19 patients most likely to benefit from treatment with either corticosteroids or remdesivir. These results of the present study add to existing clinical evidence supporting the use of remdesivir and corticosteroids for treatment of severe COVID-19 cases in certain circumstances and patient populations. The WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group determined that patients receiving corticosteroids had lower overall mortality than the control group, though there was variation in this response in several studies for patients who were receiving mechanical ventilation. 39 A meta analysis similarly found evidence that remdesivir treatment for COVID-19 was associated with reduced mortality and faster recovery, although only for patients with specific clinical parameters. 16 These variations in the observed efficacy of remdesivir may be the result of patient heterogeneity, as well as variation in severity of infection. These findings are further complicated by the fact that studies of remdesivir have been unable to consistently show a mortality benefit in conventionally defined treatment groups (e.g. by disease severity or other patient metrics).

While the emerging body of research supports a role for both corticosteroids and remdesivir in the treatment of patients with COVID-19, further research is needed to develop methods and tools to identify which patient subpopulations are most likely to benefit from treatment. Conversely, it is just as important to identify which patient populations are not likely to benefit from these treatments, so as to prevent undue exposure to the risks associated with treatment. 13 For example, corticosteroids can interfere with regulation of blood sugar or blood pressure, compromise mental status, and render patients vulnerable to secondary infection via immunosuppression. 40 COVID-19 patients treated with corticosteroids may be at elevated risk for primary, secondary, or mixed adrenal insufficiency, particularly if also treated with antivirals, which increase the half-life and bioactivity of steroids through cytochrome P450 inhibition. 41 It has also been hypothesized that steroid-mediated immunosuppression, particularly in milder cases of COVID-19, may interfere with the host adaptive immune response to the SARS-CoV-2 virus, including delaying viral clearance and increasing infectivity. 13, 42 The possible risks of immunosuppression are illustrated by research showing that patients on chronic high dose steroids for autoimmune diseases are more likely to require hospitalization for COVID-19, and that patients with moderate to severe immunosuppression demonstrate higher mortality rates than the general population. [43] [44] [45] [46] [47] Patient selection is therefore key to balancing risks associated with corticosteroid treatment with the potential benefits of modulating the hyperactive inflammatory response to SARS-CoV-2 infection which is present in some, but not all, COVID-19 patients.

While a range of adverse effects have been reported with remdesivir treatment, meta-analyses depict a generally favorable risk profile, with fewer severe adverse events such a acute respiratory failure or septic shock reported among patients receiving remdesivir compared to patients receiving placebo or the standard of care. 16,18 However, adverse event reporting in the literature describing remdesivir trials is largely considered low-quality by Consolidated Standards of Reporting Trials standards and the scope of possible adverse effects is likely not yet understood. 48 Adverse events are relatively common and can lead to treatment discontinuation. 17, 49 Increased duration of treatment is associated with a greater risk of discontinuation due to adverse effects. 18 In addition, multiple vulnerable patient populations (including patients with pre-existing severe renal or hepatic dysfunction and pregnant women) have been excluded from completed clinical trials for remdesivir, precluding assessments of tolerability and safety in these patient segments. 50 Indeed, as alteration in liver tests is relatively common during remdesivir treatment for all patients, 51,52 caution is particularly warranted when considering remdesivir treatment for patients with impaired hepatic function. The recency of FDA approval of remdesivir 53 and multitude of ongoing clinical trials indicate that the clinical understanding of the safety profile of remdesivir is evolving, highlighting the need for judicious clinical use. In light of the limited availability and high cost of remdesivir, 50 patient selection is also critical from a resource allocation perspective. Taken together, these concerns point to the need for more targeted methods of identifying COVID-19 patients to whom corticosteroids or remdesivir should be given, and those for whom other treatment avenues should be pursued.

Projections estimate that the spread of COVID-19 will continue in the coming months, even with the adoption of public health mandates designed to limit community transmission. 2,54-56 While recent analyses of vaccine trials have raised hopes that effective vaccines are on the horizon, 57,58 widespread distribution of vaccines may take significant time and leave patients vulnerable during the interim. 59 Further, even with an available vaccine, herd immunity may not be achievable in the near future, due to the high rates of community vaccination required. 60,61 The use of effective therapeutics may therefore aid with reducing morbidity and mortality 55 throughout the remainder of the COVID-19 crisis.

This research has several limitations. First, because of the retrospective nature of this work, we cannot determine how treatment recommendations may influence clinician action and patient outcomes in clinical settings. Additionally, because this study utilized data from a cohort in which treatment was not randomized, it is possible that residual confounding may influence the results of this study despite the efforts we have made to adjust for confounding variables. Due to the limitations in the data available in the EHR, we were unable to determine whether supplemental oxygen was delivered at the time of final oxygen saturation measurements for all patients. Similarly, we could not determine supplemental oxygen status at the time that peripheral oxygen saturation measurements were delivered as model inputs for all patients. Supplemental oxygen status was therefore not used to normalize either of these oxygen saturation measures; this information could have provided additional meaningful information during the model training phase, had it been available. Additionally, as the primary outcome of this study was survival time, we did not examine frequency of adverse events between study groups, nor did we develop the algorithm to identify patients at increased risk of adverse events explicitly. However, because adverse events are likely to be associated with poorer patient outcomes, the algorithm may have inherently selected for patients with a reduced risk of adverse events. Future work exploring adverse event risk in the algorithm indicated population is therefore appropriate.

Other limitations of the study relate to the study sample itself. In particular, the data used to train the algorithms were collected early in the pandemic. Significant progress in understanding the disease progression and effective treatment of COVID-19 has been made since then, and patient demographics and outcomes have shifted when compared to early cases. 62, 63 For these reasons, the training data may not well reflect the data on which the algorithms were tested. Further, the algorithms may perform less accurately on data collected farther in the future and which may be even more dissimilar from the training data. The small study sample used to test the remdesivir algorithm is another limitation of this work; replication of these findings on a larger cohort to confirm these results is warranted. Additionally, the small sample size of this study precluded any analysis of combinatorial treatments. Given the potential for drug-drug interactions, 64 future work exploring the ability to identify patients who may benefit or be harmed by combinations of therapies would be of significant clinical interest. Finally, although the focus of this work was on patient survival times, there are other clinically relevant endpoints related to COVID-19. However, because MLA systems can be readily retrained, these systems likely have the potential to identify a population that would experience symptomatic improvement, such as oxygenation, as well as improved odds of survival to help with treatment selection for clinical trials and clinical care. Additional work assessing the algorithms' ability to identify patients for whom treatment is associated with additional endpoints may be an important area of future work.

Due to the continued global threat posed by COVID-19, effective treatment for patients hospitalized with COVID-19 remains an important area of research and a key consideration for clinicians treating COVID-19 patients. Our study has shown that machine learning is capable of identifying patients most likely to derive a survival benefit from treatment with either corticosteroids or remdesivir, both of which are recommended for treatment of COVID-19. These machine learning algorithms have implications for improving patient outcomes and appropriately allocating resources. To the authors' knowledge, this report is the first description of the use of machine learning as a method of evaluating the effectiveness of COVID-19 treatments for individual patients. This finding supports that precision medicine approaches are viable for treating patients during the COVID-19 pandemic. writing -review and editing. HB: supervision; writing -review and editing. JH: project administration; supervision; writing -review and editing. JC: software, writing -review & editing. QM: conceptualization; data acquisition; formal analysis; supervision. RD: conceptualization, data acquisition, supervision.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. 

Characteristics of SARS-CoV-2 and COVID-19

We Need a Precision Approach, Not One Size Fits All

Dexamethasone in hospitalised patients with COVID-19: addressing uncertainties

Efficacy and safety of remdesivir in hospitalized Covid-19 patients: Systematic review and meta-analysis including network meta-analysis

Information on COVID-19 Treatment, Prevention and Research. COVID-19 Treatment Guidelines

New Perspectives on Antimicrobial Agents: Remdesivir Treatment for COVID-19

Activating mTOR mutations in a patient with an extraordinary response on a phase I trial of everolimus and pazopanib

Tumor genetic analyses of patients with metastatic renal cell carcinoma and extended benefit from mTOR inhibitor therapy

Pathogenbased precision medicine for drug-resistant tuberculosis

Precision Medicine Goes Microscopic: Engineering the Microbiome to Improve Drug Outcomes

Machine learning predicts individual cancer patient responses to therapeutic drugs with high accuracy

Artificial Intelligence for COVID-19: Rapid Review

Repurposing therapeutics for COVID-19: Rapid prediction of commercially available drugs through machine learning and docking

Screening of Therapeutic Agents for COVID-19 Using Machine Learning and Ensemble Docking Studies

Application of Artificial Intelligence in COVID-19 drug repurposing

Shape-based Machine Learning Models for the Potential Novel COVID-19 Protease Inhibitors Assisted by Molecular Dynamics Simulation

Artificial Intelligence for COVID-19 Drug Discovery and Vaccine Development

Masoudi-Nejad A. A multimodal deep learning-based drug repurposing approach for treatment of COVID-19. Mol Divers

A Scalable Tree Boosting System

Association Between Hypoxemia and Mortality in Patients With COVID-19

A Proportional Hazards Model for the Subdistribution of a Competing Risk

A Unified Approach to Interpreting Model Predictions

Association Between Administration of Systemic Corticosteroids and Mortality Among Critically Ill Patients With COVID-19: A Meta-analysis

What we have to know about corticosteroids use during Sars-Cov-2 infection

Repurposing existing drugs for COVID-19: an endocrinology perspective

Characteristics associated with hospitalisation for COVID-19 in people with rheumatic disease: data from the COVID-19 Global Rheumatology Alliance physician-reported registry

COVID-19 in persons with haematological cancers

Survival study of hospitalised patients with concurrent COVID-19 and haematological malignancies

COVID-19 in solid organ transplant recipients: Initial report from the US epicenter

Covid-19 and Kidney Transplantation

Quality of adverse event reporting in clinical trials of remdesivir in patients with COVID-19

Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial. The Lancet

Remdesivir for Adults With COVID-19

Hepatic Disorders With the Use of Remdesivir for Coronavirus

Commissioner Office of the FDA. FDA Approves First Treatment for COVID-19

COVID-19 Cases, Deaths, and Trends in the US | CDC COVID Data Tracker

Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period

Moderna's COVID-19 Vaccine Candidate Meets its Primary Efficacy Endpoint in the First Interim Analysis of the Phase 3 COVE Study | Moderna

What can we expect from first-generation COVID-19 vaccines? The Lancet

Changing Age Distribution of the COVID-19

Younger Adults Caught in COVID-19 Crosshairs as Demographics Shift

An overview of the safety assessment of medicines currently used in the COVID-19 disease treatment