key: cord-0927536-4x4yl9yb
authors: McCaw, Zachary R.; Tian, Lu; Sheth, Kevin N.; Hsu, Wan-Ting; Kimberly, W. Taylor; Wei, Lee-Jen
title: Selecting appropriate endpoints for assessing treatment effects in comparative clinical studies for COVID-19
date: 2020-09-12
journal: Contemp Clin Trials
DOI: 10.1016/j.cct.2020.106145
sha: 98eb158e7d3c86134663794a074ba7ed459818b5
doc_id: 927536
cord_uid: 4x4yl9yb

To evaluate the efficacy and safety of a new treatment for COVID-19 vs. standard care, certain key endpoints are related to the duration of a specific event, such as hospitalization, ICU stay, or receipt of supplemental oxygen. However, since patients may die in the hospital during study follow-up, using, for example, the duration of hospitalization to assess treatment efficacy can be misleading. If the treatment tends to prolong patients' survival compared with standard care, patients in the new treatment group may spend more time in hospital. This can lead to a “survival bias” issue, where a treatment that is effective for preventing death appears to prolong an undesirable outcome. On the other hand, by using hospital-free survival time as the endpoint, we can circumvent the survival bias issue. In this article, we use reconstructed data from a recent, large clinical trial for COVID-19 to illustrate the advantages of this approach. For the analysis of ICU stay or oxygen usage, where the initiating event is potentially an outcome of treatment, standard survival analysis techniques may not be appropriate. We also discuss issues with analyzing the durations of such events.

When assessing a new treatment vs. standard care in comparative clinical trials for COVID-19, certain key secondary endpoints are related to the duration of specific, undesirable events over the study period. Examples include the durations of supplemental oxygen usage, invasive mechanical ventilation, hospitalization, and ICU stay. [1] [2] [3] [4] Heuristically, the longer the duration, the worse the study therapy. However, since death is a possible study outcome, if a patient dies early in the study, their length of hospital stay is rather short. On the other hand, if a patient survives longer, they may spend more time in the hospital. Therefore, if the new treatment tends to provide a survival benefit over standard care, treatment efficacy as evaluated by the duration of, for example, hospitalization, can be misleading. This "survival bias" can make a treatment that prolongs survival appear to be less effective or worse than standard care. Common strategies for mitigating this bias may not be appropriate. For example, one may consider the durations of hospitalization among survivors only, or one may impute the durations of hospitalization for patients who die to an arbitrarily large value. However, neither of these strategies will provide a fair comparison if the mortality rates of the two arms differ appreciably. In this paper, we present a valid and clinically meaningful alternative based on the mean event-free survival time.

To illustrate this bias and present alternative endpoints, Figure 1 shows 4 possible observation patterns for a typical COVID-19 study monitoring the duration of hospitalization. For case 1, the patient was discharged alive from the hospital on day 7. Across the 28 days of follow-up, their duration of hospital-free survival was 21 days. For case 2, the patient died after 14 days in hospital. Although the duration of hospitalization was 14 days, their duration of hospital-free survival was 0 days. For case 3, the patient was censored after 21 days in hospital, due for example to late entry into the study. Their duration of hospital-free survival is less than 7 days. Lastly, for case 4, the patient remained in hospital across the 28 days of follow-up. Their duration of hospital-free survival was 0 days. It is important to note that although the length of hospitalization may be misleading as an endpoint in the presence of death, the hospital-free survival time is a clinically interpretable endpoint that is not subject to the survival bias issue. 

We now use a recent large clinical trial, ACTT-1, 1 to illustrate the advantages of using the event-free survival time for assessing treatment efficacy. Adaptive COVID-19 Treatment Trial (ACTT-1) is an ongoing, double-blinded, randomized, placebo-controlled trial of remdesivir versus placebo among patients hospitalized with COVID-19. Across the 28 days of follow-up, patient health was classified on an 8-point ordinal scale, spanning from category 1, discharge from hospital with no limitation of activities, to category 8, death. The primary endpoint was time-to recovery, defined as the first time during follow-up on which the patient reached categories 1, 2, or 3. Patients in these categories were either no longer hospitalized, or hospitalized but no longer requiring supplemental oxygen or attentive medical care (e.g., remained hospitalized for infection-control reasons). There were 538 and 521 patients assigned to remdesivir and placebo, respectively. Although not yet reported, key secondary endpoints J o u r n a l P r e -p r o o f discussed in the trial protocol included the durations of hospitalization, supplemental oxygen, and mechanical ventilation. 5 Since the paper presented no results for the length of hospitalization endpoint, we considered a similar endpoint, the time patients spent in hospital receiving attentive medical care, to demonstrate the appropriate analysis of event-free survival data. This undesirable endpoint is complementary to the study's primary endpoint of time to recovery. To this end, we scanned the cumulative recovery rate curves in Figure 2A and also the survival curves in Supplementary Figure S3 of the original paper. 1, 6 The reconstructed data consist of the patient's duration of the above undesirable event (hospitalized and receiving supplemental oxygen or ongoing medical care). As illustrated in our Figure 1 , the patient's observed duration of the undesirable event ended upon recovery, death, or censoring. There were 44 and 60 deaths and 132 and 169 censorings during follow-up for remdesivir and placebo. To analyze the reconstructed data, Figure 2A presents the cumulative incidence curves for the proportion of patients not requiring attentive hospital care (including deceased patients). The areas above the cumulative incidence curves (Figures 2 B & C) represent the mean times, up to day 28, that patients spent in hospital receiving attentive care. 7-10 These were 13.4 days for placebo, and 12.0 days for remdesivir. The difference was 1.4 days (95% CI, 0.19 to 2.6 days; P=0.024) in favor of remdesivir. That is, on average, patients receiving placebo spent an additional 1.4 days hospitalized and receiving oxygen or medical care across the 28 days of follow-up. Note that, in this case, the length of hospital stay was truncated by death. Since there were more deaths in the standard care arm, this analysis penalizes remdesivir. Now, to analyze the event-free survival times across the 28 days of follow-up, Figure 3A presents the cumulative incidence curves for the proportion of patients surviving and not receiving attentive care. In contrast to Figure 2A , these curves are constructed while treating death as a competing risk. [11] [12] [13] That is, patients who die were removed from the risk set and no longer eligible to experience recovery. The areas below these cumulative incidence curves (Figures 3 B & C) represent the mean event-free survival times up to day 28. 14 These were 11.9 days for placebo, and 14.2 days for remdesivir. The difference was 2.2 days (95% CI, 0.89 to 3.52 days; P<0.001) in favor of remdesivir. Thus, on average, patients receiving remdesivir survived event-free for an additional 2.2 days across the 28 days of follow-up. Note that the difference in mean event-free survival is highly statistically significant in favor of remdesivir in contrast to the moderately significant difference obtained using the duration of the need for 

Shortening the duration of hospitalization is not equivalent to extending the duration of hospital-free survival. A harmful treatment may shorten the duration of hospitalization by increasing the mortality rate without increasing the recovery rate. Such a treatment would reduce the duration of hospital-free survival. Conversely, a beneficial treatment may extend the duration of hospital-free either by reducing the mortality rate or by increasing the recovery rate. Only in the absence of death is reducing the hospital stay equivalent to extending the duration of hospital-free survival. However, if the mortality rates of the two arms are similar, then comparing the durations of hospitalization may be appropriate.

When dealing with, for example, the duration of ICU stay or oxygen utilization, the data are more complex than those from the endpoint discussed above since patients may enter the ICU during the study. This complexity extends to other cases where initiation of the event of interest does not necessarily start when the patient enters the study. 15 Figure 4 provides various possible patterns with respect to the ICU endpoint. For example, in case 4, the patient was in ICU from day 7 to 14. For case 5, the patient died during the ICU stay. The ICU-free survival days are given on the right-hand side of Figure 4 . The complexity of such data arises because entry into the ICU is likely dependent on the patient's eventual outcome, such as hospital discharge or death. Therefore, like case 6, the censoring was not independent of the ICU-free survival time and one cannot analyze the duration of ICU stay or ICU-free survival time via standard survival analysis methods. Further statistical research is needed to delineate the analytic procedures needed to obtain valid inference on the average ICU-free survival time.

Note that if there are no censored observations before Day 28, then the analysis of ICUstay is straightforward. In future trials for COVID-19, the trial size may be larger and the length of study follow-up may be longer. Therefore, at the interim analysis, there are likely to be a sizable number of censored observations, and one is likely to encounter the same challenges faced in the ACTT-1 trial. In conclusion, the conventional approach to analyzing the duration of an undesirable clinical event may not be appropriate. On the other hand, a simple conversion of this endpoint

Remdesivir for the treatment of Covid-19 -A preliminary report. NEJM

Remdesivir for 5 or 10 Days in Patients with Severe Covid-19. NEJM

A Trial of Lopinavir-Ritonavir in Adults Hospitalized With Severe Covid-19. NEJM

Remdesivir in Adults With Severe COVID-19: A Randomised, Double-Blind, Placebo-Controlled, Multicentre Trial

Enhanced Secondary Analysis of Survival Data: Reconstructing the Data from Published Kaplan-Meier Survival Curves

Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis

Interpretability of cancer clinical trial results using restricted mean survival time as an alternative to the hazard ratio

Restricted Mean Survival Time as a Measure to Interpret Clinical Trial Results

Applying evidence-based medicine to shared decision making: value of restricted mean survival time

Introduction to the Analysis of Survival Data in the Presence of Competing Risks

Competing Risk Regression Models for Epidemiologic Data

Estimating Treatment Effect With Clinical Interpretation From a Comparative Clinical Trial With an End Point Subject to Competing Risks

How to Quantify and Interpret Treatment Effects in Comparative Clinical Studies of COVID-19

Analysis of Response Data for Assessing Treatment Effects in Comparative Clinical Studies