key: cord-0894331-mhhok65s
authors: Albuquerque, Arthur M.; Tramujas, Lucas; Sewanan, Lorenzo R.; Williams, Donald R.; Brophy, James M.
title: Mortality Rates Among Hospitalized Patients With COVID-19 Infection Treated With Tocilizumab and Corticosteroids: A Bayesian Reanalysis of a Previous Meta-analysis
date: 2022-02-28
journal: JAMA Netw Open
DOI: 10.1001/jamanetworkopen.2022.0548
sha: d9e565a4e67eeff6c837df2cbdc00a87a42979b0
doc_id: 894331
cord_uid: mhhok65s

IMPORTANCE: A World Health Organization (WHO) meta-analysis found that tocilizumab was associated with reduced mortality in hospitalized patients with COVID-19. However, uncertainty remains concerning the magnitude of tocilizumab’s benefits and whether its association with mortality benefit is similar across respiratory subgroups. OBJECTIVE: To use bayesian methods to assess the magnitude of mortality benefit associated with tocilizumab and the differences between respiratory support subgroups in hospitalized patients with COVID-19. DESIGN, SETTING, AND PARTICIPANTS: A bayesian hierarchical reanalysis of the WHO meta-analysis of tocilizumab studies published in 2020 and 2021 was performed. Main results were estimated using weakly informative priors to exert little influence on the observed data. The robustness of these results was evaluated using vague and informative priors. The studies featured in the meta-analysis were randomized clinical tocilizumab trials of hospitalized patients with COVID-19. Only patients receiving corticosteroids were included. INTERVENTIONS: Usual care plus tocilizumab in comparison with usual care or placebo. MAIN OUTCOMES AND MEASURES: All-cause mortality at 28 days after randomization. RESULTS: Among the 5339 patients included in this analysis, most were men, with mean ages between 56 and 66 years. There were 2117 patients receiving simple oxygen only, 2505 receiving noninvasive ventilation (NIV), and 717 receiving invasive mechanical ventilation (IMV) in 15 studies from multiple countries and continents. Assuming weakly informative priors, the overall odds ratios (ORs) for survival were 0.70 (95% credible interval [CrI], 0.50-0.91) for patients receiving simple oxygen only, 0.81 (95% CrI, 0.63-1.03) for patients receiving NIV, and 0.89 (95% CrI, 0.61-1.22) for patients receiving IMV, respectively. The posterior probabilities of any benefit (OR <1) were notably different between patients receiving simple oxygen only (98.9%), NIV (95.5%), and IMV (75.4%). The posterior probabilities of a clinically meaningful association (absolute mortality risk difference >1%) were greater than 95% in patients receiving simple oxygen only and greater than 90% in patients receiving NIV. In contrast, the posterior probability of this clinically meaningful association was only approximately 67% in patients receiving IMV. The probabilities of tocilizumab superiority in the simple oxygen only subgroup compared with the NIV and IMV subgroups were 85% and 90%, respectively. Predictive intervals highlighted that only 72.1% of future tocilizumab IMV studies would show benefit. The conclusions did not change with different prior distributions. CONCLUSIONS AND RELEVANCE: In this bayesian reanalysis of a previous meta-analysis of 15 studies of hospitalized patients with COVID-19 treated with tocilizumab and corticosteroids, use of simple oxygen only and NIV was associated with a probability of a clinically meaningful mortality benefit from tocilizumab. Future research should clarify whether patients receiving IMV also benefit from tocilizumab.

Our random-effect model is defined as:

where is the observed mean log odds ratio of tocilizumab versus control and 2 is the known sampling variance in study . Because this is a random-effect model, each study has its own distribution, where represents its mean effect. All s are drawn from normal distribution where the mean effect is and the variance 2 , which represents the between-study heterogeneity. is predicted by a no-intercept linear regression, where each subgroup (simple oxygen only; noninvasive ventilation; invasive mechanical ventilation) has its own parameter , representing the effect of each respective subgroup.

In this case, we can assess tocilizumab's effect in each subgroup while assuming a common between-study heterogeneity:

• Simple oxygen only = • Noninvasive ventilation = • Invasive mechanical ventilation =

Because we applied the Bayesian framework, we assigned a prior distribution to each parameter. In our main model, we implemented priors that cover plausible values for all parameters, assigning limited density to impossible values, and thus employed little influence in the results (hereafter, known as weakly informative priors). 1,2 These are our weakly informative priors:

Now, we will explain the rationale underlying these distributions.

We find highly unlikely that a pharmacological treatment, such as tocilizumab, will yield a 80% odds reduction in 28-days all-cause mortality regardless of the subgroup of patients, as suggested by empirical evidence. 3 Thus, for , we set a prior distribution of (0, 0.82) in the log odds ratio scale. Another way to assess the plausibility of the priors is to perform a prior predictive check 4 , which can be visualized below:

Point estimate depicts the median and interval bar depicts the 95% credible (quantile) interval.

As expected, the distribution approximately ranges from 0.2 to 5.0.

Lastly, we will now discuss the weakly informative prior distribution for . Because we wanted to perform unconditional inferences beyond the included studies, we fitted a random-effect meta-analysis. In this model, one assumes there is within-study heterogeneity (represented by 2 , the known sampling variance in study ) and the between-study heterogeneity (represented by ).

Although the definition of small or large between-study heterogeneity is arbitrary, previous work suggests cutoff values ("reasonable" heterogeneity between 0.1 and 0.5, "fairly high" between 0.5 and 1.0, and "fairly extreme" for values larger than 1.0 log odds ratio). 2, 5 We added a category for low heterogeneity (between 0 and 0.1).

The Half-Normal(0.5) distribution yields plausible probabilities in each of these ranges:

Here are the corresponding probabilities within each of the heterogeneity ranges:

Low Reasonable Fairly High Fairly Extreme 16% 52% 27% 5%

To check whether the choice of weakly informative priors meaningfully impacted our results or our conclusions, we also fitted models using vague or informative priors.

Vague priors:

Informative priors: 6

Here are graphical representations of these normal distributions (along with the weakly informative mentioned before):

Log scale Linear scale

Here are graphical representations of distributions for the between-study standard deviation ( ) (along with the weakly informative mentioned before): 

We used the odds ratio as our primary estimand. 11, 14 We derived the risk in the tocilizumab group using the following formula: 11

where is the mortality risk in the tocilizumab group, is the mortality risk in the control group and is the odds ratio.

We then calculated the risk difference (RD) with the following formula: 11

We assumed different mortality risks in each subgroup. For the simple oxygen only and noninvasive ventilation subgroups, we used the average mortality risk in each subgroup based on the data included in this reanalysis. In contrast, regarding the invasive mechanical ventilation (IVM) subgroup, we found a striking discrepancy between the control mortality risk in the data included in this reanalysis (52%) in comparison to another previously published meta-analysis (34% in patients on IVM and using corticosteroids). 10 Thus, we have decided to use 43% (arithmetic mean between 34 and 52) as our reference risk in the IVM subgroup. Recognizing the potential variability of the subgroup baseline risks, we estimated the risk differences with twenty different plausible baseline risks for each subgroup (spanning +-10% change from the reference risks mentioned above).

Simple oxygen only 20 +-10%

Noninvasive ventilation 34 +-10%

Invasive mechanical ventilation 42 +-10%

In brief, we will update our current evidence (as modeled in our main model) with generated randomized clinical trials (RCTs) of different sample sizes comparing tocilizumab to control on patients on invasive mechanical ventilation.

We will use the estimated marginal posterior mean and standard error on this subgroup to create a prior distribution. Then, we will use normal conjugate analyses to update this prior with new data (likelihood) and form updated posterior distributions.

As described before, we fitted a Bayesian meta-analysis model, from which we estimated marginal posterior distributions on different subgroups. Once again using a Bayesian approach, we updated our current belief, as expressed by the results of our current Bayesian meta-analysis, and which has now become our new prior, with the results of these new generated RCTs to arrive at revised posterior distributions.

Of note, the only subgroup of interest now is the invasive mechanical ventilation:

Marginal posterior distribution of the invasive mechanical ventilation subgroup (also depicted in Figure 1A ). The interval bar depicts the mean and 95% credible (quantile) interval.

In the linear scale, the mean of this marginal posterior distribution is 0.89. Because we will use normal conjugate analysis, it is of greater interest to evaluate this distribution on the log scale, which is approximately normally distributed. In this case, the mean is -0.12 and the standard error is 0.17.

Thus, in the following normal conjugate analyses, we will use the following distribution as our prior:

Normal(−0.12, 0.17 2 )

We created six different RCTs and update the prior distribution mentioned above six separate times.

Assuming the prior is normally distributed and so is the data (likelihood).

The mean and variance of the posterior distributions can be estimated by the following formulas: 5 In summary, we can update a normally distributed prior distribution (shown in the Figure above) with normally distributed data to generate a normally distributed posterior distribution. Based on the posterior's mean and variance, we generated 100,000 random samples (seed number of 123). Now, we must decide the mean and standard deviation of the likelihood. All RCTs will have a mean of -0.26 (log scale). This value is the equal to 0.77 in the linear scale, which was chosen based on WHO's meta-analysis (page 14 in their Supplement 2). 7 This is the mean odds ratio of tocilizumab vs. control in patients using corticosteroids (overall results). We decided to use this value to reflect a skeptical view to heterogeneity of treatment effect across subgroups, 8 and thus the "real" effect in this subgroup would be equal to the largest body of evidence available for tocilizumab in all hospitalized COVID-19 patients on corticosteroids.

Given that all six generated RCTs were set to find the same effect size, the only difference between them was the total number of included patients: 200, 500, 1000, 1500, 2000, or 4000. To calculate the standard deviation of each corresponding prior based on the number of total patients included, one must also assume the proportion of patients included in each treatment arm and the mortality risk in the control arm:

We assumed equal allocation in both treatment arms 2.

Adapting from the suggestions in the GRADE guidelines, 9(p12) we found a striking discrepancy between the control mortality risk in the data included in this reanalysis (52%) in comparison to another previously published meta-analysis(34% in patients on IVM and using corticosteroids). 10 Thus, we have decided to use 43% (arithmetic mean between 34 and 52) as our reference risk in the IVM subgroup 3.

The mortality risk in the tocilizumab was calculated using the following formula: 11

where is the mortality risk in the tocilizumab group, is the mortality risk in the control group and is the odds ratio mentioned above. Thus, the tocilizumab risk is equal to 37%.

In summary, we are generating RCTs with mean OR equal to 0.77, control risk mortality of 43%, and tocilizumab risk of 37%.

Based on these values, we can estimate the standard deviation (SD) with the following formula: 12 = √ 1 + As previous work has shown, 13 we can estimate these values as:

where and are the sample sizes in the tocilizumab and control arms, respectively. As mentioned above, we assume equal allocation in both treatment arms, thus = .

Finally, the based on the 6 different sample sizes mentioned above are: 

A Tutorial on Using The Wambs Checklist to Avoid The Misuse of Bayesian Statistics

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

Performance of informative priors skeptical of large treatment effects in clinical trials: A simulation study

Visualization in Bayesian workflow

Bayesian Approaches to Clinical Trials and Health Care Evaluation

Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis

Association Between Administration of IL-6 Antagonists and Mortality Among Patients Hospitalized for COVID-19: A Meta-analysis

Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects

GRADE guidelines: 12. Preparing Summary of Findings tablesbinary outcomes

Association Between Administration of Systemic Corticosteroids and Mortality Among Critically Ill Patients With COVID-19: A Meta-analysis

Questionable utility of the relative risk in clinical research: a call for change to practice

Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials

Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myocardial infarction

The OR is "portable" but not the RR: Time to do away with the log link in binomial regression

Posterior probabilities of benefit per subgroup in the risk difference scale assuming weakly informative priors. Each line represents the posterior probability of benefit for a specific cutoff, such as risk difference greater than 0% or 1%, across plausible ranges of mortality risk under control treatment. Underlying weakly informative priors are N(0, 0.82) for the coefficients, and HN(0.5) for the between-study standard deviation. N(mu, sigma) = Normal(mean, standard deviation); HN(sigma) = Half-Normal(standard deviation)

Posterior distributions for comparisons of effect sizes between subgroups while assuming weakly informative priors. Each distribution represents the ratio of odds ratios of two subgroups. On top of each distribution, there is a percentage representing the posterior probability of a ratio of odds ratios greater than 1.0 (eTable 2). Arrows on the bottom represent -in that comparison -which subgroup benefited to a greater extent tocilizumab's effect on mortality reduction (lower odds ratio [OR]). For example, according to our model, there was an 85% probability that tocilizumab reduces mortality to a greater extent in the simple oxygen subgroup in comparison to noninvasive ventilation. Underlying weakly informative priors are N(0, 0.82) for each coefficient, and HN(0.5) for the between-study standard deviation. N(mu, sigma) = Normal(mean, standard deviation); HN(sigma) = Half-Normal(standard deviation).Posterior distributions (log scale) of the between study standard deviation (tau) upon different underlying prior distributions (weakly informative, vague or informative). Tau is a proxy for the between-study heterogeneity in random-effect meta-analyses. Weakly informative priors: Coefficients N(0, 0.82); Between-study standard deviation HN(0.5) / Vague priors: Coefficients N(0, 4); Betweenstudy standard deviation HN(4) / Informative priors: Coefficients N(0, 0.35); Between-study standard deviation LN(-1.975, 0.67). N(mu, sigma) = Normal(mean, standard deviation); HN(sigma) = Half-Normal(standard deviation); LN(mu, sigma) = Log-Normal(mean, standard deviation). Although the definition of small or large between-study heterogeneity is arbitrary, previous work suggests cutoff values: "reasonable" heterogeneity between 0.1 and 0.5, "fairly high" between 0.5 and 1.0, and "fairly extreme" for values larger than 1.0 log odds ratio for the between-study standard deviation (tau). 2, 5 We added a category for low heterogeneity (between 0 and 0.1). Thus, in this table, we present the posterior probabilities within each one these ranges. Because we fitted three different models using different prior distributions (weakly informative, vague, and informative), here we depict tau posterior distributions/probabilities regarding each model. Results from the normal conjugate analyses updating current evidence on invasive mechanical ventilation (used as the Prior) with generated RCTs (used as Data, eTable 6). These analyses yield posterior distributions as depicted on Panel A. In contrast to the results shown in eFigure4, the generated RCTs in these analyses are centered at 1.0 odds ratio. Panel A: Each panel represents a different model. The label on top of each panel depicts the number of total patients on invasive mechanical ventilation included in each respective model (current plus generated patients). Point estimates depict the median and interval bars represent the 95% credible intervals for both prior, data (likelihood) and posterior distributions. Panel B shows the posterior probability of benefit for different thresholds (OR < 1.0 and < 0.9).