key: cord-1014423-vw831wu0
authors: FW, Hamilton; TC, Lee; DT, Arnold; R, Lilford; K, Hemming
title: Is convalescent plasma futile in COVID-19? A Bayesian re-analysis of the RECOVERY randomised controlled trial
date: 2021-06-19
journal: Int J Infect Dis
DOI: 10.1016/j.ijid.2021.06.034
sha: 97b7367465a4b428e9460d75ab333ff47be07a21
doc_id: 1014423
cord_uid: vw831wu0

BACKGROUND: Randomized trials are generally performed from a frequentist perspective, which can conflate absence of evidence with evidence of absence. The RECOVERY trial evaluated convalescent plasma for patients hospitalized with coronavirus disease 2019 (COVID-19) and concluded that there was no evidence of an effect. Re-analysis from a Bayesian perspective is warranted. METHODS: Outcome data were extracted from the RECOVERY trial by serostatus and time of presentation. A Bayesian re-analysis with a wide variety of priors (vague, optimistic, sceptical, and pessimistic) was performed, calculating the posterior probability for: any benefit, an absolute risk difference of 0.5% (small benefit, number needed to treat 200), and an absolute risk difference of one percentage point (modest benefit, number needed to treat 100). RESULTS: Across all patients, when analysed with a vague prior, the likelihood of any benefit or a modest benefit with convalescent plasma was estimated to be 64% and 18%, respectively. The estimated chance of any benefit was 95% if presenting within 7 days of symptoms, or 17% if presenting after this. In patients without a detectable antibody response at presentation, the chance of any benefit was 85%. However, it was only 20% in patients with a detectable antibody response at presentation. CONCLUSIONS: Bayesian re-analysis suggests that convalescent plasma reduces mortality by at least one percentage point among the 39% of patients who present within 7 days of symptoms, and that there is a 67% chance of the same mortality reduction in the 38% who are seronegative at the time of presentation. This is in contrast to the results in people who already have antibodies when they present. This biologically plausible finding bears witness to the advantage of Bayesian analyses over misuse of hypothesis tests to inform decisions.

Randomised trials are generally performed from a frequentist perspective, which can conflate absence of evidence with evidence of absence. The RECOVERY trial evaluated convalescent plasma for patients hospitalised with COVID-19, concluding there was no evidence of an effect. Re-analysis from a Bayesian perspective is warranted.

Outcome data was extracted from the RECOVERY trial by serostatus and time of presentation. A Bayesian re-analysis with a wide variety of priors (vague, optimistic, sceptical and pessimistic) was performed calculating the posterior probability for: any benefit, an absolute risk difference of 0.5% (small benefit, number needed to treat 200), and an absolute risk difference of 1 percentage point (modest benefit, number needed to treat 100).

Across all patients, when analysed with a vague prior the likelihood of any benefit or modest benefit with convalescent plasma was estimated to be 64% and 18% respectively The estimated chance of any benefit was 95% if presenting within 7 days of symptoms, or 17% if presenting after this. In patients without a detectable antibody response at presentation, the chance of any benefit was 85%. However, it was only 20% in patients with detectable antibody response at presentation.

Bayesian re-analysis suggests that convalescent plasma reduces mortality by at least 1 percentage point among the 39% of patients who present within 7 days of symptoms, and a 67% chance of the same mortality reduction in the 38% who are seronegative at the time of presentation. This is in contrast to the results in people who already have antibodies when they present. This biologically plausible finding bears witness to the advantage of Bayesian analyses over misuse of hypothesis tests to inform decisions.

Convalescent plasma (CP) -blood components from patients recovered from an infection -has been used for more than a century to treat infections, with widespread use in the 1920's and 30's for pneumococcal infections and scarlet fever, before falling out of favour with the development of antibiotics (The Lancet Haematology 2020). The principle is that of 'passive immunisation' -passing antibodies from those recovered from infection to those naïve to it, thereby providing a degree of protection from that specific agent (Keller and Stiehm 2000) . It is therefore unsurprising that interest in the use of CP to prevent and treat COVID-19 has been widespread (The Lancet Haematology 2020). Unfortunately, despite best efforts, most of this usage has occurred outside of randomised controlled trials (RCT) with >100,000 doses given in the US alone. (FDA 2021) Fortunately, the RECOVERY collaborative group have recently reported the largest randomised control trial of CP in hospitalised patients with COVID-19.(The RECOVERY Collaborative Group et al.

2021) The authors conclude that CP provided no benefit, with observed mortality equal in both arms: 1399 (24%) of 5795 patients allocated to convalescent plasma and 1408 (24%) of 5763 patients allocated to usual care died within 28 days (rate ratio [RR] 1·00; 95% confidence interval [CI] 0·93 to 1·07; p=0·95). They also conclude there was no difference across pre-specified subgroups including those with detectable SARS-CoV-2 antibody tests at the time of randomisation (seropositive group), 19% versus 18% (RR 1·06; 95% CI, 0·94 to 1·19) and seronegative patients, 32% versus 34% (RR 0·96; 95% CI, 0·85 to 1·07); with test for interaction p=0.23. In particular, they note, on the advice of the Drug Safety and Monitoring Committee (DMC), that: "there was no convincing evidence that further recruitment would provide conclusive proof of worthwhile mortality benefit either overall or in any pre-specified subgroup." In the United Kingdom, this data has been taken by the regulator as strong evidence of a null effect, leading the Medicines Health Regulatory Authority (MHRA), the UK medicines regulator, to recommend against the use of CP in patients hospitalised with COVID-19, effectively removing the therapy in the National Health Service (NHS), with many editorials agreeing with the authors this proves no effect ( Liu and Aberg 2021) .

Before accepting that CP is ineffective in hospitalized patients, it is important to recognise the clear distinction between patients who are likely to benefit and those who are not. The therapeutic mechanism of convalescent plasma and monoclonal antibody (e.g. REGN-COV2) treatments is by passive immunisation -the gifting of antibodies. These antibodies (collected from donated patients), develop in most people by 7-10 days, as part of the normal immune response. It is not surprising to think that the greatest (or any) benefit of CP would only occur in patients who present early or are seronegative, or conversely, that there will be little to no benefit in giving antibodies to those who already have antibodies or have developed their own immune response. Previous literature from SARS supports this distinction (Cheng et al. 2005; Yeh et al. 2005) , as well as data clearly identifying a protective effect of monoclonal antibodies (manufactured antibodies, rather than donated) in early COVID-19 trials, with much weaker effects in hospitalised patients later in the disease course(ACTIV- Immunological data and cases of persistent infection show that failure of an early antibody response is associated with both severe disease and, in patients without any antibodies, the risk of persistent disease. (Kemp et al. 2021; Sette and Crotty 2021) Others have also argued that seropositivity is a reason for failure of convalescent plasma. (Bajpai et al. 2020) On that background, it is logical to analyse the data from patients who are seronegative (hypothesed more likely to benefit) separately from those who are seropositive (hypothesed less likely to benefit). Likewise, it is rational to analyse the data by time from symptom onset, given that the only positive trial of convalescent plasma occurred with treatment given within 72 hours (Libster et al. 2021 ).

Although subgroup analyses can be complicated by chance imbalances, lower power, and issues of multiple testing, they are appropriate to generate hypotheses and could be used in support of the argument of not disregarding CP as a potential treatment too soon (Jones et al. 2011; Lee et al. 2021 ). Moreover, conflating absence of evidence for a small effect with evidence of no effect further risks discarding a therapy which could still have a meaningful benefit. We therefore sought to undertake a Bayesian re-analysis to estimate the probability of (a) any benefit (b) a small benefit (which we define as equivalent to a number needed to treat of at most 200), and (c) a modest benefit (equivalent of a number needed to treat of at most 100) for the all patients and for both subgroups specified above.

We extracted the intention to treat results from the RECOVERY trial both overall and for prespecified subgroups: seronegative, seropositive, less than or equal to seven days since symptom onset and more than 7 days since symptom onset. These two subgroups (antibody status and time from onset) were selected on the basis of the scientific justification described above. There was no granular data available to combine these two subcategories.

We used the 'bayes' function STATA version 16 (Statacorp, College Station, TX) to calculate posterior probabilities. We calculated the probabilities of (a) any benefit (OR <1) and (b) a small but arguably clinically important benefit estimated as absolute risk difference of at least 0.5% (number needed to treat <=200) and (c) a modest benefit which we define as a risk difference of at least 1% (number needed to treat <=100). These risk differences were chosen after internal discussion between the study authors regarding what would be considered an important effect size considering the complexity and challenges in using convalescent plasma. By nature, they are subjective, but reflect effect sizes that might be salient to patients, their families and clinicians.

As suggested by a recent review on Bayesian re-analysis in COVID-19 (Zampieri et al. 2021 ), we chose four probability assumptions to account for varying prior views: 1. Vague (no information (mean risk difference (RD) 0, SD 10,000)) 2.Optimistic (10% risk of harm: mean RD 0.01, SD 0.007) 3.

Sceptical (tightly around the null: mean RD 0, SD 0.007) 4. Pessimistic (10% chance of benefit (mean RD -0.005, SD 0.0036). Posterior probabilities were computed from binomial regression models.

Posterior density function graphs were produced for each prior assumption.

All code used to generate these figures is available in the appendix.

Table 1 presents the posterior probabilities of benefit for each prior.

Across the whole trial population, the estimated chance of any benefit is around 65%, with little difference across all prior assumptions. The posterior probability of a modest benefit (preferring treatment arm) is around 19 % across all prior assumptions. The associated posterior density functions are available in the supplementary appendix, as supplementary figures.

In the seronegative subgroup, the estimated likelihood of any benefit is greater, at around 90%, across all prior assumptions. The estimated chance of a risk difference (modest benefit) of >1% is also high (more than 66% across all three priors), and varied little between prior assumptions. This contrasts with the seropositive arm, where the estimated chance of any benefit is only 20%, and with a very small (3%) chance of a modest benefit (NNT<=100).

These results are mirrored in the early treatment sub-group, with an around 95% chance of benefit in patients who were treated within 7 days of symptom onset. The chance of a modest benefit (NNT<=100) was about 80% across all prior assumptions. However, in patients who present after 7 days, the chance of convalescent plasma providing any benefit is small (17%), with a very low chance (~2%) of a modest benefit (NNT<=100). Vague prior: N(0, SD=10,000); Optimistic prior: N(0.01, SD=0.007); Skeptical prior: N(0,SD=0.007);

Pessimistic prior N(-0.01, SD=0.0036). Small benefit defined as a risk difference >0.5% (equivilanet to a NNT <=200); Moderate benefit defined as a risk difference >1% (equivilanet to a NNT <=100).

The RECOVERY trial has been a paradigm in a rapid, pragmatic, approach to trialling new therapies in a pandemic. Good practice requires a firm, pre-specified analysis plan with clear, pre-defined subgroup analysis. (Schulz et al. 2010 ) However, the conclusions drawn by the authors and Medicines

Health Regulatory Authority (MHRA) with respect to convalescent plasma risks conflating absence of evidence of a small effect with evidence that there is no benefit. However, re-analysis of the original data using Bayesian methods yields a small probability (>15%) of an effect of with an NNT of 100 across the whole trial, with an even higher probabilities of 90% and 75% respectively in patients who present within 7 days of symptoms, or who are antibody negative in presentation.

The population who present early are easy to identify (from history alone) and constitute more than a third of the whole trial population. The estimated chance of a benefit with an NNT of 100 changes from ~7% in those who present late to ~90% in those who present early.

Many clinicians, patients and their families might consider benefits in the region of one life saved for every 100 or 200 people treated as meaningful benefits. From a societal perspective, the treatment would need to achieve a mean of only one quality adjusted life year to justify a £20,000 treatment cost. However, it is not our intention to prove that CP is a cost effective treatment -at heart that is a value judgement. We wish only to show that the conclusion that the treatment is in-effective is unlikely to be true for people who have not developed immunity at the point where the therapeutic decision is made. It is always important to consider the literature in the round when making policy recommendations. Previous trials have been small and underpowered, with a recent meta-analysis of evaluations of CP in COVID identifying less than 2,000 patients across all RCT's prior to RECOVERY (Janiaud et al. 2021) . Only one previous trial of high titre CP has reported data based by antibody status (Simonovich et al. 2021) ; with 34/105 deaths (Rodionov et al. 2021) (Joyner et al. 2021 ). Yet further support for our conclusion can be found from secondary outcomes in the RECOVERY trial that we would expect to correlate with the primary outcome if the hypothesis that CP is particularly effective in immunologically naïve patients.

Both secondary outcomes (discharge home by day 28, and invasive mechanical ventilation or death)

in the original study showed heterogeneity with respect to serological status and intervention effect with impressive p values of 0.008 and 0.01 respectively in favour of CP. Although we do not focus on this to avoid accusations of 'cherry picking' the data, this is entirely consistent with and supportive of a causal path by with CP reduces mortality, and both of these are critical outcomes relevant to both patients and clinicians.

We recognize that there may have been chance imbalances in age or comorbidity within the seronegative or early subgroup of patients since randomization was not stratified on serological status or time from presentation.(The RECOVERY Collaborative Group et al. 2021) However, they were pre-specified sub-groups and make up a substantial proportion of all participants as stated above. We also recognise that both these groups double count a number of patients for the reason given above. An alternative approach would have been to create four non-overlapping groups; both (serologically positive and late presentation), neither and two either groups. However, we could not construct these groups because we did not have the raw data. In any event, both serological negativity and early presentation tap into a shared pathway concerning the development of immunity, even if there are other pathways involved.

The RECOVERY trial for CP reported no benefit. Recognising the changing literature since the trial started and using a variety of priors, we suggest the reporting of no effect may be premature. It remains plausible that CP has a small, but clinically important effect on mortality in those who have not already developed an antibody response or who present early. It is clear that any effect is likely small, but we would argue clinicians, scientists, and government agencies to review all trial data in totality, rather than regarding the null result as fixed.

No formal funding was used for this piece but FH's time was covered by the GW4 CAT Wellcome Doctoral training scheme

A Neutralizing Monoclonal Antibody for Hospitalized Patients with Covid-19

Efficacy of convalescent plasma therapy compared to fresh frozen plasma in severely ill COVID-19 patients: A pilot randomized controlled trial [Internet]. bioRxiv. medRxiv; 2020

SARS-CoV-2 Neutralizing Antibody LY-CoV555 in Outpatients with Covid-19

Use of convalescent plasma therapy in SARS patients in Hong Kong

Convalescent Plasma COVID-19 Emergency Use Authorization

Association of Convalescent Plasma Treatment With Clinical Outcomes in Patients With COVID-19: A Systematic Review and Meta-analysis

Bayesian models for subgroup analysis in clinical trials

Convalescent Plasma Antibody Levels and the Risk of Death from Covid-19

Passive immunity in prevention and treatment of infectious diseases

SARS-CoV-2 evolution during treatment of chronic infection

Remdesivir and systemic corticosteroids for the treatment of COVID-19: A Bayesian re-analysis

Early high-titer plasma therapy to prevent severe Covid-19 in older adults

Convalescent plasma in patients hospitalised with COVID-19

Blood plasma treatment has limited effect for sickest COVID-19 patients

Potential benefit of convalescent plasma transfusions in immunocompromised patients with COVID-19

CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials

Adaptive immunity to SARS-CoV-2 and COVID-19

A Randomized Trial of Convalescent Plasma in Covid-19 Severe Pneumonia

The Lancet Haematology. The resurgence of convalescent plasma therapy

Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label

REGN-COV2, a Neutralizing Antibody Cocktail, in Outpatients with Covid-19

Experience of using convalescent plasma for severe acute respiratory syndrome among healthcare workers in a Taiwan hospital

Using Bayesian Methods to Augment the Interpretation of Critical Care Trials. An Overview of Theory and Example Reanalysis of the Alveolar Recruitment for Acute Respiratory Distress Syndrome Trial

This research was funded in whole, or in part, by the Wellcome Trust. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

No formal ethical approval was required for this work, as it is was re-analysis of published data

No authors declare any conflicts of interest.