key: cord-0705680-ee5wathm
authors: Hsiehchen, David; Espinoza, Magdalena; Hsieh, Antony
title: Deficiencies in the designs and interventions of COVID-19 clinical trials
date: 2020-07-04
journal: Med (N Y)
DOI: 10.1016/j.medj.2020.06.007
sha: 358bd2722bc3cc527146403a2c7573dba71c0a66
doc_id: 705680
cord_uid: ee5wathm

nan

The coronavirus disease 2019 (COVID-19) pandemic has caused an unprecedented rush of clinical investigations. However, this urgency has also triggered concerns regarding the quality of evidence being generated. [1] [2] [3] [4] Particularly concerning is that the dearth of high-quality medical evidence has contributed to a lack of confidence in existing guidelines on COVID-19 therapies, the off-label use of unproven therapies by patients and clinicians, and predatory behaviors exploiting the lack of strong scientific rationale for many tested interventions. 3, 5, 6 Crucial to the evaluation and implementation of COVID-19 interventions is the rigor, quality, and credibility of clinical trials. We assess these characteristics in a systematic analysis of all registered interventional trials on COVID-19 to date.

We identified all interventional trials registered up to 5/5/20 from the WHO International Clinical Trials Registry and the Global Coronavirus COVID-19 Clinical Trial Tracker. Trials were manually reviewed to exclude non-interventional trials and duplicate entries. Additional trial characteristics were abstracted from individual trial registries from the European Union, Australia/New Zealand, China, India, Germany, Iran, Japan, Netherlands, Cuba, Thailand, ISRCTN, and clinicaltrials.gov.

Characteristics of 1,029 interventional trials registered between 1/23/2020 and 5/5/2020 organized by country or region are shown in Supplementary Table 1. The top countries by registered trials were China (337), US (185), and Iran (117). Most trials tested treatments (93%), planned to enroll less than 1000 patients (88%), included hospitalized patients (71.8%), and were early phase studies (58.6%). Trials were often single-center (64.4%), randomized (78%) and open-label (59.9%). Placebo comparisons were infrequent (24.2%), and trials rarely excluded prior or concurrent use of hydroxychloroquine/chloroquine (7.9%), or non-study antiviral agents (3.6%) based on stated inclusion and exclusion criteria.

Trial characteristics differed by geography. European trials had larger enrollments, multiple sites, and more than 3 arms. US trials tested more prevention interventions, enrolled healthcare workers, applied blinding, used placebos, had industry sponsors, and included international sites. China and Iran trials were associated with smaller enrollments, single-center enrollment, testing of Traditional medicines, and rare international collaborations. Of note, many clinical trials in China were registered prior to the spread of the pandemic beyond China and neighboring countries, likely making it difficult to involve international sites. Iran's strained international political relations to other countries including sanctions likely affects its ability to create clinical studies including other countries.

Supplementary Figure 1A and 1B show a positive and significant correlation between trials and COVID-19 infections and deaths (Spearman r, 0.70 [P < .001] and 0.71 [P < .001], respectively) across countries. However, the percent of explained variance for either correlation was poor, with infections and deaths explaining only 29.5% and 26.9%, respectively, of the variance in trials between countries. This indicates that in many areas effected by COVID-19, clinical trials are generally not accessible or disproportionately fewer than would be expected based on disease burden. Trials examined 309 unique COVID-19 interventions, with the 10 most prevalent interventions being tested in 65% of studies (Supplementary Figure 1C) . Controversial or undefined interventions including hydroxychloroquine, Traditional medicine, plasma products, and stem cell products were tested in a substantial number of trials (217, 130, 58, and 53 trials, respectively). 4, 6 We assessed the strength of preclinical evidence supporting the most popular interventional drugs purported to have direct anti-viral activity by examining all related PubMed entries (Supplementary Figure 1D) . PubMed queries were conducted by manually searching for the intervention name and COVID-19 OR SARS-CoV-2 and reviewing each entry. Hydroxychloroquine, Traditional medicine, and remdesivir had more than one publication describing experiments that directly assessed In vitro or in vivo anti-viral activity. Preclinical evidence for lopinavir/ritonavir, azithromycin, and arbidol were limited to single publications.

These findings indicate that COVID-19 interventional trials have heterogenous designs, but are predominantly open-label, single-center studies regardless of geography. Thus, most studies are unlikely to be generalizable, susceptible to reporting bias, and may not be feasible due to the large planned enrollment sizes. The infrequent use of placebo for comparisons or exclusion of other commonly used COVID-19 interventions limits the interpretation of many trials as interactions between drugs in both safety and efficacy cannot be discounted, and the additive or synergistic effects of treatments cannot be distinguished. We also noted that endpoints varied widely and included diverse surrogates, which will confound the interpretation and comparison of trials.

Geographic differences in trial characteristics, the poor alignment between trial numbers and COVID-19 disease burden, and lack of international collaborations suggest that better integration of clinical research efforts is warranted to combat a global public health crisis. In particular, given the widespread impact of the COVID-19 across nearly all continents, it is surprising that there is very little evidence of any international collaboration. This has undoubted contributed to many studies examining the same interventions, which is likely wasteful particularly because the design and endpoints in many trials are generally unique and preclude comparisons. Internal collaboration should be encouraged as it may improve the quality of trials by involving additional investigators, and global participation may help address health needs across the world and improve the generalizability of results.

This analysis also highlights neglected aspects of COVID-19 research. For instance, multiplicity of interventions being tested was prevalent, as well as a lack of attention to the efficacy of nonpharmaceutical and prevention interventions. The scientific rationale of many trials appears poor, as even the most prevalent interventions purported to have anti-viral activity were infrequently associated with preclinical evidence substantiated by independent groups. Our results call for a more careful assessment of hypotheses prior to clinical testing and a shift of research efforts from theory to scientific experimentation. While there is a clear need for speediness, safe and effective therapies can only be discerned from rigorously designed and conducted clinical trials. We believe that an assessment of the quality and rationale for COVID-

Randomized Clinical Trials and COVID-19: Managing Expectations

The Urgency of Care during the Covid-19 Pandemic -Learning as We Go

Waste in covid-19 research

COVID-19 coronavirus research has overall low methodological quality thus far: case in point for chloroquine/hydroxychloroquine

What is the best drug to treat COVID-19? The need for randomized controlled trials

Preying on Public Fears and Anxieties in a Pandemic: Businesses Selling Unproven and Unlicensed "Stem Cell Treatments" for COVID-19

Supplemental Table 1. Characteristics of COVID-19 interventional trials

Scatter plot of COVID-19 trials and infections (data obtained from the Johns Hopkins COVID-19 dashboard on 5/10/2020) across 60 countries with available data

Scatter plot of COVID-19 trials and deaths across 58 countries with available data

Number of trials associated with each COVID-19 intervention. The 5 most frequent interventions tested are highlighted

PubMed entries associated with the indicated intervention are shown, including publications describing experiments that directly assessed In vitro or in vivo anti-viral activity