key: cord-0974092-yc9yns3y
authors: Hill, Andrew; Mirchandani, Manya; Pilkington, Victoria
title: Ivermectin for COVID-19: Addressing Potential Bias and Medical Fraud
date: 2022-01-17
journal: Open Forum Infect Dis
DOI: 10.1093/ofid/ofab645
sha: aac230deb89348bfc19f9667d7dadb96590b8ddf
doc_id: 974092
cord_uid: yc9yns3y

Ivermectin has become a controversial potential medicine for coronavirus disease 2019. Some early studies suggested clinical benefits in treatment of infection. However, the body of evidence includes studies of varying quality. Furthermore, some trials have now been identified as potentially fraudulent. We present a subgroup meta-analysis to assess the effects of stratifying by trial quality on the overall results. The stratification is based on the Cochrane Risk of Bias measures and raw data analysis where possible. The results suggest that the significant effect of ivermectin on survival was dependent on largely poor-quality studies. According to the potentially fraudulent study (risk ratio [RR], 0.08; 95% CI, 0.02–0.35), ivermectin improved survival ~12 times more in comparison with low-risk studies (RR, 0.96; 95% CI, 0.56–1.66). This highlights the need for rigorous quality assessments, for authors to share patient-level data, and for efforts to avoid publication bias for registered studies. These steps are vital to facilitate accurate conclusions on clinical treatments.

In June 2020, ivermectin, a Food and Drug Administration (FDA)-approved antiparasitic drug, was shown to have antiviral effects against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in vitro [1] . Following this, ~86 clinical trials investigating ivermectin for coronavirus disease 2019 (COVID-19) have been registered globally. In late 2020, clinical trials began reporting very compelling clinical benefits for ivermectin in the treatment of COVID-19. From late 2020 onwards, multiple groups produced meta-analyses that reported that ivermectin had a significant effect on survival, hospitalizations, clinical recovery, and viral clearance [2, 3] . Our meta-analysis was first presented in January 2021 and published in July 2021 [4] . It suggested that ivermectin resulted in a significant 56% improvement in survival, favorable clinical recovery, and reduced hospitalizations. Such optimistic results from multiple meta-analyses have escalated public interest in using ivermectin for the treatment and prevention of COVID-19, despite the World Health Organization (WHO) only recommending its use within clinical trials [5, 6] .

However, as with all meta-analyses, a key limitation is the quality and completeness of the available evidence. During our original assessment of studies, standardized Cochrane Risk of Bias measures (RoB 2) had classified several studies as "high risk of bias" [7] . A study by Niaee et al. from Iran that reported a randomized methodology was found to have significant differences in baseline characteristics across treatment arms [8] . This suggests that the participants were not randomized appropriately, which could bias the outcomes. Second, a study by Okumus et al. from Turkey did not provide any information on allocation concealment, and it was unclear if the participants or investigators were blinded, which risks introducing observation bias [9] . Lastly, a study by Hashim et al. from Iraq provided insufficient details on the randomization process, had a lack of clarity on participants who were analyzed, and involved unblinded assessment of a subjective outcome [10] .

Furthermore, some studies were then identified to be potentially fraudulent. For example, on July 15, 2021, a study by Elgazzar et al. from Egypt was retracted from the preprint server Research Square due to "ethical concerns" [11] . It has been reported that the data for ~79 participants were duplicates, some deaths were recorded on dates before the trial had started, and instances of plagiarism were identified in the text [12] . Similarly, a study conducted in Lebanon by Raad et al. was also identified to have duplicate data for multiple participants when the patient-level database was analyzed in September 2021 [13] . Before these inconsistencies were identified, the Elgazzar and Raad studies had been included in multiple meta-analyses, which suggested significant benefits for ivermectin P E R S P E C T I V E S in the treatment of COVID-19 [2, 3] . In our original meta-analysis, the Raad study accounted for 11.8% of the effect of ivermectin on hospitalization, and the Elgazzar study accounted for 12.6% of the effect of ivermectin on survival [4] .

These instances suggest that the data available to the support the use of ivermectin for COVID-19 are not reliable. In July 2021, after the potentially fraudulent studies were identified, we retracted our published meta-analysis and began working on an updated analysis assessing the effects of stratifying by trial quality on the overall results. Clinical trials evaluating ivermectin for the treatment of COVID-19 had been identified by systematic searching of 8 databases. An in-depth evaluation of study quality was conducted, in addition to the standard Cochrane RoB 2 and CONSORT checklist [7] . First, we evaluated trials based on the effectiveness of their randomization process by comparing baseline characteristics across treatment arms using the chi-square test. Second, randomization dates were checked to ensure that patients were randomized into the treatment arms on similar dates. Third, checks were conducted to evaluate if recruitment to treatment arms was balanced at each investigational center. Furthermore, we analyzed patient-level databases, where available, to check for any evidence of duplicate participants and unexpected homogeneity or heterogeneity. From this, a meta-analysis was conducted with subgroups of clinical trials at different risk of bias levels.

Looking at the key survival outcome, the analysis includes 12 studies with a total of 2628 participants (Table 1 ) [4] . This included 4 studies at a low risk of bias, 4 with some concerns, 3 at a high risk of bias, and 1 potentially fraudulent study. The analysis demonstrates that on including all 12 studies, ivermectin results in a significant 51% increase in survival (95% CI, 0.28-0.86; P = .01) ( There are added challenges with clinical trials investigating COVID-19 treatments. In a rapid response to COVID-19, many small-scale studies have been conducted globally for potential agents. However, not all trials have reported findings. An example is a trial for nitazoxanide in Brazil with 600 participants that was completed in October 2020 but has not reported any results yet [14] . Publication bias impacts meta-analyses, with positive and significant results more likely to be reported. Therefore, we believe that by including publication bias, it is even less likely that ivermectin will show significant benefits for COVID-19 treatment. Some nonrandomized trials may also be overinterpreted. For example, in a nonrandomized retrospective cohort study, remdesivir demonstrated an improvement in clinical recovery and reduced mortality risk by 62% [15] . However, when evaluated in the WHO's randomized placebo-controlled SOLIDARITY trial, remdesivir had little to no effect on mortality [16] . Any initial promising findings from a small number of sources need to be interpreted with caution, studied further, and considered within the wider body of evidence.

The results from this analysis highlight the need for rigorous quality assessments when evaluating clinical trials of drugs for COVID-19. Existing and widely used risk of bias assessment tools are not enough. These tools provide a systematic framework for identifying potential key sources of bias in a trial's internal methodology but work on the fundamental assumption that a published study is reporting accurate and complete findings. They allow reviewers to make judgments on the assumption that basic standard procedure is followed, the data are real, and no information is being intentionally hidden. With cases of potential medical fraud now identified, it is essential that access to patient-level databases be provided. If authors fail to provide these data, the study should be considered with a higher index of suspicion. Additionally, it should be mandatory that all registered trials report their findings. We understand that these are substantial changes to established procedures. However, the failure to recognize the potentially fraudulent studies, which led to multiple meta-analyses suggesting significant benefits of ivermectin for COVID-19, indicates that the tools currently used to evaluate the quality of clinical trials are insufficient. These events warrant our stringent recommendations. 

The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro

Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines

Review of the emerging evidence demonstrating the efficacy of ivermectin in the prophylaxis and treatment of COVID-19

Meta-analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection

Desperation, misinformation: how the ivermectin craze spread across the world. The Guardian

Therapeutics and COVID-19: living guideline 2021. 2021. Available at

RoB 2: a revised tool for assessing risk of bias in randomised trials

Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: a randomized multi-center clinical trial

Evaluation of the effectiveness and safety of adding ivermectin to treatment in severe COVID-19 patients

Controlled randomized clinical trial on using ivermectin with doxycycline for treating COVID-19 patients in Baghdad

Notice of retraction: Efficacy and safety of ivermectin for treatment and prophylaxis of COVID-19 pandemic

Flawed ivermectin preprint highlights challenges of COVID drug studies

Effects of a single dose of ivermectin on viral and clinical outcomes in asymptomatic SARS-CoV-2 infected subjects: a pilot clinical trial in Lebanon. Viruses

Nitazoxanide therapy for patients with COVID-19 pneumonia -full text view -ClinicalTrials.gov

Gilead presents additional data on investigational antiviral remdesivir for the treatment of COVID-19 2020

World Health Organization. Repurposed antiviral drugs for Covid-19 -interim WHO solidarity trial results

We would like to thank Gideon Meyerowitz-Katz, Kyle A. Sheldrick, and Jack M. Lawrence for their help with the quality assessments.