key: cord-0694706-uhn0th6h
authors: Toumi, Mondher; Biernikiewicz, Malgorzata; Liang, Shuyao; Wang, Yitong; Qiu, Tingting; Han, Ru
title: A critical analysis and review of Lancet COVID-19 hydroxychloroquine study
date: 2020-09-10
journal: Journal of market access & health policy
DOI: 10.1080/20016689.2020.1809236
sha: 7eb8080f9fbc67545e5187c6961a603f20ac160e
doc_id: 694706
cord_uid: uhn0th6h

Purpose A international registry analysis led by Mehra et al. to investigate the use of hydroxychloroquine (HCQ) and chloroquine (CQ) with or without a macrolide in 96,032 hospitalised COVID-19 patients were published on Lancet, which has raised considerable discussions in the public health community. This study aimed to critically review the quality and limitations of the Mehra et al. publication and discuss the potential influences on the use of HCQ/CQ worldwide. Method A critical review of this publication was conducted to examine the potential study bias in the study objectives, methodology, confounding factors and outcomes and summarise the external reviews. Results The very high homogeneity of the patients’ characteristics at baseline was inconsistent with region specific epidemiology and several critical confounding factors. The results indicated that angiotensin converting enzyme inhibitors were associated with a hazard ratio of 0.5, which suggested a technical problem in the estimation of the propensity scores. Several major risk factors for mortality identified in the analysis were treated as a minor risk or neutral or even protective factors. Antiviral treatments were recognised as an effective method to reduce mortality and were neither further studied nor integrated in the multivariate Cox model. Conclusion This research appeared to carry multiple biases. An extensive audit of the study, conditions of review and acceptance for publication in the Lancet of that study are requested to avoid damage to the publics’ trust on the scientific community at this critical time of COVID-19 pandemic.

Coronavirus disease 2019 , caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was initially reported in China in December of 2019. COVID-19 was declared a pandemic by World Health Organization (WHO) because of the fast-worldwide spread on 12 March 2020.

As of this writing, there is no vaccine for the prevention of COVID-19 and no proven, effective pharmacologic treatment for COVID-19. CQ and HCQ, known as the treatment for malaria and rheumatic diseases, have been considered to be potentially useful in treating COVID-19 patients. Within the current context of the COVID-19 pandemic, however, CQ and HCQ were recommended by the Chinese [1] as well as several other drug agencies across Europe Africa and America [2, 3] in national guidelines for the treatment of COVID- 19 . Several clinical trials of small sample size reported the improvement of hospitalisation days, time to clinical recovery and time to nucleic acid negativity in COVID-19 patients treated with CQ and HCQ, compared with standard of care (SoC) [2] [3] [4] . However, the controversy over the efficacy and safety of CQ and HCQ is increasing. Inconsistent with the previous clinical trials, an open-label randomised controlled trial conducted by Tang et al. showed that HCQ treatment did not significantly increase the negative conversion rate compared with SoC and resulted in higher adverse events. An observational study with a large sample size conducted by Geleris et al. [5] found that there was no association between the shorter hospitalisation days and the administration of HCQ in COVID-19 patients, which aligned with another observational study designed by Rosenberg et al. [6] .

The results from a multinational registry analysis of 96,032 hospitalised COVID-19 patients conducted by Mehra et al. [7] raised considerable attention in public health practice and research that the usage of HCQ and CQ substantially increased mortality and the risk of cardiac arrhythmias, leading to the suspension of several CQ and HCQ clinical trials. The WHO temporarily suspended the recruitment to the HCQ arm in their multinational SOLIDARITY trial [8] and the French authorities changed its national recommendation for the use of HCQ in COVID-19 treatment while the French drug agency launched the suspension of all clinical trials using HCQ and the Public Health Council requested forbidding the use of HCQ for COVID-19 [8, 9] .

This study aimed to critically review the study of Mehra et al. in terms of study objectives, methodology and results and summarise the external reviews relating to the study.

A critical review of the studies of Mehra et al. [7] was conducted. A descriptive analysis of the study methodology including source of data, inclusion/exclusion criteria, intervention/control group, data collection, outcomes and statistical analysis was performed followed by a critical analysis of the domains including study objectives, methodology, confounding factors, study outcomes, results and external reviews relating to the study.

This multinational study was conducted using a patient register comprising 671 hospitals in 6 continents. The Surgical

Outcomes Collaborative (Surgisphere Corporation, Chicago, IL, USA) provided the real-world evidence, which was collected from electronic health records, supply chain databases, and financial records.

All PCR-confirmed COVID-19 patients, that were hospitalised between 20 December 2019 and 14 April 2020, and had a clinical outcome of either hospital discharge or death during hospitalisation were included.

Patients who received HCQ/CQ starting later than 48 h after COVID-19 diagnosis were excluded. Patients for whom treatment was initiated while they were on mechanical ventilation or if they were receiving therapy with the antiviral remdesivir were also excluded.

Four treatment groups, including CQ alone, CQ with a macrolide, HCQ alone, or HCQ with a macrolide. All other included patients served as the control population.

Patient demographics (e.g., including age, body-mass index (BMI), sex, race), underlying comorbidities and other medicines at the baseline were collected. The details on the use of HCQ/CQ and second generation macrolides were recorded.

The primary outcome was the association between the use of treatment regimens containing HCQ/CQ (with/ without macrolides) with in-hospital mortality. The secondary outcome was the association between these treatment regimens and the occurrence of clinically significant ventricular arrhythmias. Other outcomes included rates of mechanical ventilation use and the total intensive care unit (ICU) lengths of stay.

The Cox proportional hazards regression analysis was conducted to investigate the effects of demographic variables, comorbidities, baseline disease severity, and other medication uses on in-hospital mortality and significant ventricular arrhythmias. To minimise the effect of confounding factors, propensity score matching analysis was done individually for each of the four treatment groups compared with the control group. Additional analyses were done to examine the robustness of the initial estimations, which included individual analyses using Cox proportional hazards models and a tipping-point analysis.

The authors described the available evidence on HCQ or CQ as anecdotical. Despite several trials being ongoing they considered pressing on to provide guidance on how to use CQ or HCQ with macrolide as it is widespread with no attention to safety. They clearly expressed a pre-conceived negative attitude about CQ or HCQ safety in association with macrolide in the introduction.

The widespread use of HCQ or CQ in association with macrolide is not consistent with the authors finding as this association represented 10,004 patients, accounting for 10% of the study sample, while patients receiving an antiviral treatment at the exclusion of remdesivir represented 40% of the sample. No supportive evidence exists for any of the antiviral (except remdesivir) and their safety profile is far from benign. However, this did not capture the attention of the authors to assess the benefit and toxicity of antiviral therapies prescribed to 40% of the population.

Moreover, using antivirals as an additional treatment arm would have served to validate the whole study as these products do have a well-established safety that should have been retrieved in such study.

The information source is unknown and does not appear in the supplementary material.

Further, the authors have not provided the list of hospitals included, neither acknowledged the list of contributors in these hospitals who generated these data as it is good publication practice in large trials to express acknowledgement to all contributors.

When data are missing, i.e. not documented, the authors considered this as a negative response. If a patient had diabetes but not reported in his medical record, then the patient would be considered as not diabetic. While physicians are overloaded, such missing information may likely be the norm for all medical records in all geographies, which opens the door for massive biases.

Independence of variables used for the multivariate Cox model was not tested, and obviously, several variables are correlated, such as pathognomonic treatments and comorbidities, chronic obstructive pulmonary disease (COPD) severity and peripheral capillary oxygen saturation (SpO2), etc.

Antiviral treatment was not integrated in the survival model but in arrhythmia model, while it was shown in the bivariate analysis to be associated to mortality reduction and is present in 40% of the population.

The centre effect or at least country effect was not part of the analysis while centre behaviour, practice and patient's selection are systematic centres specific confounding variables well known in randomised and nonrandomised studies.

Wang et al. [10] reported that high-sensitivity C-reactive protein (hsCRP), SpO2, neutrophil and lymphocyte count, D-dimer, aspartate aminotransferase (AST) and glomerular filtration rate (GFR) had a significantly stronger discriminatory power than the clinical model (p = 0.0157). Therefore, such confounding factors should have been reported and integrated in the analysis. Only Sp02 was used in the Cox model developed by Mehra et al. [7] , while other variables were not considered even though they were important confounding factors.

The QT interval at the baseline was not reported, as well as the QT interval lengthening during treatment. It is expected that QT lengthening is a cause of arrhythmia caused by CQ or HCQ with or without macrolides. This is important information as well as the baseline QT, which may be a contraindication for prescription of HCQ or CQ with or without macrolides.

Comorbidity severity was not considered while it has been described as a critical confounding factor [11] .

Using SPO2 as a categorical variable below and above 94% induce a loss of predictive value, as it was found in this study that SpO2 was one of the most critical mortality predictors. Authors should have used SpO2 as a continuous variable and not as a categorical variable.

Cancer was also considered as a significant risk factor for mortality in COVID-19 patients but was not reported [12, 13] .

The arrhythmia report and assessment raised two interrogations: How the duration of arrhythmia was collected and reported, and why the type of arrhythmia was not collected and used in the study?

It is very difficult to collect from a patient's medical record the information of non-sustained ventricular arrhythmia that last at least 6 seconds. It is expected to be a categorical variable (yes/no) information with no more details on the duration. This assumed that all patients received ongoing cardiac monitoring with recording to be able to identify arrhythmia and its duration, which is highly unlikely in all countries, including the most developed, let alone developing countries.

If this was available, it would be expected that the nature of the arrhythmia would also be reported, such as, for example, Torsade de Pointe (TdP), which is the expected arrhythmia consecutive to QT interval lengthening, which may further develop in to another ventricular arrhythmia. To ascertain the cause of arrhythmia to QT lengthening the authors should have considered the type of arrhythmia that occurred. It is surprising that reputed cardiologists did not put attention to the type of ventricular arrhythmia. The nature of the ventricular arrhythmia is critical information because it is an end point. The crude appreciation of arrhythmia with no more specification represents a significant potential for misclassification bias on a critical study end point.

The Cox proportional hazards model results for inhospital mortality in different countries and regions were summarised in Table 1 . There were no country region or side effect in this study which may represent a major confounding and should have been integrated as a confounding factor. It is expected that patients' profiles may be disproportionately distributed in different country centres, including comorbidities.

The other end points may be available in electronic medical records even though the duration of ventilation and the stay in ICU are driven by heterogeneous practice and availability of beds and ventilators that were not equally distributed across regions. Moreover, what an ICU differs across regions depending on the availability of technical resources. Obviously, what an ICU is in Berlin and New York may not be the same as what it is in Yaoundé or Bamako. It is not specified what defined an ICU?

It is not specified if patients may have been ventilated outside of an ICU and if this was balanced between treatment arms.

It is well established that the use of therapies prolonging the QT interval exposes patients to the risk of TdP that may precipitate into ventricular arrhythmias. Therefore, the prescription of a combination of products susceptible to lengthen the QT interval may raise concerns about the risk of arrhythmia [14] [15] [16] .

There is a well-established contraindication to such products and there is a need to assess the QT interval before administration and monitoring during treatment. If the QT interval is lengthening, treatment should be discontinued.

It would be of the utmost importance to clarify if patients presenting arrhythmia had been screened for contraindications and well monitored while on treatment. This is essential information that was not reported in that study while likely available in the medical records.

Baseline patients' disposition is totally unexpected raising questions of severe bias in the patients sampling. Indeed, for the six considered geographies, North America, South America, Europe, Africa, Asia and Australia, the patients baseline presentation was very homogeneous except for Australia, while substantial differences were anticipated because of heterogeneous epidemiology, population, weight, age, etc.

The age of hospitalised patients was similar while it would be expected to be younger in Africa and Asia. BMI was strictly similar in Europe and North America with the same standard deviation while it is expected to be significantly higher in North America.

Cardiovascular disease, COPD and diabetes prevalence in this sample are the same in all countries except for Australia.

Similarly, the treatment with angiotensin-convertingenzyme (ACE) inhibitors, statins, angiotensin receptor blockers, and antiviral therapies were strictly similar across geographies with very different clinical practice and very different epidemiology for non-transmissible diseases. Despite extremely different hospital capacities, ICU beds and ventilators availability, the patients hospitalised did have strictly the same severity, same length of stay in non-ICU and ICU and the same recourse to ventilators. The outcomes are also surprisingly very similar for mortality and arrhythmia. For example, between two North American cities, there were double the proportion of hospitalised patients were in ICU in Vancouver over New York. We can imagine a difference between Africa and North America [17] . So, the figures retrieved in the study are totally unexpected.

The very high homogeneity between the population hospitalised for COVID-19 in the six regions assessed, as well as the outcomes are not consistent with what is expected because of the difference for population, socio-demographic, epidemiology of a non-transmissible disease, hospital resources available, clinical practice and medical expertise and technicity.

These results could only be explained by a major selection bias or data quality that may have been compromised at one or several steps in data processing, making the data source unreliable. Data integrity is at stake for this study. This raises high doubts on the overall reliability of the Surgisphere database. The company should act professionally to clarify this serious concern. This put to question the reliability of all the projects performed to date using this database.

Despite antiviral therapies appearing to have a protective effect in this study, the authors did not attempt to identify and quantify this benefit. At the time where all scientists involved in this field are desperately looking for an effective therapy the authors did not consider the identification of death sparing therapies as an important point for the scientific medical community and public. This is not consistent with literature as antivirals, except for remdesivir, have been shown to be ineffective. Such a finding would have deserved more attention and information sharing with the scientific community.

Some results are totally inconsistent between geographies: for example, diabetes was a significant risk factor in North America and Europe, while it had an important protective statistically significant effect (hazard ratio 0.75) in South America and Africa, while it was neutral in Asia and Australia. There is no biological or epidemiological rationale to explain the inconsistency of such results. Moreover, this is inconsistent in all research performed so far and all published evidence that acknowledges diabetes as one of the most important risk factors for fatality [18] together with age [19] , BMI [20] , and COPD [21] . In England, one-third of coronavirus related deaths in hospitals was associated with diabetes [18] . This is in contradiction with the current findings by Mehra et al. [7] .

In the Mehra et al. study [7] , COPD was also inconsistent as it was a significant risk factor in Australia and North America, while it had no effect in Europe, Africa, and Asia. COPD was described across the geography as a critical risk factor for fatality [21] . This is inconsistent in the study findings that SpO2 at baseline was one of the most important risk factors for mortality.

It is also interesting to notice that in all geographies, age and BMI had no impact on mortality. This is totally inconsistent with all evidence published so far [18] [19] [20] .

Treating patients with ACE inhibitors reduces mortality by about 50% across all regions and statin by about 25%. While at the same time hypertension and other cardiovascular diseases are risk factors for death.

Being immunocompromised happens to be in most geographies a modest risk factor with a hazard ratio of 1.1, while it had no effect in Asia and Australia. This is also a surprising result.

While antiviral treatments are associated to mortality reduction, they have not been integrated in the cox model.

Mortality rate at a hospital in this study was 11% which seems inconsistent with the reported results in the UK where 35% to 40% have died [22] . It was 23.5% in another survey in the UK [23] , 20% in France [24] and overall ranged from 50% to 23% in other countries [25] . The mortality in hospitals in this study is unusually low, suggesting a bias in the sample. This is consistent with the low severity of the study sample described above which is inconsistent with hospitalisation in all geographies.

The study results call for a systematic use of ACE inhibitors, which are shown to reduce mortality by 50% and eventually, in combination with statin, reach a potential of 75% reduction in mortality if their effect is additive. This is the most critical finding of that study and it seems to have been overlooked by the authors.

It is very difficult to trust these results as they are inconsistent between regions with no rationale. They are totally disconnected from current scientific wellestablished evidence and they are totally counterintuitive.

This could only be explained by major selection biases or data quality that may have been compromised at one or several steps in data processing.

A publication was immediately submitted by Funck-Brentano et al. and accepted expeditiously for publication in the Lancet to endorse the results of Mehra et al., and call for not using CQ or HCQ, with no critical review of the study. This provided echo to the Mehra et al. study and was largely disseminated by the media [7, 14, [26] [27] [28] [29] .

After deciding to discontinue the HCQ arm in the discovery trial [26] , the monitoring committees scrutinised the Solidarity data and discovered that they were inconsistent with the data from Mehra et al. [30] The monitoring committee informed the investigators of the reverse decision to continue the HCQ treatment arm [31] . This has been very destabilising for patients and investigators.

However, most reactions expressed doubts suggesting some scientists critically read Mehra et al. An open letter to the editor and authors [32] , signed by more than 100 reputed experts, questioned the data integrity, and the results arguing for implausible data on the dose of HCQ and the ratio of use in some continents, the low likelihood of some data for Africa and Australia, the unusual standard deviation for several variables while from very heterogeneous sources, errors on the Australian data set, etc. The authors were denied access to the data source or the code despite the Lancet was a signatory on the Wellcome statement on data sharing in case of emergency [33] .

In response, Harvey A. Risch wrote an article to support HCQ continuation [34] .

The excess mortality was not consistent with several published studies, which may or may not have found improved efficacy with HCQ or CQ [5, 6, 35, 36] .

This study appeared to carry multiple biases: selection biases, classification biases, and several critical confounding factors that are not included in the study or in the analysis. The statistical analysis is flawed.

The very high homogeneity of patients' disposition at baseline is inconsistent with region specific epidemiology and socio-demographic. It is very difficult to believe in the data. Beyond the several biases, failure at one or several steps of the data processing is a likely explanation.

The heterogeneity of the results between regions where the same factor may constitute either a risk for fatality in some countries or a protection in others with no biological or epidemiological rational to support it, raise questions on the data integrity and the quality of the analysis.

The fact that ACE inhibitors are associated with a hazard ratio of 0.5, while hypertension is a risk factor for death, suggests a technical problem in the estimation of the propensity scores. If there are no technical problems in the propensity score estimation, then this call to strongly advocate for ACE inhibitors as the most effective treatment for COVID-19 identified so far are valid. Obviously, authors did not feel confident doing this. This suggests a limited trust in their own data.

The fact that several major risk factors for fatality systematically recognised in all studies are identified in that study as neutral or as a protective factors or eventually as a minor risk, represent a real nonsense that should have triggered a lot of doubt on the reliability of the results. Antiviral treatments are recognised as an effective way to reduce mortality and are neither further studied nor integrated in the multivariate cox model. Surprisingly while ACE inhibitors were found to reduce mortality by 50% and statin by 25% the authors ignore this major information as they did not trust it to be correct. That was the price to pay to achieve the premeditated outcome.

This study questioned the validity of the Surgical Outcomes Collaborative (Surgisphere Corporation, Chicago, IL, USA) that provided the real-world evidence, which was collected from electronic health records, supply chain databases, and financial records. There is serious doubt about the validity of the data collected as well as the data processing. This question the reliability of all studies published so far using this data source. We call for an audit of all these studies.

Finally, a severely flawed study widely advocated through a well-orchestrated international media campaign was published in one of the most reputed medical journal leading to the suspension of HCQ or CQ trials around the world and forbidding use of CQ or HCQ for COVID-19 patients in several countries. This decision from the editorial committee, to publish this flawed study with little precaution, may prevent ever getting access to a well-conducted study to inform the medical community and the public on the actual efficacy and safety of CQ or HCQ in treating COVID-19 patients.

An extensive audit of this study, as well as the conditions of review and acceptation for publication in the Lancet, one of the most prestigious medical journal, are requested to avoid the public to lose trust in the scientific community at a time when it has become difficult to access to reliable information on COVID-19.

Chinese guidelines related to novel coronavirus pneumonia

Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

Clinical and microbiological effect of a combination of hydroxychloroquine and azithromycin in 80 COVID-19 patients with at least a six-day follow up: an observational study. Mediterranee Infection

Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial

Observational study of hydroxychloroquine in hospitalized patients with Covid-19

Association of treatment with hydroxychloroquine or azithromycin with in-hospital mortality in patients with COVID-19 in New York state

Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. The Lancet

ANSM annonce à son tour suspendre par précaution les essais cliniques évaluant l'hydroxychloroquine dans la prise en charge des patients atteints de Covid-19

COVID-19 Treatment Guidelines Panel. Coronavirus disease 2019 (COVID-19) treatment guidelines. National Institutes of Health

Clinical and laboratory predictors of in-hospital mortality in patients with COVID-19: a cohort study in Wuhan, China

Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis

Clinical impact of COVID-19 on patients with cancer (CCC19): a cohort study

COVID-19 mortality in patients with cancer on chemotherapy or other anticancer treatments: a prospective cohort study. The Lancet

Chloroquine or hydroxychloroquine for COVID-19: why might they be hazardous? The Lancet

Cardiotoxicity of antimalarial drugs

Additive proarrhythmic effect of combined treatment with QTprolonging agents

Little S Study finds lower COVID-19 death rate in metro vancouver ICUs than other cities

Does poor glucose control increase the severity and mortality in patients with diabetes and COVID-19?

Risk factors for adverse clinical outcomes with COVID-19 in China: a multicenter, retrospective, observational study

Obesity is associated with worse outcomes in COVID-19: analysis of early data from New York city

Prevalence, severity and mortality associated with COPD and smoking in patients with COVID-19: a rapid systematic review and meta-analysis

Third of UK Covid-19 patients taken to hospital die, study finds 2020

COVID-19 in-patient hospital mortality by ethnicity. Wellcome Open Res

19 en France: les statistiques du mercredi

COVID-19 death rate at Metro Vancouver hospital ICUs notably lower than in other countries: study

Kunzmann K WHO suspends hydroxychloroquine treatment in COVID-19 solidarity trial

Yasgur BS More evidence hydroxychloroquine is ineffective, harmful in COVID-19

No improvement in death rate for COVID-19 patients who received hydroxychloroquine

No evidence of benefit for chloroquine and hydroxychloroquine in COVID-19 patients, study finds

Sandercock P RECOVERY trialDMC chairman's report

Recruitment to the RECOVERY trial (including the Hydroxychloroquine arm) REMAINS OPEN. 2020

An open letter to Mehra et al and The Lancet. Zenodo

Statement on data sharing in public health emergencies

Early outpatient treatment of symptomatic, high-risk COVID-19 patients that should be ramped-up immediately as key to the pandemic crisis

Raoult D Early treatment of 1061 COVID-19 patients with hydroxychloroquine and azithromycin

Azithromycin plus chloroquine: combination therapy for protection against malaria and sexually transmitted infections in pregnancy

Mondher Toumi contributed to the study conception, design, analysis and interpretation of data and original draft preparation. Malgorzata Biernikiewicz contributed to editing and review of the article. Shuyao LIANG contributed to the study conception, and original draft preparation and review of the article. Yitong WANG, Tingting QIU and Ru HAN contributed to the study conception, acquisition of data and review of the article. All authors read and approved the final manuscript.

The authors claim that they have no conflicts of interest.

This study did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Not required as this study was just a critical review that examined the quality and limitations of the Mehra et al. publication and discussed the potential influences on the use of HCQ/CQ worldwide.

http://orcid.org/0000-0001-7939-7204 Malgorzata Biernikiewicz http://orcid.org/0000-0003-2377-5290 Shuyao Liang http://orcid.org/0000-0002-0627-8922