key: cord-0049015-swha1e4m
authors: Huizinga, Tom W J; Knevel, Rachel
title: Interpreting big-data analysis of retrospective observational data
date: 2020-08-21
journal: Lancet Rheumatol
DOI: 10.1016/s2665-9913(20)30289-7
sha: 88836c0bf58514ee5abc5aecd040703b8bb815db
doc_id: 49015
cord_uid: swha1e4m

nan

Interpreting big-data analysis of retrospective observational data Current data technology offers fantastic new oppor tunities to generate data that can inform us about the safety of drugs. These data will affect the way we use drugs by balancing benefits of specific agents with better and more information on their associated risks. Nowadays, possibilities to use big data to deal with safety concerns are enormous, and it is difficult not to get enthusias tic reading papers that take this approach. An out standing example is the use of claims data of 160 000 patients with rheumatoid arthritis to assess the risk for lowertract gastrointestinal perforation associated with tocilizumab and tofacitinib in comparison with other biological drugs. 1 In The Lancet Rheumatology, Jennifer Lane and col leagues present a study using claims data and elec tronic medical records (mostly of patients with rheuma toid arthritis) to analyse the longterm risks of cardiovas cular complications (among other outcomes) in about 1 000 000 users of hydroxychloroquine compared with more than 300 000 users of sulfasalazine. 2 This analy sis is relevant because the European League Against Rheumatism (EULAR) guidelines for the treatment of patients with systemic lupus erythematosus (SLE) recom mend hydroxychloroquine for all patients with SLE, and in practice the drug is given for decades. 3 Most doctors will feel that a study as large as that of Lane and colleagues is most likely relevant, and they will try to weigh the information presented to optimise treatment strategies for their patients. It has been convincingly shown that most published data are false, 4 and the corollary that the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true is a relevant consideration given the recent discussions around use of hydroxychloroquine in patients with COVID19.

So what considerations can be made? Might this be a false positive result? In such a retrospective analysis of observational data, there can of course be confounding by indication. It is important to note that the authors used stateoftheart methods to deal with the chal lenges of studying retrospective electronic medical record data; they did a newuser cohort study and a selfcontrolled case series to avoid the risk of bias in a casecontrol design, using propensity scores, fitting models with ten-fold cross validation, and negative control outcome analyses. The study thus provides a relevant guide for researchers in the field of electronic medical record analyses. Still, the question remains whether the results should guide our current standards of care.

As the authors state in their discussion, the cohort included patients who were new users of hydroxy chloroquine or sulfasalazine with a diagnosis of rheuma toid arthritis, without medication use in the previous 365 days, and with at least 365 days of continuous observa tion time before the index event. In general, one expects hydroxychloroquine to be used in patients with more comorbidities and, from clinical reasoning alone, there is high potential for differences in the cohorts. This is a limitation of the study, as the authors correctly emphasise. Thoroughly constructed propensity scoring was used to adjust for confounders, but this approach cannot control for all differences and could accidently include intermediary variables. 5 It is also useful to look at the absolute numbers; the numbers of events for cardiovascularrelated mor tality was 4·39 per 1000 personyears for patients tak ing hydroxy chloroquine compared with 2·00 per 1000 personyears for patients on sulfasalazine. Given these very low absolute numbers, one needs to consider that if bias between the groups exists, then the differences between 4 per 1000 and 2 per 1000 years of observation might also be caused by bias. Although the selfcontrolled case series analysis overcome many of these possible biases, the indication for hydroxychloroquine use could still be a confounder. Another unfortunate fact is that normal indicators of causality such as dose-response were missing from the study because of apparent lack of variation in dose of hydroxychloroquine or the inability to obtain data on the association between the duration of hydroxychloroquine use and cardiovascular event rate.

The study by Lane and colleagues also lacks con trols to show that the database yields what it should. Maculopathy is a wellknown adverse effect of hydroxy chloroquine, but the authors were not able to observe an association between hydroxychloroquine use and maculo pathy in their databases. This might have been caused by positive control surveillance bias, but the absence of a positive control decreases the convincing ness of the data.

Finally, the key question for longterm hydroxy chloroquine prescription for patients with SLE is how the benefits balance the risk. The current study did not (and did not intend to) address this question. So although we feel that the study by Lane and colleagues is extremely interesting with regard to methodology, and we foresee the rapid growth of studies linking of electronic health record data and claims data, it is difficult to weigh the current data in the context of daily care of patients with SLE, in which so much convincing evidence exists for the positive effects of hydroxychloroquine as recommended by EULAR.

Risk for gastrointestinal perforation among rheumatoid arthritis patients receiving tofacitinib, tocilizumab, or other biologics

Risk of hydroxychloroquine alone and in combination with azithromycin in the treatment of rheumatoid arthritis: a multinational, retrospective study

2019 update of the EULAR recommendations for the management of systemic lupus erythematosus

Why most published research findings are false

On the estimation and use of propensity scores in casecontrol and casecohort studies

We declare no competing interests.Copyright © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license.