key: cord-0719198-xhhurg4u
authors: Elgendy, Islam Y.; Nimri, Nayef; Barakat, Amr F.; Ibrahim, Joseph; Mandrola, John; Foy, Andrew
title: A systematic bias assessment of top-cited full-length original clinical investigations related to COVID-19
date: 2021-01-22
journal: Eur J Intern Med
DOI: 10.1016/j.ejim.2021.01.018
sha: 600a11dee754c0a3487cf21e83d1e3f291f2239a
doc_id: 719198
cord_uid: xhhurg4u

nan

With the emergence of coronavirus disease 2019 (COVID-19) pandemic, there has been an exponential rise in the number of scientific publications to convey relevant information about this novel disease. 1 Some data suggest that original investigations related to COVID-19 are published in a fairly shorter time frame by some journals compared with non-COVID articles, which might affect the quality and rigor of these original investigations. 2 Despite these concerns, there is a lack of an objective analysis to study the rigor of original clinical investigations related to COVID-19.

Using Dimensions, an online searchable platform that collects data on >100 million publications, 3 we identified the top 50 cited COVID-19 full-length original clinical investigations on June 24, 2020. The following types of studies were included: observational studies (i.e., diagnostic, prognostic, and non-randomized studies of intervention), randomized clinical trials (RCTs), and meta-analyses. Case reports, case series, and descriptive studies were excluded since these categories of studies are not typically published in high impact journals unless for emerging or exceptional conditions. Modelling studies were excluded since there are no formal quality assessment tool for these studies. Systematic reviews without quantitative data synthesis were excluded since there is a lack of consensus if these types of investigations are considered as original investigations. 4 Brief research reports and research letters were also excluded since we expected that various aspects of the methods might be not fully discussed given the brevity of these publications. Finally, animal and pre-clinical studies were excluded since the focus of this study was on clinical investigations. Studies in language other than English were excluded. In order to obtain a 1:1 matched historical control group, for each COVID-related study, we screened consecutive articles published in the same journal in 2019 until a full-length original investigation of the same aforementioned study design category was identified. If a matched control article could not be identified, then the corresponding COVID-19 related article was excluded. The quality assessment of the articles was performed by 2 independent investigators (NN and JI), and verified by a third investigator (IE). The following checklists were used to evaluate the studies: i) ROBINS-I for non-randomized studies of intervention; 5 ii) the Center for Evidence-Based Medicine tool for diagnostic studies; 6 iii) the Center for Evidence-Based

Medicine tool for prognostic studies; 7 iv) RoB 2 for RCTs; 8 and v) AMSTAR 2 for metaanalyses. 9 The percent agreement between reviewers for each study type was calculated. The observations from this analysis supports the notion that an accelerated process of handling COVID-related articles might have compromised the peer review process and facilitated the publication of some studies at much higher risk for bias than what is typically accepted by the same journals. While there is a timely need to deliver medical knowledge through scientific publications, the introduction of lower quality studies to the medical literature might lead to misdirected academic efforts, drawing inaccurate conclusions, and retractions. 10 This study has limitations that are worth mentioning. The findings of this analysis apply to the included journals, and might not be generalizable to other journals. Some of the included studies were conducted early during the pandemic. Many ongoing high quality RCTs and prospective studies are on the way. It is reassuring that the included RCTs in this study seemed to be at low risk of bias. 

The landscape of medical literature in the era of COVID-19: Original research versus opinion pieces

Timeline from receipt to online publication of COVID-19 original research articles

Attitudes of editors of core clinical journals about whether systematic reviews are original research: a mixed-methods study

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

The Center of Evidence-Based Medicine critical appraisal worksheet for diagnostic accuracy studies

The Center of Evidence-Based Medicine critical appraisal of prognostic studies

RoB 2: a revised tool for assessing risk of bias in randomised trials

AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

Publishing in the time of COVID-19