key: cord-1055176-1zpcnn4l
authors: Martins, Russell Seth; Cheema, Daniyaal AHmad; Sohail, M. Rizwan
title: The Pandemic of Publications: Are We Sacrificing Quality for Quantity?
date: 2020-10-01
journal: Mayo Clin Proc
DOI: 10.1016/j.mayocp.2020.07.026
sha: 2a6343b6936b3e89bfb0c5114539ec160a33e194
doc_id: 1055176
cord_uid: 1zpcnn4l

nan

The Pandemic of Publications: Are We Sacrificing Quality for Quantity?

To The Editor: The coronavirus disease 2019 (COVID-19) pandemic has sparked an explosion in biological and medical research worldwide, with more than 30,000 COVID-19 PubMed-indexed publications and 6200 medRxiv and bioRxiv preprints this year as of July 9th, 2020. 1,2 However, while immediate access to the latest research is undoubtedly crucial in combating this pandemic, the scientific community must be wary of the potential tradeoff between quantity and quality. In this letter, we discuss the implications of this surge in the rate of publications and suggest measures to minimize undesirable consequences.

During the first week of June 2020, the editors of The Lancet and The New England Journal of Medicine (NEJM), 2 premiere medical journals, both published expressions of concern regarding the questionable quality and reliability of data presented in articles published in their respective journals. 3, 4 Both of the articles in question were published using data from the Surgical Outcomes Collaborative (Surgisphere Corporation, Chicago, IL). In addition, a third article, first uploaded as a preprint in the Social Science Research Network, also used data from Surgisphere. 5 All 3 articles concerned drug therapy in COVID-19 and findings reported in these publications had a major impact on both ongoing clinical patient management and the direction of several clinical trials. The World Health Organization chose to err on the side of caution and suspended the hydroxychloroquine arm of the Solidarity Trial (an ongoing international clinical trial exploring treatments for COVID-19), thus delaying the provision of evidence-based guidelines to practitioners worldwide. The consequences of this occurrence can be exceedingly harmful, and the significant doubts raised regarding these papers has called into question the publishing behavior of both researchers and journals.

With the eagerness to publish COVID-19 research, there is a looming potential for errors on the part of both the researchers and the journals. For the researcher, this may involve publishing research of subpar methodological quality or doubtful ethical considerations, or publishing in predatory medical journals. 6, 7 In addition, researchers are rushing to upload their work as preprints, opening the floodgates to a "preprint surge." While preprints do offer a legitimate platform to share and gain quick feedback on novel results, the haste to preprint articles invites an inherent tradeoff between speed and accuracy. 8, 9 The non-peer-reviewed nature of preprints may often promote the spread of inaccurate information, with flawed data often finding itself under the public spotlight through social media and news outlets. 8, 9 As the death toll from the pandemic surges, desperation may force clinicians to turn to insufficiently peer-reviewed research for decision-making purposes. However, even peer-reviewed research published by legitimate journals may be called into question. With many highly reputable journals offering fast-tracked services for COVID-19 publications, the robustness of the methodological examination and critical appraisal within the peer review of submissions may be compromised. 9 This is evident in the redundancy of COVID-19 research, which in addition to being repetitive, may also be inaccurate, inconsistent, biased, and biologically implausible. 10 The torrent of exaggerated or subpar research being published may lead to dangerous consequences and threaten the evidence-based response to COVID-19 globally.

With this pandemic leaving health care systems in disarray, healthcare professionals on the frontline face many challenges, ranging from the lack of protective equipment to the overwhelming patient load. It is the collective responsibility of the academic community to make sure that accessing accurate information does not become another challenge. Much of the responsibility for ensuring high quality research lies with the researchers themselves. While one would like to believe that global health and advancing science are the sole motivators to conduct and publish research, some of the COVID-19 research may be due to researchers wanting to jump on the bandwagon for personal gain, such as improving one's resume or attracting media and public attention. This behavior must be countered by a paradigm cultural shift within the academic community, where the quality of research and publishing journals adds more to a researcher's value than the quantity of publications. Additionally, the academic community must work together towards identifying and denouncing journals that are of a predatory nature and perhaps discount the value of publications in such journals when considering matters such as academic appointments and promotions.

Given the potential harm posed by the publication of articles of subpar quality, reputable medical journals must also consider additional mechanisms to ensure the quality of their fast-tracked peer review process and publication services. These measures could include a preliminary assessment of the submitted work for data sources, data integrity, and research methodology by either editorial staff or external service providers with expertise in this domain. This process may be similar to how news organizations have either internal or external mechanisms for fact-checking news reports or political speech.

In the particular case of Surgisphere, the health care analytics company that provided the data for the two fast-tracked manuscripts published in The Lancet and NEJM, 3, 4 the enticing data set provided by the company included nearly 100,000 detailed patient records from 700 hospitals on 6 continents. This should have raised some red flags as it is questionable how a small analytics firm with few employees, an almost nonexistent track record for publications, and no publicly named scientific board, could have obtained reliable patient-level data from hundreds of hospitals on 6 continents in a regulatory environment that strictly limits sharing of protected health information. Moreover, none of the hospitals confirmed working with Surgisphere. The repute of the analytics company and the trustworthiness of the provided data was never questioned because such source checking is not part of the routine peer review process and reviewers are neither trained nor have expertise in this domain. Careful scrutiny of the databases and their sources is even more important as medicine moves towards big data, machine learning, and artificial intelligence. On the other hand, analyses based on data from more reliable and vetted sources such as the National (Nationwide) Inpatient Sample, 11 Center for Medicare and Medicaid Services data, 12 and data from clinical trials registered at ClinicalTrials.gov 13 could be used more reliably for fasttracked publications in reputable journals.

The surge in the use of preprint servers during this pandemic also warrants a cautious approach. While preprint servers provide a valuable service and are critical in disseminating timely information to readers during a pandemic such as COVID-19, the non-peer-reviewed nature of the manuscripts posted on these servers, their findings, and the resultant conclusions must be fully acknowledged by the medical community and media. Reputable medical journals should clearly highlight the citing of preprints in the manuscripts submitted to their journal for publication. Similarly, both clinicians and researchers must exercise caution when disseminating unvetted evidence to the public, especially on social media platforms.

The race to publish during this pandemic may cause both the medical community and the public to lose faith in medical research. Therefore, it is imperative that the academic community takes full responsibility for the work that is published, as well as its potential implications, and strives towards ensuring the publication and dissemination of only the highest quality research during these trying times.

National Library of Medicine National Center for Biotechnology Information. NCBI SARS-CoV-2 Resources

COVID-19 SARS-CoV-2 preprints from medRxiv and bioRxiv

Expression of concern: hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis

Expression of Concern: Mehra MR et al. Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19

Usefulness of Ivermectin in COVID-19 Illness

COVID-19 coronavirus research has overall low methodological quality thus far: case in point for chloroquine/hydroxychloroquine

Publishing research during pandemics: are you vulnerable to the COVID-19 or predatory publishers

Coronavirus and the risks of 'speed science

Parallel pandemic: the crush of covid-19 publications tests the capacity of scientific publishing. The BMJ Opinion website

Redundancy in reporting on COVID-19

Healthcare Cost and Utilization Project. NIS Database Documentation

Center for Medicare and Medicaid Services Database

Publication Rate and Journal Review Time of COVID-19eRelated

 Figure A) . Of 2427 journals in the LitCovid archive, 1294 (53.3%) listed bibliographic information. These journals published 7798 COVID-19 publications and 340,032 noneCOVID-19 publications. The average time to review an article was significantly fewer days for COVID-19erelated publications (11.3 days vs 106.3 days; P<.001) ( Figure B ). Given the COVID-19 crisis, these findings may be appropriate. Editors should be commended for eliminating unnecessary administrative barriers and reviewers should be lauded for rapidly reviewing COVID-19erelated manuscripts. However, potential consequences of these data should be considered.First, thousands of COVID-19 publications have been disseminated in preprint archives, which have not undergone peer review. Second, the volume of submissions might render it difficult to identify reporting of the same patients in different