key: cord-0687605-9534p64h
authors: Brandt, Justin S; Grover, Sonal; Ananth, Cande V
title: Dissemination of research during the first year of the coronavirus disease 2019 pandemic
date: 2021-05-28
journal: J Investig Med
DOI: 10.1136/jim-2021-001923
sha: 23176e6c1d5255c5b7ff2564a70fe37cbeaa8626
doc_id: 687605
cord_uid: 9534p64h

nan

The rapid and effective dissemination of research during the coronavirus disease 2019 (COVID-19) pandemic is critical if healthcare providers and public health officials are to remain aware of new developments. Several organizations have collected relevant COVID-19 articles to facilitate data sharing, including the World Health Organization 1 and the National Institutes of Health (NIH) . 2 Yet, in this time of remarkable research productivity and social media influence, how peer-reviewed research disseminates to the global community remains poorly understood.

In order to ascertain how published research disseminated during the COVID-19 pandemic, we examined the world's peerreviewed literature on COVID-19 through the application of alternative metrics (altmetrics) based on social media engagement and evaluative bibliometrics using citation rates. We utilized almetrics and citation rates to analyze COVID-19 articles indexed in the NIH's iSearch COVID-19 portfolio 2 and the Almetric Explorer 3 (study flow chart, figure 1 ). The iSearch COVID-19 Portfolio contains peer-reviewed COVID-19 articles from PubMed and preprints from several sources. We restricted our analysis to articles and used this dataset to ascertain citation rates. These data were merged by articles' unique PubMed identification numbers with altmetrics from the Altmetric Explorer, where we ascertained articles' Almetric Attention Scores (AAS; composite score of social interest that includes mentions in newsfeeds, Twitter, Facebook, and Google, among other sources). 4 We analyzed COVID-19 articles in the ISI Web of Science (WoS) Core Collection 5 to evaluate article characteristics, including study type, authorship, and funding sources. We queried the ISI WoS Core Collection using the search terms "coronavirus disease 2019", "COVID-19", "severe acute respiratory syndrome coronavirus 2", "SARS-CoV-2", and "novel coronavirus". The ISI WoS query was limited to publications from January 2020 to February 2021 (with no language or article type restrictions). Although there are several search engines that can be used for bibliometric analysis, including Scopus, Medline, and Google Scholar, 6 7 we selected the ISI WoS Core Collection because this database was classically used by Eugene Garfield, the developer of the impact factor metric, to identify "citation classics". 8 Research data were downloaded from iSearch and merged with Almetric Explorer on February 26, 2021. The results of the ISI WoS query were also downloaded on February 26, 2021. This analysis was based on a total of 87,643 articles in iSearch that were merged with data from the Altmetric Explorer (75,960 (86.7%) published in 2020 and 11,682 (13.3%) published in 2021) and COVID-19 articles were published rapidly during the first months of the pandemic, peaking in April 2020, and then plateauing at persistently high rates (figure 2A). Trends in citation rates and AAS mirrored each other during the study period ( figure 2B ). There were 48 articles in iSearch with unique PubMed identification numbers that were retracted or were retraction notices, corresponding to 34 (0.04%) articles (geometric mean citation rate (95% CI) 7.7 (3.5 to 16.5) and arithmetic mean (SD) 30.3 (80.3) and geometric mean AAS (95% CI) 26.9 (8.5 to 84.9) and arithemetic mean AAS (SD) 1243.3 (3784.0)). Some of these articles received Letter to the editor substantial social media attention. For example, among the top 10 articles with highest AAS, one of the articles was retracted and another article was the official retraction notice from the journal that published that article. The top 10 articles with the highest citation rates and highest AAS, after excluding retracted articles, are described in table 1. The ISI WoS query revealed the most common document types were "articles" (47,717; 52.7%), "editorials" (14,491; 16.0%), and "letters" (14,073; 15.5%). The most frequent WoS categories were "Medicine, General & Internal" (11,111; 12.2%) , "Public, Environmental & Occupational Health" (7281; 8.0%), and "Infectious Disease" (4790; 5.2%). Over half of all articles originated from four countries: the US (25,312; 27.9%), China (10,535; 11.6%), Italy (8899; 9.8%), and England (8759; 9.7%). The top funding agencies were the NIH, the National Natural Science Foundation of China, and the European Commission.

In this study of the world's peer-reviewed COVID-19 literature in the first year of the pandemic, we observed a dramatic explosion of research output. With rapid publication of approximately 90,000 peer-reviewed articles that addressed all facets of COVID-19, the global community has been inundated with data. Notably, nearly 60% of the world's research output originated from four countries that were hit first and hardest by COVID-19. Three countries are geographically distinct from the origination site of the virus, highlighting the global impact of the disease.

We observed that mean citation rates and AAS mirrored each other. Although there is debate about whether altmetrics correlate with citation rates, 9 10 this study suggests these metrics have correlated during the pandemic.

While citation rates and altmetrics reflect influence, they do not provide insight into research quality. The explosion of COVID-19 publications has raised legitimate concerns about research quality 11 as well as misconduct. 12 Many journals, particularly top tier journals, prioritized submissions of COVID-19-related articles, potentially at the expense of other topics, and expedited their peer review and publication. The rush to publish on the part of investigators and journals may have encouraged suboptimal research designs and methods as well as suboptimal peer review. Retracted articles perhaps reflect the most egregious examples of how the push to publish during the pandemic promoted poorquality research. While the proportion of retracted articles in this study was small, these articles received substantial social media attention as well as high citation rates. This study underscores the need for a novel metric that prioritizes research quality rather than quantity.

Global research on coronavirus disease (COVID-19

2 National Institutes of Health. iSearch COVID-19 Portfolio

Almetric Explorer

Web of Science Core Collection

The Altmetric score: a new measure for article-level dissemination and impact

A bibliometric analysis of topcited journal articles in obstetrics and gynecology

A bibliometric analysis of obstetrics and gynecology articles with highest relative citation ratios

What is a citation classic

Association between citations, altmetrics, and article views in pediatric research

Correlation between Altmetric score and citations in pediatric surgery core journals

COVID-19 impact on research, lessons learned from COVID-19 research, implications for pediatric research

The carnage of substandard research during the COVID-19 pandemic: a call for quality

Patient consent for publication Not required.Provenance and peer review Not commissioned; externally peer reviewed.This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.

Justin S Brandt http:// orcid. org/ 0000-0002-3194-1087