key: cord-0751497-a18a9spe authors: Galickas, Donatas; Flaherty, Gerard T title: Is there an association between article citations and altmetrics in travel medicine research? date: 2021-08-20 journal: J Travel Med DOI: 10.1093/jtm/taab115 sha: e23d8cd3bfa453dfb006e14697fb47135bbc3406 doc_id: 751497 cord_uid: a18a9spe nan Traditional measures of the impact of academic scholarship, such as citation counts, have been supplemented in recent years by additional indices that reflect wider societal engagement and the emerging influence of social media platforms in disseminating research findings to the public. Thus, article page views and alternative-level metrics (altmetrics) are now being reported by academic journals to quantify research output. The Altmetric Attention Score (AAS) uses public application programming interfaces (APIs) to gather data using weighted algorithms from diverse sources, including news outlets, Twitter, Facebook, blogs and research websites. It demonstrates both the article impact and its composition by providing an analysis of the sources of online attention and their geographic origin. Previous studies have suggested a modest correlation between traditional bibliometric indices like citations and newer measures such as altmetrics. 1 We aimed to investigate whether there was any relationship between these impact measures in the field of travel medicine research. We examined all articles published in Journal of Travel Medicine in 2019 and 2020. All data were extracted between April and May 2021. All categories of journal article were included. Articles were categorised according to the body of knowledge domains of the International Society of Travel Medicine (ISTM), 2017 revision (https://www.istm.org/bodyo fknowledge2). Citation data were obtained from Web of Science (Clarivate), and the AAS and total cumulative page views were recorded from the journal website (https://academic.ou p.com/jtm/issue). The geographic origins of the AAS were also recorded. Pairwise correlations were performed using Spearman's rank correlation coefficient (r-value). Statistical analysis was performed using the IBM Statistical Package for the Social Sciences (SPSS), version 26. Correlations were interpreted as small (r < ± 0.29), medium (r = ±0.30-0.49), strong (r = ±0.5-0.99) or perfect (r = near ±1) according to standard statistical practice. Of 363 articles published in 2019 and 2020, 60 were original research articles, 54 were research letters and 40 were review articles. The remaining articles were non-research letters to the editor (59), perspectives (58), clinical pearls (39), editorials (27) and rapid communications (26). The most frequently occurring primary subject matter of articles, based on the 2017 ISTM body of knowledge classification, was 'diseases contracted during travel' (40.8%, n = 148). There were 126 COVID-related articles accepted by the journal in 2020, comprising 52.7% of the total publications in that year. There was a medium correlation (95% confidence interval) between ranks of 0.40 (0.30-0.49) for citations and the AAS (P < 0.001) for all articles published in 2019 and 2020. There was no statistically significant difference between medium citation-AAS correlations for articles classified as COVIDrelated (0.46, 95% confidence interval 0.30-0.60, n = 126) and non-COVID-related (0.39, 95% confidence interval 0.27-0.52, n = 237). Figure 1 (panels A-D) shows the correlation between the AAS and citation counts for the 100 most-cited articles published in Journal of Travel Medicine in 2019 and 2020, for COVID-related and non-COVID-related articles, and between AAS and articles views for all articles published during this period. The median number of citations per article was highest for rapid communications (median = 3) than for other article categories. Rapid communications and perspectives were the most frequently viewed article type with a median number of views of 896 and 622, respectively. The most cited article, most viewed article, and the article with the highest AAS communicated information about the COVID-19 pandemic. The most cited (n = 560) article, a research letter based on the reproductive number of COVID-19, also received the most page views on the journal website (n = 162 311). The article with the highest AAS was an original research article that reported on the COVID-19 outbreak aboard the Diamond Princess cruise-ship. Japan contributed the highest proportion of the overall altmetric attention given to this article. Overall, the countries from which most online article attention originated were the USA (n = 50 articles) and the UK (n = 31 articles). We performed the first correlation analysis of conventional and non-traditional bibliometric indices in travel medicine. Previous bibliometric and citation analyses in this field have focused on the systematic mapping of content to an external body of knowledge 2 ,3 and on citation-level metrics. 4 The moderate degree of correlation between citation counts and Altmetric Attention Score for articles published in 2019-2020 in Journal of Travel Medicine, both COVID-related and non-COVID-related, suggests that these metrics may reflect independent elements of an article's impact. Giustini and colleagues (2020) found a higher but still modest correlation of 0.53 between article citations and AAS for four paediatrics journals. 1 The authors concluded that citations may not fully convey the impact of an article and that newer metrics should be evaluated when considering the overall impact of scholarly work. In a study of dissemination metrics in the field of medical education, the authors reported weak to moderate correlations between altmetrics and citation counts. 5 An analysis of the gynaecologic oncology literature determined that early indices of online attention, such as the AAS, might be useful in the prediction of future citation counts. 6 While citations accumulate over time, the AAS peaks early in an article's history and may be considered as a surrogate marker of an article's immediate impact. The relative alignment of articles with the ISTM body of knowledge domains is broadly consistent with that observed in a previous bibliometric analysis of all Journal of Travel Medicine articles published up to 2017. 2 The most well-represented domain remained 'diseases contracted during travel'. The profound influence of the COVID-19 pandemic on international travel and travel medicine activity was demonstrated in the disproportionate attention given to this topic in 2020. 7 An end-of-year editorial published in 2020 in this journal reflected on this trend. 8 The AAS is a composite measure of all online attention received by an article, with greater weighting applied to news reports, blogs and policy documents than to social media commentaries. 9 Given the global preoccupation with the COVID-19 pandemic throughout 2020, this phenomenon is hardly surprising. While the article with the highest share of online attention had not accrued the highest number of citations by the end of April 2021, it remains to be seen how many citations such articles will accumulate over time. Our research has several limitations. While the COVID-19 effect was very much in evidence in 2020, it would be worthwhile repeating our analysis to include publication years prior to the pandemic and observing the long-term citation performance of COVID-related articles published in 2020. Altmetrics are subject to bias and are vulnerable to gaming so we cannot exclude this possibility in interpreting the inflated AAS scores of some articles. However, citation counts are also prone to manipulation by authors and journal editors. We did not perform a correlation of citations with article views, as the latter only relate to articles accessed via the journal website and are influenced by the open access status of articles. Since all COVID-19 articles were made freely available without embargo, this may have contributed to their high AAS scores. We recommend that similar metric correlation analyses be extended to other travel medicine journals, including Travel Medicine and Infectious Disease, to gauge the true influence of non-traditional metrics in assessing the impact of travel medicine research. It would also be of interest to correlate impact metrics at the level of individual article categories to discern if there is a difference between scholarly and editorial-type publications in this respect. Correlation analyses of impact metrics represent snapshots of dynamic processes and they should be repeated to observe longitudinal trends over time. It would also be interesting to investigate whether open access journals differ from subscription only journals in respect of the online attention commanded by their articles. In summary, there was only a medium correlation between citation counts and altmetrics for travel medicine articles published over a two-year period, incorporating the first year of the COVID-19 pandemic. Altmetrics may provide additional insights into the societal impact of scholarly research output. Further research should elucidate the value of triangulating from multiple measures, including citations, AAS, and article views, in an effort to better quantify the overall influence of published work. We found a medium correlation between citation counts and altmetrics for travel medicine articles published over a two-year period, incorporating the first year of the COVID-19 pandemic. Altmetrics may provide additional insights into the impact of scholarly research. Further research should explore the value of triangulating from multiple impact measures. This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors. Association between citations, Altmetrics, and article views in pediatric research Bibliometric analysis and curriculum mapping of travel medicine research Travel medicine research in the new millennium: A bibliometric analysis of articles published in Travel Medicine and Infectious Disease Citation analysis of the most influential publications in travel medicine Comparing alternative and traditional dissemination metrics in medical education Examining the correlation between Altmetric Attention Score and citation count in the gynecologic oncology literature: Does it have an impact? COVID-19's impact on travel medicine surpasses that of all other emerging viral diseases End of year editorial: hot topics in travel medicine How is the Altmetric Attention Score The authors have no conflict of interests to declare.