key: cord-0856579-i9dlniac authors: Mondal, Pritish; Mazur, Lauren; Su, Lilly; Gope, Suparna; Dell, Esther title: The Upsurge of Impact Factors in Pediatric Journals Post COVID-19 Outbreak: A Cross-Sectional Study date: 2022-03-29 journal: Front Res Metr Anal DOI: 10.3389/frma.2022.862537 sha: 9dd6819ae46c066af282009658fcc2349dbeafa4 doc_id: 856579 cord_uid: i9dlniac BACKGROUND: Impact factor (IF) is a quantitative tool designed to evaluate scientific journals' excellence. There was an unprecedented upsurge in biomedical journals' IF in 2020, perhaps contributed by the increased number of publications since the COVID-19 outbreak. We conducted a cross-sectional study (2018–2020) to analyze recent trends in standard bibliometrics (IF, Eigenfactor, SNIP) of pediatric journals. We also estimated reference and publication counts of biomedical journals since publication volume determines the number of citations offered and IF. METHODS: Various bibliometrics of pediatric journals and reference/publication volumes of biomedical journals were compared between 2020 vs. 2019 and 2019 vs. 2018. We also compared open access (OA) and subscription journals' trends. Finally, we estimated IF changes in the journals of a different specialty, pulmonology. RESULTS: The study included 164 pediatric and 4,918 biomedical journals (OA = 1,473, subscription = 3,445). Pediatric journals' IFs had increased significantly in 2020 [median (IQR) = 2.35 (1.34)] vs. 2019 [1.82 (1.22)] (Wilcoxon: p-value < 0.001). IFs were unchanged between 2018 and 2019. Eigenfactor remained stable between 2018 and 2020, while SNIP increased progressively. Reference/publication volumes of biomedical journals escalated between 2018 and 2020, and OA journals experienced faster growth than subscription journals. IFs of pulmonary journals also increased considerably in 2020 vs. 2019. CONCLUSIONS: We report an upsurge in pediatric journals' IF, perhaps contributed by a sudden increase in publication numbers in 2020. Therefore, considering this limitation, IF should be cautiously used as the benchmark of excellence. Unlike IF, Eigenfactor remained stable between 2018 and 2020. Similar changes in IF were also observed among the journals of another specialty, pulmonology. JCR (journal citation report) impact factor (IF) is considered a standard quantitative tool designed to evaluate and compare the quality of scientific journals (Clarivate, 2021a) . IF is estimated by the ratio of total citations received by a journal in a given year to the number of citable articles published in two previous years (Calculator Academy Team, 2021) . Unlike IF, other bibliometrics such as the Eigenfactor score weighs citations depending on citing journal's ranking (EIGENFACTOR.org), while source normalized impact per paper (SNIP) score is adjusted to the variations among the different scientific fields (Elsevier, 2012) . Moreover, Eigenfactor and SNIP incorporate data over the last five and three years, respectively. The value and misuse of IF has been under scrutiny for several years (Bornmann et al., 2012) . Readers often select an article based on the number of citations it has received and the prestige of the publishing journal (Lemke et al., 2021) . Brown and Gutman compared the bibliometrics of occupational journals and concluded that metrics such as Eigenfactor, SNIP, and Scimago journal ranking should be used in addition to IF to evaluate journals' quality (Brown and Gutman, 2018) . However, most of the research on bibliometrics was conducted prior to the COVID-19 outbreak (Kianifar et al., 2014; Yuen, 2018) . The latest JCR was published in June 2021, based on the 2020 data, and defined an unprecedented upsurge in pediatric and other biomedical journals' IF. Several publishers have been avidly marketing this report (EDP Sciences, 2021; ESA, 2021; Weingarden, 2021) . Since the number of citations determines the IF, a sharp increase in publication volume during the COVID-19 outbreak possibly led to an upsurge in the citations offered to the biomedical journals (Aviv-Reuven and Rosenfeld, 2021). Between 2012 and 2019, the number of open access (OA) journal publications had increased considerably from one to four million articles, influencing citation volumes (Morrison, 2021) . However, comparative analysis between OA and subscription biomedical journals' recent trends has not been reported. We conducted a cross-sectional study to estimate IF and other bibliometrics (IF ranking, Eigenfactor, and SNIP) of pediatric journals between 2018 and 2020 and compared the rate of changes in the bibliometrics between 2020 and 2019. The secondary objectives of our study were to compare the growth of biomedical journals in 2020 with the previous years and estimate the difference between OA and subscription journals' trends. Finally, we analyzed IF changes among the journals of a different specialty, pulmonology, to understand whether IF upsurge was specific to the field of pediatrics or a general trend. We accessed the JCR (Clarivate, 2021a) through Penn State Library and collected data (the year 2018-2020) on IF, IF ranking, Eigenfactor scores (Web of Science Group, 2021), as well as SNIP of the pediatric journals (CWTS Journal Indicators, 2021) ( Table 1) . It was not feasible to estimate individual citation counts received by each of the 50,000+ articles published between 2018 and 2019 in 164 pediatric journals. However, an overall increase in the number of published references across the medical fields in 2020 would perhaps reflect an increased volume of citations offered to the pediatric journals. Hence, we collected data on total reference (citation) counts from all biomedical Abbreviations: IF, impact factor; OA, open access, JCR, journal citation report; SNIP, source normalized impact per paper; IQR, Interquartile range. journals since citing articles could be from any specialty. We accessed the Scimago journal rank (SJR) database (SJR, 2021b) and assembled the reference and publication volume estimates (2018-2020), of all 'Web of Science' indexed biomedical journals (Clarivate, 2021b; SJR, 2021c Table 2) . The pediatric journals were divided into four groups (from first to fourth quartiles), based on the 2020 IF score. For example, out of 164 pediatric journals, 41 journals with the highest IFs were included in the cohort of first quartile journals. We further compared the changes in IF ( -IF) between 2019 and 2020 among the four groups. c) Biomedical journals: Reference and publication counts of biomedical journals were compared between 2020 vs. 2019 and 2019 vs. 2018. We also compared the rate of acceleration ( ) between 2020 and 2019. Biomedical journals were further divided into primarily OA or subscription journals, following SJR website classification (SJR, 2021a). The growth ( ) among the OA and subscription journals were compared between 2020 and 2019, respectively ( Table 3) . d) Generalizability of the results: we collected data on pulmonary journals' IF from JCR 2021 to comprehend whether the recent changes in IFs are a general trend vs. applicable to the field of pediatrics only. Pediatric pulmonary journals were excluded to avoid an overlap between pediatric and pulmonary journals. IFs of pulmonary journals were compared between 2020 vs. 2019 and 2019 vs. 2018. The rate of acceleration in pulmonary journals' IFs ( -IF) in 2020 was also compared with 2019. e) Choice of statistical tests: Since the data were not normally distributed, Wilcoxon signed-rank tests were used to compare between individual years' bibliometrics (paired difference between two measurements of a group) and Mann-Whitney test to compare between OA vs. subscription journals (difference in an outcome variable between two independent groups). We compared the changes in IF ( -IF) between four quartiles of pediatric journals with the Kruskal-Wallis test (difference in an outcome variable between three or more independent groups). We used IBM SPSS 27 for data analyses. (b) The rate of changes in bibliometrics: -IF and -SNIP were significantly higher in 2020 compared to 2019 (Wilcoxon test: p-value < 0.001), reflecting a higher rate of acceleration in 2020; however, -Eigenfactor remained stable ( Table 2) . The Kruskal-Wallis test demonstrated that -IF were not statistically different among the four quartiles of pediatric journals in 2020 (Kruskal-Wallis H: 1.96, degree of freedom = 3, p = 0.58), or in 2019 (Kruskal-Wallis H: 1.27, degree of freedom = 3, p = 0.73). (a) Average reference and publication counts were progressively and significantly escalated between 2018 to 2020 (Tables 1, 2) . -references and -publications in 2020 were significantly higher compared to 2019, reflecting a higher rate of acceleration in 2020 than in 2019. (b) OA vs. subscription journals: OA journals (reference and publication count) experienced higher growth than subscription journals with every passing year ( Table 3 ). In 2020, the rate of acceleration for reference counts ( -references) was also higher in OA than in subscription journals (Mann-Whitney: p-value = 0.013) however, -publications were not statistically different (Mann-Whitney: p-value = 0.38) during the same period. IF increased considerably in 2020 compared to 2019 in pulmonary journals. -IF was also significantly higher in 2020 than in 2019 (Tables 1, 2 ). This cross-sectional study demonstrated a sudden uptrend in IFs of pediatric journals in 2020 compared to 2019. In contrast, other bibliometrics including IF ranking and Eigenfactor scores, were relatively stable within that time interval. IF and Eigenfactor also did not change meaningfully between 2018 and 2019. However, SNIP scores of pediatric journals progressively increased from 2018 to 2020. We also determined that the uptrend in IF was not limited to the top-ranked pediatric journals; instead, the phenomenon was universally observed irrespective of journals' ranking. A similar trend was also observed in other biomedical fields beyond pediatrics. The scientific community entrusts IF as a reliable tool to estimate journals' significance, and the sudden uptrend might have conveyed biased messages. Merits of IF and Eigenfactor have been compared in the past. Eigenfactor is an efficient scientific metric since it is also adjusted for citing journals' quality. Journal rankings often change depending on whether IF or Eigenfactor were used (Rizkallah and Sin, 2010) , indicating that journals should not be evaluated based on a single metric. Additionally, our study demonstrated an advantage of the Eigenfactor because of its ability to resist year-to-year variation due to increased publication volume. SNIP, designed by Scopus, was primarily intended to adjust for the variation in citation patterns among different scientific fields (Leydesdorff and Opthof, 2010) . However, we found that SNIP also became vulnerable like IF to the sudden upsurge in publication numbers. A considerably higher volume of biomedical articles and references were published in 2020 compared to 2019, which perhaps prompted the upsurge of IFs across the various fields of biomedical sciences (Figure 1) . The rationale behind increased publication volumes in 2020 was beyond the scope of this study. However, it might be related to markedly escalated research productivity since the COVID-19 outbreak (Nigrovic and Napper, 2021) . Both OA and subscription journals experienced a considerable increase in publication volume in 2020. However, the OA journals perhaps were more lenient with acceptance due to lack of space constraints and experienced a higher growth rate than the subscription journals. Our study has several limitations. This project focused solely on citation counts and not how the journals could influence the IF. For example, some journals no longer publish case reports as those types of articles would count toward the citable papers but are infrequently referenced (Rison et al., 2017) . Journals may also prefer to accept reviews (more often cited) or publicize aggressively in social media. Second, we considered all biomedical journals as the source of citations for pediatric journals, which could be an overgeneralization. Moreover, traditional subscription journals often publish few OA articles, as several publishers have been leaning toward a hybrid model in recent years (Pinfield et al., 2017) . Thus, OA vs. subscription journal classification could be an oversimplification. Nonetheless, this study has few strengths. To the best of our knowledge, no previous study reported the sudden uptrend in IF since the outbreak of COVID-19. Our study showed that increased IF affected not only all tires of pediatric journals but other specialties too. Henceforth, it is likely to be a general phenomenon. Finally, we analyzed other bibliometrics in addition to IF and found that Eigenfactor scores were more stable than IF. CONCLUSION A significant upsurge in pediatric journals' IF in 2020 was perhaps related to the sudden increase in overall publication volumes since the COVID-19 outbreak. Due to its vulnerability, IF should be cautiously reported as a benchmark of excellence. Unlike IF, Eigenfactor scores of pediatric journals remained unchanged in 2020, citing an additional advantage of Eigenfactor compared to other bibliometrics. A similar trend in IF was observed among all quartiles of pediatric journals, irrespective of quality and prestige. A similar uptrend in IF was also observed in journals of another specialty such as pulmonology. Both OA and subscription journals experienced a considerable increase in publication volume in 2020. However, the OA journals experienced even higher growth than the subscription journals. The data will be made available upon request. PM conceptualized, designed the study, supervised data collection, analyses, drafted the initial manuscript, reviewed, and revised the manuscript. LM, LS, and SG conceptualized, designed the data collection instruments, collected data, conducted initial analyses, reviewed, and revised the manuscript. ED conceptualized the study, supervised various aspects of accessing the data, and critically reviewed the manuscript for important intellectual content. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. Publication patterns' changes due to the COVID-19 pandemic: a longitudinal and short-term scientometric analysis Diversity, value and limitations of the journal impact factor and alternative metrics Impact factor, eigenfactor, article influence, scopus SNIP, and SCImage journal rank of occupational therapy journals Impact Factor Calculator Web of Science 2020 Journal Impact Factors From Clarivate Reveal an Increase for All EDP Sciences Journals Indexed in Web of Science Available online at Elsevier Announces Enhanced Journal Metrics SNIP and SJR Now Available in Scopus R . Available online at ESA Journals Achieve Historically High Impact Factors Comparison between impact factor, eigenfactor metrics, and SCimago journal rank indicator of pediatric neurology journals Conjoint analysis of researchers' hidden preferences for bibliometrics, altmetrics, and usage metrics Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations Dramatic Growth of Open Access Marked escalation in journal submissions during COVID-19 pandemic A "Gold-centric" implementation of open access: hybrid journals, the "Total cost of publication, " and policy development in the UK and beyond How to choose the best journal for your case report Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores Open Access Biomedical Journals SJR. Available online at Scimago Journal and Country Rank Browse, Search, and Explore Journals Indexed in the Web of Science 2020 Journal Citation Report Includes Large Impact Factor Increases for AMA Journals Comparison of impact factor, eigenfactor metrics, and Scimago journal rank indicator and h-index for neurosurgical and spinal surgical journals The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frma. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.