key: cord-0292597-4sb9nvw7
authors: Maggio, Lauren A.; Costello, Joseph A.; Ninkov, Anton; Frank, Jason R.; Artino, Anthony R.
title: The Voices of Medical Education Science: Describing the Published Landscape
date: 2022-02-10
journal: bioRxiv
DOI: 10.1101/2022.02.10.479930
sha: 31b8af49e66f5ffdda679bc975621013f674bb10
doc_id: 292597
cord_uid: 4sb9nvw7

Introduction Medical education has been described as a dynamic and growing field, driven in part by its unique body of scholarship. The voices of authors who publish medical education literature have a powerful impact on the discourses of the community. While there have been numerous studies looking at aspects of this literature, there has been no comprehensive view of recent publications. Method The authors conducted a bibliometric analysis of all articles published in 24 medical education journals published between 2000-2020 to identify article characteristics, with an emphasis on author gender, geographic location, and institutional affiliation. This study replicates and greatly expands on two previous investigations by examining all articles published in these core medical education journals. Results The journals published 37,263 articles with the majority of articles published in 2020 (n=3,957, 10.7%) and the least in 2000 (n=711, 1.9%) representing a 456.5% increase. The articles were authored by 139,325 authors of which 62,708 were unique. Males were more prevalent across all authorship positions (n=62,828; 55.7%) than females (n=49,975; 44.3%). Authors listed 154 country affiliations with the United States (n=42,236, 40.4%), United Kingdom (n=12,967, 12.4%), and Canada (n=10,481, 10.0%) most represented. Ninety-three countries (60.4%) were low- or middle-income countries accounting for 9,684 (9.3%) author positions. Few articles were written by multinational teams (n=3,765; 16.2%). Authors listed affiliations with 4,372 unique institutions. Across all author positions, 48,189 authors (46.1%) were affiliated with a top 200 institution, as ranked by the Times Higher Education ranking. Discussion There is a relative imbalance of author voices in medical education. If the field values a diversity of perspectives, there is considerable opportunity for improvement.

Academic publishing is the primary means of disseminating scientific knowledge in medical education, and the researchers who author those publications drive scientific advancement. However, recent research across scientific disciplines suggests that author diversity, as measured by gender and race, 1,2 as well as international representation, as measured by country of origin and institutional affiliation, 3, 4 are lacking. This lack of diversity and representation is important because, as many scholars have argued, more diverse teams do better science. 5, 6 For example, scientists with different perspectives often ask different questions, and different questions can lead to novel insights that propel science forward. What is more, modern science is a team sport, as evidenced by the expanding size of research and authorship teams over the past 20 years. 7 Therefore, today's research enterprise is largely a group problem-solving activity where diversity of perspectives is a key aspect of creativity, innovation, and overall excellence. 5, 6 In 2021, Maggio and colleagues examined author diversity and representation through two bibliometric studies describing 963 knowledge syntheses appearing in 14 medical education journals between 1999-2019. 4, 7 In these two studies, we attempted to describe the medical education community writ large by providing a bibliometric snapshot of the field's knowledge syntheses, with a focus on publication characteristics and author gender, country affiliation, and institution. The intent of this work was to provide insights into the landscape of knowledge syntheses, with an emphasis on understanding the authors whose voices are shaping the medical education discourse and evidence. We found that across all authorship positions there is a similar ratio of men and women; however, when considering country and institutional affiliation, there is an overrepresentation of authors from North America and Western Europe working in highly-ranked institutions. 4 While eye opening, these studies are limited to a single publication type: knowledge syntheses. And thus, we do not know if these findings mirror the broader landscape of medical education publishing, which has implications for understanding which researchers' voices are most present in our scientific discourse.

Our previous work built on previous studies that have examined slices of medical education literature using bibliometric methods. [8] [9] [10] [11] [12] For example, in 2013 Lee and colleagues investigated the evolution of medical education publications over a 50-year period that were identified as "medical education" based on search terms. 8 Three years later, Azer produced a ranking of medical education articles from 13 journals based on citations, 9 and more recently Madden et. al. described the gender of authors and editors in four medical education journals. 12 These studies each provide a valuable glimpse into the field's evidence base; however, we are unaware of any research that provides a current, comprehensive overview of these topics in medical education.

The purpose of this study was to replicate and expand on our previous bibliometric work 4,7 by describing the authorship landscape of the last 20 years of medical education literature, using the largest database of scholarship assembled to date. In the present study, we examined all articles published between 2000 and 2020 in the Medical Education Journals-24 (MEJ-24) to identify article characteristics, with an emphasis on author gender, geographic location, and institutional affiliation.

We conducted a bibliometric analysis of all articles published in 24 medical education journals published between 2000-2020, thereby replicating, combining, and expanding the approach used in our previous work. 4, 7 Our sample consisted of articles published between 2000-2020 in the 24 journals featured on the MEJ-24 (See Appendix for journal listing). 13 The MEJ-24 has been proposed as "a seed set of journals" that constitutes the field of medical education and was developed by our team using a bibliometric co-citation approach. 13 For these 24 journals, on August, 27, 2021, we retrieved metadata for 22 of these journals from the database Web of Science (WoS). On the same day, for the remaining two journals not indexed in WoS (Journal of Graduate Medical Education (JGME) and The Canadian Medical Education Journal (CMEJ), we downloaded article metadata from the Crossref REST API. For all 24 journals, we downloaded the following: journal name, author names and affiliations, publication date, open access status, author keywords, and funding details. We selected WoS based on its well-defined metadata and long history as a valuable tool for bibliometric analyses across multiple disciplines. 14 For the 22 journals indexed in WoS, we also downloaded the number of times an article was cited, its cited references, and the references of any articles that had cited it. These citation data were unavailable for JGME and CMEJ. All metadata was organized in an Excel spreadsheet. 15 From each article, we extracted the names of all authors. However, in order to accurately analyze author-level data it is essential to adequately disambiguate author names that represent the same (or different) people. 16 Thus, we created a thesaurus of author names to reconcile name permutations (e.g., Artino, AR, Jr and Artino, Anthony R, Jr were both reconciled to Artino, Anthony). Complete details of author name disambiguation and the full author thesaurus are freely accessible on Zenodo. 17 Once the author data was cleaned, to make a prediction of the authors' gender using their first name, we utilized the tool Genderize.io. 18 We recognize that our effort to predict gender is an oversimplification of a complex social construct, especially because an individual's gender is best described by that individual, and our method did not allow us to capture authors who identify as non-binary. However, there currently does not exist a resource that provides gender information for individual authors. Therefore, we relied on the Genderize.io tool, in keeping with work done previously by authors using bibliometric methods. 4, 11, 19 Genderize.io bases its predictions on a database of over 100 million names, and for each name provides the "percent confidence" that the gender prediction is correct. We accepted the tool's gender prediction if and when it was over 70% confident of the gender prediction.

For all articles, we identified country and institutional affiliations. To identify these affiliations, we included any article with at least one author-institution affiliation. As institutional affiliations are often variably reported in paper, we created an additional thesaurus of affiliations, which enabled us to accommodate these variations (e.g., Univ Nebraska and Nebraska Univ reconciled to University of Nebraska) and to associate institutions (e.g., university hospitals) with a parent institution. The thesaurus and full details of its creation are available on Zenodo. 17 To provide context for institutions, we identified each institution's ranking on The Times Higher Education World Ranking 2022, 20 which includes over 1,600 institutions worldwide. Additionally, we identified the World Bank Classification, 21 for each country which is based on their gross domestic product per capita and assigns countries to four regions (low income, lower-middle income, upper middle income, and high income).

We used Excel 15 to calculate descriptive statistics.

Across the MEJ-24, 37,263 articles were published. Of the 37,263 articles, 47 listed no author and 212 were anonymously written. We excluded these articles (0.7%) from our analysis given our emphasis on authorship characteristics and to facilitate consistency across analyses. Of the remaining 37,004 articles, the largest number of articles were published in 2020 (n=3,957, 10.7%) and least number in 2000 (n=711, 1.9%). This difference in articles published represents a 456.5% increase over the time period examined. Academic Medicine (n=7760, 21%), Medical Education (n=5499, 14.9%), and Medical Teacher (n=5029, 13.6%) published the most articles, accounting for 49.4% of all publications over this 20-year period in the 24 journals analyzed.

Articles published in the 22 journals with citation data (n=34,652) were cited 548,358 times. On average, articles were cited 15.8 times (SD=45.54 Range:0-3,310). There were 6,464 (18.7%) articles with no citations, of which 1,333 were published in 2020. "Making sense of Cronbach's alpha" published by the International Journal of Medical Education in 2011 was the most cited article in the dataset (n=3,310). 22 Table 1 lists the top 10 authors by article citations (see Supplemental Table 1 for a list of the top 20 authors).

Nearly half of the articles published in all 24 journals (n=16,817; 55.5%) were publicly accessible. For those that were publicly accessible and that also had citation data (n=34,652), the average citation rate was 15.0, which was just below the overall average citation rate. Table 2 ). In addition, males wrote 67.1% of single-authored articles. Multi-authored articles with female first authors (n=12,739) were cited on average 16.3 times (37.7=stdev) in comparison to 17.7 average (54.4=stdev) for males (n=13,632). Single author citation averages for males and females were 10.4 (34.2=stdev) and 8.6 (40.4=stdev), respectively. Figure 1 provides a comparison of author gender over the 20-year time period for first and last authors on multi author articles. Table 3 ).

Authors listed affiliations with 93 LMIC accounting for 9,684 (9.3%) author positions. BMC Medical Education included the most representation of authors listing LMIC affiliations (n=2,579, 28.1%). For first authors, only 2,552 authors (9.0%) were affiliated with LMIC. (See Table 3 for more details on geographic affiliations) 

This study, which leverages a unique and sizable data set, provides a snapshot of the last two decades of medical education research published in a core set of journals, with a focus on author characteristics. Similar to our previous bibliometric work on knowledge syntheses, 4, 7 we found that the number of medical education articles has grown substantially over the past 20 years. What is more, similar to our previous work, there appears to be a trend towards greater gender parity overall; although male-dominated imbalances in specific authorship positions still persist. Additionally, we observed that the majority of authors list affiliations in Western, English-speaking countries, and these authors tend to be affiliated with highly ranked institutions. Taken together, our findings suggest a relative imbalance of author voices in our medical education science. If diversity of perspectives is important for creativity, innovation, and overall excellence in science, 5, 6 this data indicates considerable room for improvement.

Over the last two decades there was a 456.3% increase in published medical education articles. Although this growth is much less steep than that which we found in published knowledge syntheses (2,620%), 7 this increase remains impressive and suggests that our field is growing at a relatively fast rate. This increase is likely due to a variety of factors, including the advent of several new journals (e.g., Focus on Health Professional Education, which was founded in 2014), a rise in the number of graduate programs in medical and health professions education, 25, 26 and the ever-present pressure to publish felt across many fields. 27 Looking ahead, it is unclear whether or not this growth trajectory will continue. For example, the journal BMJ Simulation and Technology Enhanced Learning ceased publishing articles at the end of 2021. 28 Additionally, there has been a rise in alternative forms of knowledge dissemination in medical and health professions education (and across science); these alternatives include, for example, podcasts, blogs, and wikis, all of which continue to grow in popularity. 29 We identified that, overall, there were more males across all authorship positions. However, we also found that the number of female authors is on the rise. Both of these findings generally align with the trends observed in science more broadly, 30 and specifically in medical education. 4, 12 While these results are encouraging, future work should continue to monitor this trend, especially in relation to the impact of COVID-19. Research conducted to date suggests a downward trend in female authors' productivity, 31-34 especially for those with young dependents and those in the early stages of their career. 35 For example, an analysis of over 40,000 articles published before and during COVID-19 indicated a widening gender gap of 14% percentage points for all authorship positions; that number increased to 24.6% when considering the first author position. 36 Beyond the basics of author order, our findings suggest that articles solo authored by females, or published with a female as first author of a team, were cited less often than those solo authored by males or published with a male first author, a finding that parallels results from biomedicine. 37 This finding has implications for the success of women in the field, since citations are often equated with impact and are used to inform high-stakes promotion and funding decisions. 38 Additionally, fewer female citations in the literature overall indicates that female voices may be less integrated into the field's evidence-base, which could have important, negative consequences on a field that is increasingly populated by female educators, clinicians, and scientists. 39 Although author affiliations represented over 80% of the world's countries, this representation was dominated by authors from English-speaking, Western countries. This finding aligns with previous research in medical education. 4, 11 What is more, only a minority of articles (16.2%) represented multi-national collaborations. This result is somewhat concerning, as articles written by authors from Western countries may fail to recognize non-Western norms and values, which can hinder the translation or adoption of article's content outside of the local context. While this finding aligns with previous research in medical education, it runs counter to science more broadly, which features a great deal of cross-border collaboration, 40, 41 creating what is considered an "international system of collaboration". 42 Adding to the benefits of integrating a variety of international voices, multi-national team research has been positively associated with greater research impact and less research waste. 43 We propose that medical educators consider approaches to increase international collaborations, which may include future research to understand facilitators and barriers to such collaborations. It might also include the creation of initiatives, such as international funding mechanisms, that encourage cross-border collaboration. It could also include efforts to harmonize research logistics related to data sharing and human subjects protections. For example, in Canada, a recently published consensus statement includes recommendations for researchers to engage in best practices for inter-institutional sharing of medical education data. 44 Similar to what we found in medical education knowledge syntheses, 4 we observed in the current study that just under half of authors were affiliated with highly ranked academic institutions, suggesting that these institutions may have a disproportionate level of influence in the field. However, we were encouraged to find contributions from a variety of entities, such as professional associations, federal agencies, and nonprofits. These types of collaborations might help to facilitate knowledge transfer and exchange, and also have the potential to raise the societal value and relevance of research. 45 As such, these types of collaborations may be worthy of continued growth. That said, in order for authors to reach beyond their national borders and collaborate outside of academia will require support to overcome institutional barriers. 46 For example, early-career medical education scientists may be less familiar with how to establish and foster collaborations outside of academia. This suggests the need for education on how to negotiate technology transfer and non-disclosure agreements. Currently, such education programs do exist to train academic researchers to partner with individuals outside academia; 47, 48 however, to our knowledge, no such programs exist presently in medical education.

Our findings should be considered in light of several limitations. First, we used the tool Genderize.io to predict author gender based on first names. However, we realize that gender is a complex social construct that is best determined by the individual. Nonetheless, we are unaware of currently available alternatives to establishing author gender in datasets of this scale. Future initiatives might consider encouraging researchers to self-identify their gender in resources such as ORCID. Second, we created our sample based on the MEJ-24, which proposes a seed set of 24 core medical education journals. 13 While this study expands upon previous efforts by our author team and others to describe a set of core medical education journals, 8, 9 we recognize that our MEJ-24 sample does not include relevant medical education articles published in clinical and other specialty-focused journals. Third, 22 of the included journals publish articles solely in English, with the exception of the GMS Journal for Medical Education and the Canadian Medical Education Journal, which simultaneously publishes articles in German and French respectively. However, there is some representation from national professional associations such as the African Journal of Health Professions and Focus on Health Professions Education, which is based in Australia.

Another important limitation is the fact that we were reliant on metadata from CrossRef and WoS, which, by definition, represent a subset of the world's literature. Additionally, while these databases are often used in bibliometric research, these tools are not infallible and may include data errors that we were unable to detect. In all cases, however, we worked to "clean" our data prior to analysis. The cleaning process required that we create standardized rules. For example, some journals allow authors to list multiple institutional affiliations on the same article. In such instances, which were a minority of cases, we chose to use only the authors first listed institutional affiliation. Notwithstanding this limitation, and in the interest of transparency and the facilitation of future replications, we have provided details for each rule and how it was applied in our dataset. 17

Researchers who author medical education articles are the primary drivers of knowledge dissemination and scientific advancement in the field. In this study, we describe the voices of those who have contributed to medical education by examining two decades of the medical education literature, with an emphasis on article and author characteristics. Our findings indicate that medical education scholarship is rapidly growing and evolving, with more female voices being heard, but with a Western country viewpoint that still predominates. Going forward, our community should seek to expand its diversity of voices through collaborations and explicit campaigns that solicit new scholarly perspectives. 

The gender gap in science: How long until women are equally represented?

Longitudinal Analysis of Gender Differences in Academic Productivity Among Medical Faculty Across 24 Medical Schools in the United States

Explicit Bias Toward High-Income-Country Research: A Randomized, Blinded

Knowledge syntheses in medical education: Meta-research examining author gender, geographic location, and institutional affiliation

Diversity Makes Better Science. Association for Psychological Science Observer

How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies

Knowledge syntheses in medical education: A bibliometric analysis

50 years of publication in the field of medical education

The top-cited articles in medical education: a bibliometric analysis

The themes, institutions, and people of medical education research 1988-2010: content analysis of abstracts from six journals

Gender Disparities in Invited Commentary Authorship in 2459 Medical Journals

Gender in authorship and editorship in medical education journals: A bibliometric review

Delineating the field of medical education: Bibliometric research approach(es)

Web of Science use in published research and review papers 1997-2017: A selective, dynamic, cross-domain, content-based analysis

Distortive effects of initial-based name disambiguation on measurements of large-scale coauthorship networks

Gender Differences in Publication Authorship During COVID-19: A Bibliometric Analysis of High-Impact Cardiology Journals

The Times Higher Education. World University Rankings 2022

The World Bank. GDP per capita (Current US$)

Making sense of Cronbach's alpha

Responding to experienced and anticipated discrimination (READ): anti -stigma training for medical students towards patients with mental illness -study protocol for an international multisite non-randomised controlled study

Knowledge of dental academics about the COVID-19 pandemic: a multi-country online survey

AM last page: master's degree in health professions education programs

Preparing health professions education leaders worldwide: A description of masters-level programs

Intended and unintended consequences of a publish-or-perish culture: A worldwide survey

Social media in knowledge translation and education for physicians and trainees: a scoping review

Historical comparison of gender inequality in scientific careers across countries and disciplines

Challenges for the female academic during the COVID-19 pandemic

Gender Differences in First and Corresponding Authorship in Public Health Research Submissions During the COVID-19 Pandemic

Female authorship of covid-19 research in manuscripts submitted to 11 biomedical journals: cross sectional study

Gender gap in journal submissions and peer review during the first wave of the COVID-19 pandemic. A study on 2329 Elsevier journals

Unequal effects of the COVID-19 pandemic on scientists

Longitudinal analyses of gender differences in first authorship publications related to COVID-19

Gender Disparity in Citations in High-Impact Journal Articles

Using publication metrics to highlight academic productivity and research impact

The State of Women in Academic Medicine: Exploring Pathways to Equity

International collaborations growing fast

Globalization of science and international scientific collaboration: A network perspective

Growth patterns of the network of international collaboration in science

Multinational teams and diseconomies of scale in collaborative research

Data-Sharing in Canadian Medical Education Research: Recommendations and Principles for Governance. Hamilton, Ontario. www.dataconnection.ca

Informal collaborations between social sciences and humanities researchers and non-academic partners

Crossing the Rubicon: exploring the factors that shape academics' perceptions of the barriers to working with industry

Academic work-integrated learning (WIL): Reengaging teaching focused academics with industry

A Community-Academic Partnered Grant Writing Series to Build Infrastructure for Partnered Research