key: cord-0888170-667kmcgn
authors: Grunhut, Joel; Wyatt, Adam TM; Marques, Oge
title: Educating Future Physicians in Artificial Intelligence (AI): An Integrative Review and Proposed Changes
date: 2021-09-06
journal: J Med Educ Curric Dev
DOI: 10.1177/23821205211036836
sha: 53b710539309c1e4585b545d5227b78a972d876b
doc_id: 888170
cord_uid: 667kmcgn

BACKGROUND: As medicine and the delivery of healthcare enters the age of Artificial Intelligence (AI), the need for competent human–machine interaction to aid clinical decisions will rise. Medical students need to be sufficiently proficient in AI, its advantages to improve healthcare's expenses, quality, and access. Similarly, students must be educated about the shortfalls of AI such as bias, transparency, and liability. Overlooking a technology that will be transformative for the foreseeable future would place medical students at a disadvantage. However, there has been little interest in researching a proper method to implement AI in the medical education curriculum. This study aims to review the current literature that covers the attitudes of medical students towards AI, implementation of AI in the medical curriculum, and describe the need for more research in this area. METHODS: An integrative review was performed to combine data from various research designs and literature. Pubmed, Medline (Ovid), GoogleScholar, and Web of Science articles between 2010 and 2020 were all searched with particular inclusion and exclusion criteria. Full text of the selected articles was analyzed using the Extension of Technology Acceptance Model and the Diffusions of Innovations theory. Data were successively pooled together, recorded, and analyzed quantitatively using a modified Hawkings evaluation form. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses was utilized to help improve reporting. RESULTS: A total of 39 articles meeting inclusion criteria were identified. Primary assessments of medical students attitudes were identified (n = 5). Plans to implement AI in the curriculum for the purpose of teaching students about AI (n = 6) and articles reporting actual implemented changes (n = 2) were assessed. Finally, 26 articles described the need for more research on this topic or calling for the need of change in medical curriculum to anticipate AI in healthcare. CONCLUSIONS: There are few plans or implementations reported on how to incorporate AI in the medical curriculum. Medical schools must work together to create a longitudinal study and initiative on how to successfully equip medical students with knowledge in AI.

The United States is projected to see a growth in healthcare expenditure at a rate of 5.4% to reach $10 trillion by 2022 thereby constituting 9% of gross domestic product. [1] [2] [3] The expansion of healthcare has permeated many aspects of medical practice. For physicians, there are many concerning trends in healthcare including the growth of information and information technology. Ten years ago it was predicted that the doubling time of medical knowledge in 1950 was 50 years; in 1980, 7 years; and in 2010, 3.5 years, and in 2020 it was projected to be 0.2 years-just 73 days. 4 At the given rate, entering medical students will experience a doubling of medical knowledge 5 times in their first year alone. Already strained under a regular medical school curriculum, accumulating dynamically changing information without proper aid will present an unprecedented challenge in medical education and medical careers. 5, 6 Rise of AI Artificial Intelligence (AI) is defined as the ability of a machine to imitate human intelligence. From its inception, AI has permeated everyday life, business and in particular, AI is increasingly applied to healthcare. 7 The technology has the potential to impact many aspects of both institutionalized and private healthcare, from diagnosis to treatment options and from benchwork research to pharmaceutical design. AI is being tested in healthcare to assist and replace repetitive tasks such as image recognition in diagnosis and augmentation of images in radiology. AI is expected to augment healthcare workflow through automated triage, improve the productivity of individual physicians, reduce human errors, discover better patterns of patient care, defray medical costs, perform minimally invasive surgery, and reduce mortality rates. [8] [9] [10] [11] [12] [13] [14] Ultimately, physicians roles are changing due to the superior efficacy of AI in dealing with many established physician practices, 15 chemical synthesis planning, and drug discovery.

In the past, as with the introduction of robotics, physicians roles were under similar pressure from change. 16 In response, physicians adapted their roles to best match their tools to further advance patient care. 17 However, with the rapid advancement of AI, physicians face an unprecedented stress to learn entire new skill sets and to manage an exponential growth of medical knowledge. 18 Fortunately, solving the challenge of learning about AI can provide a useful solution to the issue of expanding medical knowledge. AI has the ability to equip physicians with an alternative to efficiently deal with and translate medical knowledge into practice. [19] [20] [21] Transforming Education

Healthcare is advancing and along with it is the necessity to improve medical education. As the rate of discovery of medical knowledge grows, physicians will have trouble managing, organizing, and retaining information. Technologies such as AI are needed to enable healthcare professionals to effectively use this knowledge to practice medicine. 22 As medicine and the delivery of healthcare enters the age of AI, the need for competent human-machine interaction for the use of data to aid clinical decision making will rise. 23 Medical students need to be sufficiently proficient in understanding the fundamental concepts of how AI functions, its advantages to improve expenses, increase quality, and ease access to healthcare. 24, 25 Similarly, students must be educated about the shortfalls of AI such as transparency and liability. [26] [27] [28] [29] Finally, overlooking a technology that will be transformative for the foreseeable future would place medical students at a disadvantage.

We aim to provide an extensive literature review of previous studies describing the role of teaching AI in medical education as a curriculum subject. This includes teaching AI as a concept, tools, principles, methodologies, implications, risks, and applications. Medical students should be able to use, interpret, and explain the results of an AI output. 30 Although teaching medical students with AI tools is meaningful and important, that is beyond the scope of this paper. This paper also aims to further transition from a call to teach medical students about AI to an advanced path of implementation. In this paper, we summarize recent relevant papers in the field of medical education that have addressed the main topics of interest in this paper, particularly the curricular, cultural, and attitudinal changes required to successfully incorporate the teaching of AI contents in undergraduate medical education (UME).

An integrative review approach was chosen to identify the past and current literature from different types of research. Integrative review is a unique approach to combining different approaches to research including experimental and nonexperimental. 31 The sequential approach to integrative review is problem identification, literature search, data evaluation, data analysis, and presentation. 31 The problem identified was to understand the lack of AI implementation in medical education.

A search design was agreed upon by all authors. The designed strategy was to search Pubmed, Medline (Ovid), GoogleScholar, and Web of Science. Key terms searched were ("AI" or "artificial intelligence" or "machine learning") AND ("medical education" or "undergraduate medical education") AND ("curriculum" or "learning" or "teaching"). Resulting article lists were saved, reviewed, and analyzed for selection criteria on Excel.

This study included national and international experimental and nonexperimental studies published in English. We selected articles dating from 2010 in order to include the period before the major growth in electronic health records and digital information in 2011. 32, 33 Article inclusion criteria included studies discussing the need for AI to be incorporated in the medical curriculum either at undergraduate level. We included literature reviews, studies that repeated prior speculative suggestions of curriculum reform, dissertations, letters, opinions, perspectives, and commentaries. We included articles with at least speculative suggestions of curriculum reform, medical student and faculty attitude towards AI, planned out curriculum changes, reports of implemented changes, and institutional level programs to train students in AI.

However, we excluded research that did not mention the need for education of AI in medical school or provide new educational strategies or insight into the actual progression of AI in medical education. Additionally, we excluded research focusing on incorporation of AI tools to enhance medical learning as this was not the purpose of our study. Finally, we excluded research on graduate level medical education or continued medical education.

Using Excel functions, duplicate research studies were screened out. The authors went through each match and analyzed manuscript title and abstract based on predetermined inclusion and exclusion criteria for relevance. Two of the authors independently screened the articles for inclusion and exclusion criteria. They discussed their results and resolved any discrepancies using the third author. Individual articles were then reanalyzed for inclusion criteria accuracy based on article content in the same manner.

Articles were then coded according to relevance for this integrative review. Relevance was scored on a 4-point scale: 1-articles call for change with speculative reform, 2-determining student and faculty attitudes, 3-thorough plans for a curriculum change, 4 -reports of implemented curriculum change or institutional programs. This was determined through careful reading of the full manuscript and determining the best fit category. The scores were then verified by another author who had no previous exposure to the previous values scored. Any discrepancies were discussed holistically and rescored jointly.

The initial searches performed resulted in a total of 729 articles. Removal of duplicates resulted in a total of 588 articles ( Table 1) . Medline search yielded 289 matches. Pubmed resulted in 251unique matches. Google Scholar search yielded a targeted group of 37 new matches. Web of Science search resulted in 11 new matches. We excluded titles that failed to mention AI in medical education (n = 463). Authors individually determined the significance of each article based on title and then compared results to find discrepancies. We then reviewed abstracts and excluded any articles that failed to describe the focus of AI as part of the medical curriculum (n = 86). At this point, the authors excluded any articles that did not focus on UME (n = 1). This resulted in 39 articles total that is displayed in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart (Figure 1 ).

The articles' relevance based on the 4-point scale for integrative review revealed 26 articles calling for change with speculative reform, 5 articles determining student and faculty attitudes, 6 articles with thorough plans for a curriculum change, and 2 reports of implemented curriculum change or institutional programs ( Table 2 ). Both of these articles were not changes brought to an entire medical school or medical class but pilot programs for individual students.

Our results showed an overall consensus that the current knowledge of AI among physicians is at an alarmingly low level and insufficient for future physicians. 55 Previous literature has proclaimed the importance of AI incorporation into the medical school curriculum. 37 AI holds the potential to transform lives across the United States through increased economic prosperity, improved educational opportunities, and quality of life. At the same time, the potential capabilities and complexities of AI make it critically important to further advance our understanding of AI. 41 Medical educationists throughout the world are of the view that a reboot of medical education and curriculum shift from "Knowledge acquisition" to emphasis on "Knowledge management and communication" is needed to address the emerging challenges of 21st century. 39, 45 Furthermore, with the increasing integration of big data and AI in professional practice there is a gap in physician knowledge in the matter. Specifically, one result of our review reported that students in an UME program should acquire sufficient knowledge of basic and clinical medicines, data science, biostatistics, and evidencebased medicine. 28 As a medical student, one should learn to create thoroughly validated, trustworthy information for patients and the public through learning about the fundamental concepts of AI. 28

Our results show that the literature suggests a consensus of opinions regarding the importance of AI education in medical curriculum but a void in actual planning and implementation of this. 43, 46, 48 Of the 6 articles mentioning plans for curricular change, there is no indication of published articles following up on these plans. 38, 44, 50, 51 It is not fully understood what is preventing medical schools from implementing these changes. It is likely only a matter of a few years before physicians will be ill equipped to handle the change on healthcare AI will bring.

Based on our results of the previous literature, we found that the lack of knowledge in AI among medical school faculty is the single most important hindrance in incorporating it among the curriculum. 34, 47, 49, 55 Medical school faculty simply have no understanding of how to implement this change. 35 Therefore, we propose an advanced initiative to propel this change.

In order to anticipate changes and challenges in medical education environments, this review integrated the opinions, studies, and reported changes of implementation of AI in medical education. The results of our systematic review show that no institutional comprehensive studied and researched initiatives were conducted. Among the 2 reports of implemented changes, there were no previous reported studies researching what students hope to gain from such changes or what methods would be most effective. 36, 47 Similarly, among the 6 articles with proposed changes to medical curriculum, there was no evidence-based method backing these proposals. 40, 54 The detailed programs introduced in this study could be useful for medical educators in the development of curricula. Further research is required to integrate the educational trends into graduate and continuing medical education, and to investigate the status or effects of innovative educational programs in each medical school. Going forward, we propose a longitudinal study searching for a comprehensive understanding of how to conduct a large and unparalleled change by incorporating AI in the medical education curriculum.

The exposure of gaps in medical faculty knowledge of AI is in the process of research but is likely to conclude that there is a need to restructure faculty positions to bridge these gaps; these gaps may be inhibiting learners who have an active interest in AI applications but lack exposure to knowledgeable faculty to help them understand, access, and apply them. 42, 56 A 2015 publication noted that 30% of US medical students surveyed had interest in clinical informatics, but were not able to identify training opportunities to help. 57 These knowledge gaps, however, should not be solely characterized in a negative fashion, as they also present important opportunities for new AI added to the curriculum for doctors-to-be. 34 Brouillette M 2019 1

Applications and challenges of implementing artificial intelligence in medical education: integrative review. 33 Chan KS, Zary N. 2019 1

Artificial intelligence and deep learning: the future of medicine and medical practice. 5 Sanal MG, Paul K, Kumar S, Ganguly NK. 2019 1

Artificial intelligence and healthcare professional education: superhuman resources for health? 25 Walsh K. 2020 1

Artificial intelligence education and tools for medical and health informatics students: systematic review. 35 Hasan Sapci A, Aylin Sapci H. 2020 1

Artificial intelligence in health care: insights from an educational forum. 19 Barbour AB, Frush JM, Gatta LA, et al 2019 2

Artificial intelligence in medical education and the meaning of interaction with natural intelligence-an interdisciplinary approach 36 Lang J, Repp H. 2020 2

Artificial intelligence in medical education: Are we ready for it? 37 Imran N, Jawaid M. 2020 1

Artificial Intelligence developments in medical education: a conceptual and practical framework 32 Masters K. 2020 1

Artificial intelligence in medicine: today and tomorrow. 38 Briganti G, Le Moine O. 2020 1

Artificial intelligence in radiology: who's afraid of the big bad wolf? 39 Gallix B, Chong J. 2019 2

Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. 23 Sit Developing an artificial intelligence-enabled health care practice: rewiring health care professions for better Care. 17 Wiljer D, Hakim Z. 2019 3

Implications of artificial intelligence (Al) on dynamics of medical education and care: a perspective 41 Srivastava TK, Waghmare L. 2020 1

Institutionalizing healthcare hackathons to promote diversity in collaboration in medicine. 42 Wang JK, Roy SK, Barry M, Chang RT, Bhatt AS.

Integrating digital health into the curriculum-considerations on the current landscape and future developments 26 Aungst TD, Patel R. 2020 3

Introducing artificial intelligence training in medical education. 43 Paranjape K, Schinkel M, Panday RN, Car J, Nanayakkara P 2019 1

Introductory machine learning for medical students: a pilot 44 Fan KY, Hu R, Singla R 2020 4

Keeping pace: the need for digital health education in medical schools 24 Spigner ST. 2020 1

Looking ahead: The dynamic nature of health systems science, future trends, and the role of learners as change agents 20 Borkan JM, Stagg Elliott V, Wolpaw T, Hassan I, Lomis KD.

Machine learning and medical education. 45 Kolachalama VB, Garg PS. 2018 1

Machine learning: the next paradigm shift in medical education 46 James C, Wheelock K, Woolliscroft J. 2021 1

Medical education and the war with the machines. 47 Ellaway RH. 2014 1

Medical education must move from the information age to the age of artificial intelligence. 48 Wartman SA, Donald Combs C. 2018 1

Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. 49 Han ER, Yeo S, Kim MJ, Lee YH, Park KH, Roh H.

National Cancer Institute Workshop on artificial intelligence in radiation oncology: training the next generation. 50 Kang J, Thompson RF, Aneja S, et al 2020 3

On artificial intelligence and deep learning within medical education. 51 Carin L. Employing a new cohort of educators with skills in new areas is necessary but also innovative. 58 The AMA Council on Medical Education wrote that institutional leaders and academic deans must proactively accelerate their inclusion of nonclinicians, such as data scientists and engineers, onto their faculty rosters. 52 The proper plan should incorporate a longitudinal model to study AI across the medical education continuum. This study should begin with national surveys of medical students and educators on the attitudes and expectations of learning AI in medical school. These surveys should identify the realistic goals and expectations of what future physicians should expect and what faculty members need to meet these expectations. These surveys should be measurable and have interpretable data to advance the implementation of AI in UME. A concerted multiinstitutional study is a logical approach to achieve these goals. Thereafter, multidisciplinary research, collaboration, training, and learning about the role of AI in medicine should be incorporated through previously proven methods in similar drastic curricular changes.

As the push for AI in medical education continues to make advancements, it is not without challenges and barriers. Change is met with resistance from faculty and physicians ill equipped for such an evolution of clinical practice. 49 Fear of eventual physician replacement by machine may prevent these opportunities. Furthermore, misunderstanding and lack of background in AI can create skepticism in the trustworthiness of a machine learning prediction. This is compounded by the regular difficulties in organizing large curricular change. Further research on how best to incorporate these changes is warranted.

A school wishing to keep up with the technologies and advances in healthcare should heed the warnings of numerous physicians about the role of AI as a partner in medicine. Not doing so would disadvantage the future generation of physicians and place them at the mercy of catching up to technology.

Physicians should be trained to understand the technological capabilities and limitations of AI so that they can properly employ it as a safe tool in practice. Physicians need the ability to interpret the results and communicate a recommendation to the patients. Furthermore, physicians should be exposed to the emerging ethical challenges that AI will bring to the workplace. How can a physician untrained in the field of AI expect to navigate ethical scenarios such as if a computer algorithm predicts a high chance of death for a patient.

The implementation of AI in medicine will change the way care is delivered going forward. Medicine is headed along with most other fields, into the age of information technology. Using the Normalization Process Theory, we perceive that AI will become embedded in society both individually and collectively and ultimately will be a necessity in the daily routine of a physician. 53 However, this process can be accelerated in extreme circumstances that call upon the need for AI. Playing to our human strengths to prepare medical students for the future. 15 Chen J. 2017 1

Preparing medical students for the impact of artificial intelligence on healthcare 52 Reimagining medical education in the age of AI. 6 Wartman SA, Combs CD. 2019 1

Technology can augment, but not replace, critical human skills needed for patient care 53 Alrassi J, Katsufrakis PJ, Chandran L. 2021 1

The empirical challenge of 21st-century medical education. 54 Wartman SA. 2019 3

The importance of introducing artificial intelligence to the medical curriculumassessing practitioners' perspectives 22

Dumić-Č ule I, OreškovićT, BrkljačićB, Tiljak MK, OreškovićS 2020 2

The role of artificial intelligence in learning and professional development for healthcare professionals. 21 Randhawa GK, Jackson M. 2020 1

What should medical students know about artificial intelligence in medicine? 27 Park SH, Do KH, Kim S, Park JH, Lim YS. 2019 1

Where medical education meets artificial intelligence: 'Does technology care?' 28 van der Niet AG, Bleakley A. 2021 1

Why we needn't fear the machines: opportunities for medicine in a machine learning world. 16 Li D, Kulasegaram K, Hodges BD. 2019 1

Will artificial intelligence solve the human resource crisis in healthcare? 14 Meskó B, Hetényi G, Győ rffy Z. 2018 1 a 1-articles call for change with speculative reform, 2-determining student and faculty attitudes, 3-thorough plans for a curriculum change, 4-reports of implemented curriculum change or institutional programs.

During COVID-19, medicine changed and it was accelerated by AI and other advanced technologies. We saw entire practices and hospitals switch over to telehealth and this exemplifies the need to train doctors in advanced healthcare technology. The future of medical education and medicine is uncertain after the pandemic resolves but an AI-centric future is likely to be the disruptive change that will guide the future of learning and medicine.

Some of the large problems in healthcare, such as patient wait times, cost of healthcare, and insufficient care may find AI to be extremely helpful. 59 But the costs of misusing AI may be equally problematic. 60 These advantages and liabilities call for future physicians to begin to be properly trained in AI's uses and pitfalls.

Our study has its limitations including a limited amount of databases searched and limited search queries. We accounted for this by looking through the resulting included studies' references for possible missed studies. Additionally, we tried to limit bias in the relevance and scoring process by using a third author opinion and discussing all discrepancies.

There is little doubt among the literature we reviewed that AI will be the future of medicine. In order to continue to efficiently train physicians to be able to handle today and tomorrow, a national and multiinstitutional initiative is warranted.

World Health Organization. Global Health Observatory data repository. World Health Organization

American Health Care: Health Spending and the Federal Budget

Health Care Spending in the United States and Other High-Income Countries

Challenges and opportunities facing medical education

Artificial intelligence and deep learning: the future of medicine and medical practice

Reimagining medical education in the age of AI

The potential for artificial intelligence in healthcare

Artificial intelligence in healthcare

When machines think: radiology's next frontier

The application of medical artificial intelligence technology in rural areas of developing countries

3 ways medical AI can improve workflow for physicians

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

Prediction of premature all-cause mortality: a prospective general population cohort study comparing machine-learning and standard epidemiological approaches

Will artificial intelligence solve the human resource crisis in healthcare? BMC

Playing to our human strengths to prepare medical students for the future

Agents and robots for collaborating and supporting physicians in healthcare scenarios

Why we needn't fear the machines: opportunities for medicine in a machine learning world

Developing an artificial intelligence-enabled health care practice: rewiring health care professions for better care

High-performance medicine: the convergence of human and artificial intelligence

Artificial intelligence in health care: insights from an educational forum

Looking ahead: the dynamic nature of health systems science, future trends, and the role of learners as change agents. Health Systems Science

The role of artificial intelligence in learning and professional development for healthcare professionals

The importance of introducing artificial intelligence to the medical curriculum -assessing practitioners' perspectives

Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey

Keeping Pace: The Need for Digital Health Education in Medical Schools

Artificial intelligence and healthcare professional education: superhuman resources for health?

Integrating digital health into the curriculum-considerations on the current landscape and future developments

What should medical students know about artificial intelligence in medicine?

Where medical education meets artificial intelligence: 'does technology care?

What do medical students actually need to know about artificial intelligence? npj Digit Med

The integrative review: updated methodology

The inevitable application of big data to health care

Big data deep learning: challenges and perspectives

Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review

Medical education and the war with the machines

AI Added to the curriculum for doctors-to-be

Applications and challenges of implementing artificial intelligence in medical education: integrative review

National cancer institute workshop on artificial intelligence in radiation oncology: training the next generation

Artificial intelligence in medical educationare we ready for it? Pakistan

Artificial intelligence education and tools for medical and health informatics students: systematic review

Foundation NS. National Artificial Intelligence (AI) Research Institutes Accelerating Research, Transforming Society, and Growing the American Workforce

The empirical challenge of 21st-century medical education

Machine learning and medical education

On artificial intelligence and deep learning within medical education

Implications of artificial intelligence (AI) on dynamics of medical education and care: a perspective

Artificial intelligence in radiology: who's afraid of the big bad wolf?

Introductory machine learning for medical students: a pilot

Machine learning: the next paradigm shift in medical education

Artificial intelligence in medicine: today and tomorrow

Introducing artificial intelligence training in medical education

Medical education must move from the information age to the age of artificial intelligence

Medical education, professional development and credentialing

Exploring the implementation of an electronic record into a maternity unit: a qualitative study using normalisation process theory

Artificial intelligence in medical education and the meaning of interaction with natural intelligence -an interdisciplinary approach

Artificial intelligence developments in medical education: a conceptual and practical framework

Barriers and solutions to online learning in medical education -an integrative review

Medical student awareness of and interest in clinical informatics

Institutionalizing healthcare hackathons to promote diversity in collaboration in medicine

Training for the future: preparing medical students for the impact of artificial intelligence

Technology can augment, but not replace, critical human skills needed for patient care

All authors have participated in the research for this paper, and the final draft has been read and approved by all authors.

Adam TM Wyatt https://orcid.org/0000-0003-3160-6147 Joel Grunhut https://orcid.org/0000-0002-8104-9679 Oge Marques https://orcid.org/0000-0003-4321-2719