key: cord-0876355-fg3cxflr authors: Ryznar, Elizabeth; Wright, Scott M.; Roy, Durga title: The Current State of Journal Clubs in Psychiatry Residency Programs: Results from a National Survey of Program Directors date: 2022-02-23 journal: Acad Psychiatry DOI: 10.1007/s40596-022-01600-9 sha: 87b9a480afc393524fef652efd177f86f9fbaf0a doc_id: 876355 cord_uid: fg3cxflr OBJECTIVE: This study aimed to investigate journal club prevalence, implementation, and perceived effectiveness among psychiatry residency program directors in order to highlight best practices. METHODS: The authors distributed a 44-item thoughtfully designed and peer-reviewed questionnaire electronically via Qualtrics to 235 accredited U.S. psychiatry residency program directors identified using the American Medical Association database. RESULTS: Eighty-nine programs (38%) responded. Of these, 83 (93%) had a journal club. Journal clubs were mandatory in 80 programs (96%), met biweekly or monthly in 62 programs (75%), and lasted 46–60 min in 66 programs (80%). Twenty-three programs (28%) offered a list of articles to choose from, and 22 programs (27%) provided a critical appraisal tool. Only 7 programs (8%) measured learner outcomes from journal clubs. Respondents believed that promoting lifelong learning and practicing evidence-based psychiatry were the most relevant educational goals of journal club (2.57 and 2.51 on a Likert scale of 0 to 3). Journal club’s effectiveness in achieving those goals was believed to be lower (2.16 and 2.09). CONCLUSIONS: Journal clubs are common in U.S. psychiatry residency programs and tend to follow a traditional format. In order to boost journal club’s effectiveness in achieving the desired educational goals, more programs might elect to infuse elements known to augment learning. journal club described above, it is important to consider how to enhance learning and improve the overall journal club experience for psychiatry trainees. This is especially timely now: the American Board of Psychiatry and Neurology (ABPN) introduced the Article-Based Continuing Certification (ABCC) Pathway, making literature appraisal a critical element of Maintenance of Certification (MOC). Literature appraisal is a skill that may be ideally honed during journal club. Because little is known about current journal club practices in psychiatry residency programs, this study aimed to understand current implementation. A secondary goal was to identify the extent to which some of the established best practices for this conference are being integrated. Because residency program directors are charged with overseeing the education and development of trainees, this study also endeavored to understand their perspectives about the importance and value of journal clubs within psychiatry residency programs in the USA. We developed a 44-item questionnaire covering the presence, purpose, implementation, and evaluation of journal clubs (the survey is available by request from the corresponding author). We defined journal club as "a regularly scheduled educational activity in which a group of residents meets to discuss and critique published papers from psychiatric journals." Responses were multiple-choice, free response, yes-or-no, and 4-point Likert scales. We developed the survey using our expertise in medical education, curriculum development, survey design, and journal clubs. The survey instrument was drafted after a literature review of journal clubs in medical education, with a specific focus on previously identified best practices. Survey items were subsequently refined at "research in progress sessions" with medical educators across 2 distinct departments and pilot-tested with 3 psychiatry residency program directors. These steps served to confer content validity evidence to the instrument. We identified programs and program directors using the American Medical Association's Fellowship and Residency Electronic Interactive Database Access (FREIDA) tool in September 2020. Program directors' direct email addresses were secured from FREIDA, or by Google search when not listed in FREIDA. At the time of initial search, there were 269 psychiatry residency programs in the USA on FREIDA. The personal email addresses of 22 program directors could not be identified, and 12 emails consistently bounced from Qualtrics, leaving 235 program directors to whom we could send the survey link. We distributed the survey electronically between October 2020 and February 2021. Program directors were given the option to forward the survey to an assistant/ associate program director or to a faculty member in charge of overseeing journal club. Program directors were not offered any incentive for participating. This study was reviewed by the Johns Hopkins Medicine Institutional Review Board and deemed exempt (IRB00244982). Chi-squared test was used to compare distribution of type and location of responding programs relative to all U.S. psychiatry residency programs. Location was based on the 9 regions reported by FREIDA, though these were collapsed into 4 categories in Table 1 for ease of interpretation: Northeast (New England + Mid-Atlantic), Midwest (East North Central + West North Central), South (South Atlantic + East South Central + West South Central), West (Mountain + Pacific). Likert scales for relevance of journal club to various educational goals and effectiveness of journal club in achieving those goals were converted numerically from "none, low, high, tremendous" to 0, 1, 2, and 3 respectively. Eighty-nine programs responded-for a response rate of 38%. The characteristics of responding programs are shown in Table 1 . The responding sample is representative of all residency programs in terms of geography and program type (p = 0.37 and 0.78). Currently, 93% of responding programs have a journal club. Of these 83 programs, 76 (92%) exist as a standalone curriculum; the remainder are included as part of a larger psychopharmacology or research/scholarship curriculum. Of the 6 programs (7%) that do not currently have a journal club, 4 had a journal club within the past 10 years. Reasons for termination of journal club included lack of dedicated faculty champion (n = 2), low resident interest (n = 1), lack of prioritization in the residents' schedule (n = 1), and disruption from COVID-19 pandemic (n = 2). Respondents reported that the journal club is most relevant to the educational goals of teaching evidence-based psychiatry and encouraging a practice of ongoing continuing education ( Table 2) . Less relevant, but still important goals, were teaching general and statistical methods, exposing residents to alternate viewpoints, and presenting classic or landmark papers in the field. In the free-text comments, four respondents identified additional social goals (like presenting to a multidisciplinary audience or building camaraderie) and scholarly goals. Journal club's perceived efficacy in achieving those educational goals was lower than perceived relevance ( Table 2 ). Logistical details of the journal clubs' oversight, frequency, length, location, and attendance are provided in Table 3 . Respondents estimate that on average 54% of residents who attend journal club participate in the session (range: 8-100%; standard deviation: 26%). Faculty facilitators were present at 77 journal clubs (93%), either as regular facilitators or as guests (Table 3) . Several program directors commented that the presence of a regular faculty facilitator and journal club champion was integral to their journal club's success. One program specifically mentioned the inclusion of the medical librarian as a co-facilitator. Details regarding article selection, presentation, and discussion are provided in Table 3 . Only 22 programs (27%) distributed a guide or checklist to help residents critically appraise the selected article. Ten programs experimented with alternate formats for journal club, such as using a debate format and adding media clips. Resources that were specifically mentioned to enhance learning were the Journal Club Superstar Curriculum [11] and the National Neuroscience Curriculum Initiative (specifically their "Neuroscience in the News" format) [12] . The COVID-19 pandemic disrupted journal club in several ways. Seventy-six (92%) programs moved to an online (primarily synchronous version). Four programs (5%) decreased the frequency of their journals clubs while one program cancelled it entirely. Three programs (4%) changed the content of their journal club to focus on COVID-19 related articles. Resident evaluation of journal club occurred after every session in 27 programs (33%) and annually in 31 programs (37%); 13 programs (16%) never evaluated their journal clubs. Residents rated journal club as average or above average relative to other residency curricula (Table 3 ). Only 7 programs (8%) measured resident outcomes in journal club knowledge or skills. To our knowledge, this is the first in-depth study examining journal clubs in U.S. psychiatry residency programs. Our data demonstrate that the sessions continue to be held within most psychiatry residency programs in hopes of promoting a commitment to lifelong learning and evidence-based clinical practice. However, program directors believe that their journal clubs have lower impact on achieving those goals. Oversight, regularity, and structure are variable across programs, though many programs follow a traditional model (monthly meetings lasting approximately 1 h in which a previously distributed article is summarized and discussed). The results of this study match the broader literature on journal clubs. The prevalence of journal club in U.S. psychiatry residency programs matches that reported in earlier surveys assessing the presence of journal clubs/evidence-based psychiatry curricula in U.S. psychiatry residency programs (98%) and psychiatric rotations in the U.K./Ireland (97%) [10, 13] . Its popularity is much higher relative to other residency program types (25% in emergency medicine [3] , 60-65% in general surgery [4, 5] , and 73% in plastic surgery [14] ). The paucity of outcome data and learner assessment mirrors the literature from other training programs [9, [15] [16] [17] . In one systematic review, only 8/306 articles published skills and behavioral outcomes; in those few studies, assessments were based exclusively on self-report and the actual impact was unclear [9] . The popularity of the traditional journal club model may represent corroboration that the traditional format is ideal or, more likely, evidence of a lack of continuing improvement of [15] . Individual psychiatry programs have previously published about their innovations, which mirror these recommendations, but these have not been widely incorporated into most programs' journal clubs as shown by our survey data. For example, one program-after determining that article selection contributed to low resident attendance/engagement-asked faculty to create a list of recent clinically relevant or innovative articles that residents could choose from [8] . This modification increased resident attendance and participation in journal club (but not the percentage of residents who read the article). In our survey, less than a third of responding programs provided such a list. In a different article, another program showed that a structured guide or checklist helped residents prepare for the session and improved resident satisfaction with journal clubs [18] . In our survey, less than a third of responding programs provided such a guide. The "Journal Club Superstar" is a model curriculum that provides this type of scaffolding for learners and can be incorporated into existing journal clubs [11] . Other educational supplements can also be used, such as media/film clips or the textbooks Landmark Papers in Psychiatry [19] and 50 Studies You Should Know [20] . Our survey did not explicitly ask about the presence of clearly stated objectives for journal club sessions. With regard to having a dedicated leader, 44 (53%) Large (more than two thirds of the allotted time) 9 (11%) Critical appraisal tool provided 22 (27%) Resident rating of journal club*** Rated as one of the best residency curricular experiences 8 (11%) Rated better than average 32 (45%) Rated on par with other residency curricular experiences 27 (38%) Rated worse than average 3 (4%) Rated as one of the worst curricular experiences 1 (1%) * One program did not respond because its journal club did not exist prior to the pandemic, so n = 82 for this category ** Percentages do not add up to a hundred due to missing data *** Twelve programs do not have resident rankings of journal clubs, so n = 71 for this category several program directors commented that the presence of a regular faculty facilitator and journal club champion was integral to their journal club's success. Furthermore, two programs reported termination of their journal clubs within the past decade due to lack of a faculty champion. Beyond these recommendations, programs can also experiment with alternatives to the traditional journal club format. A few programs in our study mentioned a debate format; other programs in psychiatry and internal medicine have described structuring journal club as a team-based friendly competition to answer a clinical question [21, 22] . Having a nuanced clinical discussion about treatment might help reinforce a critical approach to psychopharmacology in residents, who may rely heavily on their own preferred medication choices or algorithms for treatment. Finally, the landscape of medical education is changing in the digital age and there are alternative platforms available for discussion, such as Twitter [1, 23] , blogs [24] , or podcasts [25] . These platforms have the added advantage of sourcing expertise from different institutions [26, 27] . Given the exponential rise in published studies, some critics have suggested replacing journal club with a dedicated virtual platform for critical appraisal [2] . Several limitations of this study should be considered. First, the response rate of 38% is low. However, the responding programs were representative of all U.S. psychiatry residency programs based on program type and geographic region. Moreover, this response rate is on par with other published surveys of psychiatry program directors, which had response rates ranging from 28 to 44% [28] [29] [30] . Second, the structured format of a survey eliminated the possibility of nuanced responses. For example, two programs reported having various journal clubs depending on the class year. Nonetheless, the survey was detailed (with 44 items) and had several opportunities for free-form response. Third, this survey was aimed at program directors and did not capture the direct resident experience of journal club. Several studies have discussed residents' discontents around journal club [6] [7] [8] . In this study, most program directors reported that residents rated journal clubs on par or better than other didactic educational offerings. In conclusion, journal clubs remain a common session held regularly in U.S. psychiatry residency programs. Journal clubs in psychiatry continue to follow a traditional format. Educational leaders should ensure that journal club is designed to maximize its overall educational impact; suggestions include use of structured guides, an interactive approach, relevant article selection, and frequent program evaluation and learner assessment. Additionally, leaders should ensure that journal club continues to add value in the changing landscape of medical education. Future studies of journal club impact will need to incorporate learner assessment more rigorously. The evolution of the journal club: from Osler to Twitter Rethinking medical journal club Incorporating evidence-based medicine into resident education: a CORD survey of faculty and resident expectations Understanding the modern surgical journal club The role of the journal club in surgical residency programs: a survey of APDS program directors Resident run journal club: a model based on the adult learning theory Teaching evidence based medicine to surgery residents-is journal club the best format? A systematic review of the literature Refreshing a resident-run journal club: a focus on article selection The journal club in postgraduate medical education: a systematic review What to learn and how to teach it: five years of pre-meetings for training directors in psychiatry National Neuroscience Initiative Curriculum National survey of training needs for evidencebased practices How well are we doing at teaching critical appraisal skills to our residents? A needs assessment of plastic surgery journal club Acad Psychiatry How to run an effective journal club: a systematic review A review of journal clubs in postgraduate medical education Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal The self-directed, structured summary as a teaching tool in a psychiatry journal club Landmark papers in psychiatry 50 studies every psychiatrist should know Revamping journal club for the millennial learner Journal Club 2.0: using teambased learning and online collaboration to engage learners #GeriMedJC: the Twitter complement to the traditional-format geriatric medicine journal club Available from: closler.org Electronic journal clubs for capacity building: a case study in psychiatry as a model for medical disciplines in developing countries Another time, another space: the evolution of the virtual journal club LGBT-specific education in general psychiatry residency programs: a survey of program directors Training residents in high-value, cost-effective care: a national survey of psychiatry program directors Reproductive psychiatry residency training: a survey of psychiatric residency program directors Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Acknowledgements The authors wish to thank Margaret S. Chisolm and the participants of the monthly Johns Hopkins General Internal Medicine Academic Conference for their helpful comments on the methodology of this study. Disclosures E. Ryznar receives royalties from Oxford University Press as an editor of "Landmark Papers in Psychiatry." S. Wright receives support as the Anne Gaines and G. Thomas Miller Professor of Medicine supported through the Johns Hopkins Center for Innovative Medicine. D. Roy has no disclosures.Ethical Approval This study was reviewed by the Johns Hopkins University School of Medicine Institutional Review Board and deemed exempt (IRB00244982) on 6/12/2020.