key: cord-0846890-ceu9x6tn
authors: Frye, William S.; Feldman, Marissa; Katzenstein, Jennifer; Gardner, Lauren
title: Modified Training Experiences for Psychology Interns and Fellows During COVID-19: Use of Telepsychology and Telesupervision by Child and Adolescent Training Programs
date: 2022-01-27
journal: J Clin Psychol Med Settings
DOI: 10.1007/s10880-021-09839-4
sha: 5a38f93bebb6b1d56f5c3a90d15047728bf41b13
doc_id: 846890
cord_uid: ceu9x6tn

Coronavirus (COVID-19) has affected opportunities available to psychology interns and postdoctoral fellows completing capstone training experiences during culminating training years. While research supports COVID-19 has increased the use of telepsychology services amongst psychologists, there is a paucity of research regarding how COVID-19 has altered training and use of telepsychology by psychology trainees. The current study includes survey responses from 59 psychology training directors and 58 psychology internship and postdoctoral fellowship trainees at pediatric sites throughout the United States. Results support changes in telepsychology training provided during COVID-19, including increased use of telepsychology for clinical service delivery and increased use of telesupervision for training. As expected, findings suggest novel training experiences in telepsychology for trainees within the last two years as a result of COVID-19. Given ongoing need for telepsychology services to assure access to psychological care during the pandemic and beyond, results provide support for graduate and advanced training programs to provide formal training in best-practices for utilization of telepsychology and telesupervision.

Following the initial reports of coronavirus disease 2019 in December 2019, the coronavirus pandemic has become one of the deadliest pandemics in history. The toll of attempting to continue to navigate everyday life during an ongoing global pandemic has resulted in increased psychological distress for both adults and children (Brooks et al., 2020; Tang et al., 2021) . COVID-19 has far reaching effects on the medical, social, emotional, and psychological health of children (Bartek et al., 2021) . Children receiving medical care due to the COVID pandemic present with mental health needs including reduced life satisfaction and increased rates of depression and health-related anxiety (Korte et al., 2021; Tang et al., 2021) . To meet the needs of these ever-increasing mental health concerns, service providers have had to provide clinical services in novel ways. To support the increased demand for therapeutic service providers, psychology training programs must continue to prepare advanced level psychology trainees to become practicing psychologists. Psychology internship and fellowship programs provide capstone training experiences for emerging psychologists; however, many of these training programs were impacted by COVID-19. While research has illustrated how licensed psychologists have increased use of telepsychology services during COVID-19 (Frye et al., 2021a) , a critical question remains: how has COVID-19 affected the training experiences of pediatric psychology trainees?

In order to reduce transmission of COVID-19, many healthcare sites limited or discontinued in-person services in outpatient and elective settings and transitioned to telepsychology services. The use of telepsychology services by psychologists increased from approximately 7% prior to the pandemic to 85% during the pandemic, with approximately 67% of psychologists performing all clinical work via telepsychology (Pierce et al., 2020) . While many providers were able to transition services to telepsychology, limitations such as confidence in providing services, performing comprehensive exams, regulatory barriers, remuneration, and technological issues in telepsychology have been identified as barriers by pediatric providers (Balestra, 2018; Frye et al., 2021a) . Psychology trainees who completed internship and fellowship during COVID-19 have likely experienced vastly different training opportunities than trainees in the past. We know that licensed psychologists who train interns and fellows significantly increased their use of telepsychology for service provision throughout COVID-19, and therefore it stands to reason that interns and fellows increased their use of telepsychology as well. However, little remains known as to how interns and fellows at pediatric training sites are utilizing telepsychology, or how the use of telesupervision vs. in-person training by supervisors has affected trainee's learning and acquisition of profession-wide competencies during the pandemic (see APA, 2018) .

Prior to contact restrictions resulting from COVID-19, psychology and medical training programs primarily relied on in-person services for training. However, literature has documented that many medical training sites modified clinical and educational opportunities due to COVID-19 precautions including canceling outpatient services, elective surgical procedures, or even removing trainees from clinical environments or rotations (Rose, 2020) . For medical residents and fellows, these restrictions have led to fewer in-person clinical contact hours, decreased experience with in-person examination, and reduction in exposure to diverse pathologies (Edigin et al., 2020) . While these studies support that services provided by medical trainees have changed, published studies describing changes in the experiences of psychology trainees during the pandemic are emerging (Gardner et al., in press) .

The Association of Psychology Postdoctoral and Internship Centers (APPIC), conducted a survey to understand how COVID-19 has affected health service psychology training and received responses from 432 internship and 222 postdoctoral fellowship training directors from sites working with patients of all ages (APPIC, 2021). Responses from internship training directors noted the greatest adjustments to their training programs included increased use of telepsychology for clinical services-particularly intervention, increased use of telesupervision, and disruption to development of trainees' competency in assessment. Similar trends in service delivery modalities were reported by both internship and fellowship training directors, with almost half reporting off-site video-based telepsychology was the most commonly utilized mode of service delivery, followed by in-person patient care, and on-site video-based telepsychology services. A survey of training directors from child and adolescent internship and/or fellowship training sites by (Gardner et al., in press) found that training directors described numerous altered training opportunities due to COVID-19 including increased use of telepsychology by trainees, telesupervision, modified safety precautions/ procedures, reduced in-person clinical opportunities, and decreased morale.

Prior to COVID-19, psychology trainees primarily provided in-person services to patients under the direct supervision of a licensed psychologist. For training sites accredited by the American Psychological Association (APA), the Commission on Accreditation (CoA) guidelines state that psychology interns must receive four hours of supervision weekly, two hours of which must be individual supervision (APA, 2018) . Although synchronous telesupervision is allowed for individual supervision of interns, prior to COVID-19 the CoA limited use of telesupervision to one hour per week. In response to COVID-19-related concerns, in June 2021 the CoA provided updated guidance to accredited training programs supporting the use of telesupervision, telepsychology, and distance learning as modifications to minimize COVID-19 exposure; remaining in effect through December 31, 2021 (APA, 2021 .

Given the drastic increased use of telepsychology following the start of the pandemic (Pierce et al., 2020) , licensed psychologists had to quickly adapt to utilizing teleservices for both clinical practice and training purposes to continue to meet the needs of patients and trainees (Frye et al., 2021a) . As such, psychology interns and fellows who completed their training in the 2019-2020 and 2020-2021 training years have had unique training experiences, with perhaps the greatest modification being the use of telepsychology for clinical services and potential use of telesupervision from supervisors who initially may have had limited-to-no professional training utilizing these modalities prior to COVID-19 (Phillips et al., 2021) . In fact, previous authors have noted this barrier and encouraged the inclusion of telepsychology as part of psychology graduate student training (Glueckauf et al., 2018; Traube et al., 2021) and a framework of telepsychology education for trainees was recently proposed, focused upon ethics of telepsychology, synchronous and asynchronous methods of telepsychology, logistical considerations, billing, training, and supervision (Perle, 2020) .

While these articles present helpful suggestions for telepsychology training, standards for training in telepsychology education are emerging (Perle, 2020) . In addition to standards for training guidelines, there is concern for how the shift to telepsychology has impacted professionwide competencies. In a recent study examining the effect of COVID-19 upon internship and postdoctoral training, training directors' reported assessment, communication and interpersonal skills, intervention, and consultation as the profession-wide competencies most negatively impacted (Gardner et al., 2021) . The importance of gaining clinical experiences during the pandemic, while still meeting training expectations that have traditionally occurred in-person, is clear. Additionally, the current literature provides little discussion on the adoption of telepsychology services by psychology trainees, how telepsychology has been integrated into training, or how telesupervision is being utilized for trainees. Recognizing the unique training experiences inherent to working with pediatric populations, the current study aims to provide an exploratory description of changes in the pervasiveness of telepsychology usage and modified training experiences that emerged during the pandemic as reported by psychology trainees and their training directors at child and adolescent psychology training sites throughout the United States. Specifically, this study aims to describe (1) The provision of telepsychology by trainees at child and adolescent psychology training sites at the time of the survey, (2) The type of telepsychology training psychology trainees receive (3) supervising psychologists' level of training in telepsychology as reported by the program's training director, and (4) How telesupervision is utilized for training of psychology interns and fellows as reported by the program's training director and trainees.

Training directors of doctoral internship and postdoctoral fellowship programs in the United States that were identified as having a focus on pediatric and/or child/adolescent psychology training were included in the sample. Doctoral interns and postdoctoral fellows who were forwarded the survey by their training directors were also included as participants. Participants included 66 training directors, 23 postdoctoral fellows, and 35 doctoral interns. Seven training directors were excluded due to responding only to demographic questions, leaving a total of 59. While demographics were not collected to preserve anonymity, participants represented training sites in 28 states of the United States and one site in the District of Columbia. The majority of training directors had been licensed 16 or more years (40.7%), while 22% had been licensed 11-15 years, 28.8% licensed 6-10 years, and 8.5% licensed five or less years. One training director did not report years licensed. The most commonly reported program settings included Academic Health Centers (50.0%), Child/Adolescent Psychiatric or Pediatrics (16.7%), or Consortiums (9.1%) ( Table 1) . Study procedures were reviewed and approved by the authors' Institutional Review Board.

Participants completed and anonymous questionnaire consisting of up to 20 forced-choice response items focused upon the utilization of telepsychology, services provided by telepsychology, and telepsychology training for both psychology trainees and their supervisors (e.g., "Please indicate what psychology services trainees provide via telehealth: … Outpatient treatment/ psychotherapy") This questionnaire was delivered via the Qualtrics platform and used question logic to provide director-or trainee-specific questions regarding their telepsychology experiences based upon selection of the respondent.

In March 2021, approximately one year after the placement of COVID-19 restrictions throughout the United States, the anonymous questionnaire was distributed to the email addresses of 252 training directors as part of a larger study. Training directors were identified through the training database maintained by the Society of Pediatric Psychology, Division 54 of the APA, and the APPIC directory by searching for internship and fellowship sites that include training opportunities for child and/or adolescent populations. The initial e-mail asked for the training director to complete the questionnaire and forward the e-mail to their doctoral interns and postdoctoral fellows as an invitation for completion. A reminder e-mail with the questionnaire was distributed to the same training directors approximately three weeks after the initial e-mail. The survey was closed after four weeks.

Prior to COVID-19, 39.0% of training directors reported they had used telepsychology and 10.2% of training directors had used telepsychology more than 10 times per year prior to COVID-19. At the time of the survey, 66.1% of training directors reported their site had limited the provision of in-person services due to COVID-19. Of the 59 training Approximately 75.4% (n = 43) of training directors reported trainees were permitted to provide telepsychology services from both on-and off-campus locations, while 15.8% (n = 9) reported trainees were required to provide telepsychology services while on-campus, and the remaining 8.8% (n = 5) reported trainees provided telepsychology services exclusively off-campus. When asked about the use of telepsychology services by trainees in training years prior to COVID-19, only 17.5% (n = 10) of training directors reported previous trainee cohorts used telepsychology for clinical services. Training directors reported outpatient treatment/psychotherapy, evaluation feedback, and diagnostic interviews as the most common clinical services provided by trainees utilizing telepsychology. Training directors less commonly reported the use of telepsychology for inpatient consultation and psychology/ neuropsychology assessment (Table 2) . When comparing use of telepsychology for service provision, 54.4% of training directors reported telepsychology was used in over half of the diagnostic interviews conducted at their site, 70.2% reported telepsychology was used in over half of outpatient treatment/psychotherapy sessions, and 63.2% of training directors reported over half of feedback sessions were conducted via telepsychology. Only 22.8% of training directors reported telepsychology for the majority of psychology/neuropsychology assessments (Table 3) .

Trainees were asked similar questions to assess their experiences providing clinical services via telepsychology. At the time of the survey, 72.4% (n = 42) of trainees reported their training site limited the provision of in-person services due to COVID-19. Of the 58 trainee respondents, 94.8% reported they provided clinical services via telepsychology during their current training year. The majority of these trainees (89.7%; n = 52) reported they began utilizing telepsychology at their training site due to COVID-19. Only 15.5% of trainees reported any provision of telepsychology prior to COVID-19, and only 3.4% had used telepsychology more than 10 times per year prior to COVID-19. Thirty (54.5%) of the 55 trainees who reported using telepsychology during the current training year reported they did so while located both on-and off-campus, 27.6% (n = 16) reported they were required to remain on-campus while conducting telepsychology sessions, and 15.5% (n = 9) reported they only provided telepsychology services from off-campus locations. Consistent with training director report, trainees reported the main services they provided via telepsychology included diagnostic interviews, outpatient treatment/ psychotherapy, and assessment feedback sessions. Trainees also reported limited use of telepsychology for inpatient consultation or psychology/neuropsychology assessment similar to reporting by training directors (Table 2) . Of the 55 trainees using telepsychology during the current training year, telepsychology was reported to be used in over half of diagnostic interviews (60.0%) and outpatient treatment/ psychotherapy (65.5%). The majority of trainees reported not using telepsychology for consultation/inpatient services (60.0%) and psychology/neuropsychology assessment (69.1%; Table 3 ).

With the pandemic, 69.5% of training directors reported trainees received formal training in telepsychology service delivery through their program or organization, 66.1% of training directors reported trainees engaged in self-learning, 44.1% reported trainees completed formal webinar or continuing education credits in telepsychology, 8.5% reported preparing trainees through informal discussion, didactic seminars, or supervision observation, and 5.1% reported no telepsychology training was provided to trainees. Approximately 72.9% of training directors reported mandatory telepsychology training for trainees at their site. When asked about their training in telepsychology, 89.7% of trainees reported they had not received any training in the provision of clinical services via telepsychology as part of their graduate training. Of the 23 postdoctoral fellows, 60.9% (n = 14) reported receiving some form of telepsychology training during their doctoral internship which was completed during the pandemic. Trainees were given a 5-point Likert scale to rate their satisfaction with their level of telepsychology training from very dissatisfied to satisfied. Approximately 12.1% reported feeling either dissatisfied or very dissatisfied, 39.7% reported feeling neutral, and 48.3% reported feeling satisfied or very satisfied with the training they received in telepsychology during the internship or fellowship training year.

Training directors and trainees were asked about supervision of the telepsychology services provided by trainees. The majority of training directors reported supervisors utilized case review at weekly supervision (98.2%), clinical notes/ documentation review (96.5%), and joining telepsychology session via videoconferencing (87.7%) as methods of supervisory oversight for trainee telepsychology services (Table 4 ). Trainees using telepsychology reported similar methods of supervisory oversight for telepsychology services including case review with their supervisor at weekly supervision (85.5%), receiving clinical notes/documentation review (83.6%), and their supervisor joining telepsychology sessions via videoconferencing (65.5%) ( Table 4 ). The majority of training directors reported weekly supervision sessions with trainees mainly occurred through a combination of in-person and video supervision (57.6%), while 23.7% reported utilizing only video supervision (Table 5) . Trainees reported primarily receiving supervision via video only (39.6%), in-person only (20.7%), or a combination of video and in-person (24.1%) ( Table 5 ).

More than 90% of psychology training directors and trainees in this study reported initiating telepsychology to provide a variety of clinical services due to COVID-19. This finding emphasizes that at the beginning of the pandemic, psychologists and trainees had to adapt quickly to learn a method of service delivery that they were not familiar with, and determine which services were most appropriately provided via telepsychology. Literature on telepsychology services before COVID-19 has been primarily focused upon the provision of psychotherapy. Responses from training directors and trainees supported findings from the APPIC survey results that most programs increased use of telepsychology for intervention services, while reducing in-person clinical experiences that were typically available prior to COVID-19 such as assessment. More specifically, trainees in this study primarily provided outpatient therapy, diagnostic interviews, and feedback services via telepsychology, though infrequently conducted psychological and neuropsychological assessments. This finding is understandable given that administration of standardized assessment often requires use of testing books and manipulatives, which have traditionally been administered in-person. However, innovative practices utilizing telepsychology have demonstrated success for neuropsychological testing, which if implemented by programs, could assure assessment training opportunities are sustained (Ransom et al., 2020) . These types of innovative practices may be necessary as reduction in in-person experiences such as administering standardized assessment may have a potential, yet unstudied, chance of negatively affecting real-world practice. Also, not surprisingly, both training directors and trainees reported less frequent use of telepsychology services for consultation/inpatient services, which may reflect the medical setting in which the care was embedded and expectation for in-person services in certain settings. The increase in telepsychology and off-site services may also impact professional development of trainees who are missing in-person training experiences that facilitate collaboration, consultation, and relationship building with coworkers, which is often considered an integral component of multidisciplinary and interdisciplinary work. Although the transition from in-person to telepsychology services has reduced more traditional training opportunities, it also broadened training experiences in novel ways. Almost all training sites reported trainees utilized both inperson and telepsychology services, therefore trainees who completed internship and fellowship during the pandemic benefited from unique training opportunities and experiences compared to previous training cohorts who likely had more limited exposure to telepsychology service delivery. As it appears likely that the use of telepsychology will continue in some capacities after the pandemic has resolved given the benefits to patients that may otherwise go without mental health services, balancing advantages and disadvantages of this service modality should be considered. Telepsychology can offer unique experiences in service delivery and improve patient access to care, though there is a need to ensure trainees and their supervising psychologists obtain proper training in telepsychology, considering ethical issues relevant to confidentiality and safety, especially if lacking previous experience.

Prior to internship and fellowship training, almost all trainees in this study reported they had not received training in the use of telepsychology for clinical service delivery during their graduate school program, reflecting concerns from previous research (Glueckauf et al., 2018; Traube et al., 2020) that obtaining proficiency for telepsychology in psychology graduate school training is needed. For postdoctoral fellows who completed their internship during the 2019-2020 training year, approximately 60% reported they received telepsychology training during their doctoral internship training year amid the pandemic. Following the Ethical Principles of Psychologists and Code of Conduct, psychologists are to provide services, teach, and conduct research in areas only within the boundaries of their competencies. As such, although 90% of the surveyed trainees had not received telepsychology training for clinical services prior to COVID-19, many trainees reported that they actively sought to increase their competency in telepsychology by completing formal training in telepsychology service delivery, self-learning, and/or completing formal webinar or continuing education credits in telepsychology. These data should be considered by training sites to ensure their trainees, trainee supervisors, and psychologist providers have received proper training in the provision of telepsychology services. When asked about satisfaction with training in telepsychology, approximately 40% of our trainee sample reported they were satisfied with the training they received in telepsychology and 12% who felt dissatisfied with training. The general lack of satisfaction in training is concerning given these trainees may not feel adequately prepared or sufficiently trained for practicing telepsychology.

Another significant change in training practices included the use of telesupervision (video, telephone) instead of inperson meetings and live in-room clinical supervision. The majority of training directors reported supervisors provided live, direct supervision of clinical telepsychology service delivery by trainees. Interestingly, trainees reported a higher amount of supervision only through video and significantly lower amount of supervision through a combination of video and in-person meetings than training directors. This discrepancy may reflect discordance in the supervision practices training directors expect and supervisors actually provided, trainees providing more honest opinions about supervision at their site, or a bias in the participants who completed the survey. While the reason for these discrepancies in how supervision was provided cannot be ascertained from this study, it is a topic that deserves future inquiry. Supervisors remotely joining telepsychology sessions could be a promising modality for providing direct observation of telepsychology services provided by trainees. Doing so would allow supervisors an opportunity to understand barriers trainees may face such as internet connection or patient engagement, and assess trainees' profession-wide competencies when utilizing telepsychology for intervention and assessment. Direct observation of trainee's telepsychology session via remote login is especially useful given the majority of trainees lacked telepsychology experience prior to COVID-19. Providing telesupervison also complies with the APA CoA's updated guidance to accredited training programs regarding COVID-19 related use of telesupervision, telepsychology, and distance learning to minimize COVID-19 exposure.

The present study provides a description of telepsychology utilization by pediatric psychology interns and fellows during the 2019-2020 and 2020-2021 training years. These experiences are likely to change over time as contact restrictions are modified, in response to increased rates of vaccinations and reduced rates of COVID-19 in the general population.

As our survey was distributed to child and adolescent training sites approximately 1 year into the COVID-19 pandemic, the results of this study are likely not indicative of experiences and modifications implemented at all sites and with all populations (e.g., inclusive of adults). The current study provides a limited view of how telepsychology training at their program was affected from the perspective of the psychology internship and fellowship training director(s). Future research is need to provide information regarding how other direct supervisors at training programs utilize telepsychology for training for students at their training site. By surveying direct supervisors more insight could be gained regarding their own training in telepsychology and their confidence in training others in its use. Additionally, these results do not provide an exhaustive list of strategies psychology training sites have used to train and implement telepsychology to provide services to patients across the lifespan. Barriers and facilitators of telepsychology may be dependent on patient population, state, institutional policies, digital infrastructure, or needs of each training site. As this study was designed to be an exploratory description of changes in the pervasiveness of telepsychology usage and training experiences at child and adolescent training sites, it does not provide a comprehensive assessment of the depth, breadth, or quality of telehealth training provided. Additionally, less than one fourth of eligible training directors responded to this survey and responses described are not exhaustive and may reflect biases. Anonymity of the survey precludes knowing which sites forwarded this survey to trainees and the rate of trainee respondents. While the overall response rate is somewhat low, the exploratory nature of this study allows these data to provide a meaningful overview of the role of telepsychology in training.

Once contact restrictions are discontinued, it is likely that telepsychology will remain a viable service delivery modality given its positive impact on mental health outcomes, access to care, and reduced patient expenses (Egede et al., 2018; Pande et al., 2015) . Research has found that patients report similar levels of therapeutic alliance with psychologists when provided in-person or telepsychology services (Frye et al., 2021b) ; however, this finding has not yet been established with trainees, especially in the context of COVID-19. Additional studies examining the impact of telepsychology and telesupervision on training and trainees' acquisition of profession-wide competencies is needed, particularly as related to working with a child and adolescent population when patient engagement via electronic platforms may be more challenging (e.g., younger child) and preclude use of naturalistic observation in classroom or school settings to inform case conceptualization, treatment planning, and care coordination with other disciplines and professionals.

While the allowance of telesupervision appears to be successfully utilized for trainees to continue required supervision hours and monitoring of clinical services, research examining the impact of telesupervision on acquisition of competencies and the impact on trainees' overall training experience is needed. A lack of in-person interaction during training impacts opportunities for spontaneous learning such as "curbside" supervision, immediate access of supervisors for crisis situations, and the supervisor-supervisee relationship. As lifelong learners, psychologists continue to build competencies overtime through continuing education, but future research is needed to determine the impact of telesupervision on trainee's development of competencies, professional relationship building, professional identify formation, and supervisory skills to assure readiness as emerging psychologists.

Future studies may also consider the impact of telepsychology training to determine if formal training improves trainee satisfaction, knowledge, quality of service, and/or confidence in providing telepsychology. Obtaining feedback from patients receiving telepsychology may be equally important to determine their perceptions, satisfaction, and attitudes towards trainee-delivered telepsychology.

This study provides an exploratory description of changes in the use of telepsychology and telesupervision, as well as modifications in training experiences for psychology trainees throughout the COVID-19 pandemic. Training directors, psychology interns and postdoctoral fellows reported significantly altered training opportunities, which limited in-person service provision and training opportunities, and increased used of telepsychology and telesupervision. Alterations to training experiences for psychology interns and postdoctoral fellows present with potential benefits and drawbacks for individuals completing their capstone training during the pandemic. While contact restrictions reduced opportunities to provide in-person services, particularly in psychology and neuropsychology assessment and interdisciplinary consultation, it has provided the opportunity for emerging psychologists to gain new competencies in the provision of telepsychology. Reduced in-person training experiences may affect readiness for real-world practice when we return to business-as-usual; however, it also equips these emerging psychologists with the ability to provide clinical services, supervision, and training via online platforms which will likely continue to be used long after COVID-19. Data from the current study indicated many trainees and their supervisors did not receive formal training or have experience using telepsychology prior to COVID-19. These findings highlight the need for standards of training in telepsychology education throughout graduate and post-graduate training for psychologists, given competency in use of telepsychology will likely be an expectation for providers moving forward.

To assist with the continued provision of telepsychology, graduate programs and advanced training sites should consider ways to integrate telepsychology training into their curriculum to prepare future psychologists. A framework for telepsychology training has been proposed by Perle, 2020, which may help guide programs as they consider how they will address telepsychology training. Changes in curriculum may include telepsychology-focused didactics, readings, or trainings provided by APA or training institutions. While there are many online resources and trainings available, not all information may be directly applicable to trainees and how telepsychology is provided at their site. Additionally, unless widely accepted standards of training for telepsychology are established, it will be difficult to determine who has received adequate training or what adequate training entails. Integration of telepsychology into current profession-wide competencies or training benchmarks may be beneficial to determine trainees' level of professional readiness to utilize teleservices for clinical practice.

Through investigation of telepsychology use at child and adolescent training sites, it appears telepsychology has been increasingly integrated into supervision and service provision, and will continue to be a viable modality in the future. Given the novelty of telepsychology and historical lack of telepsychology training throughout pre-and postdoctoral training, training sites should consider formal training in best-practices for utilization of telepsychology and telesupervision.

Author Contributions WSF and LG contributed to the conception of the work, data collection, data analysis and interpretation, drafting the article, critical revision of the article, and final approval of the version to be published. MF and JK contributed to the conception of the work, draft of the article, and final approval of the version to be published.

Funding Not applicable.

Code availability Not applicable.

The authors declare they have no financial or nonfinancial interests to disclose. The authors have no conflicts of interest to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement with any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in the manuscript.

Ethical Approval Study procedures were reviewed and approved by the first author's Institutional Review Board.

No individual data are published. All participants agreed to participation.

The authors affirm that human research participants provided informed consent for publication.

Internship training director survey from spring 2021

COVID-19: Updates and information

Telehealth and legal implications for nurse practitioners

Addressing the clinical impact of COVID-19 on pediatric mental health

The psychological impact of quarantine and how to reduce it: Rapid review of the evidence

Impact of COVID-19 pandemic on medical postgraduate training in the United States

Cost-effectiveness of behavioral activation for depression in older adult veterans: in-person care versus telehealth. The Journal of Clinical Psychiatry

Implementation of telehealth during COVID-19: Implications for providing behavioral health services to pediatric patients

Utilizing telemental health in a pediatric outpatient psychology clinic: Therapeutic alliance and outcomes. Counselling & Psychotherapy Research

Training directors' perspectives of psychology intern and postdoctoral fellow training experiences during COVID-19: A changing landscape. Evidence-Based Practice in Child and Adolescent Mental Health

Survey of psychologists' telebehavioral health practices: Technology use, ethical issues, and training needs

Intolerance of uncertainty and health-related anxiety in youth amid the COVID-19 pandemic: Understanding and weathering the continuing storm

Leveraging remote behavioral health interventions to improve medical outcomes and reduce costs

Introduction to telehealth for clinical psychologists: a novel course designed to improve general knowledge and hands-on expertise with technology-based modalities

COVID-19 and beyond: Telesupervision training within the supervision competency. Training and Education in Professional Psychology

The COVID-19 telepsychology revolution: A national study of pandemic-based changes in US mental health care delivery

Pediatric teleneuropsychology: Feasibility and recommendations

Medical Student Education in the Time of COVID-19

Mental health and its correlates among children and adolescents during COVID-19 school closure: The importance of parent-child discussion

Telehealth training and provider experience of delivering behavioral health services

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations