key: cord-0801190-zsos7ynv authors: Oeppen, Rachel S.; Rutherford, Elizabeth; Sadler, Paul; Isaac, Robert; Brennan, Peter A. title: Virtual ARCP assessment and trainee feedback meetings: facilitating the best experience and practice date: 2020-10-14 journal: Br J Oral Maxillofac Surg DOI: 10.1016/j.bjoms.2020.09.037 sha: 01983a19828cfccce919967c50e9f5b4d15ea09a doc_id: 801190 cord_uid: zsos7ynv As a result of COVID-19, there has been an exponential increase in the use of remote technology for many local, regional and national meetings that would previously have been held on a face-to-face basis. Remote meetings have ensured that essential clinical, educational and strategic work can continue but it is not ‘business as usual’, although colleagues accept this form of communication as the new norm. In medical education and assessment, the Annual Review of Competence Progression (ARCP) meetings and other formative educational meetings are being conducted remotely. This form of communication has some advantages but may also present possible barriers for feedback and development particularly for ‘trainees in difficulty’ when there are concerns about progression, and when an unsatisfactory outcome has been awarded. It is also worth remembering that there may be generational differences with the ease of use of virtual meeting platforms. We present some of the important factors for optimising the panel of virtual ARCPs and discuss methods to improve feedback given remotely for trainees. In just a few months, the coronavirus (COVID-19) pandemic has changed our way of life and how we conduct local, regional and national meetings. There has been a significant rise in the use of virtual meeting technology using platforms such as Zoom, Microsoft® Teams and GoToMeeting.® Most platforms have a secure environment and also enable recording of the meetings when needed. Some of the significant differences between virtual and face-to-face meetings include reduced non verbal communication clues, including body language and gestures as well as limited casual interactions that are so important for building rapport and trust (1, 2) . Furthermore, the inability to meet in person either before or after meetings can J o u r n a l P r e -p r o o f have a negative effect on morale, as well as networking, socialising with colleagues and catching up on news. These and other issues suggest that virtual technology is unlikely to completely replace the traditional face-to-face meetings particularly where team interaction is important such as for multidisciplinary meetings (3) . The 'normalisation' of online technology to resemble face-to-face meetings includes positioning cameras to ensure eye contact with other participants (4). The Annual Review of Competence Progression (ARCP) meeting is an important summative assessment of a trainee's portfolio, logbook and achievements matched to the curriculum for their stage of training. Non adverse outcome ARCP (outcome 1) is required to progress to the next year of training. In a large study of over 11,000 ARCPs, 20% of trainees were awarded at least one adverse ARCP outcome (2-targeted training with no additional time needed, or 3 additional time required) during their training (5) . In the event of an unsatisfactory ARCP, most panels arrange to meet with the trainee, either on the day or subsequently to discuss the outcome and provide guidance and support. It should be noted that the ARCP and any feedback are separate processes. During the current COVID-19 pandemic, this meeting is likely to be remote in its delivery. The yearly educational review meeting may also be held remotely. In this article we discuss some of the important considerations to help ensure the best experience for trainees and trainers, particularly when the ARCP outcome is unsatisfactory. Before an ARCP panel formally meets essential for panel members to be fully prepared before any meeting but particularly when conducted virtually. This includes familiarisation with the ARCP process itself, reading the trainees' ePortfolios (ISCP for surgery) and preparing notes if necessary to aid discussion in areas that require clarification or expansion. To assist the ARCP panel, trainees for their part should ensure that correct and accurate evidence to be considered is uploaded well ahead of the ARCP date, particularly when panel members are required to review large volumes of evidence in advance of the meeting, and preferably not on the night before. and not the night before. The number of trainees being assessed is likely to influence the way in which the panel conducts the process. For example, for panels required to assess larger numbers (10+), the ARCP panel chair may delegate certain tasks or evidence review to each panel member, but with fewer trainees, panel members can usually review the evidence for all trainees. Panel members should check to make sure that their broadband is working well, and a 'test run' held at some point before the formal ARCP, is often a valuable exercise . Issues with bandwidth during virtual meetings can result in freezing of computer screens, frustration or even anger, resulting in reduced performance of the whole panel (2,6,7). From the outset, the ARCP chair should give an initial panel briefing to establish how process is to be run, ensure all the relevant regulatory requirements for panel members have been met, discuss timings and most importantly ensure that all members feel equally valued (7) . Furthermore, it is important to check that panel members are familiar with what to expect J o u r n a l P r e -p r o o f and it cannot be assumed that all are comfortable with the online platform being used. It may be that for those panel members who are less confident or experienced at virtual meetings, the chair could facilitate a more balanced discussion by inviting opinions from them. Similarly, the chair should be aware of any panel members who may have a tendency to dominate the discussion. As with face-to-face meetings, etiquette with virtual meetings should be even more tightly controlled allowing only one panel member to speak at a time. When bandwidth saturation results in slowing or freezing of video, we recommend that only the chair and one other speaker have their cameras on to improve sound quality 6). However, ideally it is better to be able to see others on the virtual platform, just as would occur in a 'normal' meeting. On many platforms, the hand up function is useful to alert the chair. Another option is to turn ones camera on so as to inform the chair that you wish to speak in a similar way to hand raising during face-to-face meetings.. This also encourages panel members to remain focussed as they are less likely to be distracted. A printed version of the ARCP checklist provided by the local Health Education England (HEE) region is a safe and reliable way to make sure all uploaded evidence has been considered and documented for each trainee (Figure 1 ). One or more panel members can be delegated beforehand to complete this. Checklists to assist and confirm that all evidence has been considered can be invaluable not only as evidence for potential appeals but also to maintain focus particularly when distracted or interrupted (8) (9) (10) . It would be naïve to think that panel members do not become distracted when not actively participating in discussions, particularly when attenting virtual meetings. It is all too easy to simultaneously multitask by using a smart phone (11, 12) , clicking on an internet link or catching up on emails while 'out J o u r n a l P r e -p r o o f of view' of other panel members. However it should be remembered that performance is undoubtedly reduced when distracted and could even be a source of missing some vital discussion points (13) . Many will be familiar with the 'what do you think Dr X?' question which comes unexpectedly, giving a startle effect and often embarrassment about how to answer. Background noise can also reduce both concentration and performance (14) so for this reason we recommend the use of headset when attending virtual meetings (6) as well as turning off the microphone when not speaking. Finally, it is a good idea for one panel member to be the designated 'note taker' which can be referred to for areas of concern or further discussion, as well as ensuring that any required actions are minuted. This is particularly important for adverse outcomes and can be used to help guide post ARCP discussions with trainees. In some schools and training schemes (such as core training, radiology, orthopaedic surgery), with many trainees to assess, the ARCP process can last for more than one day. Maintaining focus, concentration and giving the same attention to the final trainee as the first demands that panel members are optimised to give the best performance possible. It is therefore important to incorporate regular breaks throughout the day, especially since concentrating during remote meetings can be harder than face-to-face discussions for the reasons mentioned above. Unlike discussions in a room, it is harder to look for and recognise signs of fatigue in panel members. We recommend a minimum 10-minute break every 90 minutes, with a longer break of at least 20 minutes after three hours (6) . During this time, it is important to walk away, rehydrate and spend a short time relaxing in whatever way works for the individual. During the summer months, it is easy to become dehydrated, which can result in cognitive impairment, changes in mood, and loss of interest (15, 16) . Regular comfort breaks make longer virtual meetings much more enjoyable and improve efficiency, paradoxically leading to an earlier finish time! Finally, a review or discussion summary by the ARCP panel chair is a good way of ensuring that all relevant aspects have been covered while providing an opporutnity for panel members to raise any additional points, concerns or highlights. Table 1 gives a summary of recommendations for improving the virtual ARCP panel experience. There are differences in practice across training rotations and schools for providing trainee feedback following the formal, summative ARCP process, and the conduct of yearly educational reviews. As previously mentioned, the ARCP outcome should not come as a shock for trainees who should have been briefed as to the likely result by their ES before the formal panel meeting as documented in the Gold Guide (17). Even with an ARCP outcome 1, trainees appreciate receiving feedback from educators and trainers about their performance. With adoption of virtual meeting s over conventional face-to-face sessions during the COVID-19 pandemic, not unsurprisingly, there are few published studies that have evaluated this method of feedback. In one study of medical student feedback (18) , no significant difference was found in overall experience, stress levels or helpfulness between these two J o u r n a l P r e -p r o o f modalities. However, students who met in-person with their trainers rated communication effectiveness significantly higher than remote meeting. One advantage of remote meeting was convenience and reduced travel. Given that many training schemes involve rotations to hospitals that can be considerably far apart, some UK trainees have complained about travelling several hours to attend an ARCP only to spend less than 15 minutes receiving their outcome (19) with little feedback (20) . Virtual ARCPs could therefore be of considerable benefit to some trainees, as well as to panel members who might also find it easier than travelling long distances to attend. We will not consider the various feedback techniques that can be used, but focus on the use of remote technology. This should ideally be supplemented with a face-to-face meeting if at all possible. As with giving any feedback, it should be done in an unhurried manner, with the minimum stress for the trainee. Hierarchy should ideally be reduced at the outset with introductions and some non-related discussion to try and put the trainee at ease. In our opinion, one of the biggest barriers for virtual feedback to be as good as in person is the relative lack of non-verbal communication, body language, and other clues that are readily visible during a face-to-face meeting. Therefore it is important to ensure that the trainee understands what is being said, and the careful and precise use of verbal communication is Figure 1 -A standard checklist used to ensure that process is followed during ARCP. Telepsychiatry and integrated primary care: setting expectations and creating an effective process for success Getting with the times: a narrative review of the literature on group decision making in virtual environments and implications for promotions committees What is a virtual multidisciplinary team (vMDT)? Telepresence in videocommunications: a study on stereoscopy and individual eye contact Does the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination predict 'on-the-job' performance during UK higher specialty surgical training? Human factors recognition at virtual meeting and video conferencing: how to get the best performance from yourself and others Video conferences through the internet: how to survive in a hostile environment The frequency and impact of task interruptions in the ICU Interruptions and distractions in healthcare: review and reappraisal The frequency and impact of task interruptions in the ICU Does Talking on a Cell Phone, With a Passenger, or Dialing Affect Driving Performance? An Updated Systematic Review and Meta-Analysis of Experimental Studies Analysis of Smartphone Interruptions on Academic General Internal Medicine Wards. Frequent Interruptions may cause a 'Crisis Mode' Work Climate Human factors awareness and recognition during multidisciplinary team meetings Background noise lowers the performance of anaesthesiology residents' clinical reasoning when measured by script concordance: A randomised crossover volunteer study The Hydration Equation: Update on Water Balance and Cognitive Performance Looking after ourselves at work: the importance of being hydrated and fed Randomized Evaluation of Videoconference Meetings for Medical Students' Mid-clerkship Feedback Sessions Reviewing the ARCP process: experiences of users in one English deanery It was all in your voice' -Tertiary student perceptions of alternative feedback modes (audio, video, podcast, and screencast): A qualitative literature review Trainees dissatisfied with "tick box ARCP", researchers find