key: cord-0908285-ov82ae54
authors: Collins, Justin W.; Ghazi, Ahmed; Stoyanov, Danail; Hung, Andrew; Coleman, Mark; Cecil, Tom; Ericsson, Anders; Anvari, Mehran; Wang, Yulun; Beaulieu, Yanick; Haram, Nadine; Sridhar, Ashwin; Marescaux, Jacques; Diana, Michele; Marcus, Hani J.; Levy, Jeffrey; Dasgupta, Prokar; Stefanidis, Dimitrios; Martino, Martin; Feins, Richard; Patel, Vipul; Slack, Mark; Satava, Richard M.; Kelly, John D.
title: Utilising an Accelerated Delphi Process to Develop Guidance and Protocols for Telepresence Applications in Remote Robotic Surgery Training
date: 2020-11-06
journal: Eur Urol Open Sci
DOI: 10.1016/j.euros.2020.09.005
sha: aef02634c07da5dcf85e7cea22aa0c9b4fe1706f
doc_id: 908285
cord_uid: ov82ae54

CONTEXT: The role of robot-assisted surgery continues to expand at a time when trainers and proctors have travel restrictions during the coronavirus disease 2019 (COVID-19) pandemic. OBJECTIVE: To provide guidance on setting up and running an optimised telementoring service that can be integrated into current validated curricula. We define a standardised approach to training candidates in skill acquisition via telepresence technologies. We aim to describe an approach based on the current evidence and available technologies, and define the key elements within optimised telepresence services, by seeking consensus from an expert committee comprising key opinion leaders in training. EVIDENCE ACQUISITION: This project was carried out in phases: a systematic review of the current literature, a teleconference meeting, and then an initial survey were conducted based on the current evidence and expert opinion, and sent to the committee. Twenty-four experts in training, including clinicians, academics, and industry, contributed to the Delphi process. An accelerated Delphi process underwent three rounds and was completed within 72 h. Additions to the second- and third-round surveys were formulated based on the answers and comments from the previous rounds. Consensus opinion was defined as ≥80% agreement. EVIDENCE SYNTHESIS: There was 100% consensus regarding an urgent need for international agreement on guidance for optimised telepresence. Consensus was reached in multiple areas, including (1) infrastructure and functionality; (2) definitions and terminology; (3) protocols for training, communication, and safety issues; and (4) accountability including ethical and legal issues. The resulting formulated guidance showed good internal consistency among experts, with a Cronbach alpha of 0.90. CONCLUSIONS: Using the Delphi methodology, we achieved international consensus among experts for development and content validation of optimised telepresence services for robotic surgery training. This guidance lays the foundation for launching telepresence services in robotic surgery. This guidance will require further validation. PATIENT SUMMARY: Owing to travel restrictions during the coronavirus disease 2019 (COVID-19) pandemic, development of remote training and support via telemedicine is becoming increasingly important. We report a key opinion leader consensus view on a standardised approach to telepresence.

This paper is dedicated to the memory of Professor Ericsson who contributed to this project and whose research work focused on the perfection of practice in what was named "deliberate practice". This entails, agreed performance metrics on technique, immediate feedback, and clearly defined goals. His work has greatly impacted surgical training, and his contribution to a better understanding of how we learn and progress to expert levels has been successfully applied in multiple areas including sports and music. Unfortunately, Anders passed away earlier this year on June 17. His wisdom and friendship will be missed by all who knew him.

The concept of telepresence was present from the beginning of robotic surgery. The first teleoperated robotic systems developed by SRI International and the Defence Advanced Research Projects Agency (DARPA) resulted in the surgeon console systems we are now familiar with [1] . The impetus to develop these remotely controlled systems by DARPA was driven by the identified need to provide additional expertise in warzones to decrease morbidity and mortality. The principle of providing surgical expertise from a remote geographical location remains pertinent to learning curves, as well as emergency and "unfamiliar" situations, where the alternative is to convert to an open or laparoscopic procedure [2] . Historically, travelling proctors aimed to bring expertise and feedback to inexperienced surgical teams; however current travel restrictions limit this educational resource, and training often lacks standardisation and objective performance metrics [3] .

Despite this ideology that robotic surgery could enable dissemination of expertise through network development [2] , procedural training remains largely delivered via a master-apprentice model, with potential variabilities in both surgery and educator skills. The trainee is "signed off" as competent after a suitable period of time. Subjective assessments of surgical performance have been shown to be highly variable, with poor inter-rater reliability [4] . Learning of skills is more efficient when sustained deliberate practice (SDP) is enabled [5] . This requires skills to be defined with objective metrics of performance, which are agreed on by both the trainer and the student [3] . SDP states that repetition of skills with deliberate practice is key to success and that the defined metrics should be able to be replicated in laboratory settings or training environments [5] . The combination of systems thinking with a proficiency-based progression (PBP) approach delivers consistent feedback and reduced errors in aviation training [6] . A complimentary strategy to drive standardised training, with a top-down approach, is the "train-the-trainer" (TTT) courses, where trainers learn about the curriculum structure, training protocols, standardised assessment, and how to deliver feedback safely [3] .

It is recognised that errors are more common early in the surgeons' learning curve, and the combination of simultaneously learning about both technology and technique has inherent patient safety risks [7, 8] . With growing awareness of the benefits of standardised training, there are an increasing number of validated training curricula with defined metrics of surgical performance, endorsed by societies and governing bodies [9] [10] [11] [12] . Whilst different trainees benefit from varying levels of support, there are recognised weaknesses to the current gold standard. third-round surveys were formulated based on the answers and comments from the previous rounds. Consensus opinion was defined as 80% agreement. Evidence synthesis: There was 100% consensus regarding an urgent need for international agreement on guidance for optimised telepresence. Consensus was reached in multiple areas, including (1) infrastructure and functionality; (2) definitions and terminology; (3) protocols for training, communication, and safety issues; and (4) accountability including ethical and legal issues. The resulting formulated guidance showed good internal consistency among experts, with a Cronbach alpha of 0.90. Conclusions: Using the Delphi methodology, we achieved international consensus among experts for development and content validation of optimised telepresence services for robotic surgery training. This guidance lays the foundation for launching telepresence services in robotic surgery. This guidance will require further validation. Patient summary: Owing to travel restrictions during the coronavirus disease 2019 (COVID-19) pandemic, development of remote training and support via telemedicine is becoming increasingly important. We report a key opinion leader consensus view on a standardised approach to telepresence. Notably, even after the trainees have completed the training modules, they still lack experience and are confronted with a pathology, anatomical abnormality, or clinical situation that they are unfamiliar with. To improve training, we need access to expertise when required. Operative approaches should be agreed between the trainer and the trainee, with objective performance metrics that enables PBP training. Defining standardised metrics enables us to contrast and compare alternative approaches to training that aim to achieve equivocal training outcomes. Digitalisation of training will collect data to inform whether novel approaches are equivocal, better, or worse. Telepresence has the potential to deliver expertise locally, affordably, and by avoiding travel limitations. Despite the significant potential of telepresence, there remains a lack of standardised guidance for telepresence in surgery. This Delphi process aims to define the infrastructure, communication protocols, and accountability related to delivering an optimised telepresence programme.

This project consisted of three phases, where each phase informed the subsequent phase. First, the available evidence was reviewed, which then informed the Delphi consensus questionnaire development. The consensus process then resulted in the formulation of the guidance.

A systematic review of the current literature and international protocols for telecommunication in both the health care and the aviation industry was completed independently by three individuals (A.G., A.H., and J.C.). The systematic review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement [13] . The authors reviewed current published literature on PubMed, Scopus, and Web of Science databases for full-text English-language articles published between 1995 and 2020, using the key words "telepresence", "telementoring", "telesurgery", "minimally invasive surgical procedure", "robotic surgical procedure", "education", and "distance". Additional significant studies cited in the reference list of selected papers were evaluated. The reviewers independently selected papers for detailed review after evaluating the abstract and, if necessary, the full-text manuscript. Potential discrepancies were resolved by open discussion. The electronic search yielded a total of 6753 potential articles. Figure 1 summarises the selection process. Multiple prospective studies were identified, which confirmed both feasibility and benefits from telementorship and telesurgery programmes [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] . Overall, the quality of available studies was moderate to low. Available evidence consists largely of expert opinion, consensus statements, and small qualitative studies. No publications were identified that focused specifically on guidance for setting up and running a telepresence service for robotic surgery training. Ninetytwo articles were selected by the core team and were placed into three categories related to telepresence: infrastructure, protocols, and accountability.

An advisory panel was formed, which comprised global key opinion and industry leaders with a specialist interest in robotic surgery training, telepresence, robotic network development, and communication in education and training. In total, 24 experts from the USA and Europe were brought together to discuss and develop guidance on telepresence related to the three areas of interest. The medians (range) for panel members' h index and i10 index were 29 (9-96) and 69 (9-297), respectively. The panel was chaired by Dr. Justin Collins with 12 presentations on the current evidence (see the Supplementary material).

Following the teleconference, the Delphi process was conducted to drive consensus among the experts. An Internet survey (Google forms) was generated based on the current literature and expert opinion, and sent to the 24 committee members. The Delphi was divided into three sections related to infrastructure, protocols (including training techniques), and accountability (ethical and legal issues). The Supplementary material shows a full list of the survey questions. An accelerated e-consensus reaching exercise, over 3 consecutive days, using the Delphi methodology was then applied. The Delphi method structures group communications so that the process is effective in allowing a group of individuals to deal with a complex problem. Questions in which there was 80% consensus were removed from the next round of the survey. Repeated iterations of anonymous voting continued over three rounds, where an individual's vote in the next round was informed by knowledge of the entire group's results in the previous round. To be included in the final recommendations, each survey item had to have reached group consensus (80% agreement) by the end of the three survey rounds. In the Delphi process, the finding of "consensus" is more relevant than the level of consensus. Reliability of the formulated guidance was evaluated using Cronbach alpha to assess internal consistency among experts. Levels of consensus are reported in the Supplementary material.

We had 100% (24/24) response rate from the committee in all three rounds. After three rounds of Delphi surveys, consensus was obtained in 88 of 99 questions posed in the following three categories:

1 Section 1: infrastructure and functionality requirements 2 Section 2: telepresence protocols and terminology 3 Section 3: accountability (ethical and legal guidance)

There was 100% agreement within the panel that there are both potential benefits and risks to the utilisation of telementorship (telepresence) and 100% agreement on the following general aspects of developing a telepresence service for surgical training: there is currently a lack of international standardised guidelines (or guidance) on the use of telepresence in the setting of surgical training, the future success of telepresence in surgical training will require its safe deployment within health care organisations, and a stated goal of this group is to identify the safety implications and to formulate guidance on telepresence protocols in surgical training. Supplementary Tables 1-13 summarise the various elements of the Delphi process that reached 80-100% agreement and the levels of agreement reached.

There was consensus that 5G, virtual private networks (VPNs), WiFi, and cloud networks could all be potentially used for providing telepresence services. The minimal infrastructure requirements and functional elements needed to deliver a safe and effective telepresence service are summarised in Table 1 , showing both consensus level of agreement and level of importance on a scale of 0-5, where 0 is not required and 5 is essential.

The committee agreed that ideally the trainer and the trainee should have an opportunity to meet in person to prepare for teleproctorship, if not limited by travel restrictions, distance, or costs. There was consensus that the telementor/teleproctor should be able to access electronic patient records and imaging of the patient prior to both elective and unplanned (emergency dial-in) telementorship. It was agreed that there is an ethical obligation to have standardised protocols for both elective and unplanned (emergency) telementorship, and there was 100% consensus that standardised performance metrics aid in assessment. The committee agreed that telepresence services would be benefited by developing real-time metrics for telementoring, linked to automated data collection [25] . Methods to achieve this include video performance analysis software, telemetry, eye-tracking software, and analysis of operating room (OR) times [25] [26] [27] [28] . However, there was consensus that both telemetry and eye tracking data may not be mature and standardised enough to be integrated yet, and would benefit from further research. The committee also agreed that standardised performance assessments should include reporting of 30 and 90-d complications. It was agreed that successful completion of a TTT course [3] is essential for the safe implementation of telementoring in robotic training.

Agreement was reached on general terminology related to telepresence services and that descriptive terms of surgical gestures or language commonly used by surgeons and trainers during surgery should be standardised during telementoring. It was recognised and agreed that communi-cation between the mentor and the trainee during telementoring should be standardised, and that both the mentor and the trainee should verify and confirm agreement on these communication cues, prior to commencing telementorship. Table 2 summarises the agreed general terminology terms and audio communication terminology to be used. Image overlay 3

View of the surgeon's hands 3

Ghost instruments OR = operating room. The committee agreed that there was significant potential for telepresence services in multiple settings. Table 3 summarises the potential areas of training that could be impacted positively. Agreement was reached that the telementor/teleproctor should decide on a time with the trainee to discuss about the patient prior to elective telementorship. Important elements that need to be discussed include previous surgeries, comorbidities, body mass index, investigations, and tumour staging, as well as going through the robotic device, equipment, and instruments available. The committee recommended that telementorship should incorporate the Briefing, Intraoperative Teaching, Debriefing (BID) model as a teaching strategy [29] , which involves (1) briefing-setting the learning objectives before the start of surgery, (2) intraoperative teaching according to agreed technique, and (3) debriefing-simulating reflection on the learner, teaching rules, reinforcing what was right, and correcting mistakes.

For elective telepresence, the telementor should be present for the World Health Organization checklist time out [30] and for the port placement.

The committee reached 100% consensus that trainers using telepresence should undergo a TTT course that explains how to use the technology and to learn telepresence protocols, and 100% consensus that this required a standardised approach to key index procedures that is familiar to both the trainer and the trainee and that uses objectively defined metrics to train, test, and measure surgical performance. These standardised objectively defined metrics to train, test, and measure surgical performance should be familiar to both the trainer and the trainee, clearly defined, and agreed on, prior to commencing telementorship training (See Fig. 2) .

It was agreed that surgical procedures should undergo task deconstruction to identify key tasks to be completed and errors to be avoided. Defining the phases of a procedure allows the trainer to identify quickly which part of the procedure the trainee is performing, as well as enables training to be completed with a modular approach with benchmarked progression. It was also agreed that training on singular skills tasks, such as suturing or knot tying, is useful in assessing proficiency. The committee were tasked with defining metrics that would be ideal for telepresence. The consensus views are summarised in Table 4 . There was 

Do you agree that telepresence has potential to be effective in various settings?

100% * Elective telementorship * Self-reflection and assessment, where the trainee reviews his/her own surgical performance 95%

* Elective teleproctorship * Emergency (unplanned) telementorship, where the patient is stable, but the inexperienced surgeon is presented with an unfamiliar scenario 90%

* Immediate feedback delivered to the trainee, after the surgery has been completed, but not in real time * Team training video performance analysis (not in real time) 85%

* Team training in real time

also 100% consensus that the metrics that define optimised surgical performance will change over time with advancements in knowledge, device development, or other technological advances (see Fig. 3 ). The committee agreed that video performance analysis with delayed feedback has the greatest impact on learning when delivered to the trainee immediately after completing the task. The group recommended that the number of elective cases booked for remote training should be related to the trainee's performance and unrelated to case numbers, prior to the trainee being signed off as proficient. For final assessment via teleproctorship, there is potential to assess either the whole procedure or the key procedural phases and tasks, to confirm proficiency. However, the committee agreed that this aspect requires further research before partial evaluation can be used.

Several modes of verbal communication have been described during teaching in the OR. The committee rated which types of verbal communications are most suitable for telementorship (Table 5 ). Regarding teaching models based Table 4 -Classification of the surgical procedure and related performance metrics

Elements of surgical performance that can be used for enabling deliberate practice and measuring objective performance Phase A section of the procedure, with a clearly defined start and end point. Phases of a procedure enable modular training, with training commenced in the easier phases with a lower frequency of hazard steps Visual cue A visual cue such as an anatomical landmark or areas of interest that is defined as important and therefore needs to be identified within a given phase of the procedure Task Defined steps to be completed within a phase of the procedure. Tasks can be further deconstructed into manoeuvres and surgical gestures [25] Technique error An error of the technique that may or may not be associated with an event (eg, using a wrong instrument to grasp a bowel). Technique errors may be associated with near misses and have the potential to enable early warning systems Event error An error of the technique or a device error that results in a harm to tissue or the patient Robotic device-related metrics Device error Device malfunction or failure Telemetry Automated performance metrics generated by the robotic device related to kinematic data Which modes of verbal communication should be used in the OR? 4 * Pure teaching (intended primarily to benefit the learner through providing educational value) * Instrumental and teaching (intended to achieve the pragmatic goal of moving the case forward while also conferring teaching) 3

* Instrumental (goal of interaction is to move the case forward; termed instrumental because the surgeon often uses the learner like an instrument, as a means to an end) 1

* Banter (conversation unrelated to the procedure) Which modes of verbal guidance should be used in the OR? 4

* Explanatory (the attending surgeon discusses the rationale behind a particular step of the surgery) * Didactic (the attending surgeon comments descriptively about the current step or relevant anatomy) * Commanding (the attending surgeon instructs the surgical trainee what to do next with a highly specific command) * Quizzing (the attending surgeon uses Socratic-like questioning to assess the surgical trainee's knowledge, often introducing a discussion of that topic) 3

* Deictic (the attending surgeon uses words that are "flexible" in meaning and usage, points to a specific referent that the attending surgeon assumes that the surgical trainee is aware of, and relies on the context to be interpreted correctly) * Figurative (the attending surgeon uses a verbal analogy to describe a surgical step) * Utterance (the surgeon delivers a monosyllabic order, usually used to correct, direct, stop, encourage, or praise the surgical actions) OR = operating room.

on cognitive apprenticeship principles, the committee rated which types of practices are most suitable for telementorship; these are summarised in Table 6 (see Supplementary  Tables 14 and 15 ).

The group concluded that a major potential benefit of telepresence is the opportunity to share insight and knowledge at local, national, and international levels; however, it was also agreed that there are potential risks to using telehealth technology for assessment and treatment of specific cases. There was 100% consensus that surgeons have responsibility to develop processes to enable safe introduction of MedTech, with the required expert support available when needed, and that clear international guidance needs to be created to establish medicolegal responsibilities in the context of telehealth consultations between clinicians. There was further agreement that clinicians using telehealth technology must abide by the guidance of good medical practice and laws established in the country where they are practising. The group also agreed on the following general guidance (Table 7) . 1 There should be agreed standards around anonymising the patient data to protect privacy when data are collected for research purposes. 2 Practical data/findings derived from telepresence technologies should ideally be anonymous or aggregate data, before they are made available for research teams and industry to the benefit of society. 3 All data that are planned to be collected should be approved proactively and stored according to guidelines from the organisation's data protection office. 4 Organisations should aim for data minimisation: review what data you have and why. Capture only the minimum amount of data that you need. 5 Processing data has to be done for a specific lawful purpose that the user has agreed to and has to match up with how it is described. 6 Storage limitations: data that are no longer required should be removed. If kept for longer than needed, data should be pseudonymised to protect user's identity (pseudonymisation: the deidentification procedure by which personally identifiable information fields within * Coaching (the mentor observes the student performing the task, offering tips and pearls to bring the student's performance closer to expert performance) * Exploration (the mentor stimulates the student to move towards independent practice by setting general competencies [goals] and encouraging focus on personal interests and individual goals to overcome any weaknesses and build on strengths) 4

* Reflection (the mentor encourages the learner to compare his/her thought processes with those of an expert, a rule, or some other standard to help the learner develop an awareness of his/her strengths and weaknesses) * Articulating (the mentor stimulates the learner to explain his/her knowledge, clinical reasoning, and decision-making strategy) * Modelling (the expert demonstrates the task; the expert verbalises his/her thought processes while performing the task, explaining his/her clinical judgement and reasoning) * Scaffolding (the mentor diagnoses the student's current skill level and the appropriate level of difficulty of the target activity. The expert provides support for parts of the task the student cannot yet manage independently; over time, the support is gradually faded) Table 7 -General guidance for telepresence service deployment 1 Hospitals and organisations providing telepresence services should follow agreed standards about the infrastructure required and follow agreed protocols about the service delivery.

2 Successful completion of a TTT course with benchmarked assessments, which explains these protocols, is a prerequisite before the trainer can commence remote training via telepresence.

3 Informed consent needs to be acquired from patients when telehealth modalities are used for assessment, management, and treatment purposes, and for the purpose of recording telehealth consultations.

4 Healthcare organisations would benefit from generic consent forms that ask patients to consent to the use of telepresence technologies, to collect video and data for supporting surgical technique, as well as for audit and research.

5 Parties to be approached to sign informed consent for a telepresence service include the (1) hospital/trust organisation, (2) patient, (3) surgeon, and (4) whole of the OR team if they are being videoed.

6 Patients have the right to refuse to undertake an assessment or receive treatment via a telehealth platform. 7 Before embarking on the development of telepresence services, it should be clear from the onset who will be responsible for the patient in case harm is caused during telementorship. In most cases, this should be the lead surgeon who is physically present in the operating room [43] .

8 Clinicians need to maintain accurate and complete records of any telehealth consultations that they perform. 9 Patients have the right to request access to the recording of their telehealth consultation. 10 The committee also agreed that the ethical issues regarding the evaluation of telepresence are reduced in a laboratory training environment that does not involve patients [38] . OR = operating room; TTT = train the trainer.

a data record are replaced by one or more artificial identifiers or pseudonyms. A single pseudonym for each replaced field or collection of replaced fields makes the data record less identifiable, whilst remaining suitable for data analysis and data processing). 7 In practice, data/findings derived from telepresence technologies should be, as much as possible, open label and made available for research teams and industry to the benefit of society. 8 Data integrity should be maintained. Processors should protect user data against unlawful processing or loss, ideally by having encryption of user data and privacy by design processes. 9 In the USA, voice recordings of key milestone events should be recorded in a HIPAA-protected manner during procedures. 10 According to organisational accountability under General Data Protection Regulation, an organisation is legally obliged to put into place comprehensive governance measures, privacy impact assessments, and privacy measures by design. 11 Organisations delivering telepresence services that collect data for research require a data protection officer (DPO). Everyone handling the data has shared responsibility to anonymise data where appropriate, but the DPO has overall responsibility for data protection compliance matters.

3.1.4.2. Dealing with difficult or struggling trainees. It was agreed that the "six-step" approach may be useful for safe telementorship delivery [31] . If telementorship utilised in training identifies a gap in knowledge or skills, a remediation programme should be developed and available for the trainees. In cases where failure of established protocol occurs by a trainee during mentorship, the remediation plan should be to ask the trainee to complete a review, with reflection and exploration of the reasons for divergence, with plans for future change. The review should check for understanding on the established protocol, prior to the next session. In cases where there is a failure of a trainee to follow clear instructions during telementorship, the trainee should be asked to stop immediately and given a warning that if he/ she continues not to comply with instructions, the telementorship session will be terminated by the remote trainer. In cases of repeated offences by the trainee, the connection should be stopped by the remote trainer and the reasons for closing the telepresence session should be documented clearly. If divergence from the agreed protocols by the trainee results in patient harm and the trainee continues not to follow instructions, then the telementorship session should be stopped by the trainer and the reasons for closing the telepresence should be documented clearly.

It was agreed on that there is a need for developing international guidance for clinicians, to evaluate the risks and benefits of employing telehealth technologies in specific cases. The development of a network of experts in telepresence protocols will aid in the standardisation of data collection and data labelling. There should be guidance on data aggregation strategies when video performance analysis data are combined with other forms of data.

Regarding novel technologies, there was agreement that eye tracking on the screen using head-mounted eye trackers would likely be beneficial in telementoring and that realtime automated performance feedback in training, driven by artificial intelligence, is both viable and ethical.

The coronavirus disease 2019 (COVID-19) pandemic has resulted in global guidance that restricts travelling proctors and limits training in the OR [32] . Telementoring has existed in various forms for >20 yr [33] and has been adopted slowly despite indications that it both is feasible and can impact patient outcomes positively [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] . Robotic surgery provides an ideal environment for surgical telementoring and telesurgery. Following the successful deployment of a robotic network in the early 21st century, with telementoring and telesurgery capabilities, there was much optimism that telepresence services would become common practice [20] . Despite robotic surgery having the concept of telepresence embedded from the start, the potential benefits of robotic surgery have not yet been realised at scale. Telemedicine adoption has been expedited greatly in many aspects of health care during the COVID-19 pandemic [34] . Necessity is a strong driver of change and is accelerating a growing interest in the synergies between telemedicine and minimally invasive surgery that are available in an increasingly connected world. The development of digital surgery is also resulting in new and efficient ways to train and objectively evaluate the surgeon's performance [25] [26] [27] [28] . The importance and positive impact of mentoring are well established in the literature [35] [36] [37] . Evaluation of telementoring in medical education has shown promising results [18, 19, 22] . Several studies have proved telementoring as an effective training tool. One study showed that residents in a telementoring group performed significantly better than a nonmentoring group (p < 0.001) [38] . Safety in telementoring has also been evaluated. In a systematic review comprising 11 studies, nine concluded that telementoring did not prolong surgery time compared with on-site mentoring, none of them reported increased morbidity, and only 3% of the total number of cases reported any technical issues [39] . Of note, a study conducted by Byrne and Mughal [40] included 34 telementoring cases of laparoscopic cholecystectomies. Results showed that no intervention was necessary in 68% of cases, verbal advice was given in 26% of cases, and in two cases the mentor had to come to the OR from his/her remote location and scrub into the case. The authors concluded that telementoring may be used as a bridge between on-site supervision and totally unsupervised performance. Telepresence aims to give extra levels of training support and has the potential to disseminate expertise and prevent errors. It remains an adjunct to best training practices [3] and should not replace local training support, until there is suitable evidence to indicate otherwise.

With better understanding of surgical learning curves and the ability to objectively score in performance levels, data collected via networks may also have a future regulatory role for surgeons [2, 3, 41] . Telementoring networks also have the potential to support deployment of novel technologies and drive standardisation in training via centralised education hubs, enabling dissemination of knowledge, without the need for mentor or mentee to travel. Standardisation is critical to developing cohesive networks with defined agreement between mentors and mentees [42] . The panel agreed that it was crucial to reach agreement on the procedure phases/tasks and that training aligns with previous experiences and performance. To achieve this, performance metrics need to be optimised for telepresence and aligned with prior training experiences, being aware of the benefits of a continuum of training enabling data flow from device training to basic skills to procedural outcomes and delivering data that will enable improvements to both technology and technique. Collaborative telementoring via robotic networks has the potential to drive advancement in multiple areas of robotic surgery through crowd sourcing and sharing of knowledge [2] .

In highly competitive health care systems, there is inherent resistance to sharing. With the current pandemic and the potential benefits from digitalised training, there are drivers in place to promote collaboration that will likely result in improved surgical outcomes for patients. If these benefits to surgical outcomes and improved patient safety using telementoring are realised, then legal, ethical, and reimbursement issues will likely be resolved.

Future studies should acknowledge the challenges of ethical and legal concerns, and the need to prioritise patient safety. The development of telepresence approaches to training will need careful evaluation and validation with predefined service goals.

Robotic training has traditionally been limited by various aspects such as access to expertise, integration of training into normal working patterns, and funding. Currently, travel limitations are also having a profound impact on training. Telepresence has the potential to deliver surgical expertise to underserved areas and to advance the reach of expertise to facilitate teaching of advanced surgical skills worldwide. Despite increased infrastructure requirements and the needs for retraining, telementoring in robotic surgery is becoming an increasingly practical and cost-effective option. However, significant challenges remain. Realisation of the potential of telementoring requires addressing the ethical and legislative issues, and collaboration with cybersecurity experts to ensure safety and trust. Using the Delphi methodology, we achieved international consensus among experts to develop and reach content validation for optimised telepresence services for robotic surgery training. These consensus views lay the foundation for the safe launching of telepresence services in robotic surgery. This guidance will require further validation. Funding/Support and role of the sponsor: This work was supported by CMR Surgical.

Robotic surgery: from past to future: a personal journey

Robotic networks: delivering empowerment through integration

Utilising the Delphi process to develop a proficiency-based progression train-the-trainer course for robotic surgery training

A proficiency-based progression (PBP) training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set

Deliberate practice and proposed limits on the effects of practice on the acquisition of expert performance: why the original definition matters and recommendations for future research

Training in robotic surgery, replicating the airline industry. How far have we come?

Adverse events in robotic surgery: a retrospective study of 14 years of FDA data

Top 10 health technology hazards for 2015. Health devices

Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts

Pilot validation study of the European Association of Urology robotic training curriculum

Outcomes from the Delphi process of the Thoracic Robotic Curriculum Development Committee

Recommendations for a standardised educational program in robot assisted gynaecological surgery: Consensus from the Society of European Robotic Gynaecological Surgery (SERGS). Facts Views Vis Obgyn

Transatlantic robot-assisted telesurgery

Feasibility of telementoring between Baltimore (USA) and Rome (Italy): the first five cases

Establishment of the world's first telerobotic remote surgical service: for provision of advanced laparoscopic surgery in a rural community

Telemedicine: an important aid to perform high-quality endoscopic retrograde cholangiopancreatography in low-volume centers

A model for longitudinal mentoring and telementoring of laparoscopic colon surgery

Mentoring and telementoring leads to effective incorporation of laparoscopic colon surgery

Remote telepresence surgery: the Canadian experience

Is telesurgery a new reality? Our experience with laparoscopic and percutaneous procedures

The role of telementoring and telerobotic assistance in the provision of laparoscopic colorectal surgery in rural areas

Novel telementoring system for robot-assisted radical prostatectomy: impact on the learning curve

Telementoring during endovascular treatment of abdominal aortic aneurysms: a prospective study

Use of automated performance metrics to measure surgeon performance during robotic vesicourethral anastomosis and methodical development of a training tutorial

Michigan Bariatric Surgery Collaborative. Surgical skill and complication rates after bariatric surgery

Classification approach for automatic laparoscopic video database organization

Eyetracking technology in medical education: a systematic review

The briefing, intraoperative teaching, debriefing model for teaching in the operating room

A surgical safety checklist to reduce morbidity and mortality in a global population

Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England

Global guidance for surgical care during the COVID-19 pandemic

International surgical telementoring using a robotic arm: our experience

Telemedicine during the COVID-19 pandemic: experiences from western China

A longitudinal examination of the influence of mentoring on organizational commitment and turnover

Mentorship in surgical training: current status and a needs assessment for future mentoring programs in surgery

Mentorship in surgical training: a systematic review

Telementoring versus on-site mentoring in virtual reality-based surgical training

Effectiveness of telementoring in surgery compared with on-site mentoring: a systematic review

Telementoring as an adjunct to training and competence based assessment in laparoscopic cholecystectomy

Consensus Meeting on European Robotic Training (OCERT): results from the first multispecialty consensus meeting on training in robot-assisted surgery

Globalization of surgical expertise without losing the human touch: utilising the network, old and new

The interstate telemedicine compact and the agenda of the Federation of State Medical Boards

Acknowledgements: We acknowledge the organisational role of UCL that helped co-ordinate the initial virtual meeting. No honoraria were paid to individuals participating in this research.

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j. euros.2020.09.005.