key: cord-0785347-vs8l16f7
authors: Goodman, Christopher D.; Pautler, Justin D.; Balestrini, Christopher S.; Cobos, Santiago; D’Souza, Leah; Eansor, Paige; Jaswal, Jasbir; Nichols, Anthony; Norris, Madeleine; Sharma, Manas; Willmore, Katherine; Warner, Andrew; Murrell, Donna H.; Palma, David A.
title: Evaluation of a 3D-Printed-Head Simulation Technique for Teaching Flexible Nasopharyngoscopy to Radiation Oncology Residents
date: 2020-09-04
journal: Int J Radiat Oncol Biol Phys
DOI: 10.1016/j.ijrobp.2020.08.063
sha: 0e35a5a578a537d19891184e12a010a85bb91808
doc_id: 785347
cord_uid: vs8l16f7

Purpose Simulation-based medical education (SBME) is an effective tool for medical teaching, but SBME deployment in radiation oncology (RO) is limited. Flexible nasopharyngoscopy (FNP), an essential skill for RO residents, requires practice that typically occurs on volunteer patients, introducing the potential for stress and discomfort. We sought to develop a high-fidelity simulator and intervention that provides RO residents the opportunity to develop FNP skills in a low-pressure environment. Methods CT images were utilized to create an anatomically-accurate 3D-printed model of the head and neck region. An intervention incorporating didactic instruction, multimedia content, and FNP practice on the model was designed and administered to RO residents attending the Anatomy and Radiology Contouring Bootcamp. Participants completed pre- and post-intervention evaluations of the training session and model fidelity, and self-assessments of FNP skill and confidence performing FNP. Participants were video-recorded performing FNP pre- and post-intervention. Videos were scored by a blinded observer on a pre-defined rubric. Changes in scores were evaluated using the Wilcoxon signed rank test. Results Twenty-four participants from 17 institutions and 4 countries completed the intervention, 50% were female, and most were senior residents. Post-intervention, FNP confidence and FNP performance improved significantly (mean ± SD on a 10-point scale: 1.8 ± 1.8, p < 0.001; 2.2 ± 2.0, p < 0.001 respectively). Participants felt the model was helpful (mean ± SD on a 5-point scale: 4.2 ± 0.6), anatomically correct (4.1 ± 0.9), and aided in spatial comprehension (4.3 ± 0.8). Overall satisfaction intervention was high (4.3 ± 0.8). Participants strongly agreed the intervention should be integrated into RO training programs (4.3 ± 0.8). Conclusions A 3D-printed model and associated intervention were effective at improving FNP performance and the teaching method was rated highly by participants. RO residents may benefit from broader dissemination of this technique to improve trainee performance.

Simulation-based medical education (SBME) is a key contributor to quality health professional training. SBME deployment in contemporary medical residency curricula has increased dramatically, reflecting the demonstrated efficacy of its use in improving the clinical and procedural competence of learners compared to traditional instructional methods.(1,2) SBME is most commonly utilized to improve procedural skills with the most effective interventions incorporated directly into curricula and involving deliberate practice and feedback in a non-clinical environment. (3) Effective interventions require high-fidelity simulators custom-designed for the particular skill. Threedimensional (3D) printing has emerged as a method for producing high-quality, inexpensive teaching aids that have demonstrated efficacy over traditional materials. Their use in improving procedural performance has been established secondary to the ability to precisely reconstruct intricate anatomical structures. (4) SBME use in radiation oncology (RO) is less widespread than in many other specialties. The majority of reported interventions focus on improving contouring skills via screen-based simulators (5) despite the fact that Radiation Oncologists are expected to maintain several procedural skills in the modern clinical environment. Flexible nasopharyngoscopy (FNP) is essential to the diagnosis, treatment, and surveillance of head and neck (HN) cancer patients. Mastery of this skill requires practice that typically occurs in outpatient clinics on volunteer patients, introducing the potential for stress and discomfort. A recently reported simulation workshop designed to teach FNP demonstrated an improvement in confidence and procedural expertise when administered to RO residents. (6) We sought to develop a novel intervention incorporating a high-fidelity 3D-printed simulator that provides RO residents the opportunity to develop FNP skills in a low-pressure environment.

A Radiation Oncologist identified a suitable model patient based on ease of performing FNP. A computed tomography (CT) scan of the patient used for radiation treatment planning was anonymized and imported into 3DSlicer version 4.10.2 software (The Slicer Community, open source www.slicer.org) for model design. Once imported, the 2.5mm slice scan was viewed using a window width (WW) of 350 and window level (WL) of 40. After rendering the bulk volume from the CT scan, volumes were cropped using the Crop Volume module for the creation of segmentations.

The nasal cavity required adequate diameter to allow scope passage (scope diameter = 3.6 mm) while maintaining the anatomical integrity of the model. Custom local thresholds based on the Hounsfield scale were used to generate segmentations, ensuring a viable path for the instrument. Slices were manually edited to correct irregular discontinuities on the surfaces of the nasal cavity and outer shell, and to correct abnormalities related to dental artifacts. Manual edits were also used to reshape the exterior surface of the nose and nostrils.

After initial testing, further manual edits were made to allow for additional widening of regions identified as barriers for passing the instrument. To conserve the use of filament and reduce printing time, a large hollow region was created within the head in areas not visible to the endoscope (Figure 1) . A supportive stand extending from the chin of the model was also added. 

The simulator was printed with 3 mm EcoTough polylactic acid (PLA) filament (www.filaments.ca) on a Lulzbot Taz 6 3D Printer. Due to print area size contraints, the model was designed and printed in two component pieces: the head and the neck. The GCODE was generated using Cura LulzBot Edition 3.6.13 software after the importation of stereolithography (STL) files, exported from 3DSlicer. The prints were scaled up to 1.6 times their original size to allow for improved instrument maneuverability. Both components were printed with main settings available in Table 1 . Remaining parameters used Cura default settings for the Lulzbot Taz 6 Standard Extruder. To allow for viewing of internal anatomy and easier facilitation of support material, the head print was paused at a level after the base of the nostrils had been completed (approximately z = 114). A thin piece of paper was cut and glued to the top of the incomplete model and, once dried, a modified GCODE resumed the print from this level. The head section was then separated at the level of the paper, cleaned and cleared of support material, then attached back together with hinges. The neck was printed upside-down to conserve more time and filament; once cleared of support material it was glued to the head, to create the complete final product. The final assembled model is shown in Figure 2 . Downloadable files for printing this 3D head are available at bit.ly/2z8XTWoHead (head) and bit.ly/2z3axpINeck (neck).

Participants for this study were recruited from RO residents attending the Anatomy, Radiology & Contouring (ARC) Bootcamp. Local research ethics approval and study consent were obtained.

Participants first completed a pre-intervention evaluation and were asked to perform FNP on a 3Dprinted simulator. FNP performance was recorded utilizing MobileOptx (MobileOptx LLC, Pennsylvania, USA) smartphone adaptors.

After the baseline evaluation, the participants completed a teaching intervention, incorporating didactic, multimedia and practical components of FNP. Didactic components included a review of relevant anatomy, discussion of flexible endoscope design, function, and operation, a systematic approach to FNP-aided physical examination of the upper aerodigestive tract, and tips for troubleshooting issues encountered while performing FNP. Participants also viewed the New England Journal of Medicine instructional video "Examination of the Larynx and Pharynx."(7) Finally, participants practiced on the 3D simulator, with feedback provided.

Following the intervention participants completed a post-test evaluation and repeat FNP recordings were obtained.

Evaluation questions included self-assessments of FNP skill and confidence performing FNP clinically (10point scale) and assessments of 3D-simulator fidelity and the effectiveness of the training intervention (5-point Likert scale). FNP recordings were reviewed by a single, blinded, expert FNP provider and scored on a 10-point scale using a pre-defined rubric. The expert reviewer is an experienced HN Radiation Oncologist who frequently performs FNP for diagnostic and surveillance purposes. The reviewer had experience performing FNP on the 3D-printed simulator prior to reviewing and scoring participant video recordings. Statistical analysis included comparison of pre-and post-intervention FNP scores to identify and quantify improvement in ability secondary to the intervention, using the Wilcoxon signed rank test, Wilcoxon rank sum test or Kruskal-Wallis test as appropriate. Additionally, pre-and post-test surveys were analyzed to assess simulator fidelity and attitudes towards performing FNP in a clinical setting. All statistical analysis was performed using SAS version 9.4 software (SAS Institute, Cary NC, USA), using two-sided statistical testing at the 0.05 significance level.

A total of 31 subjects consented to the intervention with 24 completing all aspects of the study and therefore available for analysis. Baseline characteristics are available in Table 2 . Participants were mostly senior residents many of whom had previously completed either a HN surgical or RO rotation. Most participants who had completed a HN rotation did not receive formal FNP training despite most performing the technique clinically. There was a wide variation in reported number of prior FNPs. Abbreviations: HN: head and neck; FNP: flexible nasopharyngoscopy; RO: radiation oncology FNP skill was objectively assessed pre-and post-intervention via blinded review of video recordings. Skill was evaluated using a standardized rubric (Appendix 1). Mean ± standard deviation (SD) preintervention score was 4.5 ± 2.0. Post-intervention mean ± SD score improved significantly to 6.7 ± 1.5 (p<0.001). This translated to a mean ± SD increase of 2.2 ± 2.0. The improvement in score did not depend on PGY level (P=0.261), whether or not the participant had previously completed a HN surgical rotation (P=0.480) or a HN RO rotation (P=0.300), or the number of prior FNPs (p=0.270).

Participants were asked to report perceived attitudes towards FNP in a clinical setting on a 10-point scale (0 = low, 10 = high). Participants reported low confidence at the time of their first-ever lifetime FNP (mean ± SD: 1.9 ± 1.6). At the time of the study, reported mean ± SD confidence was higher (5.5 ± 2.2), but participants endorsed some anxiety when performing FNP in a clinical setting (3.9 ± 2.1). Following the intervention, self-reported confidence significantly improved to 7.2 ± 0.9 (p<0.01). Participant confidence consistently improved regardless of PGY level (p=0.385) however those without a previous HN RO rotation had a significantly larger improvement in confidence (p<0.001). Participants with < 11 prior FNPs had the greatest improvement in confidence (mean ± SD increase: 3.3 ± 2.2) though a significant improvement in confidence was maintained even in those participants with ≥ 21 prior FNPs (p=0.016).

Participants evaluated various aspects of the intervention on a 5-point Likert scale (1=strongly disagree, 5=strongly agree). Participants agreed that the model accurately simulated the size and shape of human anatomy (mean ± SD: 3.8 ± 0.9) and was anatomically correct (4.1 ± 0.9). Realistic passage of the scope (2.9 ± 1.1) and simulation of human tissue (2.6 ± 1.1) were identified as potential areas for improvement. There was agreement that the model was helpful for learning FNP (4.2 ± 0.6) and should be promoted as a standard component of RO training (4.3 ± 0.8). Participants strongly agreed the intervention was a positive learning experience (4.7 ± 0.5) and would recommend the session to others (4.7 ± 0.6). Mean ± SD overall rating for the intervention (including didactic and practical components) was 8.2 ± 1.5 on a 10-point scale (0 = not beneficial, 10 = extremely beneficial).

SBME use is uncommon in RO yet there are numerous clinical and professional competencies that could benefit from simulation-based training. No participants in our study reported receiving formal training during HN rotations despite most performing the technique clinically. Instead, residents typically learn FNP in suboptimal, high-stress environments often under close scrutiny by patients, their families, and multiple members of the care team. As a direct result, provider and patient discomfort and the potential for iatrogenic injury are heightened. We report outcomes from an intervention that utilizes a customdesigned 3D-printed simulator and associated training session to teach RO residents FNP. The intervention was associated with significantly increased self-reported confidence performing FNP. When stratifying by number of prior FNPs performed we noted a significant improvement in confidence regardless of past experience but participants with less scoping experience had a greater magnitude of improvement compared to those with more experience. Objective performance, scored by a blinded, expert-reviewer, improved significantly following the intervention by a mean of 2.2 points on a 10-point scale. The improvement in objective performance was not dependent on PGY level or past experience. These findings support the model accurately simulating clinical FNP experiences. It allows for all trainees to improve by providing a systematic approach to the upper airway exam alongside deliberate practice with expert feedback. It also enables a safe, interactive learning environment for less experienced trainees to gain valuable confidence performing a highly specialized and clinically important technique without any risk to patients. This is reflected in the consistently high ratings garnered from participants when asked to evaluate the intervention.

As SBME use becomes more widespread, an increasing body of work has been published examining the hallmarks of effective simulations. (3, (8) (9) (10) In general, interventions should incorporate feedback and deliberate practice, use simulators of appropriate fidelity that are matched to the desired skills, integrate multi-modal learning aids, and rigorously measure their outcomes. The most effective simulations are incorporated as a standard component of a training curriculum. We integrated this intervention into the curriculum of the ARC Bootcamp -a 3-day intensive training course designed to assist RO residents in learning anatomy skills relevant to the modern practice of RO. (11) The session builds on anatomical knowledge gained in the course providing a bridge to clinical practice. This integration is important as effective simulations should complement clinical training rather than trying to supplant practice on real patients in clinical settings. This particular intervention is designed as a preclinical exercise allowing trainees with limited experience to gain the basic skills necessary to safely and competently perform FNP examinations prior to attending HN clinics.

An intervention designed to train RO residents to perform fiberoptic laryngoscopy has recently been described in the literature. (6) The authors created a two-phase approach that includes a workshop to provide an overview of HN anatomy and logistics of the exam. Participants subsequently perform fiberoptic laryngoscopy on a computer program and a mannequin. Post-intervention surveys demonstrated a significant improvement in mean HN anatomy knowledge and self-reported confidence performing laryngoscopy. In contrast, the present approach relies on a 3D-printed model which may confer several benefits including more accurate representation of the pertinent anatomical structures and reduced cost compared to simulation mannequins. In comparison to the present approach, the authors incorporated a follow-up period wherein participants who completed the workshop were later supervised performing FNP in a clinical setting and provided with immediate feedback. The addition of longitudinal feedback likely supports the efficacy of the intervention at improving FNP skill and should be considered for similar interventions in the future. Conversely, the authors based their conclusions on self-reported scores for confidence and HN anatomical knowledge. Our approach incorporates an objective measure of improvement in the form of scored video-recordings which strengthens the veracity of our conclusion that this intervention directly contributes to improved performance.

3D-printing is a novel manufacturing technology that utilizes an additive process to recreate intricate structures based on computer-generated models in a slice-by-slice fashion. The technology has been widely adopted for medical applications given the ability to leverage high-resolution CT imaging for model creation. (12) (13) (14) In radiotherapy, 3D-printing has been successfully used to create customized bolus (15) (16) (17) and patient-specific phantoms.(18) 3D-printed models are effective tools for medical education. In a double blind randomized controlled trial (RCT), medical students learning cardiac anatomy demonstrated significantly improved test scores when using 3D-printed models instead of cadaveric materials. (19) A further RCT demonstrated improved test scores for medical students learning skull anatomy on 3D-printed models versus cadaveric materials or traditional anatomy atlases. (4) 3D-printed models are cost-effective and can be manufactured quickly. Any institution capable of supporting the capital requirements for a 3D printer can create their own models based on source code which can be widely disseminated via the internet. Once the source files are obtained, commercial printing services can be leveraged to create models should a program not have access to an in-house printer. Given the affordable nature of most filaments, programs can easily print as many copies as required to fit their needs. The use of 3D-printed simulators is also a safe and effective way to facilitate training on invasive procedures with limited pathogen-exposure risk. During the COVID19 pandemic, there is a clear and urgent need for programs to leverage technologies such as these in new and innovative ways to continue to meet the educational needs of medical trainees. (20) This study is strengthened by its focus on high-quality SBME principles. We combined the ability to create a highly accurate simulator via 3D printing with a comprehensive training session incorporating direct feedback and rigorous outcome measurement. The use of smartphone recordings to measure objective performance strengthens our conclusion that the intervention was effective at improving FNP performance and suggests the intervention could translate to improved patient outcomes. There were some noted drawbacks to our design including the use of a single reviewer and the fact that the FNP smartphone attachments affected the balance of the endoscope which may have negatively impacted performance. Participants noted some deficits in the ability to pass the scope through certain regions of the model and also reported that the model did not accurately recreate the feel of human tissue. In the future, we aim to create a revised 3D-model which looks to improve endoscope maneuverability and plan to explore the use of different filaments to better simulate human tissue. These updated models are planned to include various pathological findings to enable more oncology-relevant training scenarios and more rigorous evaluation of participants. Future modules could incorporate training on communication of examination findings and scored assessments of examination skill based on the identification of landmark features or pathology. We have elected to make the present version of the 3D-model available via free download to promote the proliferation of this technique to more training programs. As experience with 3D-printed simulators grows within the RO community we aim to complete a multi-institutional study on the effectiveness of these and similar interventions to better characterize the optimal parameters for their incorporation into training programs. In addition, as the efficacy of these interventions is better characterized, there are opportunities to create 3D-printed simulators for other aspects of RO training. RO interventions that require invasive, uncomfortable patient contact or an understanding of complex 3D anatomy (ie. pelvic examinations, brachytherapy training) could benefit from purpose-built simulators that allow for deliberate practice in a low-stress learning environment and should be investigated as potential avenues for expanded 3D-printing use in RO.

In conclusion, a 3D-printed model and associated training intervention were effective at improving objective FNP performance and the teaching method was highly rated by participants. We believe there is excellent potential for the expanded use of SBME in RO and would advocate strongly for future interventions to be designed in accordance with the principles of high quality simulation interventions. Instructions: Evaluators are to avoid 1 point to each task required during nasopharyngoscopy procedures. 0 points will be awarded if the participant does not complete the task. There is no negative marking.

Tasks marked in bold should be examined at the time of video acquisition.

Completed For right-hand-dominant persons, the base of the scope with the direction level is held in the right hand, allowing manipulation of the level by the thumb. (The dial on the underside of the scope allows for upward and downward movement of the tip of the scope. This movement, combined with rotation of the scope allows for 360 degrees viewing of the airway.) Inspects the nares to determine which side of the nose is the most accessible to pass the nasopharyngoscope through for the examination. Uses the tripod technique to hold the nasopharyngoscope on the face while inserting into the nares to avoid undue trauma. Identifies the nasal passage along the floor of the nose, where the scope should be passed to enter nasopharynx. In the nasopharynx, visualizes: 1) all surfaces of nasopharynx 2) left fossa rosenmuller 3) right fossa of rosenmuller. Advances the scope down into the oropharynx. In the oropharynx, visualizes: 1) complete base of tongue, 2) left tonsil 3) right tonsil 4) complete vallecula (asks patient to stick out tongue) 5) complete pharyngeal wall. Advances scope into hypopharynx. In the hypopharynx/larynx, visualizes: 1) piriform sinuses (asks patient to puff out cheeks) 2) vocal cord movement (asks patient to say "eee"). Withdraws the scope carefully and safely.

Total Score (/10): ____________________ (Pass given to participants with scores > 5/10) J o u r n a l P r e -p r o o f 

Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis

Features and uses of high-fidelity medical simulations that lead to effective learning: A beme systematic review

A critical review of simulation-based medical education research

The role of three-dimensional printed models of skull in anatomy education: A randomized controlled trail

Simulation as more than a treatment-planning tool: A systematic review of the literature on radiation oncology simulation-based medical education

Development and implementation of an educational simulation workshop in fiberoptic laryngoscopy for radiation oncology residents

Examination of the larynx and pharynx

Research opportunities in simulation-based medical education using deliberate practice

Revisiting 'a critical review of simulation-based medical education research

Simulation in healthcare education: A best evidence practical guide. Amee guide no. 82

Evaluating the impact of a canadian national anatomy and radiology contouring boot camp for radiation oncology residents

Development of an innovative 3d printed rigid bronchoscopy training model

A three-dimensional pelvic model made with a three-dimensional printer: Applications for laparoscopic surgery to treat rectal cancer

3d-printed pediatric endoscopic ear surgery simulator for surgical training

Use of 3d printers to create a patient-specific 3d bolus for external beam therapy

Clinical implementation of 3d printing in the construction of patient specific bolus for electron beam radiotherapy for non-melanoma skin cancer

Fabrication of malleable three-dimensional-printed customized bolus using three-dimensional scanner

Development of patient-specific phantoms for verification of stereotactic body radiation therapy planning in patients with metallic screw fixation

Use of 3d printed models in medical education: A randomized control trial comparing 3d prints versus cadaveric materials for learning external cardiac anatomy

Using technology to maintain the education of residents during the covid-19 pandemic

J o u r n a l P r e -p r o o f