key: cord-0920976-r1pwkcfi
authors: Chawla, Shreya; Devi, Sharmila; Calvachi, Paola; Gormley, William B.; Rueda-Esteban, Roberto
title: Evaluation of simulation models in neurosurgical training according to face, content, and construct validity: a systematic review
date: 2022-02-04
journal: Acta Neurochir (Wien)
DOI: 10.1007/s00701-021-05003-x
sha: c68f1be091bdf9be7a7903d73ed52d10c64816ee
doc_id: 920976
cord_uid: r1pwkcfi

BACKGROUND: Neurosurgical training has been traditionally based on an apprenticeship model. However, restrictions on clinical exposure reduce trainees’ operative experience. Simulation models may allow for a more efficient, feasible, and time-effective acquisition of skills. Our objectives were to use face, content, and construct validity to review the use of simulation models in neurosurgical education. METHODS: PubMed, Web of Science, and Scopus were queried for eligible studies. After excluding duplicates, 1204 studies were screened. Eighteen studies were included in the final review. RESULTS: Neurosurgical skills assessed included aneurysm clipping (n = 6), craniotomy and burr hole drilling (n = 2), tumour resection (n = 4), and vessel suturing (n = 3). All studies assessed face validity, 11 assessed content, and 6 assessed construct validity. Animal models (n = 5), synthetic models (n = 7), and VR models (n = 6) were assessed. In face validation, all studies rated visual realism favourably, but haptic realism was key limitation. The synthetic models ranked a high median tactile realism (4 out of 5) compared to other models. Assessment of content validity showed positive findings for anatomical and procedural education, but the models provided more benefit to the novice than the experienced group. The cadaver models were perceived to be the most anatomically realistic by study participants. Construct validity showed a statistically significant proficiency increase among the junior group compared to the senior group across all modalities. CONCLUSION: Our review highlights evidence on the feasibility of implementing simulation models in neurosurgical training. Studies should include predictive validity to assess future skill on an individual on whom the same procedure will be administered. This study shows that future neurosurgical training systems call for surgical simulation and objectively validated models. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00701-021-05003-x.

Simulation is an educational technique where a trainee interacts with an environment that either recreates or replicates a real-world clinical scenario in a risk-free atmosphere [28, 30, 37] . Simulation models for medical education include human and animal cadavers and live animal models, An abstract including some of these findings was presented at the European Association for Neurosurgical Societies conference 2020.

This article is part of the Topical Collection on Neurosurgery Training synthetic/physical bench models, and virtual reality (VR) models [21, 30] . Simulation has been rapidly utilised in surgery, especially in high-income countries [65] .

Neurosurgical training has been traditionally based on a model of apprenticeship ("see one, do one, teach one") [12, 16] , where theoretical and practical learning take place in operating rooms (OR) [28, 30] . However, challenges associated with this apprenticeship system include financial constraints of the healthcare system and teaching in the OR [40] , restrictions on working hours [14] , worsening patient outcomes [52] , and limited clinical exposure and operation opportunities due to increasing ethical and medico-legal constraints [20, 55] . Neurosurgeons perform a mean number of 223 cases a year, which are widely varied and unique in technical competence [23] . Together with the restrictions on training hours, trainees might not encounter a similar procedure often, limiting opportunities for surgical training. This has been exacerbated by the COVID-19 pandemic, as seen by a marked reduction in elective and non-essential neurosurgical cases and redeployment of neurosurgeons to the intensive care unit, overwhelming even affluent healthcare systems [26, 56, 62, 67] . This has further reduced opportunities to perform neurosurgical procedures, impacting training [10] . Therefore, it is more relevant now than ever to continue developing simulation models to not only allow trainees to repeatedly perform procedures in a controlled environment, but also remotely (Fig. 1) .

Medical simulation serves as an alternative for time-effective acquisition of skills [2, 21, 22] , with the resultant shift of learning curve away from the patient [2, 21] . Simulation allows trainees to learn from and make errors in a safe environment as incorrect or technically demanding tasks can be performed to completion [17, 21, 30] . The efficacy of simulation on acquisition and development of technical skills is shown by improvement of objective performance metrics when skills learned from simulator-training are translated into the OR [16, 17, 21] . However, medical and surgical simulations must be evaluated as educational tools. Evaluation of simulation models is based on subjective and objective validation [68] . Face and content validity are types of subjective validity, evaluated by questionnaires. Face validity examines the realism of a simulator and difficulty level Fig. 1 Overview of the diverse range of available neurosurgical training tools. These educational approaches can generally be subdivided into physical models (including both biological tissues and synthetic constructs) and simulated approaches (which include both virtual and augmented reality) similarity in comparison to real training tasks, whilst content validity assesses the model's effectiveness during a specific skill training to improve participants' techniques [68] . Whilst these are subjective methods, construct validity is an objective method that considers the ability of a simulator to differentiate levels of skill competence [22, 47] .

Previous reviews have explored developing neurosurgical simulation-based training [16, 30, 49] , discussing the strengths and limitations of various simulation models. Simulations included in the reviews are also mostly limited to either physical or virtual reality simulators, showing that there is a gap in literature in assessing an extensive variety of simulator types. Furthermore, the type of neurosurgical procedures included is random, limiting our understanding of which neurosurgical skill or procedure can be best simulated by a specific simulator type. The ultimate goal of simulation is to ensure that skills learnt from the simulator can be transferred to the OR, inadequately explored in these reviews. Important work on simulation models is underway but current studies examining surgical simulation methods do not have a comprehensive approach to applying validation methods to assess simulator models. Whilst there is early cause for simulation to develop surgical skills, there is a gap in literature in establishing the use-case and quantifying the impact of surgical education on surgical skill acquisition and improvement.

This study, therefore, aims to reduce the gap in literature by reviewing simulation models in neurosurgical education and training for specific neurosurgical procedures: burr hole incision/craniotomy/craniectomy, aneurysm clipping, vessel suturing, skull-based tumour resection using face, content, and construct validity. These 4 skull-base neurosurgical procedures were chosen from documental analysis as they were most extensively evaluated in literature. These neurosurgical procedures are also part of the neurosurgical curriculum guidelines set by national associations including the Congress of Neurological Surgeons, American Association of Neurological Surgeons, General Medical Council, and Royal Australasian College of Surgeons.

A comprehensive search was performed in PubMed, Web of Science, and Scopus, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines up until September 2019 using predefined search terms. Search terminology for PubMed has been listed in Appendix 1. Identified abstracts and full-text papers were reviewed by two independent reviewers (SC, SD) against predefined inclusion and exclusion criteria to assess eligibility using Covidence® software [30] . A third reviewer (RR) resolved any discrepancy that occurred between the 2 authors after full-text screening. Our protocol has been published on PROSPERO under number: CRD42020141703.

Retrospective and prospective observational studies, randomised controlled trials, and case series that reported simulations used in skull-based neurosurgery education and/or training with reported outcomes were included. Case reports, systematic reviews, and articles written in languages other than English were excluded. Articles mentioning participants from any specialty and training level using the neurosurgical simulation models were included in the study. Articles that mentioned any neurosurgical simulation model for 4 such neurosurgical procedures (burr hole incision/craniotomy/craniectomy, aneurysm clipping, vessel suturing, and tumour resection) were selected. Studies that reported measurable outcomes with validated assessment tools of skill acquisition were included. Simulation models that did not report haptic feedback were excluded.

A pre-designed excel sheet was used to extract and organise data into categories by two independent authors (SC, SD). Data extracted included study characteristics (name of first author, year of publication, title, name of journal, study design), population characteristics, model characteristics (type of neurosurgical skill, neurosurgical subspecialty), and outcome evaluation (type of assessment tool, reported outcomes relating to face, content, and construct validity) (Tables 1 and 2 ).

The Medical Education Research Study Quality Instrument (MERSQI) was used for a methodological evaluation of included studies [18, 53] . MERSQI is a reliable tool created for the critical appraisal of medical education research [18, 63] . The maximum number of points scored on the MERSQI scale is 18 points.

This tool evaluates study design, sampling, data type, validity of evaluation, data analysis, and study outcomes. The MERSQI tool was used due to the high validity of this tool, assessed using 3 criteria. These included correlating global quality ratings from 2 independent experts in medical education research, examining the association between MERSQI scores and the impact factor of the publishing journal, and performing a simple linear regression to measure the association between MERSQI scores and citation rate and impact factor. This assessment found that total MERSQI scores were highly correlated with the median quality rating of the 2 independent experts (p = 0.73) and agreement between the 2 experts was excellent (ICC, 0.80; 95% CI, 0.49-0.85). The scores were significantly associated with a high 3-year citation rate and journal impact factor [53] .

A total of 1507 publications were initially identified across all searched databases. After removal of duplicates, 1204 studies were identified for title and abstract screening. Of these, 208 studies were eligible for full text screening and 18 studies were included in the final review (Fig. 2) . General characteristics of the study and the type of simulation and procedure are presented in Table 1 . Model characteristics assessed by face, content, and construct validity can be found in Table 2 . 

The neurosurgical skills assessed were aneurysm clipping (n = 6) [1, 13, 27, 39, 42, 70] , craniotomy and burr hole drilling (n = 2) [19, 46] , tumour resection (n = 4) [7, 25, 33, 69] and vessel suturing-vascular anastomosis (n = 1) [22] , vessel suturing-micro-anastomosis (n = 2) [6, 13] . Multiple neurosurgical procedures assessed by the studies included aneurysm clipping and craniotomy (n = 1) [1] and vessel suturing and aneurysm clipping (n = 1) [22] (Table 1) . These skills were simulated on animal models (n = 5), synthetic models (n = 7) [6, 7, 19, 39, 42, 60, 70] , and using VR (n = 6) [3, 25, 27, 46, 69, 72] . Animal models included cadavers (n = 2) [1, 22] , animal placenta (n = 3) [12, 13, 22] , and live rats (n = 1) [33].

All studies assessed face validity, 11 studies assessed content validity [1, 3, 7, 22, 25, 27, 33, 39, 46, 60, 69] , and 6 studies assessed construct validity of the models [6, 12, 13, 19, 25, 69] . Assessment of construct validity involved distinguishing the improvement of participants at different levels of experience and expertise. The mean MERSQI score of included studies was 10.5. Complete results of the MERSQI tool can be found in Appendix 2.

Aneurysm clipping in five included studies was simulated by a cadaver model [1] , placenta model [13] , 3D printed models (n = 3) [39, 42, 70] , and one VR model [27] (Table 1) . Belykh et al., 2017 [13] used the Objective Structured Assessment of Aneurysm Clipping Skills tool and Aneurysm Clipping Participant Survey, whilst other studies used either a 5-point [1, 27, 39] or a 4-point Likert scale questionnaire [42] . The scale used in all studies indicated that 1 referred to participants strongly disagreeing with the question, and the highest value indicating strong agreement.

All studies validated their models using face validity [1, 13, 27, 39, 42, 70 ] and 3 studies used both face and content validity [1, 27, 39] (Table 2 ). All studies conferred on participants either agreeing or strongly agreeing that the model was a true simulation of the aneurysm clipping in a surgical environment. Eighty to ninety-nine per cent of participants favourably reported that the model realistically simulated anatomy of aneurysm [1, 39] , whilst 89-100% of participants agreed that simulation models to train aneurysm clipping should be integrated in neurosurgical training [1, 27, 39] . Content validity was assessed via questionnaires on the physical aspects of the simulation model. Respondents ranked their understanding of structure and location of an aneurysm favourably as either "excellent" or with a median score of 4.8 out of 5 [42] . Compared to existing models of live animals for micro-anastomosis training, the models included were rated as superior by the majority of participants (97-99%) [1] . Only Belykh et al. (2017) assessed construct validity, showing that the participants with the least experience scored 0-28 points, whilst the group with the intermediate experience scored 29-39 and the attending group was at 40-45 point-intervals using the model [13] .

Craniotomy and burr hole drilling were assessed using two VR models and one synthetic model [19] . The VR models included a phantom-based training system [46] , and a virtual isomorphic haptic model [72] . Face validity was assessed by all studies, content validity was assessed in 1 study [46] , and another study used construct validity to assess the proficiency of craniotomy/burr hole drilling among their participants [19] (Table 1) . Using a 5-point Likert scale to assess face validity, participants found a high median visual and tactile realism with respondents ranking the tool as "useful" and "extremely useful" [19] . In terms of construct validity, the study demonstrated that the novice group experienced the greatest increase in confidence in performing craniotomy in comparison to the experienced group [19] . Upon assessment of content validity, the model scored highly on haptic feedback [46] .

Studies that assessed both aneurysm clipping and craniotomy used VR [3] and 3D printed synthetic models [60] ( [60] only considered face validity. Both studies found high ratings for "usefulness" and increasing understanding of the aneurysm ( Table 2) . Although the VR model was rated favourably for its use, it was limited by haptic realism as only 12% of participants agreed that it was realistic. Conversely, the patient-derived 3D printed aneurysm models [60] found higher ratings for realism of the artery (4.4) with lower ratings for clip application and bone drilling (4.1). This contrast between the two models is expected given the haptic realism of the 3D printed models.

Tumour resection was assessed on a synthetic model [7] , live rats [33], and two VR models [25, 69] . All studies used questionnaires to evaluate their models. [25] also reporting construct validity. In Ashour's study, the synthetic model was rated > 90% for face validity and > 80% for content validity (Table 2 ) [7] . Similarly, realism of the live rat model was rated highly [33] ( Table 2) . Content validation showed all participants upskilling after the use of the simulator (Table 2 ). Both studies using VR used the 5-point Likert scale and had favourable ratings for visual realism. However, the models did not provide faithful sensory realism. In assessing Gelinas-Phaneuf's VR model, construct validity revealed a significant difference between medical students as compared to junior and senior residents (p < 0.05) but no difference between junior and senior residents [25] . [12] assessed end-to-side micro-anastomosis techniques on synthetic and placenta models, respectively (Table 1) . Both assessed face and construct validity. Participants from both studies agreed that the respective models were suitable in replicating the real surgical technique. Both studies used the Northwestern Objective Microanastomosis Assessment Tool to assess construct validity. The mean NOMAT score of the untrained group was significantly lower than that of the trained group in the studies by Table 1) . The study used a questionnaire for face validity and ratings by neurosurgeons of the neurosurgery residents' performance for content validity. On assessment of face validity, the simulator scored > = 4 for microscope and microsurgical instruments handling (5), bipolar coagulation of bleeding microvessels (5), and aneurysm clipping (4). However, there were lower ratings for aneurysm rupture management (3.3) and aneurysm neck and dome dissection (3.8) . Regarding content validity, there was no difference between the ratings received by residents (4) for the placenta and cadaver model ( Table 2 ).

Our systematic review highlighted early evidence of the feasibility and utility of using simulation models in neurosurgical training and education. Existing systematic reviews [9, 16, 30, 44, 53, 59] have evaluated the utility and efficacy of simulation models in neurosurgical education and training. However, to our knowledge, there are no previous reviews assessing these 4 neurosurgical procedures/skills: craniotomy/burr hole drilling, aneurysm clipping, vessel suturing, and tumour resection using face, content, and construct validity. These are validation methods in medical education which provide a framework for evaluating the utility of simulation models. The MERSQI tool evidenced the quality of the studies included being moderate. The strengths of the included studies were high response rates and appropriateness of data analysis, whereas main weaknesses were failing to evaluate the simulations in more than one institution, study design, and poor validation of the qualitative and quantitative tools used for assessment. Face, content, and construct validity are discussed for all 4 simulation model types included in our study.

Both studies examining cadaver models found high ratings for face validity [1, 22] . Upon assessment of content validity, a majority of participants from Aboud et al. (2015) [1] found cadaveric intraoperative rupture to be realistic and superior to existing models for cerebral revascularisation. The cadaveric model outperformed the placental model for face validity [22] (although no differences were seen in content validity). Other studies comparing cadaveric models to physical and haptic simulators have shown they accrue the highest reported benefit for skill improvement [24] . Cadavers are known to simulate tissue dissection, bleeding, and pulsation with high fidelity [1, 54] . In line with literature, our findings demonstrate that cadavers are useful and effective for cranial procedures and manipulating soft tissues [57] . Despite their fidelity, cadaver models were the least commonly found simulation model in this review [1, 22] . This is in keeping with the decreasing prevalence of cadaver models in surgical training owing to high costs, low cadaver availability, and ethical issues [22, 49, 57] .

Similar to cadavers, all three studies using animal or tissue models found favourable results for face validity. Placenta models [13, 22] successfully discriminated between competence levels, whilst the live rat models confirmed skill improvement after practicing with the model [22, 33] . Our results are consistent with literature findings that tissue and animal models are favoured due to neuroanatomical and neurovascular similarities [44] . Specifically, the large-and small-necked aneurysms present in placenta have been an excellent model in simulating microsurgical skills such as aneurysm clipping [41] . Furthermore, the resemblance of brain tumours in rats to that in humans has served as a useful model for neuro-oncology procedures [38] . However, the paucity of animal models in our study reflects the downward trend of using animal models due to issues of animal rights, ethical concerns, and difficulty in procuring large numbers of animals [8, 54, 61] .

Six studies used VR models [3, 25, 27, 46, 69, 72] . Our study revealed favourable ratings for face validity comparing the appearance of the VR models to real surgery revealing the benefits of the model for anatomic understanding. However, participants of all six VR studies reported low haptic fidelity (Table 2) . Therefore, haptic sensation was a key limitation to the success of VR models in our study, similarly found in robotic surgery where a lack of haptics limited surgical skill development [48] . Nevertheless, actuators, which apply force-feedback, are improving with rapid response times. This includes electroactive polymers, piezoelectric, electrostatic, and subsonic audio wave surface actuations that allow for improved sensory feedback [35, 73] . With the advancement of computer graphics, the visual realism of VR simulations continues to further improve and allows for realistic anatomical representations [9, 59] . Overall, VR models provide performance metrics that have been strongly correlated to skills in the OR [9, 36] .

Synthetic models (non-flesh) are traditionally low-fidelity simulations [9] used for fine motor specific skills such as suturing [6] , tumour resection [7] , aneurysm clipping [39, 42, 70] , and burr holes [19, 60] . Synthetic models that simulated aneurysm clipping had high face validity and content validity in improving understanding of structure of the aneurysm in relation to the parent artery [39] . Similarly, synthetic models simulating craniotomy had high face validity (≥ 4) on a Likert scale [19] and provided most benefit to the novice group compared to experienced group [19] , an objective indicator to favour the model. Ashour et al. [7] also showed significant skill improvement (94%). This is similarly reflected in literature where synthetic simulators have shown to translate skills from simulations to surgical performance [5, 29, 41] . Compared to aforementioned models, synthetic models have low overall costs [49, 71] , are easily handled (avoiding the regulations around animal or cadaveric tissue) [49] , and provide a safe, controlled environment. Potential limitations include the lack of repeated use of models, increased cost of synthetic models and the lack of realism compared to human cadaveric or animal models [9] . Despite this, advancements in 3D printing technology allows for increasingly realistic and readily available simulation models. This makes training more accessible without significant financial investment.

Development of novel learning modalities such as virtual reality, tele-simulations, and synthetic models have been accelerated during the COVID-19 pandemic [31] . However, many simulation models in neurosurgery are expensive and nascent in development. Nevertheless, there is a pressing need for these models due to the concerns of surgical training education which includes being exposed to fewer cases with complex operative techniques and limited patient interaction [44, 62, 66] . Based on our review and current literature findings on residents' concerns of surgical training in the pandemic, synthetic models, due to their visual and haptic realism, are the most convenient forms of simulation models to be used safely in remote settings. However, with the rapid advent of technology spurred on by the pandemic, VR models using haptic feedback technologies show promise. Despite the anatomical and haptic realness of cadaver and animal models, there is a downward trend in their use given procurement costs, storage, and ethical issues. This is reflected in our review where VR and synthetic models were most commonly used, with only two and three studies using cadaver or animal models, respectively.

These findings provide significant benefits for remote training in lower-and middle-income countries where maximising efficiency of surgical training is essential given over 95% of the population lacks access to basic surgical care [43] . These countries face barriers to incorporate simulated training including high costs, finding an appropriate training environment, and storage of cadaveric and animal material [15] . However, development of virtual learning platforms, remote tele-simulation [34] , and the development of lowcost high-fidelity VR platforms [4, 50] could broaden opportunities for education. Our findings showed that synthetic and virtual platforms have been used to simulate key neurosurgical procedures, further demonstrating their applicability in low-and middle-income countries.

Limitations of this review must be considered, many of which are related to restrictions in medical education research and the associated ethical barriers. Additionally, there is no consensus on the gold standard to assess medical and surgical education outcomes. Construct validity has been perceived by few validity theorists to be "the whole of validity from a scientific point of view" [58] . However, only five studies assessed construct validity, whilst most studies included assessed face and content which are considered to be "subjective".

A major limitation of our paper is the lack of inclusion of predictive validity, considered an "ultimate" assessment to establish validity of an educational tool [47] . On post hoc analysis, we found that none of the included studies evaluated predictive validity. Predictive validity refers to the accuracy of prediction made by a model or test to confirm the future skill of an individual on whom the same model or test will be applied [47, 64] . However, assessing predictive validity is logistically impractical as it requires long-term follow-up. Furthermore, there is a lack of literature assessing simulation models according to the various validity types, possibly attributed to a lack of medical education tools in a surgical setting. This is also echoed by a recent systematic review [51] which calls for a prevalent use of validity scoring methodology to assess simulation tools and translation of these models in the OR. Additionally, there is a paucity of studies addressing all 3 validity scales to assess simulation models. Due to the heterogeneity in research methodology, study design, and types of outcomes reported, a meta-analysis was not conducted. This reveals the need for objective validation methodology for simulation methods.

However, the paper also had several strengths; according to our knowledge, this review is the first to address 4 major neurosurgical skills using various modalities of training, assessed by face, content, and construct validity. Existing reviews that discussed neurosurgical skills using face, content, and construct validity only cover a specific neurosurgical procedure or model type [11, 51] . Therefore, our review is the first to review these skills using an appraised method of validating simulation models. Furthermore, strong consensus in the literature shows the MERSQI tool as a robust tool of choice for critical appraisal of the medical education research included in this study [18, 63] .

Future work should aim to develop a standardised approach to assessing neurosurgical simulation tools using face, content, and construct validity. Currently, there is a lack of consistent reporting of objective validation methods. A review of 83 studies on surgical simulation reported 60% targeted construct validity, 24% targeted concurrent validity, and 5% looked at predictive validity [47] . Similarly, our review found that although a majority of included studies reported subjective validation methods such as face (n = 18), and to a lesser extent, content validity (n = 11), there was a relative lack of reporting construct validity (n = 6). We, therefore, propose developing a simulation assessment template, which incorporates both subjective and objective validation assessments, adapted for various neurosurgical skills identified in this review. This will allow simulation sites to collect adequate data to validate their models whilst saving them from time-intensive work to build new validity questionnaires for every simulation model.

Additionally, further work should examine how to facilitate the collection of data on predictive validity. Establishing predictive validity for simulators is essential given that the goal of simulation is the improved ability in the OR. This must therefore be assessed in studies evaluating simulation tools. However, doing this remains difficult for researchers to implement in practical terms. Possible options that may facilitate this are a digital survey form or a mobile app for surgeons or their supervisors to assess in-theatre performance that can then be compared to the simulator score.

This review assessed neurosurgical simulation models using face, content, and construct validity, and reported an increased use of simulation models in neurosurgical training. Whilst synthetic models are currently the most convenient and practical, especially during the COVID-19 crisis, VR models were found promising due the visual realism and improved haptic feedback technology. This also provides neurosurgical educators tools and assessment methods for simulation.

Although surgical simulation models receive generally positive feedback from trainees, comparing results among different studies were limited by the heterogeneity among studies. Moreover, studies examining simulation methods seldom use objective validation methods such as construct and predictive validity. Future work should examine how to facilitate the collection of objective validity, and aim to create a simulation assessment template that can be adapted for various neurosurgical simulation models.

Live cadavers" for training in the management of intraoperative aneurysmal rupture

The role of simulation in developing surgical skills

Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback

Use of a low-cost portable 3d virtual reality gesture-mediated simulator for training and learning basic psychomotor skills in minimally invasive surgery: development and content validity study

Assessment of technical skills transfer from the bench training model to the human model

A pilot study to assess the construct and face validity of the Northwestern Objective Microanastomosis Assessment Tool

Navigation-guided endoscopic intraventricular injectable tumor model: cadaveric tumor resection model for neurosurgical training

Microsurgical training model with nonliving swine head. Alternative for neurosurgical education

Innovations in surgery simulation: a review of past, current and future techniques

Impact of COVID-19 on neurosurgery resident training and education

Systematic review on the effectiveness of augmented reality applications in medical training

Low-flow and high-flow neurosurgical bypass and anastomosis training models using human and bovine placental vessels: a histological analysis and validation study

Face, content, and construct validity of an aneurysm clipping model using human placenta

On resident duty hour restrictions and neurosurgical training: review of the literature

An evaluation of the role of simulation training for teaching surgical skills in Sub-Saharan Africa

Simulation in neurosurgery-a brief review and commentary

The role of simulation in neurosurgery

Appraising the quality of medical education research methods: the medical education research study quality instrument and the Newcastle-Ottawa scale-education

Development of a modelled anatomical replica for training young neurosurgeons

Seven Ethical Issues Affecting Neurosurgeons in the Context of Health Care Reform

Simulation in surgical education

Learning brain aneurysm microsurgical skills in a human placenta model: predictive validity

Operative and consultative proportions of neurosurgical disease worldwide: estimation from the surgeon perspective

Neurosurgery simulation in residency training: feasibility, cost, and educational benefit

Assessing performance in brain tumor resection using a novel virtual reality simulator

Coronavirus disease 2019 (COVID-19) and neurosurgery: literature and neurosurgical societies recommendations update

Virtual cerebral aneurysm clipping with real-time haptic force feedback in neurosurgical education

Effectiveness of cadaveric simulation in neurosurgical training: a review of the literature

Implementation, construct validity, and benefit of a proficiency-based knot-tying and suturing curriculum

Developing a neurosurgical simulation-based educational curriculum: an overview

Impact of the COVID-19 pandemic on student and resident teaching and training in surgical oncology

Neurosurgical training with simulators: a novel neuroendoscopy model

Virtual learning in global surgery: current strategies and adaptation for the COVID-19 pandemic

Haptics-Touchfeedback technology widening the horizon of medicine

Psychomotor performance measured in a virtual environment correlates with technical skills in the operating room

Simulation-based learning: Just like the real thing

Morphology of cerebral arteries

Fabrication of cerebral aneurysm simulator with a desktop 3D printer

The financial impact of teaching surgical residents in the operating room

Human placenta aneurysm model for training neurosurgeons in vascular microsurgery

Development of three-dimensional hollow elastic model for cerebral aneurysm clipping simulation enabling rapid and low cost prototyping

Global Surgery 2030: Evidence and solutions for achieving health, welfare, and economic development

Neurosurgical cadaveric and in vivo large animal training models for cranial and spinal approaches and techniques-a systematic review of the current literature

Human patient simulation: educational issues and practical implications in COVID-19 times

A neurosurgical phantom-based training system with ultrasound simulation

Methodologies for establishing validity in surgical simulation studies

Haptic feedback in robot-assisted minimally invasive surgery

Simulation training methods in neurological surgery

Creating a low-cost virtual reality surgical simulation to increase surgical oncology capacity and capability

A systematic review of simulation-based training in neurosurgery, part 1: cranial neurosurgery

Morbidity, mortality, and health care costs for patients undergoing spine surgery following the ACGME resident duty-hour reform: Clinical article

Association between funding and quality of published medical education research

Testing technical skill via an innovative "bench station" examination

Vascular access in haemodialysis: strengthening the Achilles' heel

Task shifting and task sharing for neurosurgeons amidst the Covid-19 pandemic

Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery

Four tenets of modern validity theory for medical education assessment and evaluation

Virtual reality for medical training: the state-of-the-art

Cerebral aneurysm clipping surgery simulation using patient-specific 3D printing and silicone casting

Strategies for the reduction of live animal use in microsurgical training and education

Letter to the editor impact of the COVID-19 pandemic on neurosurgical residency training in New Orleans

A plea for MERSQI: the medical education research study quality instrument

Assessing the predictive validity and efficacy of a multimodal training programme for laparoscopic radical prostatectomy (LRP)

Surgical simulation: a systematic review

Innovations in neurosurgical education during the COVID-19 pandemic: is it time to reexamine our neurosurgical training models?

Challenges to Neurosurgery During the Coronavirus Disease 2019 (COVID-19) Pandemic

Construct, face, and content validation on Voxel-Man® Simulator for otologic surgical training

Virtual neurosurgery, training for the future

3D printing of intracranial aneurysm based on intracranial digital subtraction angiography and its clinical application

3D printed surgical simulation models as educational tool by maxillofacial surgeons

Comparison of cadaveric and isomorphic virtual haptic simulation in temporal bone training

Skin-integrated wireless haptic interfaces for virtual and augmented reality

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors should be commended for their effort and for drawing attention to the face, content and construct validity elements of simulation studies in neurosurgery. As in most surgical specialties, hundreds of various training tricks and techniques are published, ranging from simple exercises to highly complex simulators. Simply publishing a concept paper on a simulation idea is no longer sufficient. In the absence of validation studies, the question must always be: Why is this particular simulator/training technique necessary? Also, without validation studies, most of these simulators are no more than very expensive toys that give a training programme a "high-tech" edge.This review further exposes the most important and likely the most painful aspect of validation studies: none of them assesses predictive validity. Predictive validity refers to the ability of a simulation to lead to improved skill intra-operatively and ultimately lead to better patient outcomes. While it is evident that training alone can improve skill, a scientific manuscript claiming the validity of a certain technique claims improved skill and patient outcomes through training with that particular technique/simulator. The question is whether this effect relates to the simulator or technique itself or simply to the fact that one trains more. Until better studies appear, one must remain skeptical of different kinds of simulators that have not been validated and we must not lose sight of the fact that predictive validity in neurosurgical simulators is abysmal.Victor Volovici. Rotterdam, The Netherlands.