key: cord-0888146-u5yyww7w authors: Smith, Samuel C.L.; Siau, Keith; Cannatelli, Rosanna; Antonelli, Giulio; Shivaji, Uday N.; Ghosh, Subrata; Saltzman, John R.; Hassan, Cesare; Iacucci, Marietta title: Training methods in optical diagnosis and characterization of colorectal polyps: a systematic review and meta-analysis date: 2021-04-22 journal: Endosc Int Open DOI: 10.1055/a-1381-7181 sha: 6226b09ddfba44fdb865930783f4be552f4e52ba doc_id: 888146 cord_uid: u5yyww7w Background and study aims Correct optical diagnosis of colorectal polyps is crucial to implement a resect and discard strategy. Training methods have been proposed to reach recommended optical diagnosis thresholds. The aim of our study was to present a systematic review and meta-analysis on optical diagnosis training. Methods PubMed/Medline and Cochrane databases were searched between 1980 and October 2019 for studies reporting outcomes on optical diagnosis training of colorectal polyps. The primary outcome was optical diagnosis accuracy compared to histological analysis pre-training and post-training intervention. Subgroup analyses of experienced/trainee endoscopists, training methods, and small/diminutive polyps were included. Results Overall, 16 studies met inclusion criteria, analyzing the impact of training on 179 endoscopists. Pre-training accuracy was 70.3 % (6416/9131 correct diagnoses) whereas post-training accuracy was 81.6 % (7416/9213 correct diagnoses) (risk ratio [RR] 1.17; 95 % confidence interval [CI]: 1.09–1.24, P < 0.001). In experienced endoscopists, accuracy improved from 69.8 % (3771/5403 correct diagnoses) to 82.4 % (4521/5485 correct diagnoses) (RR 1.20; 95 % CI: 1.11–1.29, P < 0.001). Among trainees, accuracy improved from 69.6 % (2645/3803 correct diagnoses) to 78.8 % (2995/3803 correct diagnoses) (RR 1.14; 95 % CI 1.06–1.24, P < 0.001). In the small/diminutive polyp subgroup, accuracy improved from 68.1 % (3549/5214 correct diagnoses) to 77.1 % (4022/5214 correct diagnoses) in (RR 1.16 95 % CI 1.08–1.24 P < 0.001). On meta-regression analysis, the improvement in accuracy did not differ between computerized vs. didactic training approaches for experienced ( P = 0.792) and trainee endoscopists ( P = 0.312). Conclusions Optical diagnosis training is effective in improving accuracy of histology prediction in colorectal polyps. Didactic and computer-based training show comparable effectiveness in improving diagnostic accuracy. Gastrointestinal endoscopy is integral to the diagnosis and management of colorectal polyps. Optical diagnosis using advanced endoscopic technologies such as high-definition, magnification and electronic virtual chromoendoscopy permit accurate prediction of histological characteristics of colorectal lesions based on endoscopic appearances and is increasingly utilized, and its implementation across the endoscopic community is on the rise [1] . Accurate optical diagnosis allows small/diminutive colorectal polyps (< 10 mm) to be either spared or removed and discarded without the need for formal histological assessment: the "resect and discard" strategy [2] . The incorporation of optical diagnosis of small/diminutive polyps has been endorsed by The American Society of Gastrointestinal Endoscopy (ASGE) as well as recent European Society of Gastrointestinal Endoscopy (ESGE) guidelines [1] . If implemented there would be fewer specimens sent for histological analysis with substantial cost savings and reduced risk to patients with fewer unnecessary polypectomies [2, 3] . In addition, optical characterization of colorectal polyps can accurately identify malignant areas within lesions and identify lesion borders, improving correct patient management. Sessile serrated lesions (SSL) are regarded as subtle lesions that can be easily missed; however optical diagnosis training through the use of polyp classification systems such as SIMPLE [4] and BASIC [5] may facilitate enhanced detection and characterization of SSL. Given the rise of artificial intelligence and its ability to improve detection of colorectal polyps, the technology can support polyp characterization provided endoscopists are skilled in optical diagnosis. Training will be central to correctly implement optical diagnosis in clinical practice, as recognized in a recent evidencebased consensus [6] . Many different training strategies have been proposed and reported. Among them, traditional didactic training, computer-based self-learning and ad hoc training in vivo. This has become increasingly relevant following the coronavirus disease 2019 (COVID-19) pandemic, which has had an adverse impact on endoscopy training for trainees, particularly hands-on training, with a reduction in procedures of up to 96 % [7] . Societies are now recommending trainees utilize alternative learning opportunities such as cognitive-based learning [8] . The aim of this systematic review and meta-analysis was to provide an overview of training in optical diagnosis of colorectal polyps and in view of the COVID-19 pandemic complete subgroup analysis of computer-based training. Methodology of our analysis, inclusion criteria and reporting were in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations [9] and the Meta-analyses Of Observational Studies in Epidemiology (MOOSE) checklist [10] . This systematic review was registered in the International Prospective Register of Systematic Reviews (PROSPERO) with registration number CRD42020167486. We performed an electronic database search of PUBMED/Medline, Cochrane and SCOPUS databases in addition to gray literature (scanning reference lists), to identify studies reporting training for the optical diagnosis of colorectal polyps between 1980 and October 2019. The search strategy is outlined below: Search strategy in PUBMED/Medline: (Optical Diagnosis OR optical biopsy OR optical characterization) AND (colorectal polyp OR colorectal adenoma OR colorectal hyperplastic OR colorectal sessile serrated lesion OR colorectal sessile serrated adenoma), (Optical Diagnosis OR optical biopsy OR optical characterization) AND (colorectal polyp OR colorectal adenoma OR colorectal hyperplastic OR colorectal sessile serrated lesion OR colorectal sessile serrated adenoma) AND (training OR education), (Education OR Training) AND Colonoscopy AND (Colorectal polyp OR Colorectal adenoma OR Colorectal hyperplastic OR Colorectal sessile serrated lesion). Search strategy in the other databases followed a similar but simplified strategy. Studies were deemed eligible for inclusion according to the PICO statement (P, endoscopists undergoing assessment of optical diagnosis accuracy of small/diminutive colorectal polyps; I, endoscopists receiving optical diagnosis training; C, optical diagnosis as compared with histological result as gold standard; O, pre-training vs. post-training accuracy of optical diagnosis). Studies not reporting pre-training vs. post-training performance and not published in English language were excluded. Randomized-controlled trials, observational and cohort studies and abstracts were all included for analysis. Titles and abstracts of all identified articles were independently screened by two authors (SS/RC) to exclude studies not related to the topic or not meeting inclusion criteria. Potentially relevant studies were screened for eligibility by analysis of the full text. Disagreements between the two authors were referred to and discussed with the senior author (MI) and resolved with consensus. A standardized form was used to extract the data from each study. Data extracted included: a) Author name; b) Year of publication; c) Country; d) Training method (didactic, computerbased); e) Number of participants; f) Setting of study (in vivo/ ex vivo); g) Number and size of polyps; h) Number of correct histology predictions (accuracy); i) Training material; j) Endoscopic platform; and k) Duration of training. The risk of bias of included studies was assessed using the Cochrane Collaboration's tool [11] . Each study was assessed for risk of bias through study design in selection bias (random sequence generation and methods to conceal allocation), performance bias (blinding of participants), detection bias (blinding of outcomes), attrition bias (completeness of outcome data) and reporting bias (selective reporting). The overall quality of evidence was summarized using The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, categorizing the evidence as very low, low, moderate or high-quality. A small/diminutive colorectal polyp was defined as ≤ 10 mm in size. Optical diagnosis training was defined as an intervention designed to educate participants on optical diagnosis methodology. Experienced endoscopists were participants who are independent endoscopists who have completed endoscopy training but who are not considered experts. Trainee endoscopists were defined as participants who are practicing within a designated gastroenterology training program. The primary outcome of our systematic review and meta-analysis was optical diagnosis accuracy compared with histological analysis before and after training intervention. We aimed to complete subgroup analysis by endoscopists experience (experienced, trainee endoscopists), training method (didactic, computer-based) and polyp size (small/diminutive polyps). A random effect meta-analysis was performed to investigate the effect of training on the accuracy of optical diagnosis. For each study, accuracy was compared between post-training and pre-training stages and expressed as a risk ratio (RR) with 95 % confidence interval (CI), wwhich were pooled using a random-effects Mantel-Haenszel model. Forest plots were generated for all studies, followed by level of experience (trainee vs experienced endoscopists), and then for the subgroups of didactic vs. computer-based training. We also calculated 95 % CIs to determine the variation in effect between studies [12, 13] . Statistical heterogeneity was assessed using I 2 statistics, with a value of 0 % to 40 % accepted as not important, 30 % to 60 % as moderate, > 60 % as substantial and > 90 % as considerable heterogeneity [11] . Publication bias was assessed by observing asymmetry in funnel plots and sensitivity analyses performed by excluding outliers and then by year of publication. Meta-analyses were performed using RevMan v5.3 (Cochrane Collaboration, Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014) and pooled effects of didactic and computer-based training subjected to a randomeffects meta-regression model using Open Meta-Analyst (Brown University). P < 0.05 was considered statistically significant. The PRISMA [9] and MOOSE [10] checklist (Appendix 1) and flow chart (▶ Fig. 1 ) were followed to ensure compliance. The literature search yielded 1237 results. After preliminary screen-ing of titles and abstracts, 113 were selected for full text review. Of these, 16 papers [4, 14-28] (before 2010 n = 1, 2010-2015 n = 7, 2016-2019 n = 8) matched the selection criteria and were included in the systematic review. Of these studies, seven were from Europe [4, 17, 21, 23, [26] [27] [28] , six from North America [14, 16, [18] [19] [20] , 24] and three from Asia [15, 22, 25] . Among included studies, eight reported on didactic training [4, 14-16, 18, 22, 23, 27] , seven on computer-based self-training [17, [19] [20] [21] [24] [25] [26] and one on computer-based self-training vs. didactic training [28] . The majority of studies were observational in design (n = 14) and there were two randomized trials [18, 28] . Overall 11 studies were based on NBI system [14-22, 24, 25] , one on iScan [26] and one on BLI (Blue Light Imaging) [27] , two on NBI and iScan [4, 28] and one on high-definition white light (HDWL)/chromoendoscopy [23] . Sixteen studies reported pre-training and post-training values, two studies included trainees [4, 28] , eight included fully qualified/BCSP (Bowel Cancer Screening Programme) endoscopists [14, 15, [18] [19] [20] [21] [22] 25] and six studies included both groups [16, 17, 23, 24, 26, 27] . There were nine studies that only included small/diminutive colorectal polyps [4, 15, 17, 19, 20, [25] [26] [27] [28] . Study characteristics are presented in ▶ Table 1 . Fig. 3b) . The 95 % prediction interval was 0.77-1.71. We detected considerable heterogeneity in this subgroup (I 2 96 %) without significant publication bias (Supplementary Fig. 1b) . On meta-regression analysis, there was no significant difference in post-training accuracy between didactic and computerbased delivery methods (P = 0.798). We subsequently selected only studies describing the efficacy of training in experienced endoscopists. Method of training for experienced endoscopists comprised of didactic training (seven studies, 49 participants) [14-16, 18, 22, 23, 27] and computerbased training (six studies, 64 participants) [17, 19-21, 25, 26] . After pooling these studies, optical diagnosis training method on optical diagnosis in experienced endoscopists improved accuracy from 69. 8 Fig. 4b) . The improvement in accuracy did not differ significantly between the two training methods (P = 0.312). We selected studies that only included small/diminutive colorectal polyps in the assessment of training in optical diagnosis. A total of nine studies assessed the impact of optical diagnosis training on 104 endoscopists [4, 15, 17, 19, 20, [25] [26] [27] [28] . The pooled pre-training accuracy was 68.1 % (3549/5214) and post-training accuracy was 77.1 % (4022/5214) (RR 1.16 95 % CI 1.08-1.24 P < 0.001) (Supplementary Fig. 2) . We detected substantial heterogeneity in these studies (I 2 87 %). The summary of the Cochrane Collaborationʼs risk of bias tool is presented in ▶ Fig. 5 . Only two randomized trials were included [18, 28] , both of which report how the randomization process took place but both were unable to conceal the allocation to participants. Participants in all studies included were not blinded to the intervention. Participants were blinded to the out-comes of training during the study process for most studies and all studies produced complete datasets without selective reporting. Using the GRADE approach, the quality of evidence was downgraded by two points to low quality due to risk of bias and inconsistency from the heterogeneity in the studies. According to our systematic review and meta-analysis, training in the optical diagnosis of colorectal polyps was associated with improved diagnostic accuracy in both experienced and trainee endoscopists. Furthermore, when only considering small/diminutive colorectal polyps there was also a statistically significant improvement in diagnostic accuracy following training. These results are clinically relevant and important for the following considerations. First, the efficacy of training has been Tau proved for both experienced and trainees in endoscopy. Indeed, trainees showed a similar post-training accuracy level as compared to experienced endoscopists, confirming the positive impact of training on optical diagnosis. Second, irrespective of the pre-training accuracy level, training programs resulted in uniformly elevated accuracy levels. Third, there was no statistically significant difference seen in the improvement in accuracy between computer-based and didactic training. Endoscopy practice is going through major changes, which have accelerated in response to the COVID-19 pandemic. To minimize the risk of exposure to COVID-19, units are taking steps to lessen the footfall in procedural rooms, [29] . This will inevitably have a serious consequence on endoscopy training, particularly given that restrictions may be in place until 2022 [30, 31] . It is recognized that during these unprecedented times that trainees seek training in alternative means, perhaps focusing on cognitive skills [30] . This increased importance in alternative educational resources such as computer-based and simulation based training in optical diagnosis would address the unmet need during this period and likely in the longer term too [7] . Our study confirms that computer-based training is effective in improving optical diagnosis accuracy and furthermore, there is no statistical difference between computer-based and didactic training which is further confirmed by a randomized trial [28] . Although the P values and lower bound 95 % CIs of our analyses indicate a statistically significant beneficial effect of training, the 95 % prediction intervals suggest that, while the "average" training course is likely to result in an improvement in predictive accuracy after optical diagnosis training, there are likely to be courses where no benefit is observed. This is also reflected in the I 2 values which indicate considerable heterogeneity in most analyses. As such there needs to be a focus on the validation, standardization and quality assurance of training and long-term studies to assess the retention of optical diagnosis skills after a training intervention. Most studies investigating the effectiveness of training methods on the optical diagnosis of colorectal polyps were small observational studies with only two randomized trials. The studies included a baseline level of performance prior to training enabling us to fully ascertain the effectiveness of training. Including trainee and experienced endoscopists, the target of intervention was relevant to the clinical practice of endoscopy. The substantial heterogeneity of studies may be explained by the following differences: training intervention (number of sessions and duration), assessment methods (type of media used, endoscopic platform and whether material was repeated pre-and post-training), type of lesions included (some studies only included hyperplastic and adenomas [16, 17, 27] ), washout period between pre-training assessment and training, and participant characteristics (pre-training accuracy levels were as low as 45 % in one study among experienced endoscopists [16] and as high as 88 % in some trainees [27] ). The systematic review and meta-analysis has limitations. First, all included studies were unblinded due to the nature of the intervention (i. e. training methods). This however is a com-mon bias of many evaluation of technological improvements in gastrointestinal endoscopy. Second, participants were aware that their performance in assessments were being monitored and analyzed, which may introduce further bias as there may have been a subconscious increased effort during the study period. One possible solution to this would be tracking in vivo optical diagnosis accuracy for a prolonged period of time before and after training. However, this would be resource intensive and may still incur bias. Third, not all studies provided data to calculate sensitivity, specificity or negative predictive values (against the recommended ASGE PIVI threshold of > 90 %), nor was it possible to compare receiver operating characteristics curves pre-training and post-training. Given the large degree of heterogeneity, the inconsistency of results, the lack of randomized controlled trials and inherent bias the quality of evidence of the studies is considered to be low. This systematic review and meta-analysis has uncovered important gaps in evidence, which will need to be addressed in future studies. Further study on optical diagnosis training is required to establish the optimum method. There is a lack of robust randomized trials comparing training modes with only one comparing didactic with computer-based training [28] . Another important consideration is the effectiveness of training across endoscopic platforms, while initial attempts at this suggest that training is effective across NBI and iScan [4, 28] , however there is yet to be a study that includes all of NBI, iScan and BLI. The issue of retention of optical diagnosis over time is one that has yet to be thoroughly explored. The majority of studies included in this systematic review and meta-analysis involve participants scoring media on the same day as training, however none have demonstrated that optical diagnosis skills can be retained over a prolonged period of time. This is essential for optical diagnosis to be incorporated in everyday practice. Future studies should reassess participants after a prolonged period of time such as 3 to 6 months or even longer and should include an in vivo assessment component to formally investigate whether ex vivo training can translate to in vivo performance over time. A future training strategy may include a combination of methods with perhaps didactic training from experts to set a foundation of knowledge, further supported by computerbased training or artificial intelligence (AI) systems. Computerbased training could be periodically reinforced relatively easily. The ESGE suggest the most likely scenario will be as a "second reader" as opposed to a "stand alone" system [32] . Therefore, for AI to be utilized in everyday practice there is a need for endoscopists to up skill in optical diagnosis to be able to safely interpret and act upon the readings from AI systems. In conclusion, training in optical diagnosis improves the accuracy of histology predictions of colorectal polyps, including small/ diminutive polyps. The optimal method of training may include a combination of training methods augmented with continuous in vivo training, which may be provided by trainers or AI systems. Future studies need to focus on standardizing and vali- dating training modules to enhance cognitive skills of endoscopists. Correction: Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline -Update ASGE Technology Committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps Economic value of narrow band imaging versus white light endoscopy for the characterization of diminutive polyps in the colon: systematic literature review and costconsequence model Development and validation of the SIMPLE endoscopic classification of diminutive and small colorectal polyps BASIC (BLI Adenoma Serrated International Classification) classification for colorectal polyp characterization with blue light imaging Detection and characterization of colorectal polyps using high-definition white light and i-Scan: Evidence-based consensus recommendations using a modified Delphi process The impact of COVID-19 on gastrointestinal endoscopy training in the United Kingdom How to maximize trainee education during the coronavirus disease-2019 pandemic: perspectives from around the world Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group The Cochrane Collabora-tionʼs tool for assessing risk of bias in randomised trials Interpretation of random effects metaanalyses Chapter 10: Analysing data and undertaking meta-analyses Narrow-band imaging without high magnification to differentiate polyps during real-time colonoscopy: improvement with experience Diagnostic accuracy of narrow-band imaging and pit pattern analysis significantly improved for less-experienced endoscopists after an expanded training program Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps Colorectal surveillance interval assignment based on in vivo prediction of polyp histology: impact of endoscopic quality improvement program Impact of a computer-based teaching module on characterization of diminutive colon polyps by using narrow-band imaging by non-experts in academic and community practice: a video-based study Community gastroenterologists can learn diminutive colon polyp histology characterization with narrow band imaging by a computer-based teaching module Development and validation of the WASP classification system for optical diagnosis of adenomas, hyperplastic polyps and sessile serrated adenomas/polyps Effectiveness of systematic training in the application of narrow-band imaging international colorectal endoscopic (NICE) classification for optical diagnosis of colorectal polyps: Experience from a single center in China Colonic lesion characterisation skills among UK endoscopists and the impact of a brief training intervention A Web-based education program for colorectal lesion diagnosis with narrow band imaging classification Improved real-time optical diagnosis of colorectal polyps following a comprehensive training program Endoscopic characterization of small colonic polyps: baseline performance of experienced endoscopists is no different to that of medical students Optical diagnosis of colorectal polyps with Blue Light Imaging using a new international classification. United Randomized controlled study of the prediction of diminutive/small colorectal polyp histology using didactic versus computer-based self-learning module in gastroenterology trainees ESGE and ESGENA Position Statement on gastrointestinal endoscopy and the COVID-19 pandemic COVID-19 and endoscopy: implications for healthcare and digestive cancer screening Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period Advanced endoscopic imaging: European Society of Gastrointestinal Endoscopy (ESGE) Technology Review Drs. Iacucci and Ghosh have received funding from the NIHR Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The authors are grateful to Mr. James Hodson, statistician at the Institute of Translational Medicine, University Hospitals Birmingham, for providing statistical support. Tau The authors declare that they have no conflict of interest.