key: cord-0731086-7tk0xusf
authors: Harkemanne, Evelyne; Duyver, Corentin; Leconte, Sophie; Sawadogo, Kiswendsida; Baeck, Marie; Tromme, Isabelle
title: Short- and Long-Term Evaluation of General Practitioners’ Competences After a Training in Melanoma Diagnosis: Refresher Training Sessions May Be Needed
date: 2021-10-26
journal: J Cancer Educ
DOI: 10.1007/s13187-021-02063-6
sha: a6e941d32140f9d1d394796e07b03d1f52700d93
doc_id: 731086
cord_uid: 7tk0xusf

General practitioners (GPs) are first-line clinicians in melanoma diagnosis. It is, therefore, important to ensure that they maintain their melanoma diagnostic accuracy over time. The objective of this study was to assess the short- and long-term competences of GPs after a training session in naked-eye melanoma diagnosis. An interventional prospective study was conducted whereby, over a 6-month period, GPs attended a 1-h melanoma diagnostic training session. To assess their acquired competences, GPs were asked to fill in a questionnaire on basic melanoma knowledge and to evaluate 10 clinical images of pigmented skin lesions prior to training, immediately after and 1 year later. In total, 89 GPs completed the questionnaire prior and immediately after training. As expected, the number of GPs who appropriately managed [Formula: see text] 50% of the melanoma cases increased after training (P < 0.001). One year after training, only 27 (30%) of the 89 GPs completed the questionnaire. This number of participants was too low to obtain significant figures but the GPs’ mean overall score of appropriately managed clinical cases was much lower than in the immediate post-test. In conclusion, although this short training improved the GPs’ diagnostic accuracy and management of melanoma in the short-term, participating GPs do not seem to have maintained these competences in the long-term. Further studies are needed to assess whether refresher training sessions are able to sustain acquired diagnostic and management skills. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s13187-021-02063-6.

Over the last two decades, general practitioners (GPs) have been recognized to play a key role in early melanoma diagnosis [1, 2] . Early detection is essential to reduce morbidity and mortality of patients with melanoma [3] . However, melanoma diagnostic abilities of GPs in Europe have been demonstrated to be suboptimal [4] [5] [6] .

In order to help GPs deal with suspicious skin lesions, melanoma diagnostic training programs have been developed. Initially, these programs trained GPs to recognize melanoma clinically (with the naked eye) [7] . Later, programs teaching diagnosis using dermoscopy became more widespread [8, 9] . However, due to the long training period required to become competent, dermoscopy is underused for the diagnosis of melanoma in general practice [4, 10, 11] . Therefore, it is essential, at the very least, to train GPs in naked-eye recognition of melanoma.

Most training programs teaching naked-eye melanoma recognition have been shown to improve GPs' short-term diagnostic accuracy and management in a training setting (on photographs of skin lesions) [12] [13] [14] [15] [16] [17] . Nonetheless, it is also important to ensure that the competences acquired by GPs are maintained over time [18] [19] [20] . Only four studies were found in which long-term ( ≥ 6 months) accuracy of GPs in melanoma diagnosis using clinical examination alone was evaluated [21] [22] [23] [24] . The results of these studies were heterogeneous. One study showed a drop of GPs' competences at 6 months post-training [21] and a second study showed a decrease over an 8-month period [22] . However, a major French study conducted in real-life demonstrated a positive long-term impact of their training with a decrease in the incidence of very thick melanomas (Breslow thickness ≥ 3 mm) over a 3-year period after a screening campaign performed by participating GPs [23] . Finally, a recent case-control study assessing 1662 referrals to dermatologists reported better quality of referrals by GPs trained in clinical melanoma diagnosis than by untrained GPs over a 10-month period [24] . Unfortunately, these two latter studies poorly evaluated the keys to the long-term sustainability of the acquired competences.

The aim of our prospective interventional study was to determine the short-and long-term competences of GPs after a 1-h training session, designed by our study team, on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

This prospective interventional study took place over an 18-month period from November 2015 to May 2017 and was approved by the Ethical Committee of the UCLouvain, Belgium. A 1-h training session in melanoma diagnosis was integrated into the annual mandatory medical education program of eight groups of Continuing Medical Education from the five French-speaking districts of Belgium (Hainaut, Brabant-Wallon, Province du Luxembourg, Province de Liège, and Bruxelles).

A 1-h training session, designed by our study team primarily targeting GPs, was common to all participants. The session included the teaching of three widely used naked-eye melanoma diagnostic tools (the ABCD rule [25] , "pattern recognition" [26] , and the "ugly duckling" sign [27] ) and instructions in basic knowledge on melanomas and pigmented skin lesions (PSL) such as risk factors of melanoma, preferred locations of melanoma according to gender, normal/abnormal evolution of PSL according to the patient's age, and characteristics of the most common benign PSL (seborrheic keratosis, typical nevus, blue nevus, congenital nevus, halo nevus, actinic lentigo, and subungual hematoma).

Management of PSL was also taught during the training sessions. To illustrate the diagnostic tools, a large number of clinical photographs of melanomas and benign lesions were shown.

All the study data was collected using REDCap® (Research Electronic Data Capture, Vanderbilt University, USA) tools hosted at the Cliniques universitaires Saint-Luc in Brussels (Belgium) [28, 29] .

Prior to training, the GPs completed a three-section questionnaire. This questionnaire was the result of a consent between two GPs (C.D. and S.L.) and a dermatologist (I.T.). Section A of the questionnaire collected socio-demographic and medical practice data, Section B investigated the GPs' usual melanoma diagnostic practices, and Section C evaluated their basic melanoma knowledge and their diagnostic accuracy and management of PSL.

Participating GPs were asked about their gender, age, workplace (urban, suburban, or rural area), work practice (solo, pair, medical group, or local network practice), and the number of patients seen per year.

To explore the way in which GPs diagnose melanoma and manage their patients in daily medical practice, ten multiplechoice questions were formulated.

To assess the efficacy of our melanoma training session, GPs completed this section of the questionnaire before training, immediately after training, and 1 year later. The details of Section A and B are described in a previous publication [6] .

For this third section, participants were randomly subdivided into two groups, with 45 GPs in group A and 44 GPs in group B. Two sets (A and B) of similar questions were created covering each essential training topic. Each group of GPs received one of the two sets to fill in prior to and immediately after training. The questionnaires completed prior to training were collected by the organizing team before the start of the training session. The sets were switched between groups at 1-year post-training. To avoid recall bias, the questions on the same topic were similar but not identical (there was no perfect parallelism between the diagnoses and/or management proposed within these similar questions). This 1-year questionnaire was sent by return mail to the GPs.

The section C questionnaire was divided into two parts and all the questions and answers were peer-reviewed and validated by six dermatologists. The first part consisted of five multiple-choice questions that explored basic melanoma knowledge (see Appendix 1) . GPs were asked about the preferred locations of melanoma according to gender, criteria for urgent melanoma management, characteristics of high-risk melanoma patients, and patient follow-up. In the second part, GPs were asked to assess 10 clinical photographs of PSL and their clinical history (see Appendix 2) . Every clinical history included the patient's age, gender, localization, and evolution (new onset or old lesion, change in shape or size) of the lesion. For each lesion, GPs were asked to select the correct diagnosis or the appropriate patient management from among five proposals. It is important to note that the clinical photographs used in the questionnaires were similar but not identical to those presented at the training session.

The statistical analysis was performed using the SAS software (SAS© 9.4, SAS Institute Inc., USA). Descriptive statistics were used to summarize the results considering absolute numbers, percentages, and 95% confidence intervals for the three periods of assessment. In addition, the GPs' average number of correct answers to the 10 clinical cases was summarized using mean ± standard deviation and the GP's responses to clinical cases corresponding to melanomas were also grouped according to a threshold of 50% correct answers. McNemar's test for binary matched-pairs data was used to test for variations between section C of the questionnaires prior to and immediately after training with regard to the number of GPs who correctly identified or gave the appropriate management for the clinical cases. All tests were considered to be significant for P value < 0.05. Statistical inference was not made for the comparison with the 1-year questionnaire because of the low number of GPs who responded to this questionnaire and the subsequent correction to be made on the type I error in case of multiple comparisons.

As part of their mandatory continuing medical education, 89 GPs participated in the training and completed the entire questionnaire prior to training. These 89 GPs also filled in Section C (basic melanoma knowledge and PSL diagnostic accuracy and management) immediately after training. Despite several reminders, only 27 GPs (30%) completed Section C of the questionnaire and returned it by mail 1 year after training.

The GPs' characteristics are summarized in Table 1 . The overall average age of the 89 participating GPs was 45.6 years (median = 45 years) with a M/F sex ratio of 0.98. One year after training, 40% of the responding GPs were under 35 years of age and 70% of them were female.

The results of this section according to the GPs' sociodemographic background and practice type were examined and discussed in a previously published scientific paper [6] .

The data collected before the training are available in the aforementioned study [6] . Short-and long-term results are the subjects of the present study. 

Eighty-nine GPs (45 in group A and 44 in group B) responded to Section C of the questionnaire prior to and immediately after training. Table 2 shows the GPs' correct responses to the basic melanoma knowledge questions, and Table 3 shows the short-term evolution of the GPs' PSL diagnostic accuracy and management. Comparison of data from the basic melanoma knowledge questionnaire shows a significant improvement in the immediate post-training outcomes of group B for lentigo maligna management much greater than for group A, and the reverse (a greater improvement in the immediate post-training outcomes for group A than for group B) for the question on characteristics of high-risk melanoma patients. It should also be noted that pre-training results differed between the two groups for melanoma patient follow-up and characteristics of high-risk melanoma patients.

In addition, Table 4 illustrates the number of GPs who appropriately managed ≥ 50% of the melanoma cases, i.e., at least one melanoma among the 2 melanomas of the 10 clinical cases in question set A and at least two of the three melanomas (including the lentigo maligna of the ear) presented among the 10 clinical cases in set B. As the number of melanomas differed in the two sets (A and B) of the questionnaire, we chose to use ≥ 50% rather than absolute scores to present the results. This (Table 3) .

Of the 89 GPs, only 27 (30%) completed the 1-year questionnaire. In total, 18 GPs of group A filled in question set B and question set A was answered by nine GPs of group B. Table 5 shows the long-term evolution of these 27 GPs' competences in melanoma and PSL diagnosis and management.

One year after training, the GPs' correct responses to the basic melanoma knowledge questionnaire were maintained only for one question and increased for another question in group A. Unfortunately, this was not the case for the last three questions. In group B, the 1-year results were similar or even worse than the pre-training results except for one question. In the second part of the questionnaire, the number of correctly managed clinical cases significantly decreased in the long-term. In group A, the GPs' mean overall score immediately after training of 5.1 ± 1.76 out of 10 correctly managed clinical cases dropped to 3.8 ± 1.52 at 1-year post-training. For the nine 

The aim of our prospective interventional study was to determine the short-and long-term competences of GPs after a 1-h training session on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

Results of the short-term evaluations revealed a significant increase in the immediate post-training assessment on most topics in the melanoma knowledge questionnaire. The greater improvements in the immediate post-training results of group B for lentigo maligna management, and group A for characteristics of high-risk melanoma patients, are most likely due to the non-identical but similar questions on the same training topic presented to both groups and mistakenly considered of equal difficulty by the authors. The difference in the pre-test results between the two groups for the questions on melanoma patient follow-up and characteristics of high-risk melanoma patients is probably also due to more difficult questions in set A than in set B. The significant increase in the immediate post-training results of the PSL diagnostic accuracy and management are consistent with the main goal of the training session which was to teach melanoma diagnosis and management. However, most benign lesions were not diagnosed as such, and were referred to the dermatologist for further assessment or excision. Therefore, it is likely that the training did not focus sufficiently on the diagnosis of benign lesions and other differential diagnoses of melanoma, such as pigmented basal cell carcinoma. These PSL were still very poorly recognized after training which could result in over-diagnosis of melanoma. While over-diagnosis could lead to psychological stress for the patient, unnecessary referrals to dermatologist, and increased treatment costs [30] , it would be in patients' best interests for GPs not to underdiagnose melanoma, which is an aggressive cancer except when diagnosed at an early stage.

Only 27 GPs (30%) responded to the 1-year questionnaire. Our main finding was that the competences they had acquired immediately after the training were not maintained over time. Our results are in line with other studies and suggest that the long-term loss of acquired competences after training programs for GPs is common. Figure 1 shows the evolution of the GPs' mean overall scores in diagnosing and managing skin lesions after nine different training programs in melanoma diagnosis. Overall, these graphs illustrate that GPs' competences tend to improve for up to 6 months after training. However, beyond this limit, a gradual loss of competences occurs, despite the availability of refresher training material after six of the training sessions. The refresher training material varied between the different programs, and included unlimited e-learning access [31] , self-assessment paper-based training sessions [32, 33] , a second face-to-face training session at three months post-training [17] , access to teledermatology feedback services [34] , the provision of PDF-files of the course [35] , and a triage algorithm for dermoscopic assessment of pigmented skin lesions [36] . Our study was one of the three training programs that did not provide refresher material [37] [38] [39] .

There are several possible reasons for this long-term decline in the GPs' competences, such as lack of daily opportunities to use the skills and knowledge obtained during training, lack of time in clinical practice to evaluate skin lesions, lack of confidence in their diagnostic abilities, the absence of prompt dermatologist's feedback on the lesions assessed, and a gradual decline in the GPs' motivation for self-training. Similarly, the low response rate (30%) observed in our study 1 year after training may also reflect a lack of time and loss of motivation for training in melanoma diagnosis. The low response rate of GPs may also possibly reveal a lack of confidence in their ability to assess skin lesions and fear of being judged for not having maintained the acquired skills.

Robinson et al. [36] and Marra et al. [24] proposed that refresher training might help physicians retain accuracy in melanoma diagnosis. However, they did not specify the type and form of these refresher activities. However, as demonstrated in Fig. 1 , the long-term competences decreased in all studies, both in the training programs with additional refresher material and in those without it. We speculate that this decrease may be related to a progressive decline in the motivation and interest of GPs in skills they do not regularly use. In this respect, Marra et al. [24] encouraged the organization of "face-to-face" refresher training sessions at regular intervals in order to maintain appropriate management of skin lesions. This idea could be extended to interactive e-learning sessions, which have increasingly widespread since the COVID-19 pandemic in 2020. Finally, the ideal frequency of such refresher training sessions should be established. Figure 1 shows that the threshold between improvement and deterioration in competences was 6 months after training. This suggests that refresher sessions should take place at a regular frequency of at least once every 6 months.

In view of all these findings, we would recommend organizing some short interactive e-learning refresher sessions every three to 6 months after a training session. These sessions could consist of "real-life doctorpatient scenarios and problem solving" and should also include some valuable take-home messages that are easy to apply in the doctor's office. As GPs have little time to devote to training, these sessions should not exceed 15 to 20 min. We are convinced that these refresher sessions would motivate GPs to maintain an interest in the field and encourage them to apply the competences they have acquired more regularly in their clinical practice.

Due to its geographic distribution in the five French-speaking districts of Belgium, the distribution of ages and gender, and the proportion of urban and rural GPs, the study population was representative of Belgium's population of French-speaking GPs. As this study was conducted in the context of the GPs' mandatory continuing medical education program, it is reasonable to assume that their inclusion was not significantly determined by their initial interest in melanoma diagnosis. Although the 10 clinical cases and five multiple-choice questions used in the two evaluation sets (A and B) were similar but not identical, we were able to avoid recall bias at 1-year posttraining by switching the two sets of questionnaires between groups.

This study also has some limitations. Firstly, the 1-year questionnaire was sent to the GPs by return mail, which resulted in a significant drop-out of the participating GPs. Since only two of the four annual continuing medical education courses are mandatory, conducting the 12-month evaluation at one of these meetings, 1 year after the training, would have statistically reached 50% of the participants. In this context, we chose to send the questionnaire by mail in the hope of reaching more GPs. Unfortunately, this was not the case. As only 27 GPs completed the questionnaire, we could not test any significant inference on the outcomes of the three evaluation periods. Only descriptive statistics were obtained.

Secondly, the two groups of GPs who evaluated question sets A and B were constituted on the basis of their membership in the participating continuing medical education groups. Therefore, we cannot exclude socio-demographic differences between groups that could have influenced the pre-and post-training results. In addition, the process of switching the two evaluation sets (A and B) between groups made us realize that the questions in set A were more difficult than those in set B. This may also have influenced the results of the two groups.

Finally, this study assessed the GPs' diagnostic and management skills solely in a training setting. When measured on photographs of skin lesions, the competences do not always reflect the GPs' performance (changes in practice pattern) in their daily clinical practice. To further determine whether the measured competences reflect the GPs' performance in daily practice, it would have been necessary to assess the number of melanomas the participants had diagnosed or referred to dermatology during the 12-month period between the two assessments.

This study measured the accuracy of GPs in the diagnose and management of PSL in a training setting after a 1-h training session on naked-eye melanoma diagnosis. Although the GPs' competences improved in the short-term, they significantly decreased 1 year later. This is in line with other studies which showed that the skills acquired in training improved for up to 6 months, before rapidly regressing thereafter. To maintain and improve the GPs' competences over a longer period, an essential aid may be the organization of timely refresher training sessions at regular intervals. However, further studies are needed to assess whether these refresher sessions are able to prevent the loss of the diagnostic and management skills that GPs acquire during trainings.

The online version contains supplementary material available at https:// doi. org/ 10. 1007/ s13187-021-02063-6.

Acknowledgements The authors would like to thank all the GPs that kindly participated in this study, the organizers of the different GLEM (Groupe Local d'Évaluation Médicale) of each of the five Frenchspeaking districts of Belgium and the Centre Académique de Médecine Générale (CAMG) of the Université Catholique de Louvain.

Author Contribution E. Harkemanne: writing of original draft, visualization, and conceptualization. C. Duyver and S. Leconte: investigation, resources, and methodology. K. Sawadogo: formal analysis and validation. M. Baeck: reviewing and editing. I. Tromme: investigation, methodology, reviewing and editing, and supervision.

The data collected and the material used to carry out this study are available upon request from the corresponding author.

Ethics Approval The study was approved by the Ethical Committee of the Catholic University of Louvain in Brussels, Belgium.

All participating general practitioners gave their consent to the use of their data collected for the purpose of this study. The clinical images of skin lesions shown to the participants during the training and used for evaluation of their competences were photographs from the database of images of patients from the Dermatology department of the Cliniques universitaires Saint-Luc, Brussels (Belgium). The patients of the Dermatology department gave their written personal consent to the use of their anonymous images for education and clinical studies.

All participating general practitioners and patients of the Dermatology department gave their consent for publication of their anonymized data.

The authors declare that they have no conflict of interest. . 1 a Short-and long-term evolution of general practitioners' (GPs') mean overall score in diagnosing and managing skin lesions reported in nine studies on training in melanoma diagnosis. b Distribution of the corresponding learning-curve slopes ( Δ ∕Δ ) over time. The graphs make no distinction between sessions that trained GPs in naked-eye recognition of skin lesions and/or diagnosis using dermoscopy. However, we only included studies that specified the short-and long-term scores of the GPs' competences in the period after the educational program. The learning-curve slope evaluates the improvement or deterioration in the GPs' competences over a period of time (expressed in months). Figure 1b shows that the competences tend to improve when measured up to 6 months after training, while competences measured at 6 months and beyond tend to decrease. In addition, the availability of refresher training material had no impact on the sustainability of the GPs' competences over the long-term. To note that Badertscher et al. [33] and Augustsson et al. [34] used the median score instead of the mean test score, which could be a limitation in the comparison of these studies since the median and the mean are not always equal

Melanoma in primary care The role of the general practitioner

Accuracy in melanoma detection: a 10-year multicenter survey

Melanoma early detection: big data, bigger picture

Skin cancer screening by dermatologists, family practitioners, and internists: barriers and facilitating factors

Referrals by general practitioners for suspicious skin lesions: the urgency of training

Melanoma diagnostic practices of French-speaking Belgian general practitioners and the prospective study of their pigmented skin lesion diagnostic accuracy and management

Skin cancer education for primary care physicians: A systematic review of published evaluated interventions

Dermoscopy for melanoma detection and triage in primary care: a systematic review

Training primary care physicians in dermoscopy for skin cancer detection: a scoping review

Dermoscopy, a useful tool for general practitioners in melanoma screening: a nationwide survey

Examining the factors associated with past and present dermoscopy use among family physicians

Decision support software to help primary care physicians triage skin cancer: a pilot study

Impact of skin cancer education on general practitioners' diagnostic skills

Using the Internet to teach melanoma management guidelines to primary care physicians

Diagnostic ability of general practitioners and dermatologists in discriminating pigmented skin lesions

Diagnostic and referral accuracy of family doctors in melanoma screening: effect of a short formal training

Dedicated general practitioner education sessions can improve diagnostic capabilities and may have a positive effect on referral patterns for common skin lesions

The role of general practitioners in diagnosis of cutaneous melanoma: a population-based study in France

Skin lesions suspected of malignancy: an increasing burden on general practice

The current and future role of general practitioners in skin cancer care: an assessment of 268 general practitioners

Early detection of skin cancer by family physicians: a pilot project

Is an online skin cancer toolkit an effective way to educate primary care physicians about skin cancer diagnosis and referral?

Efficacy of a general practitioner training campaign for early detection of melanoma in France

The effect of a dermato-oncologic training program on the diagnostic skills and quality of referrals for suspicious skin lesions by general practitioners

Early detection of malignant melanoma: the role of physician examination and selfexamination of the skin

Superiority of a cognitive education with photographs over ABCD criteria in the education of the general population to the early detection of melanoma: a randomized study

First prospective study of the recognition process of melanoma in dermatological practice

Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support

The REDCap consortium: building an international community of software platform partners

Overdiagnosis of melanoma -causes, consequences and solutions

Effects on skills and practice from a web-based skin cancer course for primary care providers

An educational intervention to improve diagnosis and management of suspicious skin lesions

The effect of an educational programme to improve the skills of general practitioners in diagnosing melanocytic/pigmented lesions

A multifaceted intervention: no increase in general practitioners' competence to diagnose skin cancer (minSKIN) -randomized controlled trial

Effects of a 1-day training course in dermoscopy among general practitioners

A randomized trial on the efficacy of mastery learning for primary care provider melanoma opportunistic screening skills and practice

Effectiveness of a skin cancer control educational intervention for internal medicine housestaff and attending physicians

Effect of a webbased curriculum on primary care practice: basic skin cancer triage trial