key: cord-1002317-558yfisz
authors: Holfert, Jonathan; Kesting, Marco R.; Buchbender, Mayte
title: An observational study of learning effects of peer‐assisted learning tests in COVID‐19 pandemic times
date: 2021-09-20
journal: J Dent Educ
DOI: 10.1002/jdd.12790
sha: 2d1f2589df21b285e4bc91dee45bdefd17b94123
doc_id: 1002317
cord_uid: 558yfisz

PURPOSE: This study aimed to determine whether peer‐assisted learning (PAL) is a more effective learning and test method in terms of oral‐ and maxillofacial surgery. MATERIAL AND METHODS: In July 2020, a total of 267 students took a PAL‐based exam on fictional patients with surgical issues, in which they had to evaluate two fellow students and were themselves evaluated by two fellow students. The students evaluated their experience with the PAL‐based exam through a questionnaire which consisted of five given statements (answer possibilities: agree, disagree, neutral) and two questions (answer possibilities: better, equal, worse) to rate. RESULTS: In the survey, 77.9% of the students rated PAL as a better learning method and 21% rated it as at least equally effective to the known multiple‐choice (MC) test. A total of 74.9% of the students indicated that they learned more content with PAL and 20.2% said they learned the same amount; 83.7% said that their “clinical thinking has improved” through PAL. In the comments, 73% of the students noted that they think PAL is a good learning method, and at least 22% rated it as useful but in need of improvements. Only 5% did not see PAL as an acceptable learning method. In contrast to this, 1.3% saw PAL as a “bad alternative to MC tests.” CONCLUSION: PAL, especially peer assessment, might represent a better learning method as it might encourage students to deal more intensively with the learning content and to improve clinical thinking.

Training in dental clinics in times of digitalization and higher requirements of the students (i.e., fair and objective assessment) is becoming increasingly challenging. A rapid shift in thinking toward treading new paths will affect the universities due to the pandemic outbreak of the corona virus at the latest. According to the national German dental licensing laws, students should obtain theoretical knowledge through lectures and perform dental treatments for patients by themselves in a supervised course from the 3rd to the 5th year of study. Currently, the restructuring of practical and theoretical courses is difficult due to common quarantine regulations meant to guarantee the health of students, patients, and university staff. The students' theoretical education needs to be continued without any conventional lectures in the auditorium, what makes it difficult to keep the students motivated to continue learning. In general, it is challenging and hard to assess students' skills, knowledge, aptitude, and behavior with only online classes, especially during the current governmental restrictions. 1 However, technology-mediated distance learning and elearning seem to be good alternatives depending on the attitudes of both the faculty and students. 2 Redesigning the theoretical education curriculum requires the examination modalities of conventional multiple-choice (MC) tests with compulsory presence to be changed as well. 3, 4 In a survey with 69 European dental institutions, the authors determined that the lockdown led dental schools to postpone formative (46%) and summative assessments (42%) or to organize examinations entirely online (50%), and 72% of the schools considered postponing the evaluation of required clinical competences for their students. 5 There was an urgent need to develop new types of assessments for graduate students, especially during the pandemic, rather than postpone them.

Peer teaching or peer-assisted learning (PAL) could be one such alternative method. To paraphrase PAL, the large number of authors refers to Topping, who defined this term in 1998 as follows: "Peer learning can be defined as the acquisition of knowledge and skill through active helping and supporting among status equals or matched companions. It involves people from similar social groupings who are not professional teachers helping each other to learn and learning themselves by so doing." 6 PAL is widely used in medical school education, 7 for example, in preclinical anatomy 8 and problem-based learning courses. 9, 10 In addition, PAL is also the part of the practical clinical training of physical examinations, 11 learning to communicate with patients 12 or the acquisition or further development of theoretical and/or practical skills in skills labs. [13] [14] [15] This broad application is based on the fact that PAL has many advan-tages, as the literature shows. PAL can increase student confidence in clinical practice 16 and is able to be equivalent, if not more effective, in imparting knowledge of practical work compared to trained teaching staff. 13, 15, 17 Furthermore, PAL can offer a positive learning environment (cooperative learning through joint discussions, searching for solutions, etc.) and is characterized by an "openness," as students explain misunderstandings, seem to be less stressed and a certain "fear of failure" is missing. 18 Moreover, it should be mentioned that PAL can also be carried out successfully online, which is a considerable advantage in times of coronavirus but also in new ways of digital teaching and examination. 19 Scicluna et al. 20 evaluated a structured peer learning program. They distinguished between students who were taught by other students who had 1 year more experience (vertical integration) and students taught by other students who had between 1 and 6 years more experience (near peer teaching). The researchers developed a comprehensive questionnaire to evaluate the peer learning among the participants, sent it to 1606 participating medical students, and received valid responses from approximately 20% of the medical students (n = 328). In general, the respondents liked the experience of vertical integration, and the more experienced students felt obligated to teach the less experienced students, feeling personally responsible for the students' education.

One important component of PAL is peer-assisted tests. Since, to our knowledge, there is a lack of data concerning this component, this study evaluated a peer test to investigate how PAL tests are generally evaluated and how or to what extent they can improve the students' learning success.

This observational study reports findings regarding students from years 3-5 within the Department of Oral and Maxillofacial Surgery at Friedrich-Alexander University Erlangen-Nürnberg (FAU) in the summer term of 2020. The study was notified at the ethics committee of FAU, and a submission for approval was not mandatory by them. The level of difficulty of the PAL test was adjusted to the expected knowledge level of the students according to their academic year. Using different patient cases (an anticoagulated patient and tooth extraction, anaphylactic shock, oral squamous cell carcinoma, reconstructive tumor procedures, diabetic patient), the students had to answer theoretical questions (eight to 11) about the case. According to the academic year, students in the third year had a further final question related to suture materials, students in the fourth year had a question related to immune responses, and students in the fifth year had to propose a treatment for the plastic reconstruction of a facial defect. Each student had 24 h to answer the questions and upload the completed file to the internal online platform for students. Afterward, on the second day, peer tutors were selected randomly via the online platform, and each student received two independent peer tutors from the same academic year and thus acted as one of the two peer tutors for another student. Peer tutors rated their peer's work in a rating system (1-low to 5-high) and had a text field for providing constructive feedback after carefully reading the answers of their colleagues and comparing them with their own answers. All answers (the test itself and the answers given as a peer tutor) were supervised by one coordinator within the department with professional experience (oral and maxillofacial surgeon), who responded to comments if necessary to determine whether the test had been passed via the online platform during or after the test session within one day.

To assess and clarify the learning effect of PAL as our main question, we used a questionnaire, which was inspired by previous questionnaires about PAL. 11, 21 The survey was evaluated anonymously by the students and disseminated via an online link. It was mandatory to fill and included five statements and two questions.

The five statements were as follows (rated as agree, disagree, neutral):

• I think my clinical reasoning has improved due to the peer test. • Due to the peer test, I have simultaneously improved my critical ability and my ability to give feedback by giving my fellow students constructive feedback. • I believe my personal data are endangered due to the peer test. • Due to the peer test, I fear that my personal data will be exposed by my fellow students. • Due to the peer test, the community of students this semester was strengthened.

The two questions were as follows (rated with better, equal, worse):

• How would you rate the learning effect of this peer test compared to that of MC tests? • How do you think you dealt with the learning content in PAL compared to that of MC tests?

In addition, the students were able to write comments such as suggestions for improvement, their points of view, and other notes about PAL. The general message of each comment was classified as "good," "improvable," and "not acceptable," and comments were grouped by their core statements and the frequency with which each type of comment appeared. To analyze the data, we used a univariate descriptive statistic.

A total of 267 students were included and filled the evaluation questionnaire after they took PAL-based exam. The detailed results of the answers to the five statements and two questions of the survey can be seen in Table 1 . In total, 153 comments were able to be analyzed, and the general messages concerning the peer-assisted test were extracted and classified as good (73%), improvable (22%), and unacceptable (5%). Moreover, Table 2 shows that 63.4% of the students explicitly mentioned that "PAL is generally good." In contrast, 1.3% of the students rated PAL as a "bad alternative to MC tests." A total of 26.1% of the students who wrote comments indicated that they had "learned more with PAL than they would with an MC test." As for fictitious patient cases, 22.9% of the students rated the task as "more practical" than an MC test. Further, in 13.1% of the comments, students indicated that "because of the peer assessment, (they learned) new therapy approaches." PAL improved the "critical assessment" ability of 3.9% of the participants and the "self-reflection" ability of 3.3% of the participants.

Within this study, we referred to a theoretical part of PAL, examining not only the learning itself but also the assessment of other students and the resulting learning effect through a peer-assisted test. The vast majority of the students (78%) stated they gained more knowledge through PAL than they would have with an MC test; 21% of them indicated that they learned at least as much as they would have. This makes sense, as 74.9% of the students stated that they learned better or equally well (20%) with the peer-assisted test compared to an MC test. Previously, all participants had experience with MC tests within the department. It can be assumed that students prepare thoroughly in a short period just before the MC exam, take the exam, and then check to see if they passed, although they may not know which answers were correct. Thus, they may deal with the topic intensively, but this is only during their test preparation and the test itself.

In this PAL exam, in addition to studying, the assessment also required following up on the learning material, as students recapitulated their own answers, compared them with the answers of their fellow students, and identified or improved their incorrect answers. This corresponds to another study that observed a shift from an assessment of learning (founded on the idea of Martinez & Lipson) to assessment for learning. 22 In an assessment of learning, the assessment is seen as separate from the educational process and only serves to determine whether the students have acquired sufficient knowledge and skills. With an assessment for learning, however, the assessment itself is embedded in the educational process in order to teach students individually in the most effective way. Peer tutors have the opportunity to individually comment on the tests of fellow students, thus a high qualitative feedback results, which can be a powerful influence on student learning, since students are significantly more motivated for further learning. 23, 24 In a study by Cameron et al., a similar result emerged, since (bachelor degree students) BDS-1 students taught by BDS-5 students agreed that they had received useful feedback (87% of students in the peer teaching group vs. 61% of students in the traditional teaching group). 25 They concluded that students themselves might have a better ability to teach and correct colleagues and still remain objective than teachers do. Clinical reasoning resulting from PAL is not precisely defined in the literature but is associated with clinical decision-making or judgment. 26 It is crucial to solve clinical problems by collecting and analyzing patient information strategizing and finally making decisions about treatment. 27, 28 Clinical reasoning was promoted in two ways in this study. First, through the teaching method itself, since the nonanonymized evaluation simulates a kind of consultation. The comments of their peers provided new points of view and additional knowledge for students, who thus obtained a comprehensive view of the examination. In this way, they may consider other methods, perspectives, and concepts, whereby extensive knowledge and established treatment concepts have to form a common base. The second aspect that promoted clinical reasoning was the impact of the design of these exams, which contained a fictitious patient with free text answers, unlike MC tests. Such exams based on patient scenarios can be a possible scheme for PAL tests to approximate clinical situations for intensive examination of relevant topics. 29 In addition to the positive learning effects mentioned above, 64.8% of the participants agreed that two types of crucial soft skills involved should not be underestimated. In contrast to an MC test, in PAL, the students are actively included in the assessment, as they write qualitative feedback, which is a soft skill. 30 Upon receiving this feedback, the students have to rethink their answers and admit possible mistakes, which may lead to a new motivation. Thus, with PAL, students are given confidence, even if they fail, so they can remain motivated and deal with the specific learning content in a different way. 31 Possible negative effects of PAL tests should be discussed. Putting responsibility for grading an examination on the students could also lead to negative consequences, such as frustration, withdrawal, and counterproductive work behavior. 32 It is important that faculty are available to clarify unresolved questions and other issues. However, the study of Cameron et al. 25 showed that BDS-1 students in the tutor group totally disagreed with the statement "I did not feel comfortable asking questions" (61% in the peer tutoring group vs. 26% in the traditional teaching group) but agreed that they had received useful feedback (87% in the peer tutoring group vs. 61% in the traditional teaching group). Our evaluation also shows that most of the students did not fear the exposure of their personal data. Moreover, many students thought that the student community was even strengthened during the semester (43.8%). This corresponds to another study, 33 where students from the third academic year (D3) were each assigned to a student from the fourth academic year (D4) for one year and discussed topics such as maxillofacial surgery, endodontics, CAD/CAM technology, and implantology with them. Overall, 86.2% of our students recommended introducing this type of PAL general practice model into dental schools. Another negative effect, and probably the most critical point in PAL tests, is the lack of anonymization. This is reflected in the survey, as 66.3% of the students believed that their personal data were at risk of exposure. This is not solely about the students' data, but rather about the feared exposure in front of their peers and should definitely be taken seriously in further studies. But, we should consider that education also prepares for the profession and in this a presentation of the therapeutic approaches and decisions to patients and colleagues may be required at any time. 34 Moreover, anonymity in a peer-assisted test might even lead to irresponsible or exceptionally harsh feedback and prevent oral exchange and clarification between assessors and those being assessed. 35 However, this effect can also be seen bidirectionally. In the absence of anonymization and with an assigned peer with stronger social relationship, better judgments could follow than of those with no social relationship at all. The idea behind PAL is open communication among equals, as the learners are all in the same situation and because of their experience and professional status, the challenges they face, etc., and they may have a better connection with one another. 6 In further studies pseudonyms should be used instead of personal data to prevent social exposure and embarrassment. Nevertheless, this study shows that PAL test is widely accepted by students because 63.4% of them commented that their learning experience with PAL was generally good. Specifically, 26.1% of them commented that they experienced a greater learning effect with PAL than with an MC test, and 13.1% said they discovered new learning approaches. This corresponds to a study 20 in which 80% of the students stated experiencing a positive learning effect. In addition, between 86.7% and 94.2% of the participants were able to derive learning benefits from being a near-peer teacher. This correlates with our results: the students benefitted from providing feedback, deepening their knowledge through the process of teaching, and being motivated by their fellow students. A frequent comment (among 22.9% of the participants) was that a fictitious patient is much more practical and promotes clinical reasoning in a peer-assisted test.

Ultimately, it must be emphasized how well PAL test was also adapted to the restrictions imposed by the COVID-19 pandemic. In addition to its numerous negative effects, the pandemic may also catalyze a change in education. 36 PAL is not restricted by social distance, and so soft skills and knowledge can both be promoted. According to the literature, e-learning alone does not result in greater learning success, but it also does not result in inferior learning 37 ; however, e-learning does not have to completely replace conventional teaching, as it can be integrated into such teaching and lead to an advantageous blended learning strategy. 19, 38 There are a few shortcomings in this study that need to be mentioned and critically discussed. One limitation of the study is the fact that only the influence of PAL on theoretical knowledge transfer and not the influence practical skill improvement is considered. It should also be noted that PAL is a general term that summarizes various PAL programs (e.g., peer tutoring, peer-led teaching, peer-supported teaching). 39 In this study, the focus was only on peer assessment, thus reflecting only one type of PAL and making it almost impossible to gain a comprehensive understanding of PAL, its modes of operation, and the spectrum of PAL activities. Moreover, knowledge gained from staff feedback and the ability to give feedback should be further investigated in future studies. Ultimately, these results come from the subjective impressions of the students. Furthermore, this study only represents a high level of acceptance for PAL among students, but not to what extent this teaching method is more effective compared to MC tests, since no control group is integrated. A more objective approach, that is, an additional control group that takes an MC test, is needed.

The authors concluded that peer-assisted learning, especially peer assessment, results in a greater learning effect and improves clinical thinking of the students, although a small part of the students has voted PAL as a worse exam alternative.

The present work was performed in fulfillment of the requirements for obtaining the degree Dr. med. dent for Jonathan Holfert after the state examination in dentistry at the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). Open access funding enabled and organized by Projekt DEAL. 

Impact of COVID-19 on dental education in the United States

Teaching effectiveness in technologymediated distance learning

Remoteonline case-based learning: a comparison of remote-online and face-to-face, case-based learning -a randomized controlled trial

Can e-learning improve the performance of undergraduate medical students in clinical microbiology examinations?

COVID-19: the immediate response of european academic dental institutions and future implications for dental education

Peer-assisted learning. L Erlbaum Associates

Peer-assisted learning" (PAL) in the skills-lab-an inventory at the medical faculties of the Federal Republic of Germany

Peer-assisted learning in a gross anatomy dissection course

Problem-based learning: description, advantages, disadvantages, scenarios and facilitation

Problem-based learning in dental education: a systematic review of the literature

Peer-assisted learning in teaching clinical examination to junior medical students

Peer assisted learning in patient-centred interviewing: the impact on student tutors

Student teachers can be as good as associate professors in teaching clinical skills

Undergraduate technical skills training guided by student tutors-analysis of tutors' attitudes, tutees' acceptance and learning progress in an innovative teaching model

Peer-assisted versus faculty staff-led skills laboratory training: a randomised controlled trial

A systematic review of peer teaching and learning in clinical education

Clinical skills education: outcomes of relationships between junior medical students, senior peers and simulated patients

Benefits and shortcomings of peer assisted learning (PAL) in higher education: an appraisal by students. Paper presented at: Peer Assisted Learning Conference Proceedings

Peer learning a pedagogical approach to enhance online learning: a qualitative exploration

Peer learning in the UNSW medicine program

Evaluation of peer-led study groups in a PharmD program

Programmatic assessment: from assessment of learning to assessment for learning

Assessment, feedback and the alchemy of learning

Students as stakeholders in assessment: how students perceive the value of an assessment

Peer assisted learning: teaching dental skills and enhancing graduate attributes

Research in clinical reasoning: past history and current trends

Clinical reasoning skills in final-year dental students: a qualitative cross-curricula comparison

Clinical reasoning: concept analysis

Peer-assisted learning in scenariobased simulation

Same-level peer-assisted learning in medical clinical placements: a narrative systematic review

Giving feedback in clinical settings

Consequences of positive and negative feedback: the impact on emotions and extra-role behaviors

Dentists' perceptions of their peer learning experiences in dental school and effects on practice

The responsibilities and rights of dental professionals 1. Introduction

Peer assessment practices in nurse education: an integrative review

COVID-19 as a catalyst for educational change

E-learning for health professionals

The impact of E-learning in medical education

Peer-assisted learning: time for nomenclature clarification

An observational study of learning effects of peer-assisted learning tests in COVID-19 pandemic times