key: cord-0924373-epguax5p
authors: Shahba, Ahmad A.; Sales, Ibrahim
title: Design Your Exam (DYE): A Novel Active Learning Technique to Increase Pharmacy Student Engagement in the Learning Process
date: 2021-09-30
journal: Saudi Pharm J
DOI: 10.1016/j.jsps.2021.09.011
sha: c12cf951fee0bd55345a64e9be540c3012209b96
doc_id: 924373
cord_uid: epguax5p

BACKGROUND: Due to the COVID-19 pandemic, innovative e-learning solutions should be implemented to deliver knowledge to health care students remotely. Presently, there is a paucity of studies in the literature that have examined student-designed assessments in the classroom. OBJECTIVES: To examine the educational outcomes comparing Design Your Exam (DYE) activity versus instructor-designed end-of-class (EOC) quizzes and explore student perceptions and preferences for each teaching modality. METHODS: To evaluate the proposed learning model, lectures in the Industrial Pharmacy course were delivered to students by two different approaches: instructor-designed EOC assessments and student-designed DYE. Finally, the designed learning model involved a course completion questionnaire for quality assurance and future course improvement. RESULTS: Mean exam performance for content taught using the instructor-designed EOC quizzes and DYE activity were 74.4% and 71.9%, respectively (p=0.092). Average student attendance for lectures taught using instructor-designed quizzes and DYE activity were 77.6% and 72.1%, respectively (p=0.524). A post-course survey showed that 72.2% preferred the instructor-designed EOC, 5.6% preferred DYE activity, and 16.7% preferred a combination of the activities. Respondents reported that the EOC quizzes helped them to understand the lecture material and kept them focused during the lecture and that the DYE was useful in developing their personal interaction skills. CONCLUSION: DYE is a novel active learning model that can be incorporated into student courses as an alternative to the standard didactic lectures. Further development of the DYE technique, such as including supportive audio-visual resources, is necessary in order to increase student acceptance.

Passing the torch of knowledge from the instructor to the learner has evolved throughout history. Traditional lectures required learners to humble themselves and become patient in order to acquire this inheritance (Gordon, 1997) . Although instructors have occasionally adapted their teaching techniques to suit various learning styles, our generation has witnessed an overwhelming reversal of roles. More often than not, teachers are consistently seeking ways to adapt their teaching methods. They have become patient with the limited attention spans of learners and their desire for external stimuli (Oblinger, 2003) . It is imperative upon educators to continuously develop modern teaching methods to enhance learning, engage students, and use the most effective teaching techniques during lectures (Al Shammari and Al Massaad, 2019) .

Active learning techniques are commonly used in pharmacy education and are encouraged by the Accreditation Council for Pharmacy Education (ACPE) to develop problem-solving and critical-thinking skills (Allen et al., 2013; Alruthia et al., 2019; Gleason et al., 2011) .

These methods provide immediate feedback to both the instructor and the learner, enable students to take an active role in their learning, and assist students in making meaningful connections and associations (Michael, 2006; Prince, 2004) . Although instructors are enthusiastic about implementing new approaches, designing or selecting new methods and evaluating their effectiveness as compared to commonly used and/or published activities can be challenging (Alruthia et al., 2019; Gleason et al., 2011) .

Due to the COVID-19 pandemic, distance education is no longer optional. In fact, it is the only available choice to continue the education process during such circumstances. If properly applied, distance learning can serve as valuable solution in countries that have chosen to suspend physical attendance of schools and universities. Furthermore, there has never been more of an urgency to establish opportunities for supplemental e-learning (Tretter et al., 2020) . The current situation requires innovation and cooperation between various healthcare disciplines to maintain rigorous standards of education and training for health care practitioners as well as students (Chick et al., 2020) . Several efforts are being investigated to determine the best means of actively engaging learners remotely. In "Medical Education" journal has released a new series called "Medical Education Adaptations" to encourage rapid dissemination of focused articles presenting innovative methods to deliver high-quality medical education during the current crisis (Eva and Anderson, 2020) .

Gewin proposed five tips for moving teaching online during the COVID-19 pandemic (Gewin, 2020) These tips were (1) "Don't convert your entire lecture to video ", (2) "Don't rely on live video", (3) "Invite student engagement and feedback", (4) "Check in with students often" and (5) "Identify and support struggling students". The third tip advised educators to encourage student engagement and feedback because the most common mistake in the learning process is not considering your students' opinions.

Active learning strategies vary in type and the amount of effort required for effective execution. Popular examples in pharmacy education for larger classroom sizes include flipped classroom (FC) and team-based learning (TBL). The FC reverses the traditional didactic lecture with out-of-class learning activities and information reinforcement. TBL fosters an environment of learning and exploration within small student groups under the supervision and facilitation of the instructor. The utilization of pre-(individual or team readiness assurance tests, iRAT/tRAT) and end-of-class (EOC) quizzes are considered essential to TBL activities and have been associated with increasing exam scores of suboptimally performing students, assisting students in identifying areas of improvement, and motivating students to prolong their studying time (Farland et al., 2013) . Several past studies have shown the value of TBL pedagogy in the learning process. However, most TBL studies have been conducted in courses led by faculty, with minimal examination of student-led learning courses (Bouw et al., 2015; Kolluru, 2012 ).

In the current article, we introduce a novel student-led active learning technique. Design Your Exam (DYE) is an active learning model that encourages students to prepare for the lecture prior to attending by reviewing the written material similar to FC (Wilson et al., 2019) . However, instead of directing the student to read the material before attending the lecture and then assessing his/her understanding during the discussion, the student prepares the questions himself and tests his colleagues on it. Before the student reaches this stage, he/she must clearly understand the material before he/she can accurately test his/her colleagues on this information. Accordingly, this technique was hypothesized to improve students` higher-level and critical thinking skills.

The objective of this study was to introduce the DYE activity into an Industrial Pharmacy course and compare its educational outcomes against the standard didactic method with instructor-designed EOC quizzes. Student scores, perceptions, and preferences for teaching modality were compared to determine the most effective strategy.

The current study was conducted in the industrial pharmacy course in College of Pharmacy at King Saud University (Level 9, bachelor's degree, male campus). This course covers several important pharmaceutical industry processes and an overview of the various equipment utilized in these processes. By the completion of this course, the students should gain the basic knowledge and skills which they need to gain employment in the production and/or quality control of pharmaceutical products. Examples of topics covered by the course include particle size reduction, particle size analysis, mixing, crystallization, tablet manufacturing, tablet coating and capsule production.

The course was delivered live. In our college, the normal didactic method is the standard common method to deliver the lectures. On the other hand, the DYE activity was applied for the first time in the industrial pharmacy course which might raise concerns and may cause confusion for the students until they became familiar with such new activity. In addition, the lectures assigned for the DYE method were closely related and were considered to be more suitable to be delivered by the same method. Therefore, it was decided to conduct two-thirds in the traditional format and one third incorporating the DYE activity.

Accordingly, the first nine lectures of the course were delivered through the normal didactic method followed by instructor-designed EOC quizzes. Prior to the lecture, the instructor prepared 2 -4 questions using Google Forms ® that only covered the most important information from that day's lecture (Lailaturrahmi et al., 2020) . The quizzes were designed with question randomization and provided immediate grading/feedback. Ninety percent of the lecture time was dedicated to the instructor's explanation. The remaining 10% of the lecture period was reserved for the EOC quizzes.

At the end of the lecture, the instructor sent the quiz link to the students, and each student took the test using his mobile phone after entering his student ID number. Students were instructed to answer questions without assistance from others and without using their notes (i.e. closed-book) (Leonard et al., 2012) . Students learned independently and were provided with a complete explanation of the topics during the lectures; therefore, they independently answered the quizzes without any lecture information or collaboration. The students were given a maximum of five minutes to complete the quiz and were provided with their scores and any incorrect answers after completion. In addition, the instructor was able to view the students' results immediately to assess their understanding of the lecture material. Student participation was based upon whether they took the quiz.

The subsequent four lectures of the course utilized the DYE active learning technique. Prior to each lecture, the instructor assigned reading material related to the lecture and divided it among the students into several short passages so that each student would read carefully and think critically about the part allocated to him (Dobson, 2008) . After reading the material, the student prepared two questions about the part assigned to him in either a multiple choice or true/false question format. The student-designed questions were submitted to the instructor electronically with the correct answer identified. As an incentive to motivate the students to write accurate questions and to maintain confidentiality, extra credit was awarded to the student who prepared the most difficult question. Although each student was assigned a specific section of the reading material, he was still responsible for reading the entire reading assignment. After submission, the instructor reviewed the questions and revised them to prepare an electronic test to be given at the beginning of each lecture using Google Forms® consisting of 35-45 questions for each lecture. Typically, the instructor would choose the best of the two questions submitted by the student. Student participation was determined by whether or not the student submitted their questions by the specified deadline.

In order to maximize the accessibility and learning benefits of the activity, students were divided into groups of 5-6 members to compete with other groups in taking the open-book online exam (Lull and Mathews, 2016) . In contrast to fixed grouping strategy (Chan et al., 2018) , the groups were randomized each lecture by the instructor to encourage students to read all of the assigned material and not rely upon their colleagues. This strategy also minimized the opportunity for students to divide the reading material up among themselves in fixed groups prior to the exam. Since the DYE was an active learning activity without instructor clarification and/or explanation initially, students were provided approximately ten times the number of questions given on the EOC quizzes, and students primarily focused on their assigned reading material, a collaborative, open-book format was best suited for answering the DYE quiz questions. The group with the highest score was declared the winner. In the case that more than one group shared in achieving the highest score, the group that completed the exam first would be the winner. Once the specified time for the exam was over, the instructor presented the test results and identified the student who submitted the most difficult question. At the end of the lecture or in subsequent lectures, the lecturer summarized the lecture material with an emphasis upon correcting the most common mistakes among students.

The results of the EOC or DYE lecture quizzes were not subjected to score comparison due to the major difference between testing methods in each technique; however, student comprehension was assessed during a midterm exam and a final exam.

The midterm exams were administered according to the pre-specified university schedule; therefore, it only covered 7 lectures of the EOC activity. On the other hand, the two activities overlapped in the final exam. The questions related to the lectures conducted with the instructor-designed EOC quizzes and the DYE activity were grouped and analyzed separately. In addition, differential student scores in the mid-semester and final exams were analyzed and categorized based on the implemented technique in each set of lectures.

Each set of questions were analyzed and compared to determine whether there was a significant difference between student scores on either activity.

At the end of the course, a voluntary survey was sent to all students to solicit their feedback regarding both the instructor-designed EOC quizzes and DYE activities (Lull and Mathews, 2016) . The survey was in the Arabic language and designed to evaluate student feedback anonymously. The survey was created using Google Forms® and the survey link was sent by email to the student group leader and forwarded to the students. The survey consisted of three sections related to the EOC learning model, the DYE learning model, and general points and suggestions. There were a total of 20 questions; 12 were presented in a five-point Likert scale format to assess student perception on each technique. Furthermore, 2 multiple choices, 1 fill-in and 5 open-answer questions were presented to identify the preferred learning model for the student, the existence of technical problems, suggestions, and advantages/disadvantages of each model.

The protocol for this research was reviewed and approved by the King Saud University Institutional Review Board (# E-21-5718).

SPSS 26 software was utilized to test the significance of the data. The percentage of students' attendance and activity participation were compared using independent T-test.

Students' scores were compared using paired T-test. p < 0.05 was denoted significant in the study.

Fifty-two students were enrolled in the Industrial Pharmacy course in the fall 2019 semester. One student was prohibited from taking the final exam due to excessive absences that exceed the university attendance requirements. Accordingly, this student was excluded from the study; therefore, 51 students were included in the final statistical analysis.

The average attendance was approximately 78% for the instructor-designed EOC quizzes and 72% for the DYE activity (Table 1) . Furthermore, student participation in the EOC and DYE activity was 75.0% and 78.9%, respectively. Mean exam performance from the midterm and final exams was 74.4% for the didactic method and 71.9% for the DYE activity (p=0.092). The differential exam scores were generally comparable between the two techniques, but the instructor-designed EOC quizzes showed higher percentage of students earning an A grade. When comparing attendance, participation, and student scores, there was no significant difference between the two activities (Table 1) .

Eighteen students completed the survey which represented a 39% response rate. Student responses were overwhelmingly positive for the EOC quizzes as shown in Tables 2 and 3 .

Approximately 90% of the students felt that the EOC quizzes helped them to understand the lecture material and nearly 80% and 90% responded that the EOC quizzes were useful/highly useful in helping them to understand the key information and keeping them focused during the lecture, respectively. In addition, 83.3% of the participants recommended that future offerings of the course incorporated EOC quizzes. Some examples of student self-reported advantages and disadvantages of the end-of-class quizzes are provided in Table 4 .

The DYE activity was not as popular with the students (Tables 2 and 3 ). Equal proportions of the students (44.4%) stated that they either satisfactorily or poorly/very poorly understood the material when studying it using the DYE model. The majority of students reported that the DYE was not useful/absolutely not useful or neutral in motivating them to read the lecture materials before the class and motivating them to focus during the lecture. Although most students agreed that the DYE was useful in developing their personal interaction skills (72.2%), they did not recommend its use in the future (55.6%).

Selected student quotes regarding the advantages and disadvantages of the DYE activity are provided in Table 5 .

Students were also asked to compare the two models according to their preferences. When asked which model they preferred to be used in the Industrial Pharmacy course, 72.2% preferred the EOC quizzes. Approximately 16.7% recommended a combination between both models, and 5.6% chose either the DYE alone or another activity other than the current models (Table 2) .

In the current study, an instructor-designed EOC quizzes and a student-designed assessment (DYE technique) were investigated for their potential enhancement of student engagement prior to and within the lecture. Both techniques had a positive influence upon student attendance, participation, and exam scores. Interestingly, most students preferred the instructor-designed assessments for understanding the lecture, but they confirmed the benefit of DYE for enhancing their communication and interaction skills.

Student participation was higher in the DYE activities than the EOC quizzes activities; however, the differences were not significant, and this could be due to the differences in the methods of determining participation. Regardless, student comments indicated that the EOC quizzes were successful in helping students to focus during the lecture periods. This may be due to the differences in delivery. The EOC quizzes were proceeded by a lecture with the understanding that there would be a quiz on the lecture material. In this study, a lecture followed the DYE activity; however, if a recorded lecture was made available to the students ahead of time, the students would probably focus on the lecture material to assist them in formulating questions.

Several previous studies showed positive educational outcomes of "EOC quizzes". Chan et al. reported that pharmacokinetics workshops and post-workshop quizzes were instrumental in significantly increasing average final exam scores (Chan et al., 2018) .

Vinall and Kreys found in a single-blinded, randomized, controlled, crossover study that EOC quizzes encouraged student introspection related to their personal comprehension (Vinall and Kreys, 2020) . Furthermore, Hennig et al. reported that feedback quizzes could enhance learning and performance and lead to improvements in student satisfaction with clinical pharmacokinetics courses (Hennig et al., 2019) . In our study, the mean exam scores were 74.4% for the didactic method (with EOC quizzes) and 71.9% for the DYE activity (p=.092). Differences in student mean exam scores related to each activity were not significant. These results are in agreement with previous systematic reviews of the pharmacy, medical, and nursing literature which have shown that the FC instructional design has the same effect on student performance as didactic lectures (Wilson et al., 2019) .

A meta-analysis of the FC in pharmacy education did show a small, positive increase in performance, though this was not statistically significant. The non-significant differences in student attendance, participation, and exam score results indicate that the DYE activity is a novel active learning activity and can be incorporated into student courses as an alternative to the standard didactic lectures. Furthermore, these results indicate that the DYE is non-inferior to current active learning methods such as the EOC quizzes.

Student perceptions of the EOC quizzes were more favorable than for the DYE activities.

In our study, 72.2% of students preferred the didactic with EOC quizzes methods, 5.6% preferred the DYE activity and 16.7% preferred a mix of both models. This data can be correlated with the study reported by Wilson et al. (Wilson et al., 2019) which found that 5.7%% of students preferred the flipped method alone and 47.5% preferred a mix of the didactic and flipped methods. The decreased student preference for DYE might be due to their familiarity with the didactic method and the desire for instructor-led reinforcement of content (Giuliano and Moser, 2016; Khanova et al., 2015) .

On the other hand, most students agreed that the DYE was useful in developing their personal interaction skills such as teamwork and decision-making. These results are in agreement with previous studies which showed that students positively perceived studentled TBL activities as a proactive type of peer-to-peer teaching (Bouw et al., 2015) . The students perceived that peer-led activities improved their interpersonal communication, presentation, teamwork, leadership, and evaluation skills, which are all essential for future pharmacy graduates to function as a cohesive team in a wide range of healthcare fields (Valler-Jones, 2014) Several studies highlighted the importance of considering the time required for pre-course preparation when designing student-led pre-class activities (Wilson et al., 2019 ). In the current study, the assigned reading materials were planned to be short and concise for each student. This step was undertaken to minimize cheating. However, the survey comments indicated that the students focused specifically upon their assigned reading sections and neglected reading the entire assigned lecture materials in the DYE activity. This might be one the reasons for the low student perception of DYE activity in terms of understanding the scientific information of the lecture.

Although the students benefitted from the student-designed DYE assessments, incentives such as requiring students to work in teams when submitting their questions and/or further explanation of the material such as recorded lectures, instructional videos, or interactive electures may enhance student comprehension.

Several limitations should be discussed. Due to the low response rate of the students and limited number of participants, the results of the study may not be generalizable. Future studies should include larger samples, multiple class cohorts, or students from multiple universities. In addition, all questions used in the exam comparison were multiple choice.

It may be helpful to compare student performance using other assessment strategies, such

as short-answer questions, which may be better suited to evaluate critical thinking skills.

In addition, providing the students with more explanation of the material prior to submitting their questions may encourage students to review the entire assigned material, design high-quality questions, and facilitate better student comprehension. Finally, the acquisition of advanced software, such as a plagiarism checker, may prevent the duplication of student-designed questions.

There are several potential areas for future study such as assessing the impact of studentled activities such as DYE on student knowledge retention. Further studies to select content areas most suited for DYE would be helpful. A study on student burden and the optimum pre-class preparation time across the curriculum could help pharmacy faculty in implementing student-centered learning activities. More robust studies are needed to shed light on the connection between student-led activities and improvements in learning, understanding, and satisfaction. Although this study was conducted prior to the COVID-19 crisis and the lectures were delivered live, it provides a platform for improving elearning and encourages students to play an active role in distance learning. This pilot study suggests that students may perceive student-led activities positively, and that further studies are warranted.

Student-led active learning activities encourage students to play an active role in the classroom. Nowadays, instructors must explore new active learning techniques that can enhance students' engagement particularly within in the challenging distance learning environment. The DYE activity is a novel method of active learning that can be easily incorporated into the classroom setting. Further development of the DYE technique, such as including supportive audio-visual resources, is necessary to increase student acceptance.

Funding: This research was funded by Deanship of Scientific Research at King Saud University, research group number [RG-1441-414] .

The Effect of Using Flipped Classroom Strategy on Academic Achievement and Motivation towards Learning Informatics for 11th Grade Students

Teambased learning in US colleges and schools of pharmacy

The use of active learning strategies in healthcare colleges in the Middle East

Assessment of students' satisfaction with a studentled team-based learning course

Implementing a strategy for promoting long-term meaningful learning in a pharmacokinetics course

Using Technology to Maintain the Education of Residents During the COVID-19 Pandemic

The use of formative online quizzes to enhance class preparation and scores

Medical Education Adaptations: Really Good Stuff for educational transition during a pandemic

Best Practices for Implementing Team-Based Learning in Pharmacy Education

Five tips for moving teaching online as COVID-19 takes hold

Evaluation of a Flipped Drug Literature Evaluation Course

An active-learning strategies primer for achieving ability-based educational outcomes

Teachers of Color Speak to Issues of Respect and Image, The Urban Review

Quizzing for success: Evaluation of the impact of feedback quizzes on the experiences and academic performance of undergraduate students in two clinical pharmacokinetics courses

Student Perceptions of a Flipped Pharmacotherapy Course

INSTRUCTIONAL DESIGN AND ASSESSMENT An Active-Learning Assignment Requiring Pharmacy Students to Write Medicinal Chemistry Examination Questions

Google Forms as a useful tool for online formative assessment of a pharmacotherapy course in Indonesia

An introductory review module for an anti-infectives therapeutics course

Online Self-testing Resources Prepared by Peer Tutors as a Formative Assessment Tool in Pharmacology Courses

Where's the evidence that active learning works?

Boomers, Gen-Xers, and Millennials: Understanding the "New Students

Does active learning work? A review of the research

Heart University: a new online educational forum in paediatric and adult congenital cardiac care. The future of virtual learning in a post-pandemic world?

The impact of peer-led simulations on student nurses

Use of End-of-Class Quizzes to Promote Pharmacy Student Self-Reflection, Motivate Students to Improve Study Habits, and to Improve Performance on Summative Examinations

Flipped classroom versus a didactic method with active learning in a modified team-based learning self-care pharmacotherapy course