key: cord-0928062-w6p1y3qn
authors: Inoue, Natalie; Aldosari, Muath; Park, Sang E.; Ohyama, Hiroe
title: The impact of COVID‐19 pandemic on student performance and self‐evaluation in preclinical operative dentistry
date: 2021-09-18
journal: Eur J Dent Educ
DOI: 10.1111/eje.12713
sha: 89bd4ac3ffd3131fd6e80e652d4986670022967b
doc_id: 928062
cord_uid: w6p1y3qn

INTRODUCTION: During the COVID‐19 pandemic, dental schools were required to reformat their curricula to accommodate regulations mandated to protect the health of students and faculty. For students enrolled in the Operative Dentistry preclinical courses at the Harvard School of Dental Medicine (HSDM), this modified curriculum included frontloading the course with lectures delivered remotely, followed by in‐person laboratory exercises of learned concepts. The aim of this article was to determine the impact that the modifications had on student performance and student self‐evaluation capabilities. MATERIALS AND METHODS: Thirty‐eight students were introduced to this restructured course. Their performance in a final multiple‐choice (MC) examination, four preclinical laboratory competency assessments (class II amalgam preparation and restoration, class III composite preparation and restoration) and their self‐assessment of these preclinical competency assessments were then compared with the pre‐COVID pandemic (P‐CP) classes from years 2014 to 2019 (n = 216 students). Linear regressions were performed to determine differences in mean faculty scores, self‐assessment scores, student‐faculty score gaps (S‐F gaps) and absolute S‐F gaps seen between the class impacted by the pandemic and the P‐CP classes. RESULTS: The results demonstrated that students during the COVID‐19 pandemic (D‐CP) had a higher average faculty score in all four preclinical laboratory competency assessments and in the final MC examination. In addition, the S‐F gap was smaller in this cohort as compared with the P‐CP classes. CONCLUSION: Despite the challenges of restructuring the preclinical curricula, D‐CP students performed better than their P‐CP predecessors in multiple facets of this Operative Dentistry course including self‐assessment accuracy.

As a result of the recent COVID-19 pandemic, dental schools across the country have had to rapidly restructure and reformat their curriculum to ensure student safety and to meet state and national regulations on social distancing. [1] [2] [3] [4] During the initial outbreak, universities transitioned all learning to remote avenues to provide continuity in education. 5 Upon reopening of the school facilities, many universities followed the didactic lectures with preclinical exercises that divided the students into smaller subgroups to maximise social distancing. For the Operative Dentistry preclinical course taken by third-year dental students at the Harvard School of Dental Medicine Although these changes were necessary to ensure the safety of students and faculty, it also resulted in added challenges in dental education. Students reported during this time that they experienced an increased feeling of burnout, difficulties in understanding and retaining material, and reduced engagement and participation, which negatively affected their preclinical learning. [6] [7] [8] Studies suggested providing more interactive virtual classes, utilising ungraded quizzes during lecture to increase engagement and combining different virtual tools and class formats to help combat some of these challenges.

Despite recent research on the impact that the COVID-19 pandemic has had on dental education, the effects of all of these changes on student performance and self-evaluation accuracy have yet to be fully analysed. 9 In addition to preclinical and clinical skills, student self-assessment abilities are paramount in fostering healthcare professionals who are lifelong, self-directed learners. 10, 11 Competencies in self-assessment skills are prioritised and integrated in many academic programmes, and in dental education, these requirements are highlighted within the accreditation standards written by the Commission on Dental Accreditation (CODA). 12 Previous studies have found that students are often not effective in self-assessment skills upon matriculation in professional programmes. The lowest-performing students have a tendency to overestimate their performance, whilst the highest-performing students, despite more accurate, often underestimate their performance. 10, 11 Studies focusing on self-assessment in dental education in particular have also disclosed similar results. These studies have determined a significant correlation between dental students' preclinical performance and their self-assessment skills. [13] [14] [15] Taking into consideration the need for social distancing, remote learning and other protocols implemented to keep students safe during the pandemic, this article focuses on how these changes have affected student performance and student self-evaluation capabilities in an operative dentistry course. Due to the relative objectivity found in operative dentistry along with the consistency in faculty and in the course director for the course, student performance and faculty evaluations in the Operative Dentistry preclinical course at HSDM could be calibrated and compared between students 12 prior to the pandemic and the students introduced to this new, modified curriculum.

The aim of this study was to assess the differences in student performance and student self-evaluation abilities as a result of the new COVID-19 pandemic protocols. Outcome data were executed by comparing the student didactic and preclinical performance and self-evaluation skills for the P-CP classes (2014-2019) to that of the D-CP students (2020) who were introduced to the modified curriculum.

Thirty-eight third-year predoctoral students participated in the preclinical Operative Dentistry course at HSDM during the COVID-19

pandemic. To optimise learning whilst the school's physical space was temporarily closed, didactic lectures were provided remotely through distance learning on Zoom (Zoom Video Communications). Once this lecture series was completed and limited laboratory access was granted by the University, students were able to proceed with the in-person, hands-on portion of the course. The students were divided into smaller groups to maintain social distancing and to abide by new limitation on occupant capacity. In addition, access to the preclinical laboratory outside of their mandatory sessions was limited. Students were given opportunities to self-evaluate their own work throughout the course and to familiarise themselves with the self-assessment forms prior to the competency assessments at the end of the course. These assessment forms outlined the grading criteria that would be used by faculty to assess the students' work.

Upon completion of both the didactic and laboratory exercises, students were tested through a MC examination and preclinical competency assessments consisting of a class II amalgam preparation, a class II amalgam restoration, a class III composite preparation and a class III composite restoration. Assessment forms, identical to the ones used in practice, were provided to grade students on a scale from 1 to 4. These forms included detailed descriptions of the preparation and restoration parameters for each category. Students were graded on ten sections with a total possible score of 40 points for the amalgam and composite preparations, and on four sections totalling 16 points for the amalgam and 24 points for the composite restorations.

Two faculty members were chosen as graders for this study based off of their active involvement in preclinical operative courses over the past 7 years and their strong level of calibration that has been statistically confirmed in previous studies. [16] [17] [18] These graders participated as faculty in the preclinical laboratory providing feedback throughout the course's duration and remained constant throughout the course of this study from 2014 to 2020. Faculty calibration exercises were also completed throughout the course and prior to administering the preclinical laboratory competency assessments each

year. All assessments were graded blindly by removing all student identifiers from any work that was submitted. Two faculty members graded the preparations and restorations independently.

A final MC examination (120 questions in 3 h) in 2020 was administered remotely through the LockDown Browsers and a camera-monitoring system (Respondus) to maintain social distancing and academic integrity. The conventional (hard copy) method with Scantron (Eagan) answer sheets was used for the P-CP classes.

Scores on the final MC examination and the preclinical laboratory competency assessments were compared with the student performance prior to the pandemic by using the data from this course from 2014 to 2019. Other than the separation of didactic lectures and preclinical laboratory exercises, the contents of the preclinical laboratory exercises were identical to those of all previous years prior to the pandemic. Lastly, the course directors evaluated the difficulty of the final MC examination each year and maintained it at similar levels to the best of their abilities. This study was completed as outlined and approved by the Harvard University's Institutional Review Board (IRB20-1673, IRB20-1131).

In addition to the final MC examination scores, the average faculty scores and self-assessment scores were converted to percentages for each of the four preclinical laboratory competency assessments: class II amalgam preparation and restoration and class III composite preparation and restoration. The student-faculty (S-F) gap was then calculated by taking the difference between the self-assessment and average faculty scores. Because a positive gap score (student overestimating their performance) could be cancelled by a negative gap score (student under-estimating their performance), the absolute difference in the S-F gap was also calculated to measure the accuracy of self-assessment to actual performance.

In order to understand the students' population characteristics, In general, the D-CP students had a higher average faculty score in all preclinical laboratory competency assessments ( Self-assessment scores were slightly higher in the D-CP class for all laboratory exercises, but they were not statistically different when compared to the P-CP class. Although the average selfassessment scores were higher than the average faculty scores in both groups, the S-F gap was smaller between the D-CP cohort compared with the P-CP classes. The difference in S-F gap was smaller 

The year's improved student performance. 22 The amount of core difference between evaluators.

The logistics behind preclinical laboratory sessions may have also contributed towards better student performance in the D-CP class. Although students were given limited time to practice outside of mandatory class sessions, they were also given higher student-to-faculty ratios in class sessions due to social distancing limitations. This increased faculty availability could have improved the quality of instruction and feedback given to help narrow the S-F gap seen this year. It was the faculty's perception that students may have also been more engaged and efficient during these sessions knowing their access to the laboratory outside of 

Impact of COVID-19 on dental education in the United States

American College Health Association. Considerations for Reopening Institutions of Higher Education for the Spring Semester

Massachusetts Executive Office for Health and Human Services and Reopening Advisory Board. Reopening Massachusetts. 2020. https:// www.mass.gov/news/reope ning-massa chuse tts-baker -polit o-admin istra tion-initi ates-trans ition -to-first -phase -of

The impact of COVID-19 on dental education in North America-Where do we go next?

Students' perceptions on dental education in the wake of the COVID-19 pandemic

Distance learning in the COVID-19 era: comparison of student and faculty perceptions

Student perceptions of distance learning strategies during COVID-19

Connecting the dots: lessons learned from student performance in the pandemic era

Medical students' self-assessment of performance: results from three meta-analyses

The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no

Accreditation standards for dental education programs

The relationship between dental students' assessment ability and preclinical and academic performance in operative dentistry

Dental students' ability to assess their performance in a preclinical restorative course: comparison of students' and faculty members' assessments

A systematic review of the use of selfassessment in preclinical and clinical dental education

The role of digital 3D scanned models in dental students' self-assessments in preclinical operative dentistry

The effect of gender on student self-assessment skills in operative preclinical dentistry

Comparing dental student preclinical self-assessment in the US and Japan

Correlation of admission criteria with dental school performance and attrition

Metacognition and cognitive monitoring: a new area of cognitive developmental inquiry

Promoting general metacognitive awareness

Why interleaving enhances inductive learning: the roles of discrimination and retrieval

Learning, Remembering, Believing: Enhancing Human Performance

The impact of COVID-19 pandemic on student performance and self-evaluation in preclinical operative dentistry

The authors would like to thank Emily Chen and Kristie Kaczmarek for their support in conducting this study. The authors would also like to acknowledge students and faculty at the Harvard School of Dental Medicine for their participation in the Operative Dentistry preclinical course.

The data that support the findings of this study are available from the corresponding author upon reasonable request.

https://orcid.org/0000-0002-2917-7593Muath Aldosari https://orcid.org/0000-0002-2200-2408Sang E. Park https://orcid.org/0000-0003-2575-7635Hiroe Ohyama https://orcid.org/0000-0002-3599-1671