key: cord-0697641-dxp1pzg1
authors: Sletten, Sarah R.
title: Rethinking Assessment: Replacing Traditional Exams with Paper Reviews
date: 2021-06-30
journal: Journal of microbiology & biology education
DOI: 10.1128/jmbe.00109-21
sha: 4a4b1267619a436712e0618a1bdfa93d3cb361e5
doc_id: 697641
cord_uid: dxp1pzg1

The abrupt shift to remote learning due to the COVID-19 pandemic left faculty at a loss of how to administer exams, which are common methods of summative assessment in college courses. This study evaluates students’ perceptions of an alternative assessment method in which they complete Paper Review Forms on relevant primary literature in a majors microbiology course. Students indicated that they felt that Paper Reviews made microbiology content relevant to them, increased their engagement with the course content, and overall were a better way of assessment of their understanding of microbiology content than traditional exams. The format of Paper Reviews for assessment purposes makes them a good option for remote learning.

The science of teaching and learning has always had a strong focus on assessment. Best pedagogical practices include backward design (1) , which outlines course design with assessment considerations at the forefront of planning. Proctored exams have been a mainstay in college classrooms for centuries. Even with the inclusion of more project-based forms of assessments in more recent years, written exams have not lost their favor with faculty. In March 2020, COVID-19 moved higher education into remote learning environments, forcing faculty who utilized in-person exams to rethink the way they approached student assessment. According to the National Institute for Learning Outcomes Assessment's survey on assessmentrelated changes during the COVID-19 pandemic, 97% of responses made changes to their assessment strategies in some way during the spring 2020 semester (2) . Changes included things such as modifying the assessments themselves, altering assessment deadlines/grading policies, and accepting alternative forms of assessments.

In what now appears to be a serendipitous event, the 2019 spring semester brought about several changes to the General Microbiology course at the University of North Dakota (UND). Grounded in educational theory of constructivism and using backward design, the course was redesigned to better align summative assessments (previously traditional exams) to the active learning teaching style of the course. One major change was the increased use of journal articles to emphasize content in real-world scenarios. The inclusion of primary literature in undergraduate classes has been reported to help students connect course content to scientific research (3), improve critical thinking and understanding of content (4), and increase students' scientific literacy skills (5) . Additionally, Bain (6) identified embedding content into broader concerns; encouragement of students to compare, apply, evaluate, and analyze, but never to just remember; and allowing students time to reflect independently as unifying principles of good instruction.

The methodological premise for utilizing primary research for assessing students' understanding builds on Gowin's Vee scaffold, which was designed to help students make connections between science concepts and laboratory experimentation (7) . In the Vee diagram-a letter V-the left side is conceptual and contains ideas and concepts presented through lecture; the right side is where inquiry and experimental findings reside. The center of the V is where students are able to connect concepts with data to start to understand the process of science and how this drives knowledge creation.

This study was conducted in a 300-level majors General Microbiology course. Approval for use of course evaluations for this study was granted by the UND Institutional Research Board (exemption 4, IRB0002171). Participants were 31 students (43% of the total students registered) who completed the end-of-semester course evaluations for the spring 2019 semester. This course was newly redesigned using backward design; first goals and objectives were determined using ASM Curriculum Guidelines (8) , and then appropriate assessments were identified. At the end of each unit where a traditional exam would typically be inserted, a different form of summative assessment was planned-Paper Reviews. This assessment task allowed for students to choose among four primary research journal articles pertaining to the major concepts of the unit and complete a Paper Review Form (see Appendix 1 in the supplemental material) using the article to address the questions on the form. The 16-week course was divided into four units, each with a Paper Review due at the conclusion. Paper Reviews were graded using a standard evaluation rubric (see Appendix 2) . The Paper Review Form was similar across all papers/ units consisting of four identical prompts (see Table 1 ), and students were also asked to respond to a paper-specific question that varied from article to article based on the content and research discussed.

At the conclusion of the semester, students were asked the following questions specific to the Paper Reviews on their student course evaluations.

1. Journal article reviews made microbiology relevant to me (disagree, neutral, agree). 2. Journal article reviews increased my engagement in microbiology content (disagree, neutral, agree). 3. Do you feel the journal article reviews were a better assessment of your understanding of microbiology than exams would have been? Please explain (free response).

Questions 1 and 2 were quantitative in nature and were scored on a 1 to 5 scale (1, strongly agree; 2, disagree; 3, neutral; 4, agree; 5, strongly agree). Question 3 was open-ended and allowed for the collection of qualitative data.

Student course evaluations revealed that on a scale of 1 to 5, the majority of students either agreed (32%) or strongly agreed (58%) with the statement that Paper Reviews made microbiology relevant to them (mean, 4.45; standard deviation, 0.77). Most students also indicated that they agreed (29%) or strongly agreed (58%) that the Paper Reviews increased their engagement in microbiology content (mean, 4.39, standard deviation, 0.92).

The student free responses (see Table 2 ) were coded and subsequently grouped into the following themes: (i) relevant, (ii) engaging, (iii) teamwork, (iv) instructor, (v) flipped format, (vi) connectedness, and (vii) ease. From these themes, assertions were made about the student opinion of Paper Reviews over traditional exams. A large majority of students had favorable perceptions about using articles as a way to assess their understanding of microbiology. They felt that as opposed to exams that test ability to memorize material, article reviews connected course material to real-world scenarios and encouraged students to apply the course content leading to deeper learning. Students found the article reviews less stressful and more helpful, and they felt more enjoyment completing them than with exams. Students appreciated the way the article reviews aligned with structure of the flipped course, yet some found a disconnect, with the content covered in the online lectures not fully being covered in the article review questions. Pesponses to the open-ended question: Do you feel the journal article reviews were a better assessment of your understanding of microbiology than exams would have been? Please explain. Responses are unedited YES! Personally, I feel like the stress that happens when studying for exams is a leading factor to why people don't remember things afterwards. They [exams] are so into just remembering what happened for that specific amount of time and that is it. The journal articles were not even a bit stressful, making it fun and easy to read. I think I remember everything that I read throughout the semester because I didn't stress over it once and I was actually learning something relevant to the world and the class. Please keep this up, best idea for a professor to make! I don't think they were a better assessment of our knowledge, but I definitely think I learned more about microbiology from the article reviews than I would have from a test.

Based on the way you set the class up, I feel that the reviews were a better assessment of my understanding.

Yes! I didn't feel as if I was learning material just to get a question right on a test. The material stuck with me having to apply it rather than memorize it.

Yes. Exams test memorization. Article reviews test the student on if they know the topics covered in class, and also test the student to apply those topics to a real-world context. Writing paper reviews is a level above intellectual processing. When a student is required to take classroom concepts and use those concepts in out of classroom contexts, the level of learning and memorizing information is significantly higher than memorizing concepts a couple days before an exam, take the exam, and never use those concepts again. Writing article reviews greatly improved my personal learning in microbiology.

Yes, it wasn't just memorizing facts. It was applying what you learned. My nursing friend is in the other section for nursing students and said she wishes her exams would have been like that.

Yes I do because the exams are just memorizing and then forgetting information after but the paper reviews helped me learn information that I could still recite.

Definitely yes. First of all, preparing for tests more so involves memorization of material up until the test and then forgetting it. The paper reviews are helpful for learning to how read about scientific studies and showing you can understand them. Communicating science is a very important aspect and is more practical than an exam.

I'm not sure. I prefer tests, but also enjoy writing. I was just confused about what was expected from the journal reviews.

Yes, they were very helpful because they were more realistic and related to current studies and issues in microbiology.

Yes, because tests really only test memory, and understanding. The article reviews required understanding, critical thinking, and the ability to write down those ideas. The article reviews were a more fun way to use the information gained throughout the course.

I do think that journal article reviews are a better assessment because it gives us time to truly apply our knowledge. Exams can be difficult because some people may not be the best test takers and you don't just want to regurgitate information; the reviews make you apply the knowledge.

No, I believe they give practical experience but not course material specifically.

Yes, they allowed you to dig deeper to understand.

Exams would've forced me to learn the material in a way to regurgitate information, but the article reviews made me connect microbiology to my everyday life. Being able to critically review science articles improved my ability to understand science-related topics, but I feel like it didn't necessarily assess whether I knew the information or not but rather if I could understand and discuss an article.

I really enjoyed the reviews and they helped me relate microbiology to real-life events.

Yes the journal article reviews were a better assessment rather than exams, as I personally cram for exams and try to remember everything and anything by memorization and such. Having the journal article reviews made my understanding of microbiology expand as I could see all of its relations with the real world and learned about things I probably never would have.

These journal article reviews are far better than assessments. They teach us how to apply information and express ideas through writing rather than memorizing and forgetting. I learned more from these journal articles than an exam ever could. This course taught me how to apply science to the real world, something exams fail to do.

The shift away from traditional exams to Paper Reviews aimed to align the course teaching and assessment strategies. Crowe et al. (9) posit that if classroom activities focus on Bloom's taxonomy of higher-order cognitive domains (i.e., analysis and evaluation) yet assess using methods that utilize lower-order skills (i.e., knowledge and comprehension), students learn they do not need to put forth effort to fully understand the content. Paper Reviews encourage students to understand how course content relates to authentic research and develop written interpretations and assessments of scientists' work, as well as apply it to novel situations.

Incorporating primary literature in college courses is a routine practice in college courses, and many studies have assessed their benefit in the classroom, especially for upper-level STEM courses where students are often preparing for graduate schools. Nelms and Segura-Totten (10) suggest that learning how to effectively analyze papers takes practice, and students who have repeated exposure to primary literature can develop "tools" to move toward reading mastery. Additionally, Anderson (11) found that graduate students who were part of an undergraduate course that incorporated a journal club that reinforced course contents felt more prepared than their peers to discuss papers. Meaningful exposure to primary literature can benefit all students regardless of degree attainment, as it can help build STEM literacy, which is part of the National Science & Technology Council's goal to create a civil society that is better prepared for rapid technological change (12) .

The replacement of traditional exams with Paper Reviews proved to be not only an effective way to assess student understanding of content in an upper-level microbiology course but provided for an easy transition to remote learning when the COVID-19 pandemic forced students and faculty home during the spring 2020 semester. This form of student evaluation should be considered a viable alternative to in-person exams.

Supplemental material is available online only.

SUPPLEMENTAL FILE 1, DOCX file, 0.02 MB.

Understanding by design

University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment

Annotated primary scientific literature: a pedagogical tool for undergraduate courses

less is more in teaching science

Incorporating primary scientific literature in middle and high school education

What the best college teachers do

Gowin's Vee

Recommended curriculum guidelines for undergraduate microbiology education

Biology in bloom: implementing Bloom's taxonomy to enhance student learning in biology

Expert-novice comparison reveals pedagogical implications for students' analysis of primary literature

Active learning in the undergraduate classroom: a journal-club experience designed to accentuate course content

Charting a course for success: America's strategy for STEM education