key: cord-0825369-5rj76rvg
authors: Rooney, Sarah Ilkhanipour; Scott, Rebecca A.
title: Promoting Effective Student Teamwork Through Deliberate Instruction, Documentation, Accountability, and Assessment
date: 2020-11-10
journal: Biomed Eng Education
DOI: 10.1007/s43683-020-00038-5
sha: e8e9c0a3340f0ee29ea1d9ae8e7bee1d7d7b3e2a
doc_id: 825369
cord_uid: 5rj76rvg

nan

team. The course is normally hands-on and studiobased; however, due to Covid-19, we transitioned to distance learning right after the students had completed concept selection and were about to begin prototyping. Just before the University announced its decision to move to online learning, we administered an anonymous survey to gauge students' expectations for online learning and their prior Zoom experience. Teamwork was identified as one of the students' primary concerns: 15/45 survey respondents listed collaboration and teamwork in response to ''What concerns do you have about remote learning?'' When our course transitioned mid-semester to a remotelearning format, we faced the challenge of how to foster teamwork in an online setting, particularly with learners who are novices in teamwork. Our philosophy is that deliberately training students in teamwork and promoting their success through specific practices can and should transcend instructional format. In this paper, we describe and reflect upon the practices implemented to support student team learning during face-to-face and remote instruction; most of the described approaches can be implemented in either instructional format. Because this project did not originate with a research intent and instead is a description of practice, it does not meet the definition of human subjects research that would require an IRB determination.

In response to remote learning, we adapted the detailed design and testing phases, which are summarized here to provide context for the continued role of teamwork. We transitioned to virtual design with jus-tifications and simulations in lieu of a physical prototype with testing. To test risky design assumptions, teams created engineering drawings using SolidWorks or Onshape, completed motion studies in SolidWorks, modeled the physical system in Algodoo, simulated circuits using Tinkercad, and/or justified materials with Granta CES EduPack. The TAs supported this transition by pivoting to become experts on one or two of these software packages. Since most of our students had no prior experience with these programs, our TAs created a document in the first week of remote classes that summarized each program's capabilities (Supplement A).

To help students navigate the transition, we provided information about how to set up VPN, use Zoom, and access software licenses, as well as changes to the course structure (revised schedule, weekly advisor meetings, TA office hours, and updated assignment expectations and rubrics). We created a video that reinforced: (1) our course goals and that these goals could still be accomplished in a remote format, (2) the unique opportunity to acquire new skills in remote communication and project management, (3) new weekly advisor meetings and TA office hours (described below), and (4) specific instructions for teams in the first week back. In their first week of remote learning, we expected students to meet as a team, review the updated phase documents and rubrics, and develop a team plan, which they were to document in their team norms update (described below) due at the end of that week. In the remainder of the semester, we sent a weekly announcement that summarized what the teams' focus should be during that week.

Throughout the course, we promoted successful teamwork through (1) teaching about teamwork, (2) documenting teamwork, (3) promoting accountability, and (4) assessing teamwork (Table 1) . Successful teamwork post-transition to remote learning built upon the foundation laid at the beginning of the semester. Prior to the start of the semester, students were assigned to teams based on the CATME Team-Maker Tool (www.catme.org). 6 Team formation criteria were prioritized to distribute across teams experience with CAD, hand-tools, and Arduino, as well as writing expertise, and to optimize opportunities for teammates to meet outside of class hours by aligning students' schedules. Teaming began on the first day of class, when the 16 student teams participated in a lesson on teamwork that covered the following topics: a warmup activity that emphasized the value of diverse perspectives (Supplement B), definitions of ''team'' and ''group'', 5 discussions of gender-based differences in task division on teams, 8 stages of teams (forming, storming, norming, performing, and adjourning), 5 and types of intra-group conflict (task conflict, relationship conflict, and process conflict). 4 Stages of teams were addressed so students knew what to expect throughout the semester and to acknowledge that conflict is normal. In the second week of class, our lessons focused on project management methods (timelines/Gantt charts, meeting minutes, and document management), role-playing using the ''BET'' and ''BEAR'' techniques to provide effective feedback, 3 and instructions on how to complete CATME peer evaluations 9 and why they are important. These explicit lessons on teamwork provided a common language and specific tools for the students to utilize throughout the semester.

Building upon the lessons and tools discussed in class, students were required to document their teamwork. At the start of the semester, each team submitted a team norms document using a provided template (Supplement C) that outlined individuals' strengths, weaknesses, and learning goals, as well as plans for team meetings, communication, decision making, equitable division of labor, and conflict resolution. At the start of each subsequent phase of the design process (i.e., three times), students refined their team norms document and submitted progress reports (Supplement D). When we first transitioned to distance learning, teams were instructed to specifically address in their progress reports how they would maintain communication and work in an online, instead of faceto-face, format. At the end of the course, students individually submitted a final reflection paper that described how they ''function[ed] effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives'', 1 as well as what they learned about themselves and teamwork that they plan to apply in the future.

In addition to the team norms and progress report documents, students were expected to maintain detailed meeting notes and document division of responsibilities in their project timelines. Specifically, teams established project timelines at the start of each design phase to identify tasks, deadlines, and the lead team member(s). Meeting notes were used to document and justify key project decisions, as well as detail teammates' progress on assigned tasks. Teams organized these teamwork documents, along with all other project documents, in a shared Google Drive that was accessible to all teammates, instructors, and TAs. Overall, the multiple teamwork documentation methods were used to track progress, maintain accountability, and promote self-reflection and metacognition.

Accountability was critical after the switch to remote learning because instructors, TAs, and teammates no longer could ''drop in'' on each other during class work times. In a typical face-to-face format, students primarily work in the design studio during class (twice a week for~2 h each session). During this time, the instructors circulate among the student teams to get impromptu progress updates and answer questions. When all teams have been seen and there are no more questions, the instructors typically return to their offices (in the same building as the studio) so students know where to find them when they have additional needs. In the remote instructional format, all lecture content was asynchronous and informal studio meetings could not occur. Further, once we transitioned to a remote format, teams did not always choose to work during the official class times. To address these issues and still maintain instructor-team communication, we scheduled weekly Zoom meetings. Each team was required to schedule a weekly Zoom meeting (20-min time slots) with their assigned faculty instructor using Google Calendar appointment sign-ups. Meetings were student-driven and more formal than the in-person drop-in meetings; teams prepared agendas, explicitly demonstrated progress since the prior week's meeting, and created summary notes, receiving practice in running an efficient meeting. Teams also submitted weekly documentation, via email, to instructors summarizing three items: their accomplishments for the previous week, their plans for the upcoming week, and what help (if any) they needed from the instructor. These ''weekly updates'' from teams were established during the face-to-face portion of the course and continued after the switch to remote learning as a means of monitoring team progress. TAs also held virtual office hours each week (2 h each) via Zoom. To further help the instructors monitor team progress and address common concerns, TAs filled out weekly reports via shared Google documents (Supplement E).

Students were evaluated on teamwork with a rubric in each phase of the design process (Table 2) . This rubric was designed to align with ABET student outcome 5 1 for subsequent program assessment. The team norms document and progress reports, online file management system, interactions with their faculty instructor, and TA feedback were all used as evidence to inform the teamwork assessment. Furthermore, to promote both individual and collective responsibility, a hallmark of successful teams, 5 students completed CATME peer evaluations. 9 The numerical adjustment factor generated by CATME was used as a multiplier to determine each student's individual grade. Students were required to submit detailed, unique comments for each team member (including themself), which were used as justification for quantitative scores. Poor or general comments (e.g., ''we worked well together'') resulted in a grade penalty. Faculty instructors had the authority to override the quantitative score if deemed necessary. For example, if the CATME adjustment factor was 0.94 for a student, that student's individual grade would be 0.94 of the team grade, provided the Students were required to provide detailed, unique comments about their teammates and themselves via CATME peer evaluations-poor or general comments that did not provide adequate feedback were penalized Peer evaluations Students completed CATME peer evaluations 4 times CATME peer evaluations were used to determine individual grades comments submitted by their teammates were justified (e.g., student was regularly late and unprepared for team meetings and completed their assigned tasks with mediocre quality); however, if the teammates did not provide adequate justification for this score, the faculty instructors modified the score to better reflect the individual's contributions.

As BME Junior Design serves as the first truly teambased experience for most of our undergraduate students, it is critical that we provide formal teamwork instruction and tools to promote student success. Participants of the 2019 Fourth BME Education Summit noted that teamwork is often not explicitly taught, despite being required in many courses. 10 Following the recommendations posed in the BME Education Summit, we sought to instruct students in teamwork and provide them practice opportunities prior to capstone design. 10 The required team norms documents and progress reports resulted in important dialogue between teammates that was often overlooked in previous offerings of the course. The initial team norms document (pre-Covid) revealed that when not meeting face-to-face, 100% of teams planned to use group messaging/texting to communicate. Additional communication forms were proposed less frequently (video calls, e.g., Face-Time-6/16 teams; email-3/16; Google documents-3/16; phone calls-2/16; talk between classes-1/16). In response to ''How will you share documents/data and keep track of revision history so you don't accidentally overwrite something?'', 100% of teams indicated in their initial team norms document that they planned to use Google Drive and Google documents.

In reviewing the teams' three progress reports (1-pre-transition, 2-at the transition, 3-remote), we found that students largely maintained in the remote format the same communication structure they had established in the face-to-face format. Specifically, many teams used a group chat via GroupMe, iMessage, or texts for quick questions and to schedule meetings, and they used shared Google documents to comment on specific report sections and to assign and resolve tasks. Teams reserved important decisions, delegating tasks, and sharing ideas for group meetings. These meetings switched from face-to-face to Zoom when the class moved remote. Based on the anonymous survey administered just before the transition to remote learning, students had little prior experience with Zoom (24/45 respondents never used Zoom, 15/45 used Zoom as a participant once or twice, 5/45 used Zoom as a participant several times but had never set up a meeting, only 1/45 used Zoom to set up a meeting, and 0/45 regularly used Zoom). Students found the shared screen feature of Zoom helpful when working on tasks such as CAD, displaying their virtual prototypes, and using the virtual whiteboard.

Asking students to define their personal learning goals and document their progress toward those goals in the team norms documents aligns with the recommendations of Linder et al. 8 [0]

Teamwork Function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives with appropriate documentation (e.g., agendas, meeting notes, organized online file management system). [8] [7] Poor team dynamics with obvious unequal division of labor. Operated as individuals rather than as a team. Unable to arrive at team decisions or resolve conflict effectively. Team agendas and meeting notes lack key details. Online file management system is disorganized with unclear file names and is difficult for an outsider to navigate. [6] [0]

Total: / 20 directed practice. 2 When students documented their learning goals, it allowed them to gain the actionable support of their teammates and their instructors. Our students do not take a CAD course prior to Junior Design; therefore, most students are novices. Because we created teams using CATME criteria to distribute skills (including CAD), we successfully split up this strength. As revealed through team norms progress reports and CATME peer evaluation feedback, some teams distributed tasks based on existing skills and strengths, but many others aligned responsibilities, such as CAD, with individual learning goals. Additionally, some teams had problems remotely accessing our University's license to SolidWorks, so they switched to Onshape. Because Onshape is an online platform, it is easier for students to share work, which they found more conducive to a remote teamwork environment. Regardless of the program used, Zoom's screen-sharing capabilities allowed more experienced teammates to peer-instruct and allowed students to easily share their designs. Importantly, revisiting established team norms and learning goals at the beginning of each subsequent design phase provided teammates with a platform to discuss and update their team expectations, which was critical following the transition to a remote-learning format. Just like engineering design, teamwork is an iterative process. Prior published work [e.g., 2] demonstrates that engineering student teams move through different stages of teamwork. Furthermore, each phase of the engineering design process requires different skills. Compounding both factors here is the transition to remote learning. The dynamic nature of teamwork necessitates reflection for continuous improvement, and we observed that teams did adapt based on self-reflection. For example, one team began with 3-5 h meetings, but shifted to limiting their meetings to 2-2.5 h after they identified their limit in productivity. In the remote-learning format, the students no longer experienced the same accountability or structured time as with in-person instruction, and teams needed to agree upon new communication, meeting, and task distribution plans, especially since they no longer had 4 hours of required in-person meetings each week. Although completing the team norms progress reports formalized these discussions for students, anonymous end-of-semester feedback obtained through University-administered course evaluations indicated that several students felt some assignments seemed like busywork, with one student specifically commenting that the team norms progress reports often felt repetitive. In the future, we recommend evaluating ways to shorten the progress reports.

Based on the team norms progress reports, the absolute number of times teams met did not change very much as the semester progressed, which is surprising because some teams noted in their first progress report that they wanted to meet more frequently but for less time. Given that we no longer had scheduled/ required class time during the remote learning portion of the course, minimal change in the number of meetings suggests that students actually met less frequently as a team when we moved online, although teams did note that they met more frequently as deadlines approached. This less frequent meeting pattern could be a cause or an effect of more efficient delegation. We observed that teams seemed to engage in greater division of labor in a remote setting compared to in-person, which may be attributed to the fact that they arranged for fewer group meetings or because some file formats only allow one active person compared to in-person prototyping, which can accommodate multiple hands. Some teams chose to work online simultaneously, but independently, to maintain accountability and keep on task.

Anonymous end-of-semester student feedback indicated that the weekly Zoom meetings with instructors were highly valuable and several students suggested keeping these meetings (whether face-to-face or via Zoom) in the future to maintain accountability. In contrast, TA office hours were not as heavily utilized as we expected. On average, 3.0 ± 2.4 teams (out of 16) attended office hours of at least one TA during any given week. TA office hours were held during formal class times; following the transition online, teams did not always choose to work during those times, which may have contributed to their underutilization. Teams also indicated that they often used additional outside resources (e.g., YouTube) to selfteach, potentially negating their need to attend TA office hours.

The transition to remote learning did not increase conflict within teams compared to previous years. Two student teams experienced significant internal conflict due to differences in communication preferences and work habits, which required instructor intervention and began prior to the transition to online. Following the transition to remote learning, two teams struggled to maintain motivation and equitably divide work; however, this conflict resolved without instructor intervention. Review of team norms progress reports revealed that relationship conflict 4 slightly increased toward the end of the semester, which is common based on the instructors' prior experience and did not seem specific to the remote format. Process conflict 4 generally decreased as the semester progressed; one group noted it was easier to schedule meetings after transitioning to remote learning due to greater schedule flexibility, and one team identified a mismatch in preferred working hours (morning or evening) fol-lowing the transition to online. A couple teams noted in their team norms progress reports that they needed to be more deliberate in planning meetings in the remote learning format since they no longer could schedule during their time between common classes.

Analysis of CATME peer evaluations revealed that individual teamwork outcomes of ''contributing to the team's work,'' ''interacting with teammates,'' and ''keeping the team on track'' were not altered when comparing design phase 2, which was completed on our final day of face-to-face instruction, to design phase 3, which was completed one month after our course transitioned to a remote-learning format (Table 3). To draw comparisons, the average score in each criterion for each student was calculated from the ratings of their teammates, as well as the students' rating of themselves, and a Wilcoxon matched-pairs signed rank test was utilized to evaluate changes in individual teamwork outcomes following the mid-semester transition to remote learning. This finding supports that the transition to online did not cause a decline in individual teamwork performance. Individual teamwork reflections submitted at the end of the semester revealed that most students demonstrated significant growth, with students commonly citing the importance of communication, being understanding of each other's circumstances, and learning resilience as key drivers of team success in the transition online. However, despite their success, celebrating accomplishments as a team was often challenging, and future efforts should consider ways to boost morale and celebrate achievements in a remote setting.

To provide further support for students, future iterations of the course will consider increased instruction on and practice with communication methods and discussion of the five dysfunctions of a team. 7 Furthermore, we believe it would be helpful to formally reinforce mid-semester the teamwork lessons that were introduced at the start of the semester; by this time, teams have progressed out of the ''forming'' phase of teamwork and are making their way to the ''performing'' stage, which typically results in passage through the uncomfortable ''storming'' phase. 5 Additionally, individual students and teams may have different needs as they work through the various teamwork stages and design process phases, so reinforcement and accumulated, deliberate practice (such as role-playing) would be beneficial. 2 Anonymous endof-semester feedback also indicated that several students desired clearer guidelines in their teamwork evaluations; specifically, one student felt that they should not receive a teamwork score and an individual peer evaluation score. Several students also commented about the desire for clearer expectations on phase deliverables. We currently provide templates and rubrics in advance, have weekly lectures to summarize these expectations, and meet with teams to answer their specific questions. We will continue to identify ways to improve the communication of our expectations to students; however, part of this feedback may be because this course is typically the students' first experience with a fully open-ended design project, which is very different from a ''textbook problem.'' Learning to manage ambiguity is a challenging skill, and something we will continue to seek ways to address.

In conclusion, student teams effectively transitioned from face-to-face to a remote-learning format. The activities and templates described in this paper can be implemented in both learning structures. The ample evidence we have collected suggests instructors and students can implement just small changes to accommodate a remote format and still facilitate successful teamwork. Further, these techniques are readily translated to other courses that seek to support team function. Overall, the teamwork lessons, documentation, accountability, and assessments implemented in this course facilitated positive team dynamics and successful final design outcomes. For each CATME criterion, the average score for each student was calculated from the ratings of their teammates, as well as the students' rating of themselves. A Wilcoxon matched-pairs signed rank test was conducted to evaluate changes in individual teamwork outcomes following the mid-semester transition to remote learning. Specifically, the results obtained from design phase 2, which was completed on our final day of face-to-face instruction, were compared to those of design phase 3, which was completed one month after our course transitioned to a remote-learning format. All criteria have a maximum score of 5.0.

How learning works: seven research-based principles for smart teaching

Teaching the art and craft of giving and receiving feedback

Intragroup conflict in organizations: a contingency perspective on the conflict-outcome relationship

Observing engineering student teams from the organization behavior perspective using linguistic analysis of student reflections and focus group interviews

Design and validation of a web-based system for assigning members to teams using instructor-specified criteria

The five dysfunctions of a team: a leadership fable

Work in progress-taking one for the team: goal orientation and gender-correlated task division

The comprehensive assessment of team member effectiveness: development of a behaviorally anchored rating scale for self-and peer evaluation

Core competencies for undergraduates in bioengineering and biomedical engineering: findings, consequences, and recommendations

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

The authors thank Dr. Julia Maresca for inspiring the creation of the team norms template and Dr. Valerie Myers for permission to use the team warm-up activity.

Both authors contributed to the course instruction. The first draft of the manuscript was written by Dr. S.I.R. and both authors edited previous versions of the manuscript. Both authors read and approved the final manuscript.

Not applicable.

Not applicable (no data are presented, and instructional tools/templates are provided in the supplemental materials).

Not applicable.

Dr. Sarah I. Rooney is a member of the McGraw-Hill Education (MHE) Biomedical Engineering Faculty Advisory Board, and she reports personal fees, outside the submitted work. MHE was not involved in the course design, creation or implementation of instructional materials, or manuscript preparation.

This project did not originate with a research intent and instead is a description of practice; therefore, this project does not meet the definition of human subjects research that would require an IRB determination.

Not applicable.