key: cord-0694038-pfldgi07
authors: Magana, Alejandra J.; Karabiyik, Tugba; Thomas, Paul; Jaiswal, Aparajita; Perera, Viranga; Dworkin, James
title: Teamwork facilitation and conflict resolution training in a HyFlex course during the COVID‐19 pandemic
date: 2022-01-21
journal: Journal of Engineering Education
DOI: 10.1002/jee.20450
sha: 20e7f095300fc462202d6bb421e15f925f43a2ae
doc_id: 694038
cord_uid: pfldgi07

BACKGROUND: We evaluated the effect of three teaching strategies to facilitate teamwork in a systems analysis and design course during the COVID‐19 pandemic: (1) offering a HyFlex version of the course, (2) facilitating scheduled online teamwork sessions for all students, and (3) providing conflict resolution training to help teams overcome collaboration challenges. PURPOSE/HYPOTHESIS: To identify the impact of these instructional strategies and answer four research questions, we measured (1) performance, dynamics, and cooperation strategies of teams and (2) students' perceptions of their own and team members' performance along with changes in their perceptions of their conflict management skills. DESIGN/METHOD: We used a simultaneous triangulation mixed‐methods design to obtain distinct but complementary qualitative and quantitative data. We compared data from two offerings of the course: Fall 2019 and Fall 2020 semesters. In the Fall 2019 semester, an in‐person active learning strategy was used, while in the Fall 2020 semester, the course followed a HyFlex delivery mode due to the COVID‐19 pandemic. RESULTS: Findings suggest that the use of cooperative learning pedagogy along with HyFlex accommodations for safety and social distancing requirements for the Fall 2020 semester provided students with a comparable learning experience to a traditional in‐person mode. CONCLUSIONS: Learning strategies, pedagogical supports, and teamwork training can enhance social interactions, and consequently, students' social presence in online learning. Conflict resolution training could be a valuable tool for improving teamwork skills and communication among team members.

The interdisciplinary nature of 21st-century workplaces requires STEM graduates to acquire a blend of technical and professional skills, including communication, collaboration, teamwork, leadership, and problem-solving (Deming, 2017; Vogler et al., 2018) . Recognizing this need, bodies for program accreditation, such as ABET, now consider "an ability to apply knowledge of mathematics, science, and engineering" as important as "an ability to function on multidisciplinary teams" (ABET, 2016, pp. 27-28) . For this study, we specifically focus on teamwork from this list of professional skills. Teamwork is an essential skill in the workplace and education settings. It is also "effective in promoting greater academic achievement, more favorable attitudes toward learning, and increased persistence through STEM courses and programs" (Springer et al., 1999, p. 21) . Nevertheless, teamwork has valence in that it can range from being productive to unproductive. Thus, teaching undergraduate students teamwork skills requires higher education institutions to actively embed those skills directly into their curricula and ensure that students learn those skills in the context of working productively in teams. It is insufficient to expect teamwork skills to naturally appear if students are only instructed to "work in groups." Educators need innovative methods to effectively teach teamwork skills to undergraduate students throughout their degree programs.

The growth of online education in the past years (Garrison & Kanuka, 2004; Seaman et al., 2018) , and the recent need to teach online due to the COVID-19 pandemic necessitate that universities teach students teamwork skills inperson and in an online environment. Teaching teamwork skills in a traditional in-person classroom is markedly different from teaching when students connect remotely (sometimes asynchronously) and do not interact face-to-face with their instructors and classmates. Recently, in-person courses have been either modified into HyFlex courses or transformed into fully online courses. HyFlex is a form of blended learning that flexibly combines face-to-face and remote instruction (Beatty, 2014) , "blending synchronous online student attendance and face-to-face student attendance (hybrid) in a single course and allowing students to choose when and how they attend (flexible)." (Abdelmalak & Parra, 2016, p. 19) . HyFlex course delivery has primarily been implemented in graduate studies to accommodate the involved responsibilities of working adults (e.g., Abdelmalak & Parra, 2016 Beatty, 2014; Wright, 2016) . However, during the COVID-19 pandemic, the HyFlex mode became a feasible approach for delivering in-person learning experiences in a way that was safe, feasible, and accessible for students who were not able to return to campus (Lederman, 2020) .

As courses move to HyFlex and online modalities, they require new forms of presentation and interaction (e.g., discussion forums, short video lectures, and improved communication tools) (Smyth, 2011; Wallace, 2003) . As universities increasingly move toward online and HyFlex teaching, educators need to address two pressing issues:

(1) enabling higher order learning outcomes (Binnewies & Wang, 2019) and (2) implementing pedagogical approaches that are better suited to support and engage learners (Gillett-Swan, 2017) . While past research has given us compelling pedagogical frameworks, such as collaborative learning (Laal & Ghodsi, 2012) and frame-of-reference training (Loignon et al., 2017) , more research-based approaches are needed to maximize student engagement and build teamwork skills in online and HyFlex environments (Ale Ebrahim et al., 2009; Bond et al., 2020; Gillett-Swan, 2017) .

Our study addresses the need for developing effective online and HyFlex pedagogy to teach teamwork to students. Specifically, we evaluate three teaching strategies used for a systems analysis and design course during the COVID-19 pandemic: (1) offering a HyFlex version of the course combining online and socially distanced in-person instruction, (2) facilitating scheduled online teamwork sessions for all students (online and in-person), and (3) providing conflictresolution training to help teams overcome collaboration challenges. To identify the impact of these instructional strategies, we answered the following research questions: RQ1: How did the academic performance and team self-assessment change in the Fall 2020 semester during the COVID-19 pandemic as compared to teams in a previous in-person semester? RQ2: What are the differences of self-reported cooperative strategies of teams between a HyFlex delivery mode as compared to an earlier in-person format? RQ3: Within the HyFlex delivery mode, what is the effect of conflict resolution training on students' self-reported ability to resolve conflicts?

Online teamwork is becoming more prevalent in higher education due to the COVID-19 pandemic. Higher education institutions, along with other types of organizations (e.g., business and medical), benefit from online teams since they enable innovation, decision-making, and problem-solving (Curşeu et al., 2008) . Using specific training programs, organizations can improve teamwork skills. Past research has identified that teamwork training can result in positive team behaviors and performance (McEwan et al., 2017) . While improvements in teamwork have been documented in corporations, our literature review identified a need for improved strategies for teaching teamwork skills in higher education, specifically in engineering education (Paoletti et al., 2020; Woods et al., 2000) .

While meta-analyses have confirmed determinants of effective in-person teamwork, factors that impact online teams are still indefinite (i.e., Ale Ebrahim et al., 2009; Lin et al., 2008) . Recent work identified online team behaviors that are antecedents to successful team performance (Sottilare et al., 2018) , finding that behavioral states, such as trust and conflict in teams, significantly mediate performance and learning (Sottilare et al., 2018) . Furthermore, a literature review concluded that psychosocial factors, such as relationship building, cohesion, and trust, are crucial for online teams to be effective (Chang & Bordia, 2001; Lin et al., 2008) .

Communication among team members has also been identified as important in influencing the social dimensions of teams (Lin et al., 2008) . For instance, communication among team members in an online setting can be difficult in certain situations and further magnify physical distance and language and cultural differences (Ale Ebrahim et al., 2009 ). Nevertheless, training interventions can enhance team performance (McEwan et al., 2017; Salas et al., 2008) . In their meta-analysis, McEwan et al. (2017) found that teamwork interventions were effective for both new and existing teams, regardless of the type of intervention used (e.g., didactic lectures, workshops, simulation training, and review-type activities). This research implies that it is possible to develop, support, and improve teamwork through effective pedagogy (Riebe et al., 2016) , mediation (Tu, 2000) , and training (Fathi et al., 2019) .

Teamwork pedagogy is aimed at (a) addressing dynamic interactions within the teamwork process and (b) developing approaches that are related to developing students' psychological and communication-oriented capabilities (Riebe et al., 2016) . Pedagogies of engagement, which can promote effective teamwork, are problem-based learning strategies that help create learning environments where learners actively think about and apply knowledge during instruction through cooperation among students, student-faculty contact, and active learning (Smith et al., 2005) . Pedagogies of engagement have the dual goal of (1) actively involving students in the learning process and (2) resulting in meaningful experiential and reflective knowledge. The dual goal of pedagogies of engagement suggests that optimal learning experiences need to facilitate learner engagement to be fully effective. Such approaches need to build a sense of community and cultivate social interaction as it is the main mechanism for delivering social learning (Gunawardena, 1995; Tu, 2000) . Thus, active social interaction is the basis for productive engagement (Appleton et al., 2006) . Fostering student engagement is crucial because it may lead to academic achievement and contribute to students' social and cognitive development (Marks, 2000) .

An effective approach for facilitating social interaction is mediation (Gunawardena, 1995) . Mediators (or moderators) can facilitate discussions by eliciting ideas, mediating conflict, recognizing/summarizing contributions, providing immediate feedback, and weaving ideas together (Kreijns et al., 2003) . However, effective team mediation requires training and adaptation to develop relevant interaction skills (Tu, 2000) . A second approach for improving teamwork is to provide conflict management training (Casper, 2017; Fathi et al., 2019) . Teams that manage conflict become cohesive, more flexible, and more effective (Casper, 2017) .

The epistemological foundation for this work is the situative perspective of learning (Johri & Olds, 2011) . Situative learning theories assume that knowledge is distributed among people and their environments (Greeno et al., 1996) . Human knowledge derives from construction and reinterpretation within a social context (Clancey, 2009) . Within this perspective, the process of learning occurs as individuals engage in meaningful participation in a community (Greeno et al., 1996; Lave & Wenger, 1991) . This perspective emphasizes the critical role of learners working together to use their skills and knowledge to solve problems and perform tasks. Hence, meaningful learning requires approaches to socially negotiate as one works in a collaborative environment (Ferguson, 2011) . Action and interaction processes result in social negotiation, which is the underlying notion behind the social constructivist view of learning (Johri & Olds, 2011) .

Social constructivist theories of learning advocate for designing effective learning environments that foster learners to create meaning from their experiences. Meaning-making is the primary aim of constructivist learning processes, where learners engage in "knowledge construction through collaborative activities that embed learning in a meaningful context and through reflection on what has been learned through conversation with other learners" (Jonassen et al., 1995, p. 12) ; that is, such learning experiences may require a process of articulation and reflection involving internal and social negotiations, where experiential and reflective knowledge surface from the learners' interactions with the world (Jonassen et al., 1995; Norman, 1995) . Jonassen et al. (1995) described four elements to be considered when designing constructivist learning environments: (a) Context that refers to the design of real-world learning experiences; (b) Construction that involves an active process of articulation allowing individuals or groups to make their meaning for what they experience; (c) Collaboration through engagement in developing, testing, and evaluating different products, beliefs, positions, ideas, or hypotheses; and (d) Conversation where individuals and groups reflect on what is known and unknown, as well as negotiate and evaluate plans of action before carrying out those plans. Therefore, to be fully effective, optimal learning experiences need to facilitate learner engagement.

The implications of the theoretical framework to our study are that (a) it provided us with the guidance to deliver a practical teamwork experience through cooperative learning, (b) it guided our data collection and analytical approach by focusing our analysis on teams as a unit, as well as individual learning and performance, and (c) it connected our findings to situative learning in the discussion of our findings. Figure 1 depicts the alignment of our epistemological, theoretical, pedagogical, and methodological perspectives. This foundation informed the course design and the methods and lenses used for our analytical procedures.

We implemented a situative perspective to learning regarding the course design by enabling individuals to participate in professional practices with peers, materials, and informational systems (Collins & Greeno, 2011) . The goal was to allow students to engage in knowledge construction and reinterpretation processes through active social interaction. In addition, such interaction among team members should accommodate social distancing restrictions due to the COVID-19 pandemic. Specifically, we guided our course design under the principles of cooperative learning (Johnson & Johnson, 2011) , which promotes small-group learning and gives recognition based on group performance F I G U R E 1 Alignment of epistemological, theoretical, pedagogical, and methodological foundations [Color figure can be viewed at wileyonlinelibrary.com] (Slavin, 1983) . Cooperative learning involves students working together to accomplish shared goals (Johnson & Johnson, 2011) . Cooperative learning and collaborative learning are pedagogies aligned with a situative perspective. Cooperative learning differs from collaborative learning in that cooperative learning provides "a structure [emphasis added] of interaction designed to facilitate the accomplishment of a specific end product or goal through people working together in groups" (Panitz, 1999, p. 3), and collaboration is "a philosophy [emphasis added] of interaction and personal lifestyle where individuals are responsible for their actions, including learning and respect the abilities and contributions of their peers" (Panitz, 1999, p. 3) . While our goal was to provide students with a situation where they would attempt to learn together, as aligned with collaborative learning (Dillenbourg, 1999) , our goal was to provide the structure and conditions to guide students in their teamwork experience aligned more with cooperative learning.

Elements of cooperative learning include (a) requiring group members to agree on strategies for solving a particular problem, (b) promoting interaction where students support one another in their learning, (c) individual accountability by assessing the performance of each student, (d) application of social skills such as communication and conflict management, and (e) group processing by reflecting on their functioning as a team (Johnson & Johnson, 2011) . Section 3.2 and Table 1 provide further specifics of how we implemented the elements of cooperative learning in the course.

We implemented a comparative study in a classroom that focused on connecting theoretical and empirical work with a focus on situated learning (Johri & Olds, 2011) . Situativity was accomplished by having the context of learning play a central role (e.g., physical and social aspects of the environment) (Johri & Olds, 2011) . For this, we used a simultaneous triangulation mixed-methods design (J. Creswell et al., 2003) , which allowed us to obtain distinct but complementary qualitative and quantitative data to answer our research questions. As prescribed by this design, we merged our data sets by (a) transforming data to facilitate integrating the two data types during the analysis and (b) bringing the separate results together in our interpretations.

Our analysis also took a situative perspective by focusing on social and material context, activities students engaged in, outcomes of their interactions, and aspects of their forms of participation (Johri & Olds, 2011) . For this, we used a combination of different measures, including (1) the quality of students' outcomes in the form of team-based and individual academic performance, (2) assessment of their team interaction in the form of self and peer evaluation, and (3) beliefs about their participation and experience in the form of reflections on their collaboration strategies and conflict management skills.

We used two conceptual lenses to analyze students' reflections regarding their collaboration strategies and conflict management skills. To characterize teams' reported collaboration strategies, we used the Goals, Roles, Process, and Interpersonal Relationships (GRPI) model (Rubin et al., 1974) . This model has also been widely accepted as a lens for analyzing teamwork behavior (Karabiyik et al., 2020; Makhalemele & Nel, 2021) , and we selected it for our analysis because its four constructs constitute effective team behaviors (Duckworth, 2008) . Goals refer to the process of developing a shared mission as a team; roles involve developing an understanding of the team's structure; processes include common procedures, behaviors, norms, and practices accepted by all team members; and interpersonal relationships encompass aspects of team dynamics such as communication, collaboration, conflict management, and trust (Bates, 2014) .

To characterize students' beliefs about how they would manage conflict, we utilized the Dual Concern Model (Sorenson et al., 1999) . This model of conflict resolution assumes five modes of dealing with conflict based on two dimensions. One dimension is a concern for self (assertiveness), and the second is a concern for others (cooperativeness). It is best to have high assertiveness and high cooperativeness (Thomas & Kilmann, 1974) . The collaborating mode has the highest levels of cooperativeness and assertiveness, while the compromising mode has moderate levels of cooperativeness and assertiveness, and the accommodating mode has high levels of cooperativeness but low levels of assertiveness. Uncooperative strategies are competing with high levels of assertiveness and avoiding with low levels of assertiveness.

The systems analysis and design course is a required course offered to undergraduate students pursuing software engineering, cybersecurity, and network engineering majors. Students are expected to demonstrate and integrate workplace techniques and frameworks for designing, documenting, and prototyping software projects using skills acquired from previous courses such as software development, systems modeling, and project management. Specifically, the course aims to develop skills used by contemporary software developers to model requirements, perform a cost-benefit analysis, and then construct an acceptable design to implement a successful system solution in the form of a functional prototype. The course also promotes teamwork skills by implementing teamwork pedagogy (i.e., cooperative learning) and different forms of active learning (see Figure 2 ).

A typical lecture had the following structure. The instructor started class with an agenda and reminders for the week; then, she introduced the topic of the day (for about 15 min). Once the foundation was explained, the next step was to proceed with practice. For this, the instructor used a mini-case study throughout the semester, elaborating on modeling techniques and software engineering methods for systems analysis and design. When introducing a new technique or method, the instructor selected one system requirement and demonstrated how to transform it into a diagram using a diagramming software tool. Students were then asked to practice the same diagram by modeling a different requirement. For example, the professor modeled the process of constructing a sequence diagram for the requirement of making an appointment for a specific system, and then students practiced constructing a sequence diagram for the T A B L E 1 Alignment between principles of cooperative learning and elements of the course (Magana et al., 2018) Principle

Definition (Johnson et al., 1998) Course implementation (Magana et al., 2018) Positive interdependence Team members need one another to complete the task. This can be achieved by establishing a clear mutual goal

The project was divided into four clear milestones, a final design document, and five software deliverables. The deliverables for the project were established from the very beginning of the semester

The group is accountable for achieving its goals Each member must be accountable for contributing a fair share of the work Students were expected to work together as a team throughout all milestones and deliverables of the project. However, as part of the project, students were also expected to contribute individually. Twothirds of the project was graded as a team, with the remaining one-third graded individually. Specifically, the project required the implementation of 10 functional requirements for the system. Individual accountability consisted of each student being responsible for separately documenting, modeling, and implementing two system requirements. For each requirement, two use case narratives, two activity diagrams, and four sequence diagrams were constructed. In addition, students performed self and peer evaluations at the end of the semester Interpersonal and small group skills Basic teamwork skills: As a group, provide effective leadership, make decisions, build trust, communicate, and manage conflict Students were expected to work in and out of class throughout the entire semester. However, during class time, the instructor and TAs monitored group performance and facilitated conflict resolution as needed. Students were also expected to self-organize for rotating the role of the leader (i.e., scrum master) for each of the milestones Face-to-face promotive interaction A group member teaches classmates about a topic As part of the individual portion of the project, students became specialized in one aspect of the system. However, for them to complete the prototype, everyone needed to understand the system's functionality. In addition, students utilized the in-class time to work on the project and helped one another as a team

Group processing As a group, make decisions about which behaviors to continue and which to change Team retrospectives were used as a mechanism for group processing. For every milestone, students were asked to reflect on aspects that went well and what did not, using a team-based retrospective (see Table 6 for prompts provided to students). Students committed to improving at least one team behavior from milestone to milestone requirement for placing an order. Once students completed the practice, the instructor demonstrated the solution for the assignment. Students then had the opportunity to compare and contrast their solutions with ones provided by the instructor. A short discussion was facilitated immediately after the practice to address clarifications. All diagrams created by the instructor were uploaded to the learning management system for future consultation. Students, organized into teams of four or five, worked on a semester-long software development project with two main deliverables: project documentation and a software prototype. The project documentation was developed and delivered in four milestones and a final report. The corresponding software prototype was implemented and delivered in five cycles of software development product increments called sprints.

In a regular semester, teams were formed on the second day of classes. Toward the end of the class period, the instructor grouped students enrolled in the same laboratory section of a corequisite database design course that shares the same case study. Within those groups, students were free to select team members. Students worked on their design problems both in and outside of class, which culminated in them producing design specification documentation, a functional prototype, a usability evaluation of their prototype, and a sales presentation of a prototype.

The coordination of the semester-long project, grounded in a situative perspective, applied cooperative learning as a guide for realizing teamwork (see Figure 1 ). Table 1 depicts an overview of how we embodied the five principles of cooperative learning from Johnson et al. (1998) . Additional details of the implementation of cooperative learning and our enhancements and adaptations for a HyFlex delivery mode are presented in Section 3.3.

The semester-long projects were based on teaching cases published in the Journal of Information Systems Education. A teaching case presents a problem an organization may experience and asks students to solve it by implementing a F I G U R E 2 Class-time teamwork sessions during a regular in-person semester [Color figure can be viewed at wileyonlinelibrary.com] software solution. The project implementation followed two software development methodologies: the Unified Process (Jacobson et al., 1999) for delivering the project report and Scrum (Sutherland & Schwaber, 2007) for delivering the functional prototype. The Unified Process and Scrum are approaches for implementing the systems development life cycle iteratively, incrementally, and cooperatively (Harb et al., 2015) . The systems development life cycle is the process of (1) understanding how an information system can support the needs of an organization, (2) designing and building the system to fulfill that need, and (3) delivering the system to the users. Table 2 lists the project schedule and deliverables along with brief descriptions.

Changes to the course were implemented to jointly respond to the social distancing requirements of the COVID-19 pandemic while at the same time ensuring active learning. Specific changes to the course included (1) offering a HyFlex version of the course, (2) facilitating online teamwork sessions for all students (online and in-person), and (3) providing conflict-resolution training to help teams overcome collaboration challenges.

HyFlex was used as the delivery mode following three main strategies. First, a fully online version of the course was offered in parallel to the in-person section. Students from the online section joined the class synchronously via web conferencing software and attended the lecture together with the in-person students. In addition, when students were unable to attend class in person due to exposure, quarantine, or bereavement, they were provided with the link to join the class online. Furthermore, all lectures were recorded and uploaded into the learning management system, along with all notes, examples, and materials produced by the instructor before and during class time.

T A B L E 2 Implementation and deliverables for the semester-long project

Deliverable Description

Week 1 Milestone 0: Team formation Teams of four or five members were formed, and they submitted a team contract together

Week 3 Milestone 1: Business case document describing the business needs and system requirements Project vision statement, systems request, product roadmap, product backlog, context diagram, and team retrospective

Week 5 Sprint 1 Software prototype with two system requirements implemented

Week 6 Milestone 2: System proposal document describing the workplan, feasibility analysis, and requirements determination Updated product backlog, use case narratives detailing requirements of the system, project planning in a Gantt chart, project cost estimate in a cash flow, and team retrospective

Week 7 Sprint 2 Software prototype with four system requirements implemented

Week 8 Sprint 3 Software prototype with six system requirements implemented

Week 9 Milestone 3: System proposal document describing the functional, structural, and behavioral aspects of the system Updated product backlog, functional model as activity diagrams, structural model as a class diagram, behavioral model as a sequence diagram, updated project planning in a Gantt chart, and team retrospective

Week 10 Sprint 4 Software prototype with eight system requirements implemented

Week 11 Sprint 5 Software prototype with 10 system requirements implemented

Week 12 Milestone 4: System design specification document describing details of implementation Updated product backlog, data storage mechanism as an entityrelationship diagram, hardware and software deployment plan as a deployment diagram, revised Gantt chart, and cash flow

Week 13 Prototype usability report Evidence of usability testing of the prototype and a discussion of design elements for improving the prototype for the final implementation Weeks 14-16 Final presentation Final team presentation in the form of a sales pitch and a final prototype walkthrough A teaching assistant (TA) monitored the online session and responded to questions from students. This TA alerted the professor via text message when further clarification was needed. Approximately one-fourth of the students joined the class online (N o = 30). The in-person session of the course also implemented changes such as social distancing and other safety measures. To that end, the class was split into two sections: a Tuesday session (N iTu = 47) and a Thursday session (N iTh = 49). Students in the Tuesday session attended class in person every Tuesday (together with the online session), and students in the Thursday session attended class in person every Thursday. Figure 3 shows a sample of the classroom layout during an in-person lecture.

During the in-person lecture and as part of the active learning practice, students had the option to work as a team or individually during the modeling practice. If they opted to work as a team, in addition to wearing a face mask, they were asked to wear a face shield for additional protection. As described earlier, once the students tried to build a model on their own, the instructor modeled the solution for the entire class, and students had the opportunity to ask questions and request clarifications. However, for the semester-long project, all students were required to work in teams of four or five students. Team formation procedures, which also changed during the pandemic, occurred during the first week of classes. To facilitate the team-formation process, we sent out a survey where students identified (1) the time zone of their location and (2) peers they wanted to work with who were also enrolled in the same laboratory section of the database design corequisite course. Student responses were taken into consideration when the graduate TA formed the various teams. Second, on the day students were not required to attend class, they were offered the option to join a web conferencing teamwork session using Zoom to work on their project or other course assignments. The graduate TA organized the breakout rooms during the session, responded to questions regarding the project submission requirements, provided feedback on previous submissions and grading, and helped students to manage conflict. Regardless of whether they attended the optional teamwork sessions, students were required to read the teaching notes and submit the weekly assignment that counted toward class participation or complete the weekly quiz on their own time. In addition, we scheduled six mandatory teamwork sessions throughout the semester (one or two sessions right before the submission of major milestones). The instructional team, that is, the graduate TA and three undergraduate TAs who had previously completed the course, divided the work to join every team in a breakout room to assist them and to respond to questions before the project milestone submissions. As these mandatory teamwork sessions were scheduled at the same time as the in-class lecture, most students were available to join. Teams in different time zones were able to schedule teamwork sessions with the graduate TA during a feasible time (e.g., 9:00 p.m.). This graduate TA was a doctoral candidate in the Department of Computer and Information Technology who had more than 4 years of industry experience and 4 years of experience teaching systems development courses. The graduate TA was responsible for providing feedback on milestones, guiding teams through making corrections or revisions, and facilitating conflict resolution interventions as required. When the conflict persisted for more than a week or two, the course instructor scheduled a conflict resolution session where she served as a moderator, following the conflict resolution strategies described below.

The third component of the instructional change consisted of delivering teamwork conflict resolution training. Since this was a new component of the instructional design, we tested two separate approaches. All students in the class were asked to read the case study titled "Coping with Hitchhikers and Couch Potatoes on Teams" (Oakley et al., 2004) . This case exemplifies typical challenges faced by teams in education settings and proposes coping strategies. Then, students were asked to describe any parallels between situations described in the article and those experienced within their teams. Students in the HyFlex Tuesday in-person session and HyFlex fully online session of the course received live conflict resolution training delivered by an expert in labor relations, whose expertise includes collective bargaining, negotiations, and dispute resolution. The expert had more than 40 years of experience as an arbitrator, mediator, and fact-finder in various labor-management disputes in the private and public sectors. The live training began by providing students with techniques to address conflict (Thompson, 2020), followed by providing techniques to prevent conflict escalation (Lewicki et al., 2021) .

Students in the Thursday section, on the other hand, did not receive the live training. This provided us an opportunity to identify potential effects on students' conflict management skills. A week later, students from all three sections were asked to write a reflection describing a scenario where they experienced teamwork conflict (in this or other courses). Students were also prompted to describe the strategies or steps followed to address the past conflict. Finally, they were asked to reflect on the experience and then describe in detail what they would change in their strategy if presented with the same conflict again.

We collected qualitative and quantitative course assignment data to answer our three research questions. The triangulation mixed-methods design provided us with the flexibility to compare the instructional modes from two offerings of the course: Fall 2019 and Fall 2020 semesters. The Fall 2019 course was offered an in-person active learning mode, while the Fall 2020 course was altered due to the COVID-19 pandemic (see Section 3.3 for details). The Spring 2020 semester was not considered in our analysis since instruction changed abruptly toward the end of the semester. Specifically, we (a) collected qualitative and quantitative data in a single-phase timing each semester, (b) transformed qualitative data into quantitative data to facilitate analysis and interpretation of results, and (c) combined results for interpretation. Specifics of the research design implementation are described in the following sections.

The systems analysis and design course for cybersecurity, network engineering, and software engineering majors included 110-150 students, approximately 30% second-year, 40% third-year, and 30% fourth-year students. Table 3 summarizes the participants and conditions of the study.

We obtained student demographic data from the institutional office in charge of student demographics (personal communication). A total of 113 students enrolled in the Fall of 2019 in the traditional fully in-person delivery mode of the course, and a total of 126 students enrolled in the Fall of 2020 in the HyFlex delivery mode of the course. Within the HyFlex offering of the course, we were able to obtain separate student demographic data for the in-person and the online sections of the course. However, as it was not possible to obtain separate student demographic data for the Tuesday and Thursday in-person sections of the course, they are presented together in Tables 4 and 5 .

As shown in Table 4 , the Fall 2019 semester included 6% female and 84% male students enrolled in the course, while 25% female and 75% male students enrolled in the Fall 2020 semester. In comparison, our institution's undergraduate population is 43% female and 57% male. Thus, this course had significantly more male than female students. We note that the Office of Enrollment Management offers binary gender options. As such, we are unable to quantify the gender nonbinary undergraduate student population of our institution and, in turn, this course. The institution's ethnic demographics are 7% Asian, 3% Black or African American, 5% Hispanic or Latino/a/x, 59% White, 3% two or more races, <1% American Indian or Alaska Native, and <1% Native Hawaiian or other Pacific Islanders. Hence, as shown in Table 5 , our cohort contained more Asian, Black or African American, Hispanic or Latino, and students of two or more races than the institution as a whole.

The data were collected throughout the semester in the form of course assignments. The implementation and deadlines for the deliverables for the course remained the same for the two semesters, as described in Table 2 . The only change in the Fall 2020 semester occurred between Week 7 and Week 8 when the conflict resolution training was held for the HyFlex Tuesday in-person and HyFlex fully online sessions (see Table 3 ). An overview of the course assignments used in data collection and their alignment with our research questions are presented in Table 6 .

Measures collected from each student individually were the second term exam collected 3 weeks before the semester ended, self and peer evaluations at the end of the semester, and written reflections after reading the case study or reading the case study with conflict resolution training. Measures collected as a team were team performance consisting of grades from each of the four milestones and self-reported collaboration strategies in the form of four team retrospectives submitted along with each milestone. 

A simultaneous triangulation design involves transforming data to facilitate integrating the two data types (i.e., qualitative and quantitative) during the analysis (J. W. Creswell & Creswell, 2018) . To merge qualitative and quantitative data, we first transformed all our qualitative data into quantitative data and then analyzed every individual dataset using descriptive and inferential statistics.

Measures of team performance consisted of grades for team-based submissions of the project milestones describing the system documentation as described in Table 2 . We scored each milestone for a grade using a rubric developed by the lead instructor, who has more than 10 years of experience teaching systems development courses. The lead instructor refined the rubric over several semesters of instruction, but for the last 3 years, it has remained unchanged. All instructional team members were briefed and trained regarding grading standards and how the rubric was to be used while grading. As an example, Table 7 presents the rubric used for scoring Milestone 3. Once each milestone was scored for each team, we calculated measures of spread and central tendency to describe overall performance for each section. We also calculated inferential statistics to compare across all of them. Scores ranged from 0 to 100 and were interpreted as follows: scores below 70% were considered basic performance; scores between 70% and 89% were considered proficient performance, and scores 90% and above were considered advanced performance. We first used the Mann-Whitney U test to compare the performance of students in traditional and overall HyFlex environments. We then used the Kruskal-Wallis H test, which considers corrections for multiple comparisons, T A B L E 6 Research questions aligned with course assignments used in data collection

Course assignment RQ1: How did the academic performance and team self-assessment change in the Fall 2020 semester compare to team performance and dynamics demonstrated by students in a previous in-person semester?

• Team academic performance: Team-based grades from each of the four project milestones collected throughout the semester • Individual academic performance: Individual grades for a modeling term exam • Team self and peer assessment: Self and peer evaluation surveys for each team and its members were collected at the end of the semester. The questions were the following: Student [name]: Completed work on time or made alternative arrangements; was dependable in attending group meetings; did work accurately and completely; contributed positively to group discussions; helped others with their work when needed; contributed a fair share of the overall workload; overall was a valuable member of the team RQ2: What are the differences of self-reported cooperative strategies of teams between a HyFlex delivery mode as compared to an earlier in-person format?

• Self-reported collaboration strategies: Four team-based retrospectives (submitted with each milestone) were qualitatively analyzed via content analysis and then scored with a rubric. Guiding reflection questions included the following: Planning: How did you plan the organization of work for the milestone? What were team members' roles? How were activities assigned to each team member, and what was the justification for that? How was the communication handled among team members? Monitoring: What aspects of the team coordination/collaboration went well in this milestone? What aspects of the team coordination/collaboration went wrong in this milestone? What are the possible concerns? Evaluation: What do you think as a team was particularly good about the milestone you just completed? What are areas or sections of the milestone that you just completed you think could be improved? Plan of Action: What aspects do you think can be done better for the next milestone regarding team performance?

RQ3: Within the HyFlex delivery mode, what is the effect of conflict resolution training on students' self-reported ability to resolve conflicts?

• Class participation: An individual written reflection regarding the teamwork experience. Reflection prompts included the following: Scenario: Describe a scenario where you have experienced teamwork conflict and detail the context, the issue, and the individuals involved. The strategy followed: Describe the strategies or steps followed to address the conflict back then, including the leader's actions and any conflict resolution steps. Potential changed strategy: Describe what you would have changed if presented with the same dilemma/conflict and explain possible alternatives to compare team performance among the four groups shown in Table 3 . Since there were four independent groups and the grade data were not normally distributed, doing a one-way ANOVA was not feasible. Since the data met the assumptions for the nonparametric Kruskal-Wallis test (McKight & Najab, 2010), we used it for comparing team performance among the four groups.

The individual academic performance consisted of students' scores on the second term exam that evaluated their software modeling skills. The exam provided students with a case study for analysis and design that included four parts aligned with modeling practices applied throughout the semester as part of the semester-long project. The first part of the exam asked students to identify and document the requirements of the system, while the second prompted students to model the functional view of the system (i.e., use case diagrams and activity diagrams), the third required them to model the structural view of the system (i.e., class diagram) and the final part asked students to model the behavioral view of the system (i.e., sequence diagram). In the traditional in-person condition, the exam was delivered during class time with no access to course materials. In the HyFlex delivery mode, the exam was delivered as a take-home with no time restriction and access to course materials. We calculated means and SDs for each semester. We then used the Kruskal-Wallis H test to investigate whether there were any statistically significant differences in students' performance between semesters.

Students were asked to individually complete self and peer evaluations at the end of the semester by rating seven statements on a 5-point Likert scale. Survey questions were grouped into two constructs, one describing their collaboration team processes and another describing their team outcomes. Five independent raters categorized the survey questions into either team process or team outcomes with an 82.8% agreement. Statements in the team processes construct included the following: Student [name]: was dependable in attending group meetings, contributed positively to group discussions, and overall was a valuable member of the team, and helped others with their work when needed. Statements in the team outcomes construct were the following: Student [name]: completed work on time or made alternative arrangements; did work accurately and completely; and contributed a fair share of the overall workload.

We computed scores for each construct by adding scores for each survey item under the corresponding construct. The maximum possible scores for outcomes and process were both normalized to a maximum of 20. For each construct and each course section, we calculated measures of spread and central tendency. Student scores were interpreted as follows: scores between 18 and 20 were classified as high performing students, and scores below 18 were classified as moderate performing students. We then calculated inferential statistics to compare across all students. We first used the Mann-Whitney U test to compare the performance of students in traditional and overall HyFlex environments. We then used the Kruskal-Wallis H test to investigate whether there were any statistically significant differences among the four groups shown in Table 2 .

The data source for measuring team collaboration strategies consisted of each team retrospective submitted along with each individual milestone. That is, each team submitted a total of four milestones and four team retrospectives throughout the semester. Whenever a single team retrospective was missing from a team, all team data were discarded in the analysis of team collaboration. Team retrospectives were first qualitatively analyzed via a conventional content analysis (Hsieh & Shannon, 2005) . A first step in the analysis was to categorize each of the reflection questions into related constructs. For this, we utilized the GRPI model (Rubin et al., 1974) . The definition of each construct of the GRPI model, the reflection questions, and representative quotes are provided in Table 8 .

Once the team reflections for each milestone were classified into separate constructs according to the definitions provided in Table 8 , the next step was to score the quality of the cooperation strategies implemented. For this we iteratively created a rubric delineating three levels of performance (see Table 9 ). The rubric scores were then used to calculate the total mean scores obtained by each team in terms of goals, roles, processes, and interpersonal relationships for all four milestones.

We scored student retrospectives using the rubric shown in Table 9 . We scored team retrospectives for a given milestone by reading complete retrospectives and assigning a score of 0 for a basic level, 1 for a proficient level, and 2 for an advanced level for each of the four constructs of the GRPI model. Retrospectives for the 27 teams were evenly divided between two raters, who first scored a subset of retrospectives together to ensure consistency in the interpretation of scoring. After scoring each retrospective, we calculated the sum of each teams' GRPI construct scores across their four milestones; each team had a total score between 0 and 8 for each of the four GRPI constructs, meaning each could obtain a maximum cumulative score of 8 for each construct. The total average score for a construct is the mean of the total scores obtained by each team. The total average score was interpreted as follows: if a team scored in the range of 0-3.00, their cooperation strategies were categorized as slightly effective; if the scores were between 3.01 and 6.00, their cooperation strategies were categorized as moderately effective, and if a team scored between 6.01 and 8.00, their cooperation strategies categorized as highly effective.

The data source for measuring team conflict resolution strategies consisted of students' reflection assignments. As part of the course assignments, students had to describe a situation when they experienced conflict, how they reacted to it (i.e., initial applied strategy), and what they would have done differently (i.e., potential new strategy). Table 10 presents a description of the reflection prompts. Students' reflections were first qualitatively analyzed using conventional content analysis (Hsieh & Shannon, 2005) . For this, we utilized the Dual Concern Model (Sorenson et al., 1999) to categorize students' responses regarding their conflict-handling modes. We used the two dimensions of assertiveness and cooperativeness organized into five conflict-handling modes for scoring cooperative strategies that teams described. Table 10 presents and defines the five conflict-handling modes prescribed by the Dual Concern Model, the points assigned to each student response, and sample quotes from students' reflections about how they dealt with conflict in a teamwork environment. How did you plan the organization of work for the milestone?

"Work for this milestone was broken up into categories and later assigned to team members. Everyone shared relatively similar roles to ensure that no one was seen as a higher, more important piece to the case study. I feel our team has a decent understanding of what needs to be filled out, but not quite the whole picture. With a little more practice identifying these requirements, I feel our team will be able to list and categorize these much more efficiently."

All team members must know what part they play, what is expected, and how they are held responsible and accountable What were the team members' roles? How were activities assigned to each team member, and what was the justification for that?

"Activities were given out by the milestone's Scrum master. Knowing everybody in the team, I assigned tasks to each other based on each member's strengths and weaknesses."

Processes Explanation about procedures that the team has to follow, in terms of workflow or review, for current milestone or improvements to be made for future milestones What are areas or sections of the milestone that you just completed you think could be improved? What are the aspects you think can be done better for the next milestone in terms of team performance? What are the possible concerns? What do you think as a team was particularly good about the milestone you just completed?

"The one thing that stuck out to me was everyone's willingness to perform the work they were assigned." "There were a couple of things that went wrong with this milestone as well. Those that stuck out would be the procrastination of the team, the occasional miscommunication work designation, and work a man down for a portion of the milestone."

Interpersonal We categorized and scored all individual responses to the reflection prompts. Students described the initial applied strategy (i.e., describe the strategies or steps followed to address the conflict at the time it was experienced, including the leader actions and any conflict resolution steps), and the potential new strategy (i.e., describe what you would change if presented with the same dilemma/conflict and explain possible alternatives). For each of the three groups in the Fall 2020 HyFlex semester (i.e., Tuesday session, Thursday session, and fully online session), we visualized results using Sankey charts to represent flows where widths of connections are proportional to the number of students. We then compared those results descriptively. Recall that only the Tuesday session and the fully online sections of the HyFlex mode received the conflict resolution training. We used the Kruskal-Wallis H test and Wilcoxon Signed-Rank T A B L E 9 Rubric for scoring levels of quality of cooperation strategies implemented by teams

Basic (0) Proficient (1) Advanced (2) Goal Did not address the overall plan for the current milestone in terms of goals or organization Accommodating is unassertive and cooperative. When accommodating, an individual neglects his or her concerns to satisfy the concerns of the other person "I did not want to make a big deal out of it, so I continued to do the work by myself. I thought that they would all at least show up for the final."

Competing is assertive and uncooperative, a poweroriented mode. When competing, an individual pursues his or her concerns at the other person's expense, using whatever power seems appropriate to win his or her position "Multiple attempts to set up meeting times on multiple days. Remind teammates that the amount of work I am doing is unfair and that it needs to change. Not include their names on the submission document. Contact the TA/Prof to ask for assistance."

Avoiding (1 point)

Avoiding is unassertive and uncooperative. When avoiding, an individual does not immediately pursue his or her concerns or those of the other person "I think we used the 'avoiding' strategy because we decided really quickly to kick out the lazy group worker to avoid further issues. We also discussed together to make wise decisions." test to identify if the interventions had a statistically significant effect. We searched for differences in the gain per group between the initial strategy applied and the potential new strategy to be applied if faced with the same situation. The Wilcoxon Signed-Rank test was used for the following reasons: (1) data were paired and randomly derived from the same sample and (2) the data were not normal, meaning using a parametric test such as paired t-test was not appropriate (Scheff, 2016) . Therefore, the Wilcoxon Signed-Rank test (Wilcoxon, 1992) was an appropriate choice.

Three strategies were followed to ensure the validity, reliability, and trustworthiness of this study. Five independent raters categorized the survey questions into either team process or team outcomes with an 82.8% agreement for the survey instrument that measured self and peer assessments. To determine the inter-rater reliability of the self and peer assessment, we computed Cronbach's alpha for the items under outcomes (0.88) and for items under process (0.94), which indicates a high internal consistency.

During the qualitative analysis of the team collaboration strategies, 20% of the team retrospectives were coded by a second researcher. Cohen's Kappa coefficient was calculated to be 0.673, suggesting that the raters satisfactorily agreed on the coding (McHugh, 2012) . Similarly, for the qualitative analysis of the team conflict resolution strategies, 20% of the student reflections were qualitatively analyzed using conventional content analysis by a second researcher using the same conflict-handling modes prescribed by the Dual Concern Model. The percentage agreement for the subset of reflections that both researchers coded was 92.5%, indicating substantial agreement on the codes.

We present the results for each research question individually. Then, as prescribed by the simultaneous triangulation design (J. W. Creswell & Creswell, 2018) , we analyze individual results from each research question together in the Discussion and Implications section (Section 7).

The academic performance measure consisted of students' grades on the team-based project deliverables comprised four milestones. Table 11 presents descriptive statistics for each of the four milestones by section. As observed, students from all sections obtained the highest performance on Milestones 2 and 4, with average scores over 90%. Overall performance on Milestone 1 was proficient, with average scores over 80%. However, there were differences in student performance, particularly on Milestone 3. While the average score for the in-person section in the Fall 2019 semester was over 90%, for the three sections in the Fall 2020 semester, the average scores were between 76% and 87%. We conducted a Mann-Whitney U test to compare student team performances for each milestone between the traditional Fall 2019 environment and the overall HyFlex environment in Fall 2020. There were no significant differences in Milestone 1 (U = 219, p = .07) and Milestone 4 (U = 267, p = .4). However, there were significant differences observed in Milestone 2 (U = 189, p < .05) and Milestone 3 (U = 98.5, p < .01). Regarding differences in team performance in the HyFlex versions of the course, it can be observed that students performed about 80% on Milestone 1 and Milestone 3 and over 95% on Milestone 2 and Milestone 4. To determine if the differences in scores (i.e., per each milestone and for each HyFlex offering) were significant, we used the nonparametric Kruskal-Wallis H test. Based on our results, there were no significant differences among the three HyFlex groups (i.e., HyFlex Tuesday in-person session, HyFlex fully online session, and HyFlex Thursday in-person session) for 

Measures of self and peer assessment were taken at the end of the semester when students rated their performance and their team members' performances. Table 12 shows descriptive statistics for the self and peer assessment measures. We interpreted a score of over 18.5 as a high score and lower than that to be a moderate score. The overall averages of team processes and performance for the HyFlex and in-person sessions were high. Overall, students identified themselves and their team members as dependable in (1) attending group meetings, (2) contributing positively to group discussions, (3) being a valuable member of the team, and (4) helping others with their work when needed. Students in the HyFlex and in-person sessions reported completing work on time or making alternative arrangements, doing work accurately and completely, and contributing a fair share of the overall workload.

In contrast, the online session scores were moderate for team processes and team outcomes, meaning that students in the HyFlex fully online session identified themselves and their team members as somewhat dependable in attending group meetings, contributing positively to group discussions, and being a valuable member of the team. Students in the HyFlex fully online session also identified that they somewhat completed work on time or made alternative arrangements, delivered work somewhat accurately and completely, and somewhat contributed a fair share of the overall workload.

A Mann-Whitney U test was applied to test for differences in overall team milestone performance between the traditional environment in the Fall 2019 semester and the HyFlex environment in the Fall 2020 semester. No significant differences were found between these two broad groups in terms of processes (U = 7027, p = .853) or outcomes (U = 6777, p = .512). The Kruskal-Wallis H test was then conducted to investigate differences between the specific HyFlex offerings in the 2020 semester. Statistically significant differences were found between the groups in terms of peer-rated team outcomes scores (H = 11.020, df = 2, p = .004) and team process scores (H = 10.975, df = 2, p = .004). Pairwise comparisons were conducted to identify differences between specific groups. The statistically significant differences in terms of outcomes were between (i) the HyFlex Tuesday in-person session and the HyFlex fully online session (p = .011) and (ii) the HyFlex Tuesday in-person session and the HyFlex Thursday in-person session (p = .018). There T A B L E 1 2 Descriptive statistics of self and peer assessment measures Statistically significant differences between groups in terms of pairwise comparisons between processes. b Statistically significant differences between groups in terms of pairwise comparisons between outcomes.

was also a statistically significant difference in terms of processes between the HyFlex Tuesday in-person session and the HyFlex fully online session (p = .004). Overall, the HyFlex Tuesday in-person session had the highest average scores in terms of outcomes and processes, closely followed by the HyFlex fully in-person session and HyFlex Thursday in-person session. The results suggest that the cooperative learning environment, coupled with the instructional scaffolds, was effective in maintaining the perceived proficiency of student teams in terms of process and outcomes in the in-person and HyFlex sessions, but there was a small decline in perceived student proficiency in the HyFlex fully online session.

Team collaboration strategies derived from students' team-based reflections as part of the project retrospectives submitted with each milestone. We used the GRPI model as discussed in Section 5.4 to characterize team collaboration strategies. Table 13 shows descriptive statistics for teams categorized by the course section in which they were enrolled (i.e., in-person and HyFlex). For each section (e.g., HyFlex Tuesday in-person session), descriptive statistics of their four GRPI constructs are listed for the teams in that particular session. We interpreted a score less than 3.00 to be slightly effective, a score between 3.01 and 6.00 to be moderately effective, and a score greater than 6.01 to be highly effective. The in-person section scored highest on average on all four GRPI constructs. The HyFlex Tuesday in-person session scored lowest on average on the goals and roles constructs. In contrast, the HyFlex fully online session scored lowest on average on the interpersonal relationships and process constructs. Specifically, considering the score interpretations described in Section 4.3, the HyFlex Tuesday in-person session and the online session only described strategies considered slightly effective in terms of their goals. Teams in those two sections also described strategies that were considered moderately effective regarding their roles, interpersonal relationships, and processes. The HyFlex Thursday in-person session exhibited a similar pattern by describing strategies moderately effective for their goals, roles, interpersonal relationships, and processes. Regarding the in-person section, strategies described regarding their goals and roles were considered moderately effective, while interpersonal relationships and processes strategies were considered highly effective.

Team conflict resolution strategies were derived from the individual students' reflections on their conflict resolution styles. These reflections were assessed only in the Fall 2020 semester to evaluate an instructional intervention responding to the instructional changes due to the COVID-19 pandemic. Results focus on students' conflict resolution strategies consisting of instances when students experienced some form of conflict in the past and their reflection on how their strategy might change if they encountered the same situation after reading the case study and receiving conflict resolution training.

In Figure 4 , we show Sankey charts depicting counts of students for each initial applied strategy on the left and the potential new applied strategy that students thought would be better on the right. The chart also shows the directed flow of how the initial strategy transitioned to a different one. It suggests an overall increase in the number of students who adopted collaborating and compromising conflict resolution modes among the HyFlex students. It was observed that among HyFlex fully online students, there was an overall increase in the number of students who would adopt a collaborating conflict resolution mode. At the same time, there was an overall increase in the number of students who would adopt a compromising conflict resolution mode among the face-to-face students who received the conflictresolution training. This is in contrast to the group that did not receive the expert training who exhibited no notable or substantial changes in overall numbers in terms of conflict resolution modes.

Also, in all the HyFlex sessions, there was an overall reduction in the number of students who would have applied an accommodating or avoiding conflict resolution mode. Students' initial applied strategies and potential new strategies were also scored as described in Table 10 to account for a measure of the quality of their conflict resolution modes. Based on those scores, Table 14 presents descriptive statistics of students' conflict resolution strategies categorized by the course section in which they were enrolled. All sections initially described applying a conflict resolution strategy between accommodating and compromising. There was no significant change in the HyFlex in-person sessions in terms of potential new strategies applied. However, students in the HyFlex fully online session reported applying an overall higher quality in their potential uses of conflict resolution strategies in the future.

The Kruskal-Wallis H tests for the initially applied strategy among the three groups (H = 3.758, p = .153) suggest that, overall, the three groups described applying comparable strategies in solving the situation they described. Similarly, as observed by the gains described in Table 14 , all sessions showed an overall positive gain in terms of their descriptions of potential new strategies to apply, given they would face a similar situation (i.e., team conflict). However, only the HyFlex fully online session experienced statistically significant gains according to the Wilcoxon Signed-Rank test (z = 93.5, p = .007). The overall conflict resolution strategy in the fully online session moved from accommodating and compromising to between compromising and collaborating. This trend indicates a possible transition to a more effective conflict resolution strategy in future encounters with similar situations.

During the COVID-19 pandemic, it was important for the course instructor to deliver a comparable active learning experience that provided the technical and professional skills needed in the 21st-century workplace. Implementing a HyFlex instructional mode allowed for safely accommodating on-campus and remote students during the pandemic. By following principles of cooperative learning pedagogy and aided by web-conferencing technology, we were also able to provide a comparable situative learning experience for the HyFlex students similar to when the course was offered in a traditional in-person mode. This situative teamwork experience engaged students in meaningful practice where they had to coordinate and communicate better to (a) build knowledge and skills and (b) overcome challenges posed by social distancing restrictions. Research on online behaviors from distributed teams has identified that the quality of collaboration is related to high levels of coordination and communication (Serçe et al., 2011) . Students in the HyFlex instructional mode, mainly the online section students, might have put additional efforts into making their teamwork experience productive. However, we recognize that students may also have experienced challenges working on project deliverables as evidenced by performance differences on the milestones. Milestone 3 was particularly challenging, which students reported being the most difficult in terms of the amount of work and the team coordination needed. This milestone consisted of the individual project contribution (i.e., individual accountability), where all students were expected to submit a set of use-case narratives, activity diagrams, and sequence diagrams individually. Nevertheless, student performance on Milestone 3 did not significantly impact overall team performance as observed both from overall scores for Milestone 4 and on their submitted prototypes. These findings suggest that a cooperative learning environment can be adapted for delivery in a HyFlex mode of instruction (Kupczynski et al., 2012; McInnerney & Roberts, 2009; Millis, 2010) . However, educators need to provide additional coordination and team training (i.e., conflict resolution management) for teamwork support.

Through self and peer evaluations, students reported high scores on their team performance and team outcomes for the in-person and HyFlex sections, with the HyFlex Tuesday in-person session posting the highest scores. This session received conflict resolution training, which could have aided students in overcoming conflict and perhaps in developing team cohesion (i.e., a shared bond that motivates team members to stay together and to want to work together; Casey-Campbell & Martens, 2009). Team cohesion, which has been identified as a key contributor to team success (Salas et al., 2015) , can result when teams overcome conflict. For instance, a longitudinal study of 53 teams identified that conflict management had a direct, positive effect on team cohesion (Tekleab et al., 2009) . Compared to the two other formats of the course, students in the HyFlex fully online session reported moderate scores in team processes and team outcomes. This could be attributed to students not having an opportunity to get to know one another and meet in person. While the lead instructor tried to form teams with students in the same time zone, students may have experienced challenges in making time for synchronous working sessions. Furthermore, students in the HyFlex fully online session may not have developed team cohesion due to a lack of opportunities to have meaningful in-person interactions.

In the in-person Fall 2019 semester, students reported higher overall scores about setting goals, assigning roles, creating and following procedures, and establishing interpersonal relationships. Both HyFlex in-person sessions reported lower overall scores regarding setting goals and assigning proper roles. In contrast, the HyFlex fully online session reported low scores in establishing and following processes and building interpersonal relationships. This difference can be attributed to the decreased in-class time devoted to teamwork. Future research is needed to identify if differences in performance and behaviors in the online section were due to students' cultural backgrounds. In our study, students in the online section were primarily international students. While some research suggests that cultural background may affect the collaborative behaviors of teams, other studies have found no significant differences (Serçe et al., 2011) . Similarly, future research is needed to identify how transferable teamwork skills are to the workplace.

Regarding opportunities to work in teams outside of the lecture time, recall that the Fall 2020 course offering had a 50% reduced class time, and thus, less opportunity during the lecture time to work in teams. However, students from the HyFlex in-person sessions could still meet outside of class face-to-face or join the web conferencing teamwork session facilitated by the TA. The lack of social interaction was more critical for the HyFlex fully online students. Those students had only the option to join the web conferencing teamwork session facilitated by the TA, but often differences in time zones precluded them from joining the scheduled whole-class teamwork section. Some teams took advantage of requesting additional sessions with the TA on an accommodated schedule, but these instances were rare. As a result, students in the HyFlex fully online session had even fewer opportunities to build interpersonal relationships and follow collaboration processes established. This finding can also be explained under the perspective of team cohesion: "Individuals who feel no sense of cohesion with their team (whether due to distrust, dislike, disinterest, or a host of other reasons) are less motivated and less likely to participate in the 'teaming' behaviors that enable the many positive effects of teams" (Salas et al., 2015, p. 365) . While in-person and HyFlex teams described applying collaboration strategies associated with interpersonal relationships and processes, students in the HyFlex fully online session reported more strategies associated with setting goals and plans than strategies associated with building interpersonal relationships and processes.

Concerning the potential effect of the conflict resolution training that students in the HyFlex Tuesday in-person and HyFlex fully online session received, it can be observed that the intervention had a positive effect in the short term; that is, all students who received the conflict resolution training in the Fall 2020 semester showed an overall positive gain in terms of their descriptions of potential new strategies to apply, given they would face a similar situation (i.e., team conflict). Specifically, students in the HyFlex Tuesday in-person session reported potentially using a more compromising mode for conflict resolution if encountering a conflict they had experienced in the past. Students in the HyFlex Thursday in-person session reported potentially using more compromising and collaborating modes for conflict resolution. None of the students initially described applying a collaborating conflict resolution mode in the HyFlex fully online session.

In contrast, after the intervention consisting of conflict resolution training and the case study, nine students reported that they would adopt a collaborating strategy in the event of a conflict in the future. This change suggesting that students in the HyFlex fully online session benefitted the most from the expert training was statistically significant. Previous work suggests that the collaborating conflict resolution mode is the most desirable as it maximizes concern for oneself and others (i.e., Thomas & Kilmann, 1974) followed closely by compromising, which exhibits concern for self and others but not to the degree of a collaborating conflict resolution mode. Longitudinal research is needed to identify if students can transfer conflict resolution skills, manage conflict effectively, and investigate if such intent translates to actual behavior change.

Overall findings from this study are consistent with previous literature in that cooperative learning pedagogy can help teams function better. However, the learning experience for the HyFlex fully online students can be improved by enhancing their social interactions. Previous work has shown that active social interaction can be enabled by the increased social presence (Kreijns et al., 2003) . Social presence and a sense of community are key to promoting collaborative learning and knowledge building (Gunawardena, 1995) . While the course revisions effectively promoted teaching presence via cooperative learning and cognitive presence via the course project, social presence could be further supported to accommodate students' psychosocial factors. We implemented conflict resolution training to improve teamwork skills and communication among members. Our study suggests that this training was most beneficial for the HyFlex fully online students. However, students in this section of the course may not have been able to attend the weekly scheduled teamwork sessions with the TA due to time zone differences. As such, we cannot assume that students would spontaneously meet when there was no monitoring or facilitation from the instructor or TA.

Active social interaction is the basis for productive engagement (Appleton et al., 2006) . Research in online learning environments has identified that to create a deep and meaningful collaborative learning environment, the three elements of Community of Inquiry must be considered : (a) teaching presence in the design and facilitation of the learning experience (Anderson et al., 2001) , (b) cognitive presence in providing students with opportunities to construct and confirm meaning (Chen et al., 2019) , and (c) social presence in enabling communication in trusting environments that afford interpersonal relationships (d'Alessio et al., 2019) . A literature review concluded that psychosocial factors, such as relationship building, cohesion, and trust, are crucial for the effectiveness of virtual teams (Chang & Bordia, 2001; Lin et al., 2008) . Communication was identified as equally important as it directly influences the social dimensions of the team (Lin et al., 2008) . In virtual teamwork, communication among team members is often an issue, which may further exacerbate the physical distance, along with language and cultural differences (Ale Ebrahim et al., 2009) . Nevertheless, team training interventions are viable for enhancing teamwork performance (McEwan et al., 2017; Salas et al., 2008) . Research has identified that web conferencing can increase social presence among team members (Rourke et al., 1999) . In future course implementations, we will schedule online teamwork sessions at more flexible hours to accommodate students in different time zones.

Engineering education researchers have called for more empirical studies to identify and promote the adoption of effective research-based teaching practices (Finelli & Froyd, 2019) . Engineering education researchers have also called for more research studies focused on connecting theoretical and empirical work in the context of situated learning (Johri & Olds, 2011) . Additionally, research on HyFlex learning is very limited (Kyei-Blankson & Godwyll, 2010) .

This study contributes to these efforts by describing two implementations of cooperative learning pedagogy (Johnson & Johnson, 2011) . One implementation was initiated in an in-person active learning version of the course, and the other in HyFlex in-person and fully online versions of the course. This study contributes to online and blended learning research where more research-based approaches that maximize student engagement and build teamwork skills are needed (Ale Ebrahim et al., 2009; Bond et al., 2020; Gillett-Swan, 2017) .

A particular area of opportunity in engineering education research is situative work through the lens of Community of Inquiry . As mentioned earlier, this framework involves creating constructivist learning experiences involving social presence, teaching presence, and cognitive presence. We argue that although these elements have been studied separately in engineering education, more work is needed to investigate the joint integration of these three interacting elements. Specifically, research is needed to characterize the role of instructors, TAs, and course developers in selecting the content and design of practices, setting the course climate, and supporting discourses in classrooms, online, and blended environments . In addition, research should also include an investigation of the educational experience as a whole and the resulting construction of knowledge and student meaning-making .

In the context of a HyFlex delivery mode, it is important to focus on the "flexibility" aspect provided to the students and the motivation for choosing in-person or online delivery modes. Also, more research is needed to characterize the learning experience of those students who moved between the in-person and online modes intermittently (e.g., because they got sick) or permanently (e.g., because they had to go home to take care of family in the middle of the semester).

Our findings indicate that a HyFlex course implementation provided students with a comparable learning experience to traditional in-person delivery. Evidence for this included the team performance, individual exam grades, self-reported team performance and team outcomes, reported applied collaboration strategies, and potential strategies to address team conflict. Furthermore, our study also identified the need for learning strategies and supports that can improve social interactions among HyFlex fully online students to enhance social presence. As documented in previous studies, our HyFlex implementation provided benefits such as accommodating students' needs and life circumstances during the pandemic and increasing access to course content and teamwork experiences (Abdelmalak & Parra, 2018) . However, we also recognize that our HyFlex implementation came with challenges such as students' low level of engagement going unnoticed, technical difficulties with the technology, and additional work for the faculty and the TAs.

One limitation of our study was that we could not track the interaction of in-person and online students outside of class. Similarly, we were unable to document differences between how in-person and online students took advantage of the teamwork sessions provided by the TA during class time and with special accommodations to fit their time zones. Also, we could not document other difficulties students may have experienced. For example, students may have experienced problems accessing the modeling software, poor connectivity, difficult living conditions at home, substantial personal responsibilities regarding other family members, and challenges related to time management and mental health. We were also unable to control or measure the effect of students' cultural or generational backgrounds.

We also acknowledge that some of these measures are based on student perceptions (i.e., self and peer evaluations and team retrospectives). Others are hypothetical and based on reflections (i.e., applied vs. potential conflict resolution strategies). However, our performance data are more objective (i.e., scores over four project milestones and a modeling exam). Reflective practices of the course can aid learning and can be transferred to other situations. Our future implementations of the course will take advantage of the lessons learned during the pandemic. For instance, providing students a virtual working space equipped with web conferencing capabilities along with file-sharing capabilities was particularly useful for communication, documentation, and instructor feedback. Such use of technology will be permanently adopted for the course. Similarly, we will permanently adopt regular in-class sessions for teamwork facilitation, mediation, and conflict resolution training in future in-person offerings of the course. Regarding research, our future work will (1) continue to investigate ways to enhance social presence among students; (2) perform a deeper analysis on the low performing groups to identify challenges, specific needs, and supports to move forward; (3) quantify and correlate social presence with team performance and team cohesion; and (4) identify how teamwork skills learned in the context of this course can be transferred to other courses. Paul Thomas is a Doctoral Student in Computer and Information Technology at Purdue University, 401 North

Expanding learning opportunities for graduate students with HyFlex course design

Case study of HyFlex course design: Benefits and challenges for graduate students

2016-2017 criteria for accrediting engineering programs engineering accreditation commission

Virtual teams: A literature review

Assessing teaching presence in a computer conferencing context

Measuring cognitive and psychological engagement: Validation of the student engagement instrument

Committee effectiveness in higher education: The strengths and weaknesses of group decision making

Hybrid courses with flexible participation: The HyFlex course design

Challenges of student equity and engagement in a HyFlex course

Mapping research in student engagement and educational technology in higher education: A systematic evidence map

Sticking it all together: A critical assessment of the group cohesion-performance literature

Teaching beyond the topic teaching teamwork skills in higher education

A multidimensional approach to the group cohesion-group performance relationship

What if online students take on the responsibility: Students' cognitive presence and peer facilitation techniques

Scientific antecedents of situated cognition

Learning and cognition in education

An expanded typology for classifying mixed methods research into designs

Research design. Qualitative, quantitative and mixed methods approaches

How do virtual teams process information? A literature review and implications for management

Social presence enhances student performance in an online geology course but depends on instructor facilitation

The growing importance of social skills in the labor market

What do you mean by collaborative learning?

How TRW automotive helps global virtual teams perform at the top of their game. Global Business and Organizational Excellence

An interpretive structural modeling of teamwork training in higher education

Meaningful learning and creativity in virtual worlds

Improving student learning in undergraduate engineering education by improving teaching and assessment

Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education

Blended learning: Uncovering its transformative potential in higher education

The challenges of online learning. Supporting and engaging the isolated learner

Cognition and learning

Evaluating project characteristics for selecting the best-fit agile software development methodology: A teaching case

Three approaches to qualitative content analysis

The unified process

The encyclopedia of peace psychology

Active learning: Cooperation in the college classroom

Situated engineering learning: Bridging engineering education research and the learning sciences

Constructivism and computer-mediated communication in distance education

Understanding the interactions between the scrum master and the development team: A game-theoretic approach

Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research

Cooperative learning in distance learning: A mixed methods study

An examination of learning outcomes in HyFlex learning environments

Benefits of collaborative learning

Situated learning: Legitimate peripheral participation

The HyFlex option for instruction if campuses open this fall. Inside Higher Ed

A model to develop effective virtual teams

Facilitating peer evaluation in team contexts: The impact of frame-of-reference rater training

Fostering cooperative learning with scrum in a semi-capstone systems analysis and design course

Investigating the effectiveness of institutional-level support teams at full-service schools in South Africa

Student engagement in instructional activity: Patterns in the elementary, middle, and high school years

The effectiveness of teamwork training on teamwork behaviors and team performance: A systematic review and meta-analysis of controlled interventions

Interrater reliability: The kappa statistic

Collaborative and cooperative learning

Kruskal-Wallis test

Cooperative learning in higher education: Across the disciplines, across the academy

Things that make us smart: Defending human attributes in the age of the machine

Turning student groups into effective teams

Collaborative versus cooperative learning: A comparison of the two concepts which will help us understand the underlying nature of interactive learning. US Department of Education, Office of Educational Research and Improvement

A checklist to diagnose teamwork in engineering education

A systematic literature review of teamwork pedagogy in higher education

Assessing social presence in asynchronous text-based computer conferencing

Making health teams work: An educational program. Massachusetts Institute of Technology

Does team training improve team performance? A meta-analysis

Measuring team cohesion: Observations from the science

Fundamental statistical principles for the neurobiologist: A survival guide

Grade increase: Tracking distance education in the United States

Online collaboration: Collaborative behavior patterns and factors affecting globally distributed team performance

When does cooperative learning increase student achievement?

Pedagogies of engagement: Classroom-based practices

Enhancing learner-learner interaction using video communications in higher education: Implications from theorising about a new model

A test of the motivations underlying choice of conflict strategies in the dual-concern model

Designing adaptive instruction for teams: A meta-analysis

Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis

The scrum papers: Nut, bolts, and origins of an agile framework

A longitudinal study of team conflict, conflict management, cohesion, and team effectiveness. Group & Organization Management

Thomas-Kilmann conflict mode instrument

On-line learning migration: From social learning theory to social presence theory in a CMC environment

The hard work of soft skills: Augmenting the project-based learning experience with interdisciplinary teamwork

Online learning in higher education: A review of research on interactions among teachers and students. Education, Communication & Information

The future of engineering education: Part 3. Developing critical skills

The HyFlex course design: A case study on adult and career education courses

Magana is the W.C. Furnas Professor in Enterprise Excellence in the Department of Computer and Information Technology at Purdue University, 401 North Grant Street

Tugba Karabiyik is a Postdoctoral Associate in the Purdue Polytechnic Institute at Purdue University, 401 North Grant Street

This research was supported in part by the National Science Foundation under Award IIS Number 2113991 and a Protect Purdue Innovations Faculty Grant. Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or Purdue University.ORCID Alejandra J. Magana https://orcid.org/0000-0001-6117-7502