Frontline Learning Research Vol.8 No. 3 Special
issue (2020) 126 - 139
ISSN 2295-3159
aUniversity of California, Santa
Barbara, USA
bUniversity of Arizona, USA
Article received 17 May 2019 / revised 15 November/ accepted 1 January / available online 30 March
Self-report data have contributed to a rich understanding of learning and motivation; yet, self-report measures present challenges to researchers studying students’ experiences in small-group contexts. Rather than using self-report data alone, we argue that fusing self-report and observational data can yield a broader understanding of students’ small-group dynamics. We provide evidence for this assertion by presenting mixed-methods findings in three sections: (a) self-report data alone, (b) observational data alone, and (c) the fusion of both data sources. We rely on 101 students’ self-reported experiences as well as observational (i.e., audio) data of students working in their group (N = 24 groups). In section order, we found that (1) students’ self-reported small-group behavior predicted their end-of-study reported anxiety and emotion; (2) coded observational data captured five types of group dynamics that students can engage in; and (3) students’ initial group-level characteristics predicted their real-time group dynamics, and observed group regulation activity predicted students’ self-reported anxiety, emotion, and regulation moving forward. Thus, while self-report and observational data alone can each increase our understanding of student motivation and learning processes, pursuing both in tandem more effectively captures the give-and-take among students, how these experiences evolve over time, and the personal meanings they can afford.
Keywords: Self-Report; Observation; Small-Group Dynamics; Motivation; Co-Regulation
Instruments measuring students’ motivation and learning processes have contributed to a rich understanding of students’ experiences in school. Yet, the extent to which self-report measures adequately capture the learning process for all students across varying contexts remains an important concern (e.g., Urdan & Bruchmann, 2018). For researchers studying specific instructional contexts, self-report data can pose challenges to investigating motivation and strategy use in small groups. Namely, self-report measures make it difficult to investigate how students’ reported behavior and emotion occur in real-time and in relation to other people in their immediate environments. When students work together, each person brings unique experiences and characteristics into their small groups. How identity, disposition, motivation, and readiness to learn impact group functioning—and how individuals are impacted by their interactions with others over time—reflects a dynamic process that self-report data alone cannot capture.
To better understand this complex learning environment, we pursued a longitudinal, mixed-methods study of 101 students’ small-group experiences during six math lessons (n = 24 groups from two third-grade and two fifth-grade classrooms). Students completed self-report measures at pretest and at posttest. Throughout the study, students also completed an instrument describing their individual small-group behaviors after each lesson. Finally, after completing the study, students responded to items asking them how they would feel if their teacher asked them to get into small groups again. In addition to these self-report data, our project included real-time audio data of students working in their small groups.
Selected results of this study were briefly discussed as part of a larger chapter focusing on the guiding theoretical perspective (McCaslin & Vriesema, 2018); we present the full study here for the first time. The present special issue aims to better understand the impact of self-report data on theory and practice (Dinsmore & Fryer, 2020). We contribute to this goal by specifically addressing two of the three guiding questions: How does the use of self-report constrain the analytical choices made with self-report data, and how do the interpretations of self-report data influence interpretations of findings? We situate both questions within the context of small-group research. We begin by briefly introducing the guiding theory. We then present our study’s findings in three sections depicting what we learn from self-report data alone, observational data alone, and integrating both data sources.
The co-regulation model (McCaslin, 2009) that guides this research is a motivation perspective positing that learners are social, have a basic need for participation and validation (McCaslin & Burross, 2008), and differ in how and what they participate (McCaslin et al., 2016). Influenced by Vygotskian tenets, this theory describes how three sources of influence function together to inform emergent identity. These sources are cultural (e.g., norms, challenges), social (e.g., relationships, opportunities), and personal (e.g., readiness to learn, disposition). Students bring their personal backgrounds and characteristic adaptations to the classroom; yet, the opportunities presented to students and the relationships formed throughout their schooling experiences can shape who students become.
Given the dynamic processes described within this theoretical perspective, small-learning groups present an opportune setting to study emergent identity. Students in small groups each bring varying achievement levels, dispositions, and motivation to the task. However, the nature of the small-group instructional setting requires that students work together toward a common goal and negotiate challenges when necessary. When students work with each other across multiple occasions, small groups provide an opportunity to understand how student identity informs their work with other classmates and how these shared classroom experiences can shape student identity moving forward.
Some scholars have hailed small-group learning formats as the success story of educational psychology (Johnson & Johnson, 2009). Small-group activities can enhance student thinking and learning of both formal (e.g., math) and informal (e.g., appropriate social skills, motivated student engagement) content and skills (e.g., Elias & Schwab, 2006; Hadwin et al., 2018; Webb, 2008). However, while small-group learning has demonstrated benefits, there also are concerns that not all small-group activities are beneficial nor do all group members experience them similarly (Rogat et al., 2013; Webb, 2013). Naturalistic observational studies examining the processes that actually occur within small groups and what students make of them are relatively scarce. Extant research, however, suggests their importance (e.g., Hadwin & Järvelä, 2011; Tan et al., 2005; Webb, 2013).
Therefore, the dynamic processes occurring within small-group settings necessitate dynamic methodologies to study them. Asking students about their experiences in small groups can yield important information regarding students’ interpretations of events; and, researchers can investigate how students’ personal characteristics associate with these self-report data. However, self-report data alone cannot capture the give-and-take of small-group interactions. Yet, observational data alone also can fail to capture the full student experience. In the case of observation-only data, researchers rely on their own interpretations of events and fail to capture students’ own self-reported experiences of the events. Thus, combining self-report and observational data provides a foundation for more fully understanding how individual characteristics inform small-group dynamics, and how these dynamics inform student identity moving forward. To illustrate these points in finer detail, we present three sections that discuss (a) self-report data, (b) observational data, and (c) the fusion of both data sources in our research.
This section relies on self-report data to illustrate how students’ reported small-group behavior associated with their characteristics at pretest and posttest. First, we describe how students’ pretest characteristics—their teacher-ranked math readiness, self-reported anxiety and emotional adaptation (i.e., context-dependent emotion and coping strategies; McCaslin et al., 2016)—associated with their self-reported small-group behavior. Second, we show how self-reported group behavior predicted students’ posttest anxiety, emotional adaptation, and reports of how they would feel if their teachers asked them to get into small groups again. To contextualize these results, we first describe the relevant method information.
Students (N = 101) completed the pretest (October) and posttest (January) surveys that measured their anxiety and emotional adaptation. Teachers also ranked each of their students on mathematics achievement at pretest. At the end of the study, students completed an instrument asking them how they would feel if their teacher asked them to get into groups again. Throughout the study, students also completed short instruments immediately after each small group lesson to indicate their behavior during the lesson. We present students’ average reported behavior (i.e., the average across the six lessons) below in order to enhance clarity of the results.
2.2.1 What School is Like for Me (WSLM)
WSLM is the Test Anxiety Scale, a well-known, well-researched, and well-critiqued instrument (Pekrun, 2006; Zeidner & Matthews, 2005) adapted from Sarason et al., 1958). WSLM asks students to agree or disagree with 18 sentences describing anxious thoughts and feelings. Cronbach’s alpha for WSLM was α = .76 at pretest and .71 at posttest.
2.2.2 School Situations (SS)
School Situations (SS; Burggraf, 1993) is an adaptation of the Test for Self-Conscious Affect (TOSCA), a dispositional measure originally designed for adults and subsequently revised by Tangney and colleagues to include children (e.g., Tangney et al., 1995). The SS inventory asks students to use a five-point scale to endorse sentences in response to 12 written vignettes that portray routine school challenges within three contexts: whole class, small group, or private/individual. Sentences are behavioral representations of emotions (guilt, shame, or pride) and coping strategies (externalize, normalize).
Rather than consider the five SS scales (pride, guilt, shame, normalize, externalize) independently, as originally designed, we used five unique emotional adaptation profiles identified in previous research (McCaslin et al., 2016) for our analyses. The five profiles were: (1) Distance and Displace: the student attempts to withdraw from a difficult situation to care for the self and/or attempts to blame other people or things to find relief from feelings of shame; (2) Regret and Repair: the student attempts to repair or fix the situation and to care for the self through normalizing the event in order to find relief from feelings of guilt; (3) Inadequate and Exposed: the student assumes responsibility and blame for mistakes or difficulties without engaging in self-care or displacement strategies in response to negative emotion; (4) Proud and Modest: the student acknowledges success, but tempers feelings of pride with humility; and (5) Minimize and Move On: the student adopts a ‘just keep going, do not dwell, look beyond it’ escape response to mistakes and difficult situations. At pretest, Cronbach’s alpha was .75, .79, .70, .72, and .64 for Distance and Displace, Regret and Repair, Inadequate and Exposed, Proud and Modest, and Minimize and Move On, respectively. In the same order, internal consistency reliability at posttest was .75, .87, .70, .75, and .66, respectively.
2.2.3 How I was in Group Today (How I Was)
How I Was presented 20 sentences to students and asked them to underline any that described their behavior in their group that day. Sentences comprised three scales (McCaslin et al., 1994): (1) Enhancing: sentences that represent engagement from which other group members may benefit; (2) Neutral: sentences that represent participation that is neither active nor withdrawn; and (3) Interfering: sentences that describe preoccupation with concerns of the self. The Interfering scale consisted of items suggesting that students withdrew from or were unable to participate in small group activity (e.g., “My stomach felt funny”; “My head hurt”) rather than engaging in behaviors that actively distracted or interfered with others in small group. Therefore, we subsequently refer to this scale as “Withdrawn” to clarify this distinction.
2.2.4 How I Felt
How I Felt was designed to capture students’ thoughts and feelings when the teacher said it was time to get into their small group. It consisted of six items that described positive and negative emotional experiences in three relative domains: cognitive, affective, and physiological. Interested (cognitive), Happy (affective mood), and Relaxed (physiological) comprised the “positive” scale (α = .85). Confused (cognitive), Sad (affective mood), and Nervous (physiological) comprised the “negative” scale (α = .82). Students used a 3-point scale (not at all, a little bit, a lot) to respond to each item.
2.3.1 Pretest Student Characteristics and Self-Reported Small-Group Behavior
Students’ pretest anxiety and emotional adaptation did not associate with students’ self-reported small-group behavior. However, students with higher initial math readiness reported greater use of Neutral regulation strategies, such as listening, during their small groups (r = -.27, p = .007; higher numbers indicate lower rank in math readiness).
2.3.2 Self-Reported Small-Group Behavior and Posttest Student Characteristics
We pursued a series of multiple regression analyses that controlled for students’ reported pretest anxiety and pretest emotional adaptation. We did not control for group membership (e.g., using fixed effects models) because we believed that this might yield decontextualized results. In this paper, we focused on exploring how group processes shaped individual processes and vice versa; thus, we did not account for group membership in order to work toward this goal. However, we did attempt to cluster errors at the group level in our regression analyses in order to account for the shared variance within groups. Unfortunately, we did not have a sufficient number of participants for the number of groups in our study to run this analysis effectively. As a result, we proceeded to use traditional multiple regression analyses here and subsequently in the paper.
Results indicated that students’ self-reported behavior in their small-groups predicted students’ posttest anxiety (F(9, 72) = 4.16, p < .001; r2 = .34, adjusted r2 = .26), as well as two emotional adaptation profiles: Regret and Repair (F(9, 71) = 5.22, p < .001; r2 = .40, adjusted r2 = .32) and Inadequate and Exposed (F(9, 71) = 3.00, p = .004, r2 = .28, adjusted r2 = .18). Specifically, reported use of Enhancing regulation during small group predicted less anxiety at posttest (β = -2.32, p = .035). Use of Withdrawn regulation also predicted lower endorsement of Regret and Repair and Inadequate and Exposed emotional adaptation at posttest (β = -0.25, p = .01; β = -0.22, p = .051, respectively).
2.3.3 Self-Reported Small-Group Behavior and Posttest Anticipated Affect
We pursued a series of multiple regression analyses that controlled for students’ pretest anxiety and emotional adaptation to determine how students’ self-reported behavior during small group predicted their anticipated affect at posttest (i.e., when they imagined the teacher asking them to get into small groups again). Students’ self-reported behavior in small groups predicted their endorsement of both positive and negative affect (F(9,75) = 3.00, p < .001, r2 = .32, adjusted r2 = .24; F(9,75) = 3.45, p = .001, r2 = .29, adjusted r2 = .21, respectively). Reported Enhancing behavior during small group predicted greater anticipated positive affect (β = 0.50, p < .001); in contrast, reported Withdrawn behavior predicted greater anticipated negative affect (β = 0.42, p < .001).
Overall, the self-report data indicated how students’ reported small-group behavior associated with their personal characteristics and attitudes at pretest and posttest. Specifically, average student-perceived Enhancing behavior across the six lessons predicted lower anxiety at posttest and greater positive affect at the end of the study when students imagined getting into small groups again. In contrast, students who described themselves as Withdrawn during their small groups felt more negative emotion when they imagined getting into small groups again. Student-perceived Withdrawn behavior also predicted less endorsement of Inadequate and Exposed and Regret and Repair emotional adaptation; thus, while withdrawing from participation might mitigate the potential for experiencing shame in small-group settings, it also prevents students from potentially developing strategies for overcoming interpersonal challenges with peers.
Although the interpretations of self-report data provided insight into how students’ perceived small-group behavior associated with their personal characteristics and expectations (e.g., affect), there are several important limitations. First, our analyses were constrained by individual-level data. The data allowed us to examine how students’ self-reported behavior associated with their pretest and posttest outcomes; yet, students do not participate in their small groups alone. The constrained data sources prevented a more complete understanding of the give-and-take among students in these settings. Second, our interpretations of the data relied purely on student reports. Students’ individual interpretations of their classroom activities are vital to understanding their emergent identity; however, finding ways to corroborate self-report data with real-time data can enhance understanding of self- and other-awareness in small-group dynamics.
While Section 1 illustrated associations with students’ self-reported small-group behavior, Section 2 depicts students’ actual behavior during their small groups. In Section 2, we describe the types of co-regulation dynamics that emerged during students’ small-groups lesson and how the dynamics associated with the groups’ average achievement on the small-group tasks; the group is the unit of analysis. We present the observational results after describing the relevant procedures and coding systems.
Three researchers independently analyzed, transcribed, and verified audio data of small-group interactions for three lessons (representing the beginning, middle, and end of the six lessons) for each group (N = 24 groups). The three researchers remained unaware of the larger study. Two complementary observation systems were developed for analyzing the audio data. We describe the coding systems below.
3.2.1 Group Behavior Checklist (GBC)
The first system, the GBC, is a lower-inference observation instrument that captured the range of on- and off-task behaviors that students displayed when working with others in small groups. This study used four GBC variable domains: (a) planning, (b) problem solving, (c) help-seeking, and (d) feedback. Coding was completed in 30-second intervals. In total, 2,180 intervals were coded with the GBC. The percentage of exact agreement (91%) among coders was calculated on three coding pairs over three lessons.
3.2.2 Group Environment Summary (GES)
The second system, the GES, is a higher-inference system that captured students’ interpersonal and affective dynamics and expressed intrapersonal coping strategies. Variable domains included group affective climate; giggle/laugh bursts; and types of aggressive, protective, regressive/escape, and somatic expressed coping behaviors. Coding was completed in two halves: at the mid-point and end of each lesson. The percentage of exact agreement was 73% among three coding pairs over three lessons. See McCaslin and colleagues (2011) for more complete documentation of audio enhancement and transcription procedures; McCaslin and Vega (2013) for coding system design, procedures, and application in the pilot study; and Vega (2014) for implementation decisions for the revised system.
3.2.3 Group Achievement
Student activity worksheets completed “by the group” as part of each lesson were scored and verified by two math educators for correctness. Percentage correct represented students’ small-group achievement for the lesson material.
3.3.1 Group dynamics
We represented the GBC and GES data as the percentage of intervals in which a behavior occurred. We then subjected the data from both observation systems to a principal components analysis using Varimax rotation in order to develop an understanding of overall group dynamics from our discrete coding categories. Results yielded five independent factors that collectively accounted for 61.92% of the variance in student small-group interactive behavior. Factors, in order of magnitude, were: Conflict and Control, Working Together, Resource Drain, Edgy Compliance, and Scuffle and Confusion (see Table 1 for example behaviors and percentage variance explained by factor). We consider these five distinct co-regulation dynamics that students can engage in while in small groups.
Table 2
Real-Time Co-Regulation Dynamics
Note. Only the top three positively loaded items for
each factor are listed in the table. Total number of items varied
by factor: n = 18 for Factor 1; n = 11 for Factor 2; n = 7 for
Factor 3; n = 9 for Factor 4; n = 5 for Factor 5. The exploratory
factor analysis yielded 8 cross-loaded items: 5 items loaded in
the opposite direction, and 3 items loaded in the same direction.
The five small-group dynamics factors can be organized into relatively task-focused, other-focused, or the fusion of the two perspectives. In task-focused contexts, student dynamics primarily centered on the academic activity at hand, whereas dynamics in other-focused contexts reflected an emphasis on one’s group members. In addition, we can consider how types of coping behaviors typically associated with individual student behavior—aggressive, protective, and regressive—emerged as characteristics of group co-regulation dynamics. Please see Figure 1 for a visual representation of how the joint activity varied across the five small-group dynamics.
Figure 1. Small-Group Regulation Foci
Note. This figure was adapted from McCaslin and Vriesema
(2018).
The Working Together dynamic fused the demands of task and peers in small group learning within a protective press. Group members could ask for assistance, disagree with each other, and offer suggestions and solutions without concern for personal safety. In comparison, an aggressive press encompassed both the relatively task-involved Edgy Compliance dynamic (in which provocative and aggressive behaviors were related to attempts to meet task demands) and the other-involved Conflict and Control dynamic (in which aggression and protection behaviors consumed group attention). Finally, the Scuffle and Confusion dynamic in the disorganized pursuit of task demands and the Resource Drain of needy peers were each marked by regressive, or relatively immature, co-regulation dynamics. Taken together, these profiles did not represent particular groups per se; rather, they represented the types of co-regulation dynamics—that is, the types of observed behavior (e.g., communication patterns, coping strategies)—that emerged during the students’ time in small groups. Please see Table 2 for the means and standard deviations for the co-regulation dynamics.
Table 3
Descriptive statistics for students’ co-regulation dynamics
Note. Means and standard deviations reflect the
percentage of time students spent engaging in each of the
co-regulation dynamics.
3.3.2 Group achievement
Scuffle and Confusion negatively associated with the average percentage correct on group task activities (r = -.46, p = .024). Group achievement did not associate with Conflict and Control (r = -.02, p = .929), Working Together (r = .09, p = .670), Resource Drain (r = .08, p = .718), or Edgy Compliance (r = .21, p = .321).
We presented this section depicting observational data alone for two reasons. First, the extensive coding systems provided a framework for understanding the types of dynamics that can emerge in small groups. The researchers observed student behavior that informed behavioral co-regulation ranging from the ‘ideal’ Working Together dynamics to the aggressive give-and-take between students (e.g., Conflict and Control) to the disorganized task pursuits that embodied Scuffle and Confusion. These interpretations, therefore, yielded a broader understanding of students’ systematically observed behavior during their small-group lessons.
Second, we presented the observational data alone to illustrate that even with real-time data of students working in their small groups, we fail to understand what these dynamics can mean for students’ identity moving forward. Relying on observational data alone constrained our analyses to correlations between small-group dynamics and group achievement on the small-group tasks. While this has the important benefit of aligning with the types of data available to teachers when they use small groups in their instruction, we argue that fusing observational and self-report data can provide a more nuanced understanding of students’ experiences in small groups as well as insight into what these classroom experiences can mean for students’ emergent identity. We provide evidence for this argument in the next section.
Rather than constraining self-report data to individual-level analyses or relying solely on group-level observational data, Section 3 first illustrates how group-level characteristics associated with real-time group dynamics. Specifically, we took the average of group members’ individual characteristics, such as emotional adaptation, to determine how the group’s overall approach to learning tasks associated with their group functioning during the small-group lessons. Second, we describe how the give-and-take of small-group dynamics predicted students’ individual characteristics at posttest (i.e., their anxiety, emotional adaptation, and anticipated affect). Third, while we have confidence in the reliability of our systematic coding procedures, researcher perceptions of small group dynamics may not coincide with student perceptions of their small-group experiences. Therefore, we also present results that depict the alignment between student- and researcher-perceived group behaviors.
4.1.1 Pretest Group Characteristics and Real-Time Group Co-Regulation Dynamics
We created group-averaged scores for anxiety, emotional adaptation, and math readiness in order to determine how group composition associated with co-regulation dynamics.
Anxiety. Group-averaged anxiety did not associate with students’ group regulation dynamics.
Emotional Adaptation. Group-averaged endorsement of Inadequate and Exposed positively associated with Working Together dynamics (r = .47, p = .021). Group-averaged endorsement of Distance and Displace positively associated with engagement in Resource Drain dynamics (r = .41, p = .048). Finally, group-averaged endorsement of Proud and Modest negatively associated with Edgy Compliance group dynamics (r = -.42, p = .039).
Math Readiness. Groups with a greater concentration of higher-ranked math students were more likely to display Working Together and Edgy Compliance dynamics (r = -.21, p = .049; r = -.23, p = .028 respectively). In contrast, groups with a greater concentration of lower-ranked math students were more likely to engage in Scuffle and Confusion dynamics (r = .37, p = .001).
4.1.2 Real-Time Group Co-Regulation Dynamics and Posttest Student Outcomes
We pursued a series of multiple regression analyses controlling for students’ pretest characteristics to determine how real-time group dynamics predicted students’ self-reported anxiety, emotional adaptation, and anticipated affect at posttest.
Anxiety. Small-group dynamics predicted students’ self-reported anxiety, F(11, 66) = 3.97, p < .001; r2 = .40, adjusted r2 = .30. Specifically, participating in groups that displayed Working Together and Resource Drain co-regulation dynamics predicted lower posttest student anxiety (β = -.27, p =.011; β = -.27, p = .012, respectively). Although students may have used different strategies in the two co-regulation dynamics, receiving help from peers in both contexts may have associated with lower anxiety at posttest.
Emotional Adaptation. Group dynamics predicted students’ endorsement of Regret and Repair at posttest, F(11, 65) = 4.09, p < .001; r2 = .41, adjusted r2 = .31. In particular, Edgy Compliance dynamics predicted greater Regret and Repair at posttest (β = .21, p = .038). Group dynamics did not predict the other four emotional adaptation profiles.
Anticipated Affect. We explored whether small-group dynamics predicted how students would feel if their teachers asked them to get into their small groups again. Students’ observed small-group dynamics predicted their self-reported anticipated positive affect (F(11, 69) = 2.08, p < .001; r2 = .25, adjusted r2 = .13). Specifically, participating in groups that displayed Working Together and Edgy Compliance dynamics predicted greater anticipated positive affect reported by students at end of the study (β = .43, p = .004, and β = .30, p = .041, respectively). Small-group dynamics did not predict anticipated negative affect.
4.1.3 Alignment between self-report and observational group data
To examine the alignment between student and researcher perceptions, we (a) created group-averaged How I Was scores for the same lessons for which we had coded data and then (b) examined the associations between self-reported group behavior and observed group behavior. Group is the unit of analysis (N = 24).
Group-averaged reported Enhancing behavior positively associated with Working Together and Resource Drain co-regulation dynamics (r = .27, p = .010; r = .31, p = .003, respectively). Group-averaged reported Withdrawn behavior positively associated with both Resource Drain (r = .24, p = .022) and Conflict and Control (r = .23, p = .027) dynamics. Finally, group-averaged reported Withdrawn behavior also negatively associated with the Working Together dynamic (r = -.29, p = .004).
Fusing the self-report and real-time observation data provided two main insights into students’ experiences in small groups. First, results indicated that students were aware of themselves within their groups. The self-reported small group behavior aligned with the systematic observation data. For example, self-reported Enhancing behavior positively associated with Working Together co-regulation, while self-reported Withdrawn behavior associated with greater Conflict and Control and less Working Together co-regulation. Further underscoring the alignment between the two data sources, Resource Drain co-regulation—dynamics in which group members expressed needs (e.g., by asking for materials, attention, etc.)—associated with both self-reported Enhancing and Withdrawn behavior. In this instance, groups consisted of members who provided help (Enhancing) to those who needed and/or wanted it (Withdrawn). Thus, students as young as grade three appear to accurately self-monitor, and students as old as grade five appear willing to accurately report their small-group behavior.
Second, results indicated that students’ emotional adaptation and academic readiness were important features of small-group dynamics and personal learning. For example, participation in Edgy Compliance co-regulation dynamics, unpleasant as it may have been, predicted an increase in students’ posttest endorsement of the Regret and Repair emotional adaptation profile. This suggested that students were not only aware of the behavior exhibited in their small group but that they learned about themselves and others from that experience. In this instance, interpersonal dynamics informed intrapersonal endorsements that appeared to move the student away from their prior experience toward the person they wanted to become—the person who feels badly when failing to support another and works to make amends.
Students’ academic readiness also provided evidence for the press between intra- and interpersonal dynamics. Groups with higher-ranked students, for example, were more likely to display Working Together and Edgy Compliance co-regulation dynamics. This suggests that students with higher math readiness have the potential to direct their resources in more productive (e.g., offering suggestions, asking questions) and less productive (e.g., bragging, refusing others’ participation) ways. Yet, in spite of the different real-time interaction patterns, the self-report data indicated that students learned from these experiences and that the small groups shaped students’ posttest characteristics. As noted with Edgy Compliance co-regulation, these dynamics predicted greater endorsement of positive regulation strategies moving forward (Regret and Repair); and, for the groups already Working Together, these dynamics predicted lower anxiety at posttest.
In this paper, we addressed two of the three guiding questions in this special issue: How does the use of self-report constrain the analytical choices made with self-report data, and how do the interpretations of self-report data influence interpretations of findings? Our goal was to illustrate the benefits and challenges of using self-report data to understand students’ experiences in and attitudes towards small groups.
Overall, using self-report data alone provided insight into students’ self-awareness and perceptions of their own behavior during small group. While useful for capturing the perceived student experience, relying purely on self-report data constrained our inquiry to individual-level analyses in ways that ignored the mutual give-and-take between the individual and their group members. Thus, the primary challenge of using self-report data to study small groups is that we fail to capture the dynamic processes that are inherent in these social learning tasks. In other words, self-report data can illustrate how personal sources of influence (e.g., math readiness, emotional adaptation) shape students’ experiences and emergent identities; yet, we fail to also learn how students shape—and are shaped by—the mutual press between personal and social sources of influence in real time.
Although we emphasized the role of self-report data for understanding students’ experiences in small groups, this paper also identified strengths and limitations of observation-only data. For example, our real-time data corroborated research by Ladd and colleagues (2014) in which students reported the (lack of) positive small-group behaviors displayed by their peers. The researchers noted that “substantial proportions of participants received average ratings that were so low…as to imply that they “seldom” or “never” exhibited such skills during collaborative classroom activities” (p. 169). Our data provided insight into which behaviors and skills these peers might engage in instead. Working Together is great when it happens, but the pursuit of joint activity, in which disagreements are respected, questions appreciated, peer elaborations valued, and understandings deepened, does not represent the reality of the array of small group dynamics. Instead, groups also display aggressive and regressive coping behaviors that result in more or less effective group functioning.
Yet, while observation-only data yielded a broader understanding of real-time behavior in small groups, we nevertheless failed to capture students’ perceived experiences within these dynamics. Rather, by fusing self-report and observational data, we learned that small-group co-regulation dynamics were saturated with social and self-conscious emotions, and the uneven regulation of those emotions often did not proceed smoothly or turn out well. Overall, students differed in their typical need to cope with learning difficulty, but coping with lack of control and uncertainty are part of what it means to be in a small group for most. To be in a small group with peers—classmates who vary in their own learning and social skills (Ladd et al., 2014; Rogat et al., 2013)—can exacerbate or attenuate that reality. Thus, like others in this special issue (e.g., Rogiers et al., 2020; van Halem et al., 2020), we argue that using multiple data sources can yield broader understandings of student behavior in classroom settings.
The focus on self-report data in this paper and special issue warrants further discussion of survey data in particular. First, some critiques of self-report data question whether participants can provide accurate responses to researchers’ survey items. These concerns broadly reflect literature showing that participants sometimes fail to understand their own motives (Nisbett & Wilson, 1977) or provide opinions about events that did not happen (Bishop et al., 1980). However, the present study provides evidence that students in grades three and five can (and are willing to) provide accurate reports of their time in small groups. Students’ self-reported individual group behavior aligned with researcher-observed behaviors taking place during the small-group activities. For researchers, this suggests the utility of using self-report data in research on small-group processes, particularly when these survey measures can be corroborated with other data sources. Furthermore, in line with prior recommendations (Corno, 2011), teachers may also want to consider using brief surveys to better understand how small-group activities unfolded in their classrooms. Of course, while this strategy may help teachers to refine these activities in their classes, future research will also need to determine whether students’ responses vary depending on whether students are reporting co-regulation dynamics to their teachers or to researchers.
Second, in addition to considering participants’ understanding of their own attitudes, some researchers express concerns about using self-report data due to the surveys themselves. These criticisms acknowledge that there may be aspects of any given survey that can prevent participants from responding as accurately as possible (Duckworth & Yeager, 2015). In the current paper, the internal consistency reliability coefficients generally provided one source of evidence for using these survey measures in our analyses. However, it is important to acknowledge that one of our five School Situations factors, Minimize and Move On, fell below the recommended .70 for Cronbach’s alpha (Nunnally, 1978). Thus, even though this factor—capturing student escape strategies—was identified in previous research using this same instrument (McCaslin et al., 2016), we encourage researchers to replicate this work to determine whether the five School Situations factors emerge in their own samples, or whether some strategies do not translate across all contexts. In spite of the relatively low internal consistency for the Minimize and Move On factor, we are confident in our self-report measures, again due to the alignment between the survey and observational data in this study.
In sum, conceptions of small-group members in terms of their cooperative or regulatory skill set is a start that is likely to sputter without recognition of the fullness of individuals who have personal histories and concerns that make them more and less vulnerable to threat (Frijda, 2008) and making threats. This calls for expanding conceptions of small-group cooperative skills and dispositions of individuals to include, for example, considerations of power and influence among group members. Our observations of demanding behavior and provocative exchange suggest that students can consider power from a perspective of coercion and control rather than one of support and positive influence (Keltner, 2016). Students’ personal concerns and heightened perceptions of threat are part of power and influence dynamics. Both are better understood within deliberate consideration of conflicts that may underlie and result from them. We do students a disservice when we fail to acknowledge the fullness of the task of working and learning with others. We also miss an opportunity to fully learn from the potential of small-group learning for students’ personal growth and well-being when we fail to use multiple research methods fluidly. Thus, while self-report and observational data alone can each increase our understanding of student motivation and learning processes, pursuing both in tandem can yield richer understandings of students’ classroom activity, how these experiences evolve over time, and how that matters in the dynamics of being and becoming a student.
Bishop, G. F., Oldendick, R. W., Tuchfarber, A. J., & Bennett,
S. E. (1980). Pseudo-opinions on public affairs. The Public
Opinion Quarterly, 44(2), 198-209.
Burggraf, S. A. (1993). School situations. Unpublished manuscript.
Bryn Mawr, PA: Bryn Mawr College.
Corno, L. (2011). Studying self-regulation habits. In H. D.
Schunk, & B. Zimmerman (Eds.), Handbook of self-regulation of
learning and performance (pp. 361-375). New York: Routledge.
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters:
Assessing personal qualities other than cognitive ability for
education purposes. Educational Researcher, 44(4), 237-251.
https://doi.org/10.3102/0013189X15584327
Elias, M. J., & Schwab, Y. (2006). From compliance to
responsibility: Social and emotional learning and classroom
management. In C. M. Evertson & C. S. Weinstein (Eds.),
Handbook of classroom management: Research, practice, and
contemporary issues (pp. 309-341). Mahwah, NJ: Lawrence Erlbaum
Associates.
Frijda, N. H. (2008). The psychologists’ point of view. In M.
Lewis, J. M., Haviland-Jones, & L. F. Barrett (Eds.), Handbook
of emotions, 3rd ed. (pp. 68-87). New York: Guilford Press.
Fryer, L. K., & Dinsmore, D. L. (2020). The promise and
pitfalls of self-report: Development, research
design and analysis issues, and multiple methods. Frontline
Learning Research, 8(3), 1–9.
http://doi.org/10.14786/flr.v8i3.623
Hadwin, A. F., & Järvelä, S. (2011). Introduction to a special
issue on social aspects of self-regulated learning: Where social
and self meet in the strategic regulation of learning. Teachers
College Record, 113(2), 235-239.
Hadwin, A. F., Järvelä, S., & Miller, M. (2018).
Self-regulation, co-regulation, and shared regulation in
collaborative learning environments. In D. H. Schunk & J. A.
Greene (Eds.), Handbook of self-regulation of learning and
performance (pp. 83-06). New York, NY: Routledge.
Johnson, D. W., & Johnson, R. T. (2009). An educational
psychology success story: Social interdependence theory and
cooperative learning. Educational Researcher, 38, 365-379.
https://doi.org/10.3102/0013189X09339057
Keltner, D. (2016). The power of paradox: How we gain and lose
influence. New York, NY: Penguin Press.
Ladd, G. W., Kochenderfer-Ladd, B., Visconti, K. J., Ettekal, I.
Sechler, C. M., & Cortes, K. I. (2014). Grade-school
children’s social collaborative skills: Links with partner
preference and achievement. American Educational Research Journal,
51(1), 152-183. https://doi.org/10.3102/0002831213507327
McCaslin, M. (2009). Co-regulation of student motivation and
emergent identity. Educational Psychologist, 44(2), 137-146.
https:// doi.org/10.1080/00461520902832384
McCaslin, M., & Burross, H. (2008). Student motivational
dynamics. Teachers College Record, 110(11), 2319-2340.
McCaslin, M., Tuck, D., Waird, A., Brown, B., LaPage, J., &
Pyle, J. (1994). Gender composition and small-group learning in
fourth-grade mathematics. Elementary School Journal, 94,
467-482.
McCaslin, M., & Vega, R. I. (2013). Peer co-regulated
learning, emotion, and coping in small-group learning. In S.
Phillipson, K. Y. L. Ku, S. N. Phillipson (Eds.), Constructing
educational achievement: A sociocultural perspective (pp.
118-135). New York, NY: Routledge.
McCaslin, M., & Vriesema, C. C. (2018). Co-regulation: A model
for classroom research in a Vygotskian perspective. In D. M.
McInerney & G. A. D. Liem (Eds.), Big theories revisited
2: Research on sociocultural influences on motivation and
learning. Charlotte, NC: Information Age Publishing.
McCaslin, M., Vega, R. I., Anderson, E. E., Calderon, C. N.,
Labistre, A. M. (2011). Tabletalk: Navigating and negotiating in
small-group learning. In D. McInerney, R. Walker, G. Liem (Eds.),
Sociocultural theories of learning and motivation: Looking
back, looking forward (pp. 191-222). Charlotte, NC:
Information Age Publishing.
McCaslin, M., Vriesema, C. C., & Burggraf, S. (2016). Making
mistakes: Emotional adaptation and classroom learning. Teachers
College Record, 118(2).
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we
can know: Verbal reports on mental processes. Psychological
Review, 84(3), 231-259. https://doi.org/10.1037/0033-295X.84.3.231
Nunnally, J. C. (1968). Psychometric theory (2nd edition). New
York: McGraw-Hill.
Pekrun, R. (2006). The control-value theory of achievement
emotions: Assumptions, corollaries, and implications for
educational research and practice. Educational Psychology Review,
18, 315-341. https://doi.org/10.1007/s10648-006-9029-9
Rogat, T. K., Linnenbrink-Garcia, L., & DiDonato, N. (2013).
Motivation in collaborative groups. In C. E. Hmelo-Silver, C. A.
Chinn, C. K. K. Chan, & A. M. O’Donnell (Eds.), The
International Handbook of Collaborative Learning (pp. 250-267).
New York: Taylor & Francis.
Rogiers, A.; Merchie, E., & van Keer, H. (2020). Opening the
black box of students’ text-learning processes:
A process mining perspective. Frontline Learning Research,
8(3), 40–62. http://doi.org/10.14786/flr.v8i3.527
Sarason, S. B., Davidson, K. S., Lighthall, F. F., & Waite, R.
R. (1958). A test anxiety scale for children. Child Development,
29(1), 105-113.
Tan, I. G. C., Sharan, S., & Lee, C. K. E. (2007). Group
investigation effects on achievement, motivation, and perceptions
of students in Singapore. The Journal of Educational Research,
100(3), 142-154. https://doi.org/10.3200/JOER.100.3.142-154
Tangney, J. P., Burggraf, S. A., & Wagner, P. A. (1995).
Shame-proneness, guilt-proneness, and psychological symptoms. In
J. P. Tangney & K. W. Fischer (Eds.) Self-conscious emotions:
The psychology of shame, guilt, embarrassment, and pride (pp.
343-367). NY: Guilford Press.
Urdan, T., & Bruchmann, K. (2018). Examining the academic
motivation of a diverse student population: A consideration of
methodology. Educational Psychologist, 53(2), 114-130.
https://doi.org/10.1080/00461520.2018.1440234
van Halem, N., van Klaveren, C. P. B. J., Drachsler, H., Schmitz,
M., & Cornelisz, I. (2020). Tracking
patterns in self-regulated learning using students’ self-reports
and online trace data. Frontline Learning Research, 8(3),
142-164. http://doi.org/10.14786/flr.v8i3.497
Vega, R. I. (2014). The role of student coping in the socially
shared regulation of learning in small groups. Unpublished
doctoral dissertation. Tucson, AZ: University of Arizona.
Webb, N. M. (2008). Learning in small groups. In T. L. Good (Ed.),
21st century education: A reference handbook (Vol. 2., pp.
203–211). Thousand Oaks, CA: Macmillan.
Webb, N. M. (2013). Information processing approaches to
collaborative learning. In C. E. Hmelo-Silver, C. A. Chinn, C. K.
K. Chan, & A. M. O’Donnell. (Eds.), The international handbook
of collaborative learning (pp. 19-40). New York: Taylor &
Francis.