key: cord-0998060-ztmhfvq6
authors: Huang, Xiaoxia; Mayer, Richard E.; Usher, Ellen L.
title: Better Together: Effects of Four Self-Efficacy-Building Strategies on Online Statistical Learning
date: 2020-10-03
journal: Contemp Educ Psychol
DOI: 10.1016/j.cedpsych.2020.101924
sha: 7397e9eb4a83d357aec959e663820f07ed7ffd8e
doc_id: 998060
cord_uid: ztmhfvq6

The goal of this study is to test the individual and combined effects of supplementing an online statistics lesson with four motivational strategies corresponding to Bandura’s (1997) four sources of self-efficacy (anxiety coping, modeling, mental practice, and effort feedback) on cognitive, motivational, and affective outcomes. Internet participants (N = 279) completed an online statistics module in one of six conditions with one or all four self-efficacy-building strategies (5 treatment conditions) or none of these strategies (control condition). The results indicated that the four strategies worked effectively in combination, significantly improving transfer test scores (d = 0.608), increasing self-efficacy ratings (d = 0.696), and reducing task anxiety ratings (d = -0.534), as compared with the control condition. By contrast, no motivational strategy alone was effective. The results suggest the importance of taking advantage of the power of all four sources of self-efficacy information in combination when designing motivational interventions for online mathematical lessons.

anxiety has been identified as a ubiquitous phenomenon in academic settings (Ashcraft, 2002) , and extensive empirical research has demonstrated a negative relationship between math anxiety and math self-efficacy (e.g., Griggs et al., 2013; Pajares & Miller, 1994) .

Accordingly, a number of research studies have focused on developing psychological interventions to reduce math anxiety (Feng et al., 2014; Gan et al., 2015; Park et al., 2014; Sharp et al., 2000; Zettle, 2003) . These interventions can be categorized into behavioral, cognitive, or cognitive-behavioral types, with behavioral interventions focusing on the emotionality component of math anxiety, cognitive interventions focusing on the worry component of math anxiety, and cognitive-behavioral interventions including both emotionality and worry components (Hembree, 1990) . Empirical studies in computer-based learning have documented the benefits of affective coping messages delivered by pedagogical agents in reducing math anxiety or improving performance (Huang & Mayer, 2016; Im, 2012; Shen, 2009 ). There is also evidence that cognitive anxiety coping techniques, such as growth mindset development, can improve outcomes (Huang & Mayer, 2019; Im, 2012) . For example, previous research has shown a growth mindset intervention to be effective in reducing students' anxiety in learning statistics (Smith & Capuzzi, 2019) . A growth mindset reflects a belief that one's ability is not fixed, but changeable and improvable through continued efforts (Dweck, 2008a) . A coping message conveying this belief may likewise reduce math anxiety and positively influence perceived self-efficacy.

Vicarious experience is defined as experience gained through observation, which includes modeled performance (Bandura, 1997) . In most activities, people judge their capabilities based in part on the accomplishments of others as "there are no absolute measures of EFFECTS OF FOUR SELF-EFFICACY-BUILDING STRATEGIES 8 adequacy" (Bandura, 1997, p. 86) . Two types of modeling that are frequently researched differ on the basis of learner-observer similarity, for example, peer or coping models versus expert or mastery models that demonstrate differing learning processes (Bandura, 1997) . On one hand, when people see someone similar to themselves fail or succeed on a certain task, their selfefficacy can be weakened or strengthened accordingly as a result of the social comparison; on the other hand, people look for competent models who possess the expertise they aspire to attain (Bandura, 1997) . Either modeling type could be effective in increasing learning or self-efficacy (e.g., Cumming & Ramsey, 2011; Schunk & Hanson, 1985 , 1989 . Nevertheless, according to Bandura (1997) , "The instructive contribution of modeling is especially important when perceived inefficacy reflects skill deficits rather than misappraisals of the skills already possessed" (p. 88). He also emphasized that the influence of positive models could be obtained by "maximizing modeling's instructive function and minimizing its comparative evaluative function" (p. 92).

In technology-supported learning environments, modeling can be presented by a pedagogical agent -a virtual character who guides the learner for instructional purposes (Veletsianos, 2010) . More specifically, a pedagogical agent can be used to demonstrate and verbalize covert thought processes involved in a problem-solving process. Previous empirical studies have included pedagogical agents who model a mathematical problem-solving process.

Researchers then evaluate the benefits to cognitive, motivational, or/and affective learning (Huang, 2017; Huang & Mayer, 2019 ).

Mastery experience refers to one's perception of previous successes and has been identified as "the most influential source of efficacy information" because it provides "the most EFFECTS OF FOUR SELF-EFFICACY-BUILDING STRATEGIES 9 authentic evidence" of whether one has the capabilities to succeed (Bandura, 1997, p. 80) . It is how people evaluate their performance, rather than the performance per se, that strengthens or weakens self-efficacy beliefs (Usher et al., 2019) . Bandura discussed both mastery-based and mastery-oriented strategies to support mastery performance. Mastery-based strategies focus on actual mastery experiences, whereas mastery-oriented strategies may rely on experiences generated from virtual performance, such as via virtual reality technology rather than actual performance in the real world. Transfer of perceived mastery is then possible from virtual reality to actual reality (Bandura, 1997) .

Just as people can gain a sense of mastery from virtual reality experiences, they can also gain a sense of mental mastery from imagined cognitive performances. Mental practice (i.e., mentally rehearsing the process of a successful performance) can enhance a sense of mastery (Bandura, 1997; Fiorella & Mayer, 2015) . However, research on mental practice for cognitive tasks is limited despite its potential (Leahy & Sweller, 2008; Leopold & Mayer, 2015) . Some have shown that mentally practicing the procedure in an example after studying facilitates learning (Cooper et al., 2001) . However, little empirical research has examined whether this strategy would increase self-efficacy and other desired learning outcomes (Huang & Mayer, 2019 ).

Social persuasion serves as another means of developing self-efficacy beliefs. People who are convinced by others that they are capable of mastering a certain task are more likely to develop a sense of self-efficacy (Bandura, 1997) . For social persuasion to be effective in fostering self-efficacy, messages should be perceived as an authentic evaluation of one's capabilities (Bandura, 1997) .

Attributional feedback, that is, feedback that pertains to the causes of one's performance (Schunk & Rice, 1986) , is a common form of social persuasion that affects self-efficacy (Bandura, 1997) . Attributional feedback linking achievement outcomes to effort or to ability has shown different effects on motivation and performance (Schunk, 1983) . Research conducted by Schunk and his colleagues indicated that ability feedback enhanced self-efficacy and performance during initial skill development (e.g., Schunk, 1983; Schunk & Rice, 1986 ).

However, more recent research on growth mindset has shown that effort feedback, which attributes one's performance to modifiable causes (e.g., effort) increases self-efficacy and persistence (Dweck, 2008a; Mueller & Dweck, 1998; Romero et al., 2014) . Research on attributional feedback has been primarily carried out in face-to-face settings, with only limited work conducted in online settings (e.g., Huang & Mayer, 2019; Zhao & Huang, 2020) . In an online example-based learning environment, customized effort feedback can be integrated into paired problems with the goal of strengthening learner self-efficacy.

A growing body of research has shown that the four hypothesized sources of self-efficacy are core elements in the development of academic self-efficacy, but the relative strength of each informational source varies according to contextual demands and individual differences (Usher & Weidner, 2018) . Mastery experience is generally considered the most influential source as it involves individuals' direct experience, but when information from all four sources is available, mastery experience may not be the sole basis on which individuals adjust their self-efficacy. For example, Bandura (1997) acknowledged that vicarious experience, the indirect experience of observing others, typically has a weaker effect on self-efficacy than direct experience, but under certain conditions observing others can be more impactful. Ultimately, the influence of the four sources on self-efficacy depends on how people interpret the information, contextual factors (e.g., learning domain), and individual factors (e.g., gender, ethnicity, ability level; Bandura, 1997; Butz & Usher, 2015; Joet et al., 2011; Usher & Pajares, 2008) .

Survey-based research on the sources of self-efficacy in K-12 mathematics has generally shown that mastery experience is a strong source of students' self-efficacy (Butz & Usher, 2015; Joet et al., 2011; Usher & Pajares, 2009 ). In the context of college students learning algebra in a primarily asynchronous online environment, however, vicarious experiences and physiological states have been identified as significant sources of math self-efficacy (Hodges & Murphy, 2009 ). In short, although the previous literature generally supports the four hypothesized sources of self-efficacy, the relative strength of the relationship between these sources and self-efficacy has been less consistent.

One reason for these inconsistencies might be variation in how people integrate multiple sources of self-efficacy information can vary. As Bandura (1997) explained, Some may combine efficacy-relevant factors additively -the more indicants there are, the stronger is the belief of personal capability. Others may use a relative weighting rule in which some factors are weighted more heavily than others. Still others may use a multiplicative combination rule. Here the conjoint impact of factors on efficacy beliefs is greater than simply their additive effect. (p. 114) It would be worthwhile to investigate the individual and combined effectiveness of instructional strategies connected to each of the four sources of self-efficacy when applied to an online mathematical learning environment. To date, little experimental evidence has focused on self-efficacy development. Even less has focused on assessing self-efficacy-supportive interventions for online adult learners. Examining how targeted efficacy-relevant experiences affect self-efficacy individually and in combination in asynchronous online learning environments via the four hypothesized informational sources could provide helpful tips for optimizing instructional design.

This study is intended to extend Huang and Mayer's (2019) research by analyzing the integrated effect of four self-efficacy-supportive strategies, which include (1) an agent-delivered anxiety coping message informed by the physiological states source of self-efficacy, (2) the paired practice problems informed by the social persuasion source. Our goal is to test these four self-efficacy-building strategies, both individually and as an integrated set, to better understand the mechanisms of self-efficacy development. Using Bandura's (1997) theory of selfefficacy and previous findings from the literature, we predict that adding all of the four selfefficacy-building strategies to the online statistical lesson will lead to  increased performance scores on practice, retention, and transfer tests (Prediction 1),  enhanced task-specific self-efficacy ratings (Prediction 2), and  reduced task anxiety ratings (Prediction 3).

We have elected not to make predictions concerning the individual effectiveness of each strategy for several reasons. First, there is a lack of conclusive evidence concerning the individual contribution of each strategy, particularly in this novel context of adult learners studying mathematical content in an online asynchronous environment. Second, according to Bandura's (1997) theorizing, the single or combinatory influence of efficacy-relevant informational sources depends on multiple factors such as how people interpret the information, the particular context, and individual differences. It is therefore possible that any individual strategy alone will not be sufficient to change the targeted outcomes. It may be that the combined effect of the four strategies has the strongest influence. Therefore, it is an open question as to whether one or more of these efficacy-building strategies will be more effective than no strategy (i.e., a control group) in improving learning and self-efficacy and reducing anxiety in mathematics. If certain strategies are effective, we can promote their use in learning interventions. If the combined strategies are effective, we can conclude that the effectiveness of a treatment aimed at promoting self-efficacy and learning in an online lesson rests in the mutuallyreinforcing power of combining information from all four sources of self-efficacy.

The participants consisted of 296 adults recruited from Prolific, an online crowdsourcing platform for research where Internet participants can be filtered based on their demographic screeners. Seventeen participants were excluded from the data analyses due to their failure to meet the attention check criteria included in the study as described in the Procedure section (n = 16) or in the recruitment criteria (n = 1), leaving a total of 279 participants in the final data set (151 women, 128 men; M age = 23.23 years, SD = 6.24). Recruitment criteria for the study included that the participant (a) was at least 18 years old or older, (b) spoke English as their first/primary language, (c) had a current education level between, and including, a secondary school diploma and an undergraduate degree, (d) had no prior knowledge of the learning topic, and (e) had received a 90% approval rate or higher for their previous research participation via the study platform. Most of the participants were White (74.2%), followed by Asian/Pacific Islander (7.5%), Black (5%), Latino or Hispanic (5%), mixed race (5%), and other race (3.3%).

Participants were randomly assigned (stratified by gender) to one of five treatment conditions (T1-T5) or to a control group (C). In a between-subjects design, 48 participants (26 women, 22 men) were assigned to the anxiety coping group (T1), 49 (26 women, 23 men) were assigned to the modeling example group (T2), 49 (28 women, 21 men) were assigned to the mental practice group (T3), 45 (26 women, 19 men) were assigned to the effort feedback group (T4), 47 (25 women, 22 men) were assigned to the integrated strategies group (T5), and 41 (20 women, 21 men) were assigned to the control group (C).

The instructional materials consisted of six versions of an online, self-paced instructional module designed to teach two statistical rules: the empirical rule and Chebyshev's rule. The module materials were adapted from those used by Huang and Mayer (2019) . The overall module structure was: (1) demographic survey and self-efficacy pretest; (2) review of prerequisite skills (how to define and calculate a mean score); (3) introduction to the two statistical rules; (4) practice activity consisting of five condition-dependent example-problem pairs; (5) task anxiety and self-efficacy measures; and (6) posttest. All six versions of the module were identical except for the practice activity (i.e., Part 4 of the module described above). The practice activity in the treatment versions included one or all of the four strategies aligned with Bandura's (1997) four sources of self-efficacy, whereas the control version did not include any of the four strategies, as described below.

In the practice activity of the control version of the module, participants received five step-by-step worked examples (as exemplified in Figure 1 ). Each worked example was followed by a paired problem for participants to solve (as exemplified in Figure 2 ). Participants received knowledge-of-correctness feedback on their performance for each problem (e.g., "You received 3 out of 4 points").

The T1 version of the module included a practice strategy that was aligned with the physiological states source of self-efficacy. Before receiving the practice activity, participants in this condition received an anxiety coping message delivered by an animated pedagogical agent in a video (as exemplified in Figure 3 ). The purpose of the anxiety coping message was to reduce participants' anxiety level by encouraging a growth mindset (Dweck, 2008b) . The pedagogical agent delivering the message was a female character serving the role of a motivator. The video lasted for 83 s. The complete transcript is presented in Appendix A (taken from Huang & Mayer, 2019 , pp. 1014 -1015 . The rest of the practice activity was identical to that in the control version of the module.

The T2 Figure 4 ). The pedagogical agent demonstrated and verbalized the problem-solving process for each example problem, with a voice that was intended to sound confident and authoritative. Key points of the problem-solving process were displayed synchronously as text information, which appeared next to the expert model. The paired problems and corresponding knowledge-of-correctness feedback were the same as in the control group.

The T3 version included a strategy aligned with the mastery experience source of selfefficacy. The worked examples and paired problems were identical to those in the control condition. However, participants in this treatment received a mental practice activity after each worked example. Specifically, participants were asked to mentally practice the problem-solving procedure presented in the example before they moved to the paired problem. The mental practice instruction was presented by the same female motivational pedagogical agent who delivered the anxiety coping message in T1 version of the module, but in this case, she appeared as a static image accompanied by textual information (as shown in Figure 5 ). Participants were guided to refer to the example if they encountered any difficulty while they were mentally performing the procedure demonstrated in the example. This framing was used to ensure that this mental practice activity would serve as a mastery aid by enabling learners to experience a sense of success in their skill development (i.e., avoid failure in their imagined experience; Huang & Mayer, 2019).

The T4 version included a strategy aligned with the social persuasion source of selfefficacy. Participants in this treatment received the same example-problem pairs as the control group. The only difference was that they received effort feedback at the end of each practice problem following the paired worked example, in addition to the knowledge-of-correctness feedback (as shown in Figure 6 ). Effort feedback messages were integrated in the design corresponding to each of the five problems in the practice activity, so no participant would receive the same feedback message twice. For each problem, there was a message for a correct response (e.g., Good work. Your effort paid off. Keep it up!) and an incorrect response (Your answer is not 100% correct. Don't give up. Focus on the next example-problem pair. Study the example carefully. With hard work, you can improve your performance.). In other words, participants received one of the two messages in each effort feedback pair for each problem, depending on whether they solved the problem correctly or incorrectly. Appendix B includes the complete sets of the effort feedback messages used in the T4 condition.

The T5 version of the module included all of the previously mentioned four strategies in the T1 -T4 versions, which were presented as an integrated set. That is, participants in this condition received the anxiety coping message (Strategy 1) at the beginning of the practice activity followed by five pairs of modeling examples (Strategy 2) and corresponding problems.

For each modeling example-problem pair, they were asked to mentally practice the problemsolving procedure demonstrated in the example (Strategy 3) before moving to the paired problem. In addition, after each problem, they were presented with an effort feedback message (Strategy 4) either praising their effort (correct response) or encouraging them to keep on trying (incorrect response). The top portion of Figure 7 shows the study procedures once participants started the module; the bottom portion of the figure shows the similarities and differences among the six versions of the module.

Assessment materials 1 , which were adapted from Huang and Mayer (2019) , included a demographic survey, self-efficacy questionnaire, task anxiety scale, practice problems, and retention and transfer posttests.

The demographic survey consisted of questions on participants' basic characteristics, such as age, gender, ethnicity, comfort level with computer-based instruction (1 = extremely uncomfortable; 5 = extremely comfortable), and skill level of basic math calculations (1 = extremely unskilled; 5 = extremely skilled).

A 6-item, task-specific self-efficacy measure was used to assess individuals' beliefs in their capability to perform the statistical content covered in the online module. Participants were given a 101-point rating scale (0 = no confidence at all; 100 = extremely confident; Bandura, 2006 ) on which to judge their perceived confidence for performing six tasks related to the two statistical rules (e.g., "Distinguish the conditions for applying the Empirical Rule and Chebyshev's Rule"). To maximize face validity, the measure was developed based on the specific learning objectives of the online module. Participants' task-specific self-efficacy was measured twice: before the start of the instruction (α = .97) and at the end of the instructionspecifically, after the practice activity but before the performance tests (α = .96).

Task anxiety was assessed with a single item. Participants were asked to rate the amount of anxiety they felt when studying the example-problem pairs during the practice activity (1 = very, very low anxiety; 9 = very, very high anxiety). Task anxiety was measured at the end of the practice activity.

The practice activity included 20 questions (α = .91) in five word problems (e.g., "A data set with a bell-shaped distribution and size N = 400 has a mean = 3 and a standard deviation = 1.5. Find the approximate number of observations in the data set that lie between 1.5 and 4.5").

Participants received 1 point for each correctly answered question or 0 points for each incorrect response. The maximum total score of the practice problems was therefore 20 points.

The retention posttest included six questions (α = .81) that required recalling/recognizing statements concerning the two statistical rules (e.g., "Per Chebyshev's Rule, at least what percentage of the data in a distribution falls within 2 standard deviations from the mean?"). The maximum score on the retention test was 6 points, with 1 point assigned to each correct response and 0 points to each incorrect response.

The transfer posttest test (11 word problems, α =.94) included questions of both near and far transfer. The near transfer test included six word problems (with 18 questions) similar to those presented during the practice, requiring the application of one of the two rules [e.g., "A data set of size N = 100 with a bell-shaped distribution has a mean = 5 and a standard deviation = 1.5. Please determine the following: How many standard deviation(s) are 3.5 and 6.5 away from the mean?"]. The far transfer test included five real-life problems (with 13 questions) that were different than the practice problems, although the underlying principles of the problems were similar (e.g., "A total of 200 students took an IQ test. The scores showed a bell-shaped distribution with a mean of 100 and a standard deviation of 17. Please determine the following:

Approximately how many people have an IQ score between 83 and 117?"). The maximum total score for the transfer test was 66 points (i.e., the sum of the scores for the 31 individual question items pertaining to the 11 word problems; the score for a correct response ranged from 1 to 3 points per question).

The retention test and the transfer test were presented as a posttest at the end of the online module. We did not include a pretest on statistics performance because research on the testing effect shows that the act of taking a test is itself an instructional experience that can affect subsequent posttest performance (Brown et al., 2014) . Instead, we sought to ensure that our participants did not have previous statistics instruction in the topics in the lesson by asking two screening questions, which asked potential participants whether they had learned about the Empirical rule or Chebyshev's rule prior to the study. Only participants who responded "no" to both questions were eligible for the study.

Participants voluntarily completed the study through the Prolific online crowdsourcing platform where they were recruited. Participants were informed that they would be learning basic statistics content as well as completing related surveys, practice problems, and tests if they decided to participate in the study. A link to the study was created in the platform, and participants clicked the link when they were ready. Consent information was presented first, followed by instructions on how to complete the study (e.g., find a quiet place, minimize distractions, have a calculator and scrap paper ready). After responding to the demographic question on gender, participants were randomly assigned (stratified by gender) to one of the six versions of the module. The average module completion time was about one hour. Each participant was paid $15 after completion of the study. We obtained IRB approval and followed guidelines for treatment of human subjects.

Three attention check criteria were used in the study to ensure the quality of the data: the total time on task (no less than 15 minutes) and two multiple choice questions embedded in two different places in the online module. The length and format of each attention check question looked similar to that of the other questions where it was embedded (Oppenheimer et al., 2009 ).

We added the first attention check question to the first self-efficacy measure ("It is important that you pay attention to this study. Please write the number 25 in the blank next to this statement at the end of the scale"), and the other in the practice activity ("It is important that you pay attention to this study. Please check the radio button next to '2' below").

First, a number of tests were conducted to check the equivalence of participants assigned to each group in terms of their background characteristics. Chi-square tests of independence were conducted on gender and ethnicity (ethnicity categories were collapsed to White vs. Other Ethnicities due to the small numbers for the other ethnicity groups). One-way analyses of variance (ANOVA) were conducted on age, pretest self-efficacy, comfort level with computerbased instruction, and skill level of basic math calculations. Next, as the time participants spent engaging with the online module (time on task) could be a potential confounding variable influencing the outcomes, another one-way ANOVA was conducted to compare the total time participants spent on the module among the conditions. Finally, to assess the individual and combined effectiveness of the four self-efficacy-supportive strategies, separate one-way ANOVAs with Dunnett's post hoc tests were conducted to compare each of the five treatment conditions with the control condition on the intended outcome measures: performance (practice, retention, and transfer), self-efficacy, and task anxiety.

As a preliminary step, we sought to determine whether the groups were equivalent in terms of their background characteristics. A chi-square test showed that there was no significant difference among the groups in the proportion of men and women, χ 2 = .93, p = .97, or of proportion of White and other ethnicities (which we collapsed into one group due to the small numbers in the ethnic groups), χ 2 = 3.65, p = .60. One-way ANOVAs showed no significant differences among the six groups in terms of participants' age, F(5, 273) = 1.55, p = .18, comfort level with computer-based instruction, F(5, 270) = 1.04, p = .39, skill level with basic math calculations, F(5, 272) = 1.19, p = .32, or self-efficacy level prior to the instructional modules, F(5, 273) = .56, p = .73. We therefore concluded that the groups were equivalent on basic characteristics.

Next, we sought to determine whether the groups differed in the amount of time they spent on the lesson. A one-way ANOVA indicated that there was no significant difference by condition in the amount of time participants spent on the online module, F(5, 273) = 1.68, p = .14. We thus concluded that the groups were equivalent on time on task.

The first prediction of the study was that the treatment group receiving all of the selfefficacy strategies would perform better than the control group on the practice activity, the retention test, and the transfer test. In Part A of Figure 8 with the first prediction, these findings indicate that the four self-efficacy-supportive strategiesas an integrated set rather than individually-were effective in improving practice and transfer performance. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced significantly greater practice, retention, or transfer scores relative to the control condition.

The second prediction was that participants receiving all of the self-efficacy-enhancing strategies would report higher self-efficacy after taking part in the statistical learning module than participants in the control group. In Part A of Figure ps > .05. These findings indicate that the four self-efficacy strategies as an integrated set, rather than individually, were beneficial in promoting participant self-efficacy. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced a significantly greater self-efficacy rating than did the control condition.

The third prediction of the study was that the participants assigned to the learning module with all of the self-efficacy-supportive strategies would report lower task anxiety levels than those assigned to the control group module (i.e., without those strategies). The last bar in Part A of Figure 8 displays the means and standard deviations of participants' self-reported anxiety levels by condition, with the raw numbers provided in the corresponding final row of Part B. A one-way ANOVA with Dunnett's post hoc tests revealed a significant effect of condition on task anxiety: F(5, 273) = 3.821, p = .002, η 2 = .065. Consistent with the third prediction of the study, follow-up Dunnett's tests showed that the integrated treatment group (M = 3.66, SD = 2.23) reported significantly lower anxiety than the control group (M = 4.93, SD = 2.52), p = .034, d = -0.534. However, no significant differences were found between the other treatment groups and the control group on task anxiety, ps > .05. Again, these results indicate that the four selfefficacy strategies as an integrated set, rather than individually, were successful in mitigating task anxiety. In contrast, none of the four self-efficacy strategies, when presented as individual treatments, produced a significantly lower anxiety rating compared to the control condition.

This study compared four self-efficacy-supportive strategies embedded in an online statistical learning module, both individually (T1 -T4) and in a combined manner (T5), with a control condition that did not include any of these self-efficacy-building strategies. Overall, the results showed that the four self-efficacy strategies worked effectively as an integrated set but did not individually affect learner outcomes. Compared to those in the control group who received the online learning module with only knowledge-of-correctness feedback, participants who were randomly assigned to the integrated treatment group (T5) involving all four strategies designed to enhance self-efficacy performed better on practice and transfer tests, and reported higher self-efficacy and lower anxiety than the control group. These findings largely replicate previous results by Huang and Mayer (2019) and provide additional evidence of the effectiveness of instructional design that embeds self-efficacy strategies aligned with the four sources of self-efficacy information described by Bandura (1997) . In addition, the effect sizes of the present study were noticeable, with d = 0.780 for practice, d = 0.608 for transfer, d = 0.696 for self-efficacy, and d = -0.534 for task anxiety. These values fall within the medium-to-large effect size range based on Cohen's standards (1988) , higher than those reported by Huang and Mayer (2019) .

At the same time, a new finding of the present study is that when only one of the strategies was used (T1 -T4), there was no demonstrated benefit of the treatment that resulted in statistically significant differences between the treatment and control groups. In other words, when any of the four self-efficacy strategies alone was used during the statistical learning module, the effect of the individual strategy was not strong enough to significantly increase learners' test performance or self-efficacy or to reduce their task anxiety.

This research has important theoretical implications. It is among the first studies to systematically investigate the combinatory effect of efficacy-relevant information in the particular setting of adult online mathematics learning. In this way, our findings contribute to the scarce literature in this area by documenting the effectiveness of self-efficacy interventions for online lessons. Specifically, this experiment showed that instructional elements that are systematically designed to address the four sources of self-efficacy information conceived by Bandura (1997) -vicarious experience, mastery experience, social persuasion, and physiological states-enhance both learning and motivational outcomes. No single strategy alone showed these effects, however. We take this as evidence for the mutually-reinforcing power of combining efficacy-supportive strategies in the context of online mathematics instruction.

In contrast to the approach used by Huang and Mayer (2019) , the present study was designed to dissect the set of self-efficacy strategies by pinpointing the potential of each strategy to contribute to self-efficacy development and statistical content learning. The results were somewhat surprising, as they showed that alone, a design element reflecting a single source of self-efficacy was insufficient for changing learners' outcomes. Furthermore, these findings, together with those presented in Huang and Mayer (2019) , verify that mathematics self-efficacy is a malleable and dynamic construct (Bandura 1997; Gist & Mitchell, 1992) , and its development can be significantly influenced by a series of purposefully designed instructional strategies over a relatively short period of time. It is worth noting that although one of the strategies -modeling examples -served as a replacement of the written worked examples used in the module, the other three strategies -anxiety coping message, mental practice, and effort feedback -were added elements not present in the control version. For the integrated treatment group, as learners studied the online module, they continuously received new information via one strategy after another, which helped shape their self-efficacy. For the four individual strategy conditions, however, learners were only given one type of efficacy-relevant information, thus the impact of the individual strategy on self-efficacy development and related outcomes was not as impactful.

The results of this study also support Bandura's (1997) notion of context-dependent benefits. Researchers have pointed out that "people weigh and combine information from the various sources to form self-efficacy judgments" (Schunk & Usher, 2012, p. 22) , and the relative influence of each source may differ depending on contextual and circumstantial factors such as gender, ethnicity, task difficulty, and learning domain (Schunk & Usher, 2012; Usher, 2009) . In this study, the combination of multiple sources of self-efficacy information was most potent.

That is, in the context of studying mathematical content in an asynchronous online learning environment, adult learners seem to benefit most from integrating the sources of self-efficacy information additively, such that the more efficacy-supportive strategies are available, the stronger learners' self-efficacy beliefs. The findings of the study may help explain why some previous research conducted in similar contexts but targeting only one source of self-efficacy information failed to increase both self-efficacy and learning (e.g., Huang & Mayer, 2016 ).

The primary practical implication of these findings is that interventions designed to build learners' self-efficacy may be most effective when they tap all four sources of self-efficacy information as conceived by Bandura (1997) . In short, when considering self-efficacy-supportive strategies in the context of learning mathematical content in an asynchronous online learning environment, researchers should make efforts to incorporate all four sources of self-efficacy information rather than focusing on only one of these sources, as the impact on cognitive, affective, and motivational learning is much stronger when the distinct types of efficacy-relevant information are made available to learners.

In addition, the design elements of the self-efficacy intervention in this study may inform other researchers and practitioners of the strategies that can be used to effectively enhance learning and self-efficacy while reducing anxiety in an online learning environment. Selfefficacy is a powerful motivational construct that connects to a number of desired educational outcomes, including enhanced learning performance and reduced anxiety. Numerous research studies have shown the linkage between self-efficacy, anxiety, and learning performance (Griggs et al., 2013; Huang & Mayer, 2016; Pajares & Graham, 1999) . Hence, a selfefficacy-supportive intervention has positive effects beyond self-efficacy improvement. Key design elements of the self-efficacy intervention in this study included four strategies in an example-based learning environment: (a) an anxiety coping message delivered by a pedagogical agent serving a motivational role, which is aligned with the physiological states source, (b) modelled problem-solving process presented by a pedagogical agent serving an expert role, which is aligned with the vicarious source, (c) mental practice after each modelled example, which is aligned with the mastery experience source, and (d) customized effort feedback to each practice problem, which is aligned with the social persuasion source. These strategies can be easily modified and customized to fit with different learning content and in different learning contexts to improve a variety of outcomes related to self-efficacy.

In addition, the fact that we were able to test a self-efficacy-building intervention in the context of an online learning experience provides additional, and timely, practical implications.

As mentioned above, online learning has become more commonplace over the past decade, and this trend has rapidly accelerated in response to the current COVID-19 pandemic. Many institutions around the world, whether in emergency response or by preventative measure, have moved traditional face-to-face courses to online delivery formats. Many have announced plans to continue completely online or hybrid instruction indefinitely. Moreover, it is likely that this instructional impact will extend beyond this historical crisis as an increasing number of institutions, instructors, and students see the need of offering more flexible learning opportunities. Nevertheless, anecdotal evidence suggests that students' motivation and learning have suffered as the result of shifts to online instruction. As mathematical content delivery moves to online formats, instructors might want to consider how to support students' selfefficacy for mathematical learning. The set of self-efficacy-supportive strategies tested in this study may provide a helpful beginning.

We included the same strategies as were used by Huang and Mayer (2019) in order to dissect the set of four strategies and investigate their individual effects in addition to the combined effect of the four self-efficacy strategies. However, as noted by Huang and Mayer (2019) , these four strategies "are just four of many viable forms aligned with Bandura's four sources of self-efficacy information" (p. 1031). Other valid instructional strategies that align with Bandura's (1997) theorized sources of self-efficacy can be examined in future research, for example, the use of live models or pedagogical agents whom the learner selects, or a breathing exercise designed to refocus the mind and lower heartrate.

Another limitation is that the present study involved mostly White learners in their early 20s in an online statistical learning environment. Future research should test these strategies in different settings, with different learning content and target learners, and examine if the results of the study can be replicated. Participants in this study may not share similarities to learners in more taxing environments, such as those having to navigate learning complex content online every day and in multiple content areas. In addition, the fact that the participants of the study were recruited from an online crowdsourcing platform presents a potential threat to the external validity of the study. Nevertheless, the context of the study resembled an asynchronous, selfpaced, informal learning environment similar to those increasingly seen across college campuses where learning "anywhere, anytime" is emphasized. Future work should test these effects within an authentic learning setting, such as a university-level statistics class, where participants are not incentivized for their participation with monetary awards. This would help researchers eliminate the monetary motivation to participate that might have affected the emotional response, motives, or engagement level reported by participants in this study. Thus, it will be useful to see if the same results can be replicated with participants enrolled in an authentic online class.

Furthermore, qualitative data methods such as interviews and think-alouds may be used to explore whether or how different groups of learners differing in attributes such as ethnicity, age, and gender respond differently to these strategies. Insights gained through these types of data can inform the design of interventions that target specifically to the intended target learners.

An Example of Written, Worked Example in the Practice Activity

An Example of a Paired Problem in the Practice Activity

For this practice, you will work on 5 example-problem pairs. For each pair, you will study the example first, and then solve the paired problem. You will get immediate feedback for your problem solving. Do not give up if you encounter any difficulty. As long as you work hard, your effort will pay off. As Albert Einstein once said, "It's not that I'm so smart, it's just that I stay with problems longer". Your ability in solving the problems will grow with your continuous effort. If you make a mistake, do not feel discouraged. What is important is to learn from your mistake and improve as you go through the example-problem pairs.

Through repeated practice we can improve our skills and feel confident while using them.

The 5 example-problem pairs to be presented next will focus on engaging you in the repeated practice. Your goal is to study an example first, and then solve the paired problem following the similar procedure. Whether studying the example or solving the paired problem, just focus on what has to be done, one step at a time, and eventually, you can reach the mastery level on these problems. Now, click the Next button to start working on the example-problem pairs.

Highlights  Students studied an online statistics lesson with 0, 1, or 4 self-efficacy features.  The 4-feature treatment improved transfer and self-efficacy better than others.  The 4-feature treatment reduced task anxiety better than others.  Combining 4 features is more effective than implementing them separately.  Combining 4 features is more effective than including no features.

Math anxiety: Personal, educational, and cognitive consequences

Self-efficacy: The exercise of control

The effects of feedback protocol on self-regulated learning in a web-based worked example learning environment

Make it stick: The science of sucessful learning

Salient sources of early adolescents' self-efficacy in two domains

Statistical power analysis for the behavioral sciences

Learning by imagining

Math and science pursuits: A self-efficacy intervention comparison study

The impact of web-based worked examples and selfexplanation on performance, problem solving, and self-efficacy

Investigating the effect of model type and practice format on performance expectations and actual physical performance

Mindsets: How praise is harming youth and what can be done about it

Mindsets and math/science achievement

Does classical music relieve math anxiety? Role of tempo on price computation avoidance

Learning as a generative activity: Eight learning strategies that promote understanding

The relaxation effects of stimulative and sedative music on mathematics anxiety: A perception to physiology model

Enrollment and employees in postsecondary institutions, Fall 2017; and financial statistics and academic libraries, fiscal year 2017: First look (Provisional Data) (NCES 2019-021rev)

Self-efficacy: A theoretical analysis of its determinants and malleability

The responsive classroom approach and fifth grade students' math and science anxiety and self-efficacy

The nature, effects, and relief of mathematics anxiety

Sources of self-efficacy beliefs of students in a technology-intensive asynchronous college algebra course

Example-based learning: Effects of different types of examples on student performance, cognitive load and self-efficacy in a statistical learning task

Benefits of adding anxiety-reducing features to a computerbased multimedia lesson on statistics

Adding self-efficacy features to an online statistics lesson

Impact of math self-efficacy, math anxiety, and growth mindset on math and science career interest for middle school students: the gender moderating effect

The effects of emotional support and cognitive motivational messages on math anxiety, self-efficacy, and math problem solving (Doctoral dissertation)

Sources of self-efficacy: an investigation of elementary school students in France

The imagination effect increases with an increased intrinsic cognitive load

An imagination effect in learning from scientific text

Advances in designing instruction based on examples

Praise for intelligence can undermine children's motivation and performance

Instructional manipulation checks: Detecting satisficing to increase statistical power

Self-efficacy, motivation constructs, and mathematics performance of entering middle school students

Role of self-efficacy and self-concept beliefs in mathematical problem solving: A path analysis

The role of expressive writing in math anxiety

Instruction based on examples

The worked-example principle in multimedia learning

Academic and emotional functioning in middle school: the role of implicit theories

The Oxford handbook of human motivation

Ability versus effort attributional feedback: Differential effects on selfefficacy and achievement

Peer models: Influence on children's self-efficacy and achievement

Self-modeling and children's cognitive skill learning

Extended attributional feedback: Sequence effects during remedial reading instruction

Increasing mathematical problemsolving performance through relaxation training

The effects of agent emotional support and cognitive motivational messages on math anxiety, learning, and motivation (Doctoral dissertation)

Using a Mindset Intervention to Reduce Anxiety in the Statistics Classroom

Assessing mathematics self-efficacy: How many categories co we really need

A comparison of cognitive load associated with discovery learning and worked examples

Sources of middle school students' self-efficacy in mathematics: A qualitative investigation

Sources of math and science selfefficacy in rural Appalachia: A convergent mixed methods design

Sources of self-Efficacy in school: Critical review of the literature and future directions

Sources of self-efficacy in mathematics: A validation study

Sociocultural influences on self-efficacy development

Effects of worked examples, example-problem, and problem-example pairs on novices' learning

Example-based learning: Integrating cognitive and socialcognitive research perspectives

Contextually relevant pedagogical agents: Visual appearance, stereotypes, and first impressions and their impact on learning

Online mathematics achievement: Effects of learning strategies and self-efficacy

Structuring effective worked examples

Acceptance and commitment therapy (ACT) vs. systematic desensitization in treatment of mathematics anxiety

Individual differences in response to attributional praise in an online learning environment

Your answer is not 100% correct. Don't give up. Focus on the next example-problem pair. Study the example carefully

Keep on trying, and your effort will pay off

Although your performance is not perfect, you can get there if you work hard on studying the next example and solving the paired problem

You can improve if you spent more effort on the next example-problem pair

The assessment materials also included a participant reactions questionnaire (including measures on motivation, task difficulty, and effort on the learning module) and a cognitive load scale