key: cord-0781006-ghjq4i2j
authors: Hopcan, Sinan; Polat, Elif; Albayrak, Ebru
title: Whether to flip Extreme Apprenticeship: which is more effective in programming instruction?
date: 2022-04-20
journal: Educ Inf Technol (Dordr)
DOI: 10.1007/s10639-022-11055-y
sha: 3eca7b1e20e1fdc3b29904e3cdf88f96138236f5
doc_id: 781006
cord_uid: ghjq4i2j

Programming knowledge is more important than ever in the digital world. However, teaching programming can be challenging, especially with novice learners. Considerable research has been conducted into the most effective methods for teaching programming. Extreme apprenticeship, a variation of cognitive apprenticeship, is a method that has been used in teaching programming at university level in recent years. Because this method focuses particularly on completing lots of exercises with coaching and guidance, it may solve many problems related to learning programming. Flipped learning can be useful for student preparedness and providing sufficient theoretical knowledge at the beginning of the course. This study compares the applications of the extreme apprenticeship method, flipped extreme apprenticeship, and traditional classroom, analyzing them at the university level in terms of their effects on academic achievement and engagement coupled with gender differences. The findings of the study indicate that the extreme apprenticeship and flipped extreme apprenticeship instructional methods improve academic achievement and student engagement in introductory programming more than the traditional method. The results of the research point to important directions for the development of the extreme apprenticeship method in programming instruction and provide a guide for instructors. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10639-022-11055-y.

In the knowledge era in which we currently live, rapid scientific and technological advances have made programming abilities essential for a skilled workforce and have emphasized the need for education in such skills (Campe et al., 2020; Witherspoon et al., 2016) . People may need to use their programming skills to be more successful in their future work, putting more importance on the training they receive. Introductory programming courses have therefore become increasingly important over the last few years and have become a focus of researchers' attention (Giannakos et al., 2014a; Lahtinen et al., 2005; Nikula et al., 2011) .

Programming is a multi-step process that involves analysis, strategy development, algorithm development, and code writing (Falloon, 2016; Hwang et al., 2012) . To master programming, learners must learn both syntax and how to set up many logical structures, such as variables, loops, arrays, and recursions (Korhonen & Malmi, 2000) . According to research, programming is an area in which students fail most (Hanks et al., 2011; Robins et al., 2003) and face disappointment (Bravo et al., 2005) . Novice learners, in particular, may suffer from the process of learning programming skills (Koulouri et al., 2014) .

Investigations of novice learners' learning methods found that students had significant difficulties in their courses because they tended to write codes line by line rather than establishing a logical structure (Robins et al., 2003) . Many methodologies have been used in the literature to make programming teaching more effective in order to overcome this situation. Some of these methodologies are executing robotic applications (Witherspoon et al., 2016) , pair programming (Lee, 2011) , flipped learning (Zha et al., 2020) , demonstration (Bean et al., 2015) , collaborative learning (Rodríguez et al., 2017; Williams et al., 2003) , and learning by creating games or stories (Kafai & Burke, 2013) . However, the traditional method is still used by universities in introductory courses for programming. Rather than developing novice learners' problemsolving skills, many introductory programming courses teach them the syntax and meaning of programming (Iqbal Malik, 2016; Vihavainen et al., 2011a) . This format includes lectures, take-home assignments, and sometimes demonstration sessions where exercises are shown as models (Vihavainen et al., 2011a) .

Introduction to programming courses need to be redesigned and instructional methods based on a constructivist understanding should be adopted so that novice learners in particular can learn and internalize the logic of programming in greater depth based on their previous experience. For this, teaching with guidance, putting the student at the center of learning, will be beneficial. However, the nature of exercises and guidance is critical in this regard. According to Bruhn and Burton (2003) , traditional lectures do not provide students with practical knowledge and students fail to understand how to apply programming concepts in their assignments. Given these disadvantages, unguided or minimally guided problem-solving exercises can be intimidating and confusing for inexperienced students (Bruhn & Burton, 2003) . In fact, educational psychologists argue that minimal guidance is less effective, particularly for novice learners, in challenging subjects such as programming due to humans' cognitive architecture (Kirschner et al., 2006) . With minimal guidance, students may be unable to complete an exercise, increasing the dropout rate and feelings of inadequacy (Vihavainen et al., 2011a) . Furthermore, exercises that are not too difficult can give learners a higher quality learning experience by increasing their self-confidence and belief in their ability to solve problems (Wiedenbeck, 2005) . Educators can improve their students' performance by teaching them strategies to put together pieces of code rather than having them memorize programming theory, as this helps them learn the syntactic and semantic structures of language (Spohrer & Soloway, 1986) .

In general, a programming teaching approach should primarily focus on providing adequate support for novice learners (Kirschner et al., 2006) . Researchers who design environments for teaching programming to novice learners have been drawn to the extreme apprenticeship (XA) method, which has been used in educational environments for the past ten years. Flipped learning is another method that can provide support to novice learners in terms of preparing them in advance for lessons. So far, however, the XA environment has not been applied in combination with flipped learning in the literature. This study adds to the field's innovation by testing the effects of these two environments on programming instruction.

The theoretical background for the current discussion is structured through a review of the literature. Research has proven that student participation is weak in traditional methods, often due to the dullness of the content, especially when applied in disciplines with complex concepts and plenty of theoretical topics (Tang et al., 2020) . In this environment, learners also cannot develop their high-level skills in programming due to the lack of practical knowledge and sufficient feedback (Bruhn & Burton, 2003; Carbonaro, 2019) . Therefore, there is a need for environments that will increase learner engagement beyond the traditional to gain learning experience by doing (Yıldız Durak, 2018) . In addition, because out-of-school experiences are limited to reading resources shared by the instructor, learners are often not willing to advance their learning processes. Because of all these shortcomings, the method is not as effective as the flipped (Taşpolat et al., 2021) and XA methods (Rämö et al., 2015) and can mean students lose their way in complex and challenging content such as programming (Hopcan et al., 2022) . Studies that compare the traditional method with flipped (Amresh et al., 2013; Chen et al., 2014; Souza & Rodrigues, 2015) and XA methods (Plonka et al., 2015; Rämö et al., 2015; Vihavainen et al., 2011b) also found that the traditional approach did not contribute as much as others in essential factors such as engagement and achievement.

This article examined the effects of XA, flipped XA, and traditional methods relative to each other. In XA, plenty of supervision is given to learners at first and support is gradually withdrawn as the process progresses (Vihavainen et al., 2011a) . Also, in this approach, less time is given to theoretical lectures than in the traditional method, giving more time for application. Flipped XA is an environment where these processes are reversed and contains the applications of XA in the classroom. Here, there is pre-lesson preparation in which learners benefit from videos prepared by the instructor. The traditional method, on the other hand, is a more teacher-centered approach where lectures are predominateand few exercises are provided. The following section explains XA and f lipped XA and presents how these two methods go beyond the traditional method. Research on the intertwined relationship between methods and learner achievement, engagement, and gender differences are described.

Extreme apprenticeship, based on the cognitive apprenticeship approach, is a new methodology (Del Fatto et al., 2016) first used in introduction to programming courses at the University of Helsinki in 2011 (Vihavainen et al., 2011a) . Cognitive apprenticeship is essentially an approach in which skills are acquired under the supervision of a subject matter expert who has prior practice and experience, with an emphasis on the learning the process rather than outcome . Accordingly, cognitive apprenticeship enables effective cognitive development to take place under the supervision of an expert in learning environments that necessitate the use of metacognitive abilities (Plonka et al., 2015) . With the potential to take programminginstruction to the next level, XA entails organizing programming instruction in such a way that it can more effectively address instruction, respond to needs, and deal with problems (Rämö et al., 2015) . This method ensures that novice learners attain programming skills by completing a series of small exercises under the supervision of an expert (Del Fatto et al., 2016) . These exercises are initially very simple. Support is gradually reduced as the learner progresses, allowing them to improve their cognitive processes on their own and eventually become an expert in the subject (Lee, 2020) . Learners get plenty of feedback on their progress during exercises (Vihavainen et al., 2012) ; although the instructor does not give learners the answer but rather manages the scaffolding, process by providing hints (Vihavainen et al., 2011b) . For example, instead of giving the answers directly to the learners, providing a concrete example of the solution enables them to discover the specific answer themselves. With these small hints, learners can advance their problem-solving skills. According to Sinha and Kapur (2021) , it is important for learners to produce their own solutions, a process that helps them achieve higher order thinking. In this way, XA allows students to go beyond memorization and gain a deeper understanding of concepts (Rämö et al., 2015) .

According to one research conducted on the XA method in higher education, learners' achievement rates did not decrease, they felt confident about learning, and the process was satisfactory even though their workload increased (Hautala et al., 2012) . In another study, XA enabled learners to have a high level of self-efficacy and gain positive experiences (Lahdenperä et al., 2019) . When XA was tried in programming courses, learners were found to perform satisfactorily . Keijonen, Kurhila, and Vihavainen (2013) tried the XA method in their study and found that the learners were successful. Vihavainen et al. (2011b) found that in the programming course they taught using XA through a MOOC, learners felt that they had a quality learning experience.

Flipped learning is an instructional method that promotes XA by encouraging students to prepare before class and actively participate in class with the knowledge they have gained (Rämö et al., 2015) . In this method, learners work on exercises in a classroom setting and study passive course content in their own environment to achieve high-level learning (Sarawagi, 2013) . Among these preparations are watching subject-related videos, studying shared resources, and doing subject-related homework prior to the lesson. Any questions that come to mind while students work individually are brought to the lesson and discussed (Amresh et al., 2013) . As a result, it is possible to ensure that students perform mental exercises at a higher level.

It is critical that materials in the flipped environment are interesting and that the student receives feedback after watching videos to ensure quality of learning (Yıldız Durak, 2018) . The role of the educator shifts from the role of the mere instructor to the role of a mentor. (King, 1993) . It is important, in the constructivist understanding, that the instructor ensures that learners find answers on their own rather than providing a direct solution. In XA, too, after initially providing a large amount of support, learners become expert in what they do as the instructor gradually withdraws support while at the same time learners' progress from simple to complex via a master-apprentice relationship. Programming instruction requires long periods conducting a large number of exercises. In order to address the excessive amount of time required, this study involved the design and implementation of a flipped XA environment in which flipped learning was blended with the XA method. According to research conducted in higher education, flipped learning has a positive impact on programming instruction. Amresh et al. (2013) investigated the effects of the flipped learning environment and the traditional environment on the programming instruction of university students and discovered that the flipped environment resulted in higher achievement. Souza and Rodrigues (2015) found not only higher achievement but also higher self-efficacy among learners, while Chen et al. (2014) found that learners were more satisfied, motivated, and successful.

According to research in the field, there is a significant dropout rate in programming instruction (McKinney & Denton, 2004) . In this regard, learners' engagement levels as well as their achievement indicators should be closely monitored. Students' learning processes can be influenced by whether or not they attend school, complete their homework, and carry out the necessary preparation at home. Pattanaphanchai (2019) and Chen et al. (2014) found a significant relationship between achievement and engagement in the flipped environment. Maher, Latulipe, Lipford, and Rorrer (2015) used flipped learning and paired programming, as well as scaffolding. They noted that the students enjoyed and engaged well in this environment. In general, research has shown that when flipped learning is used, achievement and engagement increase. When the XA environment is combined with flipped learning, it is expected that students will arrive prepared for class, allowing for a reduction in the amount of theoretical teaching and the completion of a large number of targeted exercises, making this a beneficial approach. This is because novice learners require plenty of resources and individualized instruction (Lahtinen et al., 2005) . In addition, learners are motivated to prepare for class, thus learning in their own time and at their own speed. However, the XA environment has not previously been studied in combination with flipped learning. This study is therefore the first to compare the effects of this process with both the XA environment and the traditional environment.

Gender differences in programming instruction have been documented and should not be overlooked. Kay (2006) came to the conclusion that male perceptions and interests in programming skills differ from females' due to their prior experiences. Furthermore, some studies have found that male students are more successful and engaged more in programming than female students, while others found no difference (Alvarado et al., 2017; McDowell et al., 2006; Tyler & Yessenbayeva, 2018) . According to the literature, female students tend to have a lower rate of completion of programming courses (Black, 2007) . These differences in programming instruction may be due to female students not interacting with computers as much as male students in their early years (Kay, 2006; Loftsson et al., 2019) . Because of this, any new method for programming instruction should be assessed in terms of gender differences. The ultimate goal of this research was to create the most successful learning environment possible for students to gain valuable programming experience. Accordingly, the XA model was tried both alone and with a flipped classroom approach in an introduction to programming course, taking into account the gender factor in terms of achievement and engagement, and the effects of both learning environments compared to the traditional method.

For this purpose, answers to the following research question were sought: Is there any significant difference in the engagement and achievement of students in terms of programming instruction method and gender?

This study applied extreme apprenticeship (XA), a variant of cognitive apprenticeship, in the flipped classroom and traditional classroom to teach the basics of programming at the university level. The study followed the matching-only posttest-only control group design as one of the quasi-experimental designs. The study followed a quasi-experimental design. Experimental design is defined as research designs that explore cause-effect relationships between variables (Fraenkel et al., 2012) . The purpose of quasi-experimental design is the same as experimental design. The difference between them is that in quasi-experimental design, control and experimental groups are chosen not by chance but by measurement (Fraenkel et al., 2012) . No random assignment was made in the selection of experimental and control groups and the prior performance test of the groups were checked for being equal in terms of academic achievement, which was the dependent variable of the research. The instructional strategies, XA, flipped XA, and traditional, were considered independent variables, and achievement and engagement were considered dependent variables.

The procedure was carried out over a period of eight weeks. In both experimental and control groups, lessons were conducted by the two researchers, who have eight years of experience in their field. In order to prevent factors arising due to instructor differences affecting the research and threatening internal validity, the two instructors followed previously determined procedures and met on a weekly basis. A prior performance test was applied to ensure that the participants had similar prerequisite knowledge. Figure 1 demonstrates model of the study implementation.

The XA classroom involved short lectures, exercises, course book, and homework. The flipped XA classroom included pre-lesson videos, a brief review in a lecture, exercises, course book, and homework, while the traditional classroom included lectures, one exercise per lesson, and homework (see Figs. 2, 3 and 4). As previously mentioned, XA contained a lot of exercises. Lecturing was kept shorter than would be Since exercises are a vital part of this process, a number of exercises were prepared to cover each topic. In addition, the exercises were prepared in relation to the students' departments in order to better hold their attention. Examples of exercises are given in the Appendix A (see Appendices). The instructors provided a plenty of guidance to the students during the first exercises. As the student progress, the guidance was gradually reduced to have students construct a deeper understanding. Scaffolding was conducted by instructors and mentors. Mentors were all active and eager to help. Student teaching assistants, or mentors, scaffolded students in carrying out the exercises, monitoring as they walked around the laboratory. They provided continuous feedback in XA and flipped XA, guiding the students to the answers rather than directly providing solutions as instructors did. In addition, both instructors and mentors naturally guided the practices in the laboratory in the traditional classroom. There was traditional guidance as it is a hands-on course. Therefore, there were four or five mentors in the laboratory to each of the three groups. Communications of mentors and instructors were carried out with the instant messaging program What- sApp. During the lesson, those with questions and answers relating to the exercises wrote to the group and received immediate feedback.

Explanation Avoiding long lectures Long lectures were avoided throughout the course. Lectures were given only to enable students to complete the exercises. Exercises related to the lectures Exercises related to the lessons taught were included. Starting early (starting exercises from the first week)

After the first lesson, the students began the exercises.

Constant help in labs While the students were doing the exercises in the laboratory, the instructor and volunteer mentors, who were experienced with and knowledgeable about the programs, guided the students.

The exercises were designed from easy to difficult to give students a sense of achievement. They were also divided into parts leading to the final destination.

Since the exercises were the main instrument of the course, the students were required to complete the exercises. Plenty of practice with lots of repetition As many exercise as possible were completed. Some of them were repetitive.

Clear guidelines were given for the exercises. 

Write the city whose license plate is entered on the screen. List.

Writing the list elements to the screen. Loop (two application).

Calculation game based on random operations and numbers (math and function). The extreme apprenticeship approach was used in the study for developing and delivering the course content (Vihavainen et al., 2011b) . The approach used is presented in detail in Table 1 .

The course content, lecture videos, in-class exercises, and out-of-class homework were developed by instructors with eight years of experience in programming instruction. While preparing the course content, the literature and practice area were examined (see Table 2 ).

The participants were 124 freshmen students enrolled in the Information Technologies (IT) course, an undergraduate program of the faculty of education at a state university in Istanbul. Istanbul is Turkey's most densely populated and metropolitan city. It was therefore considered appropriate to choose a state university based in Istanbul to ensure as much generalizability as possible. (See the conclusion for limitations to this.) Generally, research has been conducted on a single case (one faculty of education), whereas this current study provides a deeper and richer understanding in examining a situation (Turnbull et al., 2021) . The students were from four different departments: Social Studies Teaching, Elementary Education Teaching, Preschool Teaching, and Turkish Language Teaching, which were randomly assigned to the three groups of the study. The students were of similar socio-economic level, had similar university entry scores, and were from similar ethnic backgrounds. Table 3 demonstrates information of study participants. Of the participants, 76.61% (n = 94) were female and 24.19% (n = 30) were male. The number of participants in the experimental groups was equivalent, with 43 (34.68%) in the XA group, 44 (35.48%) in the f lipped XA group, and 37 (30.65%) in the traditional group.

In this study, demographic form, the engagement data, a prior performance test, and a multiple-choice exam were employed to recruit data. Ethics approval was obtained before commencing the study. The study used a prior performance test and a multiple-choice exam as an achievement test to measure participants' level of programming knowledge. Since the number of correct answers in the prior performance test was very low, internal consistency analysis could not be performed. Sample items/ questions from the instruments can be seen in Appendix B (see Appendices). Prior performance test Since students from different departments participated in the study, the Basic Programming Prior Performance test was applied to reveal the students' basic programming and algorithm pre-knowledge. It consists of 17 questions. The opinions of two educational technology experts were obtained for the test.

The achievement test was developed by the researcher in order to measure the level of students' learning in basic programming. It consists of 20 multi-choice questions. Expert opinions on the questions were obtained from five information technology teachers and five educational technology experts. Revisions were made in line with the experts' opinions. Three questions with low item discrimination index (0.11, 0.14, and 0.17) were removed from the test. The item discrimination indexes of the remaining questions varied between 0.23 and 0.77, and the item difficulty indexes varied between 0.21 and 0.91. As a result of the KR-20 test, the reliability score was calculated as 0.80. As a result, 17 questions were asked.

The engagement data This score was found by recording students' attendance in face-to-face lessons, video watching rates, and homework completion rates. In the flipped classroom group, the engagement was the average of homework rate, video watching rate and class attendance rate; in the other two groups, it was the average of homework rate and class attendance rate.

The study continued for eight weeks for three hours a week, teaching basic programming using the programming training tool MIT App Inventor (http://appinventor.mit. edu). MIT App inventor, one of the block-based visual programming tools, is an easy tool that can be learned in a short time. For this reason, it has been chosen as a learning tool for novice learners in this study (Polat & Hopcan, 2019) . The experimental groups were XA (N = 43) and flipped XA (N = 44); the control group consisted of students who received traditional education (N = 37). The researchers took some precautions in order to avoid issues affecting internal validity and generalizability (external validity). For increasing internal validity, a prior performance test was applied to ensure that the participants had similar prerequisite knowledge. It was therefore assumed that both groups were equal. To ensure that equality was maintained throughout the study, the researchers checked that the participating students did not attend a similar course for the duration of the intervention. Each group was presented with an equal number of resources and time. In order to prevent instructor differences affecting the research and threatening internal validity, the two instructors followed previously determined procedures and met to discuss them on a weekly basis. In addition, in order to ensure generalizability (external validity), the researchers tried to obtain a representative sample.

In the first lesson, the students were informed about the purpose of the lesson and the study, the content of the lesson, and the platforms of the lesson. The participants were informed about ethical principles, the purpose of the study, and that participation was voluntary. Google Classroom was used in all groups as a learning management platform and MIT App Inventor was used as a programming learning tool. In the flipped groups, the Edpuzzle platform was used for videos before the lesson. In Edpuzzle, multiple choice, fill-in-the-blank, matching type questions etc. were added to each video to check whether the students had watched it or not and to reinforce the topic. During the first lesson, students were asked to register on these platforms. The duration of the videos was about 15-16 min. In order to control the duration of intervention in the study, lesson time was kept the same for all three groups. Since there are pre-videos in the flipped classroom, the normal lesson time was shortened by 15 min and the lesson time was equalized with the other two groups.

SPSS 21 was used to analyze the data. Descriptive statistics was employed. Two way MANOVA were used to investigate effects of three independent variables on two dependent variables. The alpha level for data analysis was set at 0.05.

The prior performance test scores presented in Table 4 show a normal distribution in the total. In the groups, the students were generally not successful in the prior performance test, and the average score of each group was low. This proves that the students did not have any prior knowledge about basic programming.

A Two-Way MANOVA test was performed to determine the joint effect of two independent variables (gender and group) on two dependent variables (achievement score and engagement score). Table 5 presents the sample size, mean, and standard deviation values of each analysis group. As can be seen in Table 5 , the highest engagement was in the flipped XA group (M = 91.89, SD = 10.24), the engagement scores of female students were higher than male students (Total M = 86.87, SD = 13.06), the highest achievement score was in the XA group (Total M = 77.84, SD = 15.97) and the female students' score (Total M = 66.08, SD = 20.64) in the groups other than in the flipped XA was higher than the male students' score (Total M = 58.63, SD = 21.15). 

Since the skewness and kurtosis values of the dependent variables (achievement score: -0.36, -0.83; engagement score: -1.14, 0.81) were between − 2 and + 2, it can be said that they showed a normal distribution. The Mahalanobis distance value was measured as 11.37 (Pallant, 2005 ) (which should be less than 13.82 when there are two variables), and a multivariate normality assumption was provided. For the assumption of a linear relationship between each pair of the dependent variables across each level of the independent variables, scatter plots were examined and generally, a linear relationship was provided. The correlation between the two dependent variables was examined and found to be 0.32. Since it is larger than 0.2 (Pallant, 2005) , relation assumption is provided, and since it is less than 0.9 (Pallant, 2005) , multicollinearity assumption is provided. Looking at the Box's Test of Equality of Covariance Matrices results, it can be assumed that the covariance matrices of the dependent variables are equal across groups. Scores on the combination of dependent variables do not differ by either gender or interaction of gender and group. As for the differences in gender sample sizes, providing the assumption of the multivariate homogeneity of variance-covariance matrices according to our Box's M test result provides evidence that sample size is acceptable for two-way MANOVA analysis (Van Huynh et al., 2018) . Also, according to Hair et al. (2009) two independent variables in our analysis (engagement and achievement). Therefore, the number of samples per cell should be at least 3. As shown in Table 5 , the lowest sample number is in the XA-male cell (n = 8). Since it was larger than 2, the number of samples was considered sufficient. In addition, because the number of samples per cell differs, Scheffe (Barnette & McLean, 2005) was used as a post hoc test.

Because all assumptions were met, Wilks' Lambda values were checked. Looking at the multivariate test results in Table 6 , the combination of achievement and engagement was not equal for all groups (λ = 0.59, F (4, 234) = 17.48, p < 0.01). A large effect size was observed (η2 = 0.23). According to gender (λ = 0.97, F (2, 117) = 1.65, p > 0.05) and the joint effect of group and gender (λ = 0.95, F (4, 234) = 1.33, p > 0.05), there was no evidence of a significant difference between the combination of achievement and engagement scores. Table 7 shows tests of between-subjects effects. The ANOVA results were significant and both achievement (F (2,118) = 30.64, p < 0.01) and engagement (F (2,118) = 8.45, p < 0.01) scores differed significantly between the groups. The achievement effect size was large (η2 = 0.34), and the engagement effect size was almost large (η2 = 0.13).

There was no evidence of a significant difference in either dependent variable according to gender.

Since the number of individuals in each group differed, the Scheffe test was performed as post-hoc (Barnette & McLean, 2005) . According to Table 8 As seen in Fig. 5 , the highest engagement is seen in the flipped XA group. While the engagement of female students in the XA group was higher than that of female students in the traditional group, the opposite was true for male students.

As seen in Fig. 6 , regardless of gender, XA was the most effective method on achievement followed by flipped XA, and traditional was the least effective. It had already been predicted that the other two methods would be more effective compared to traditional. However, while it was thought that flipped XA could be more effective than XA, the opposite was in fact true and this is worth some discussion.

This study compares the applications of the extreme apprenticeship method, flipped extreme apprenticeship, and traditional classroom, analyzing them at the university level in terms of their effects on academic achievement and engagement coupled with gender differences. The study found significant differences in terms of achievement and engagement in all three learning environments. The findings are discussed in light of the literature in the following sections.

In this study, the XA method was the most effective on achievement. Learners were able to visibly improve their programming skills by completing numerous short exercises and then receiving feedback. Programmers who have a good experience in programming tend to build stronger program structures and in a shorter amount of time (Adelson, 1981) . In this respect, the method adopted to teach programming is essential. The learners involved in this study were not only more successful but also had a better learning experience. They received immediate feedback showing them where they had gone wrong, and then had the opportunity to come up with new solutions while being coached by the instructor. Thus, learners gained valuable experience through the XA process, a method that combines theory and practice with multiple programming tasks ranging from simple to complex (Leinonen et al., 2019) . The fact that learners performed well in these environments has also been found in previous research studies, corroborating this study's findings (Keijonen et al., 2013; Pasini et al., 2016) . Vihavainen et al. (2011b) compared the programming courses they previously taught using the traditional method to the courses, they taught using XA in subsequent periods. They found that the courses using XA had a higher pass rate. Other studies have previously been done on this subject (Keijonen et al., 2013; Pasini et al., 2016; Vihavainen et al., 2011b) , but by going one step further and comparing the exam success scores of the control group and the experimental group using statistical analyses, this study obtained a higher effect size providing clearer evidence that XA is a more effective method.

When the XA and flipped class methods were used, achievement was greater than in the traditional environment. This finding is in line with research in the field comparing flipped and traditional learning environments for programming instruction (Amresh et al. 2013; Chen et al., 2014; Chis et al., 2018; Pattanaphanchai, 2019; Puarungroj, 2015; Souza & Rodrigues 2015) . What is noteworthy here, however, is that achievement in the XA group was higher compared to the flipped XA environment. However, the average in the flipped XA environment was not much smaller than the average in the XA environment. Therefore, some adjustments can be made in the flipped XA environment to achieve higher achievement. Moreover, merely examining achievement may not be adequate in terms of shaping learning environments. Observations made during the learning process are just as important as final achievement in determining the impact of a methodology used in teaching programming skills .

Because of the complexities of programming, learners can lose motivation when no good instructional method is available, and they may even drop out of a course entirely. Research shows that a significant number of students drop out of programming classes because they are struggling (McKinney & Denton, 2004) . This is alarming in today's world, where computers are used in almost every aspect of life and programming skills are crucial. This study examined the rates of doing homework, attendance, and video views in order to reveal the students' engagement status in detail. The XA method adopted in this study significantly increased the engagement of learners in the course as well as their achievement. The flipped XA environment had the highest engagement rate of the three study groups. Similarly, in their study, Chis et al. (2018) found that students were more satisfied with the flipped learning environment and engaged more in lessons compared to traditional programming lessons. However, in the current study, while students were more involved, their gains were not as great as in the XA environment. Factors affecting engagement should be considered in this case.

In this study, video watching rates, homework, and attendance determined engagement. Of these three elements, only video is unique to the flipped environment. Videos are important for learners to get a sense of course topics before class and to achieve higher levels of learning by bringing problems they have to be solved to the classroom (Yıldız Durak, 2018) . However, in spite of the high level of engagement in this study, the low achievement rate suggests that there may be issues with the students' individual learning processes. Previous studies have emphasized that learners may not adapt to the flipped environment, face problems focusing, and often do not efficiently manage their learning processes well (Vivek & Ramkumar, 2021) . In this regard, the students may have watched videos tailored to the flipped environment out of necessity, but they may not have gained meaningful learning experiences as a result. Previous research has shown that videos can increase cognitive load (Homer et al., 2008) . Above a certain level, increased cognitive load negatively affects learning performance (Sweller, 1994) . The duration of the video, the incorporation of complex subjects such as algorithms and syntax, the creation of programming structures, and the inability to ask questions about problems being experienced at the time may have increased cognitive load and reduced attention and learning.

Studies done using flipped learning show that, in general, the learning environment is more effective in terms of engagement than the traditional environment. According to Puarungroj (2015) , in flipped environments, learners can become more motivated and engaged in lessons more because of videos. Giannakos, Krogstie, and Chrisochoides (2014b) examined research results in the literature and concluded that flipped learning had a positive effect on student performance and engagement when designed well. In an introduction to programming course, Kuzminska et al. (2017) compared the flipped environment to the traditional method. According to their results, in the flipped environment, students' homework and course engagement were more positive than in the traditional environment. Since engagement is important for achievement when learning programming, the flipped XA environment is considered to offer an important infrastructure.

In future studies, learner engagement in the flipped XA environment should be closely monitored, and the difficulties encountered in the process should be identified and the reasons investigated. In this regard, the instructor's performance in the flipped environment is just as important as the learner's. According to Dimauro et al. (2019) , the flipped learning environment necessitates a significant effort on the part of instructors in their transition from knowledge-transmitters to coaches. Instructors must track students both inside and outside of the classroom to check whether they are watching videos and preparing for lessons. In addition, learners should be given the necessary guidance regarding how to work in flipped environments and the path they will take. This, however, can be exhausting for the instructor. It takes time to prepare videos before class, add questions to videos, monitor students, and plan the necessary guidance. Moreover, some students may be completely unprepared for lessons (Triantafyllou & Timcenko, 2014) , necessitating the re-teaching of basic course content. It is deemed critical that educational institutions provide necessary support to instructors in order to ensure that instructor performance does not deteriorate and engagement is increased through improved course processes.

Another consideration is that learners who are accustomed to the traditional learning environment may find the flipped learning environment challenging (Amresh et al., 2013; Loftsson et al., 2019) . This could be another reason why students did not perform as well as expected in the flipped XA environment. There is a significant decrease in the desire to learn programming, particularly when the instructional design is unsuitable for individuals (Lahtinen et al., 2005) . According to Gündüz and Akkoyunlu (2019) , it is concerning that some university students were unable to get feedback while watching videos at home prepared for programming learning in the flipped learning environment; they became bored watching videos alone, and some were unable to adequately watch videos at home due to internet problems. Loftsson and Matthíasdóttir (2019) stated that the rate of students watching the given videos same with the content to make them come to the lesson preparedly was insufficient.. According to Giannakoset al. (2014b) , students may be unreceptive to the structure of the flipped environment, and attendance may suffer as a result. Tyler and Abdrakhmanova (2016) found that the flipped environment was beneficial to student engagement and performance only on average. In this regard, the flipped XA environment would be more effective when learners are ready based on their individual characteristics and instructors have adequate resources and support.

It is also important for those who learn in a flipped environment to have control of their own learning and to possess self-directed learning skills because they will need to improve their programming skills on their own at home (Yıldız Durak, 2020). Tracking and improving self-directed learning and the learner control processes (Knowles, 1975) , which include learners determining their needs and learning goals, employing appropriate learning strategies, and evaluating their learning processes, would be beneficial in improving learning performance in a flipped environment (Çakıroğlu & Öztürk, 2017; Yıldız Durak, 2018) . While watching videos, students may become distracted by social media sites such as Facebook or use chatting apps (Puarungroj, 2015) . Students' self-directed learning skills, as well as social media addiction, can be managed in this regard. Moreover, they should be encouraged to take note of questions that arise while watching videos and bring them to the classroom.

There are some advantages to digital media in the home learning process that should be taken advantage. For example, to improve students' programming skills in an individualized learning environment, the instructional environment can be enriched with interactive animations, visuals, and voice communication. (Korhonen & Malmi, 2000) . Social networks and online discussion environments can also be incorporated into flipped XA environments (Kuzminska et al., 2017) . At the same time, engagement and participation may be increased by adding different elements attractive to learners such as gamification (Ekici, 2021) .

There was no difference in achievement and engagement levels of the students in terms of gender. Girls are more willing than boys to solve problems in complex areas such as programming (Lee, 2020) , and girls tend to take a more systematic approach to developing solutions in their programming processes, first constructing a logical structure before writing lines of code (Funke et al., 2015) . However, since females find programming more complicated and challenging than males, this may stifle their engagement, motivation, and performance (Akinola, 2015; Balanskat & Engelhardt, 2014; Giannakos et al., 2014a) . However, society in general is not supportive enough for girls to progress in fields such as engineering and science (Yu et al., 2007) . Boys, on the other hand, are more interested in and confident in programming (Maguire et al., 2014; Rubio et al., 2015; Stoilescu & McDougall, 2011) due to their prior social and cultural experiences with programming as well as their increased exposure to technology (Kay, 2006; Loftsson et al., 2019) . These two situations may have balanced the achievement and engagement in terms of both genders. Similarly, neither Tyler and Yessenbayeva (2018) , who used videos and practice quizzes in a flipped environment, nor Pasini et al. (2017) , who used the XA method in an introduction to programming course, found any differences in performance between genders. In addition, the researchers considered the different sample sizes of male and female students. The results of the study related to achievement and engagement in programming courses show similar effects due to gender difference as that reported in the literature (Choi, 2015; Pasini et al., 2017; Tyler & Yessenbayeva, 2018) . Also, as stated in the method, the analysis indicated that the difference between the sample sizes would not affect the result (Hair et al., 2009; Van Huynh et al., 2018) . However, in future studies, examining cases where sample numbers are close to each other will further contribute to the literature.

Our goal was to develop the most effective learning environment and experience that we can for novices to learn programming. The results of this study will be useful in designing programming instruction environments for novice learners. Owing to the XA environment in this study, learners were much more successful in programming and engaged at a higher level than in the traditional environment, with the help of gradually decreasing guidance ranging from very much to little, and a handful of small exercises that were not tedious. On the other hand, flipped XA was more effective than the traditional environment in terms of achievement, but it was mildly behind the XA environment. For students to be experts, they must assume the necessary responsibility in their own learning processes through participating in the course (Rämö et al., 2020) . Students who are unfamiliar with the flipped environment, however, may become distracted or lose motivation. It is also essential to plan educational environments that can be adapted to suddenly changing situations, for example pandemics such as coronavirus . In such cases, as they may have less contact with the school environment, learners may need to manage their own learning processes more by navigating resources and online environments. Studies show that the flipped learning environment positively affected learners' participation and motivation during the coronavirus pandemic by keeping them more active (Ng et al., 2022) . With flipped learning, negativities such as the dullness of the tra-ditional environment and the loneliness caused by the online environment, such as dissatisfaction and reluctance to participate, can be eliminated (Priyaadharshini & Vinayaga Sundaram, 2018; Tang et al., 2020) . In this study, the flipped XA and XA environment significantly increased engagement. What is more, the XA method may enable students to achieve a higher level of achievement as they transition from being a novice to a master in the learning area with the increase in the number of activities, increased feedback and more online and face-to-face classes, as occurred during the COVID-19 pandemic. For this reason, it is also essential to study such environments. In this regard, the focus should be on how instructors can provide the best support to learners. Instructors can create virtual environments in which students can interact in flipped environments. Specific studies on videos can be conducted in the future, such as comparing them to standard videos in terms of length and interaction, to ensure that videos in the flipped environment do not become monotonous. Studies can be carried out to investigate the effectiveness of these videos in improving self-directed learning skills. A particular limitation of this study is the fact that it only looked at gender in terms of individual differences. Further studies may be conducted with more students looking at various individual differences such as personality traits and learning preferences. Such research results may also provide insight into advanced programming instruction environments. While the study includes a representative population of a metropolitan city university, it is limited to one education faculty. Future studies may be conducted with a larger sample size, including more than one faculty of education.

Problem solving and the development of abstract categories in programming languages

Computer programming skill and gender difference: An empirical study

Gender differences in students' behaviors in CS classes throughout the CS major

Evaluating the effectiveness of flipped classrooms for teaching CS1

Computing our future: Computer programming and codingpriorities, school curricula and initiatives across Europe

Type I error of four pairwise mean comparison procedures conducted as protected and unprotected tests

Starting from scratch: Developing a pre-service teacher training program in computational thinking

Cognitive Apprenticeship and the computer programming student: How do students learn within this framework

Integrating Educational Tools for Collaborative Computer Programming Learning

An approach to teaching Java using computers

Good practices to influence engagement and learning outcomes on a traditional introductory programming course

Pair programming in middle school: variations in interactions and behaviors

Is FLIP enough? Or should we use the FLIPPED model instead?

Investigating flipped classroom and problem-based learning in a programming module for computing conversion course

A comparative analysis of different gender pair combinations in pair programming

Flipped classroom with problem based activities: Exploring selfregulated learning in a programming language course

How measuring student performances allows for measuring blended extreme apprenticeship for learning Bash programming

Experimentation of Flipped Learning in Higher Education Academy

A systematic review of the use of gamification in flipped learning. Education and Information Technologies

How to design and evaluate research in education

An analysis of young students' thinking when completing basic coding tasks using Scratch Jnr. On the iPad

Gender differences in programming: Research results and teachers' perception

Happy girls engaging with technology: Assessing emotions and engagement related to programming activities

Reviewing the flipped classroom research: reflections for computer science education

Student views on the use of flipped learning in higher education: A pilot study. Education and Information Technologies

Multivariate Data Analysis

Pair programming in education: A literature review

Extreme apprenticeship method in teaching university-level mathematics

The effects of video on cognitive load and social presence in multimedia-learning

Collaborative Behavior Patterns of Students in Programming Instruction

A pilot study of cooperative programming learning behavior and its relationship with students' learning performance

Role of ADRI model in teaching and assessing novice programmers (Doctoral dissertation

The social turn in K-12 programming: moving from computational thinking to computational participation

Addressing gender differences in computer ability, attitudes and use: The laptop effect

Carry-on effect in extreme apprenticeship

From sage on the stage to guide on the side

Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching

Self-directed learning: A guide for learners and teachers

Teaching introductory programming: A quantitative evaluation of different approaches

Flipped learning model: Tools and experience of its implementation in higher education

Supporting quality of learning in university mathematics: A comparison of two instructional designs

A study of the difficulties of novice programmers

Effects of Teaching Method on the Self-efficacy in a University Programming Class

Empowering teachers to create educational software: A constructivist approach utilizing Etoys, pair programming and cognitive apprenticeship

Admitting Students through an Open Online Course in Programming: A Multi-year Analysis of Study Success

2019 International Conference on Engineering, Technology and Education (TALE)

Enhancing collaborative learning using pair programming: Who benefits? AISHE-J: The All Ireland Journal of Teaching and Learning in Higher Education

Flipped classroom strategies for CS education

Pair programming improves student retention, confidence, and program quality

Houston, we have a problem: there's a leak in the CS1 affective oxygen tank

Engaging students in creative music making with musical instrument application in an online flipped classroom. Education and information Technologies

A motivation guided holistic rehabilitation of the first programming course

SPSS survival manual: A step by step guide to data analysis using SPSS for Windows

The Challenge of Learning to Program: motivation and achievement emotions in an eXtreme Apprenticeship experience

The role of the cognitive style in improving the learning to program

An Investigation of Students' Learning Achievement and Perception using Flipped Classroom in an Introductory Programming Course: A Case Study of Thailand Higher Education

Knowledge transfer in pair programming: An in-depth analysis

Teachers' Acceptance of MIT App Inventor as an Educational Mobile Application Development Tool

Evaluation of higher-order thinking skills using learning style in an undergraduate engineering in flipped classroom

Inverting a computer programming class with the flipped classroom

Extreme Apprenticeship-Emphasising conceptual understanding in undergraduate mathematics

The extreme apprenticeship method. PRIMUS, 1-15

Learning and teaching programming: A review and discussion

Exploring the pair programming process: Characteristics of effective collaboration

Closing the gender gap in an introductory programming course

Flipping an introductory programming course: Yes you can!

Robust effects of the efficacy of explicit failure-driven scaffolding in problem-solving prior to instruction: A replication and extension. Learning and Instruction

A "light" application of blended extreme apprenticeship in teaching programming to students of mathematics. Methodologies and Intelligent Systems for Technology Enhanced Learning

Investigating the effectiveness of the flipped classroom in an introductory programming course

Novice mistakes: Are the folk wisdoms correct?

Gender digital divide and challenges in undergraduate computer science programs

Cognitive load theory, learning difficulty, and instructional design

Efficiency of flipped classroom with online-based teaching under COVID-19

Programming language training with the flipped classroom model

Introducing a flipped classroom for a statistics course: A case study

The Use of Case Study Design in Learning Management System Research: A Label of Convenience?

Flipping the CS1 and CS2 classrooms in Central Asia

A Comparison of Flipped Programming Classroom Models-Results by Gender and Major

Vietnamese teachers' perceptions of socialemotional learning education in primary schools

Multi-faceted support for MOOC in programming

Extreme apprenticeship method in teaching programming for beginners

Extreme apprenticeship method: key practices and upward scalability

Scaffolding students' learning using test my code

Evaluation of course outcome attainment of engineering course with traditional, blended and flipped classroom approaches. Education and Information Technologies

Factors affecting the success of non-majors in learning to program

Building pair programming knowledge through a family of experiments

Gender, interest, and prior experience shape opportunities to learn programming in robotics competitions

Flipped learning readiness in teaching programming in middle schools: Modelling its relation to various variables

Modeling different variables in learning basic concepts of programming in flipped classrooms

Innovations in e-learning, instruction technology, assessment, and engineering education

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations