key: cord-0727851-vifd6xdc authors: Martinez, Mario Antonio title: Computer discourse and use as determinants of student math outcomes: performativity and action at work in the lower school grades date: 2022-02-02 journal: Heliyon DOI: 10.1016/j.heliyon.2022.e08888 sha: 76480ad5b026a7a2310a8975ea2df71aa91d43ae doc_id: 727851 cord_uid: vifd6xdc There exists a tension between discussing the use of computers in the lower grade math curriculum and computer use outcomes, as achievement results fall behind the talk of use. Purpose: The purpose of this research was to explore this proposition on a large scale limited to a United States context. To that end, two studies emerged. Methods: In the first study, a large corpus of articles encouraging computer use from Teaching Children Mathematics was used to explore talk about how to use computers in the classroom using a content analysis method. The second study used weighted sample survey methods with corrections and jackknife replications for ANOVA tests to determine whether there were statistically significant differences between exposure time to computers during 4th grade math and standardized test scores on the TIMSS math knowledge and application subtests generalized to the United States population of students. Results: The results of the first study indicated that computer use talk fell under the domains of geometry, graphing functions, and base ten blocks. The second study indicated non-significant results on both tests, which was interpreted as a valuable finding. Conclusions: Several conclusions emerged for these studies. The most poignant ones included that common core standards do not require or mention technology use in obtaining math objectives in the 4th grade. It is concluded that the TIMSS survey results suggest that some of the findings might be due to lack of availability of computers for students and professional development opportunities for faculty. It is also concluded that there is a continuity between the recent 2000s and the use of computers in the lower grades during COVID-19, where reports show learning losses, despite talk of it as a teaching method. Computers, their logic, their applications, and data are at the forefront of everyday life (Fox and Rainie, 2014a ,b, Smith, 2010 , Kohut et al., 2007 . The world uses computers for finance, health care informatics, improvements in vehicles, and mobile phones (West and Allen, 2021, Oppenheim, 2010 ). Yet computer use also belongs under the domain of educating students. From the standpoint of inculcating in young learners mathematical understandings with the assistance of computers, the curriculum has at once borne the burden of introducing students to programming and user interface software in order to accumulate learning successes across math objectives (Roblyer and Doering, 2013, Díaz et al., 2015) . The definition of computer use in classrooms involves any machinery or technology that requires a central processing unit to function, to include software and peripheral add on technology like speakers, visualization enhancements such as large computer displays. For the purposes of these combined studies the focus is on computers E-mail address: mario.a.martinez@live.com. used to teach young students in elementary school classrooms. Studying this phenomenon is complex. One of the main findings when surveying what is known about computers and elementary math classrooms is that process-based work with computers is common. Among these interventions there are those that focus on the problem solving process by reducing cognitive load on students (Chang et al., 2006) , while others have focused on thinking guidance as students discuss mathematical ideas (Kramarski and Mizrachi, 2006) , and yet others that focus on using whiteboards supported by computers to reinforce collaborative learning (Hwang et al., 2006) . Each of these instances focus on breaking down larger processes (for example, adding large numbers) into smaller ones. Process based interventions have the built-in advantage of examining student learning products that exemplify the process of learning. example, in one case, screencasts were used to examine student explanations in mathematical modeling strategies concerning multiplication, division and equal sharing problems on an app (Soto, 2015) . In another example, stages of math problem solving that caused issues for students could be focused on to hone learning skills (Chadli et al., 2018) . In yet another related example, graduated prompts were provided to students to assess their effectiveness in overall math outcomes, as with puppet series problem solving tasks, in which six puppets were presented and a participant had to create the last in the series (Veerbeek et al., 2019) . Other interventions associated learning with students making physical objects by using spatial reasoning techniques gleaned from videos of students doing tasks such as redesigning house plans with computer assisted design (CAD), or using computer programming to make computer games (Ramey et al., 2020) . These interventions are able to uncover subtle process-driven thinking and decisions that students make by slowing down the process of learning. This means recording and or guiding the steps to a mathematical conclusion or product, increasing its worth to researchers and math teacher-practitioners. Several interventions reported on using the power of the internet to create interventions that drive student learning. This might mean that learning, though starting or ending in the classroom, might be accessible anywhere there is internet access, a novelty that online learning presents. One study used enrichment activities to drive student learning using online mathematical curriculum, focusing on constructivist principles, and providing a large set of opportunities for solving math problems. The rationale was that students would learn through their personal experiences with the curriculum with the teacher poised at a distance (Frid, 2001) . The modes of interaction (viz. email correspondence and online reflection) allowed enough proximal distance between teachers and learners so students could rely on their own resources to complete activities and was seen as a driver of learning. In another online intervention, the power of audio recording was used to gain learning insights into the base ten system. Exploratory, cumulative, disputational, and tutorial discourses were typologized and measured, and it was found that during group learning in the online intervention (Orme and Monroe, 2005) , that both boys and girls contributed to rich online discussions and interactions. It can be concluded by this study that online learning was a factor in collaborative outcomes. In yet another online intervention, web-based formats were studied for their effects on student achievement (Flemming, 2011) . Findings included that nonstandard problems such as bringing elementary algebra and geometry down to the lower elementary grades helped to improve understanding of math objectives as the software was combined with an interactive online environment, and the introduction of advanced topics to an early developmental level. It is concluded that the uniqueness of online environments such as online chats, group learning, and email correspondence facilitates learning in students that participated in studies. Many computer interventions in the math content area focus on the gamification of learning. One intervention focused on elementary students' multiplication reasoning (Bakker et al., 2015) . The rationale behind the use of games in learning derives from reports of strong student motivation and engagement during such activities (Beal and Rosenblum, 2018, Denham, 2015) . It has been shown that computerized math games intensify competition, along with cooperation between students working on group activities (Es-Sajjade and Paas, 2020, Ke, 2008) . High quality games that position students in virtual environments attempt to make artificial meaning out of learning (doing math to progress to the next task in everyday activities) (Plass-Nielsen and Wolter Nielsen, 2019), and stand in contrast to those that focus on repetition (Gregersen et al., 2019) . It is harder to show proof of gains in learning problem solving approaches. However, computer software featuring metacognitive and cognitive approaches have shown efficacy in reaching students to great effect (Carr et al., 2011, Kramarski and Mizrachi, 2006) . Some studies have shown increases in understandings of concepts such as the distributive and associative properties of mathematics (Denham, 2015) . Other studies have shown relationships between computer use for problem solving and gains in argumentative skills and mathematical problem solving (Hwang et al., 2006) . Finally, other studies demonstrate positive changes in student reasoning overall (Root et al., 2019 , Vrugte et al., 2015 . For all of the studies showcasing computers advancing the math curriculum, there are other studies showing evidence of no to very little effect upon student achievement. In one example, a computer program built to assist with mathematical concepts through dynamic illustrations of concepts was found to have no effect on math achievement scores (Rutherford et al., 2014) . In some interventions, there appeared to be a gender divide in outcomes, favoring boys (Carr et al., 2011) . Observational studies have shown that popular k-12 math game website searches were related to low scores in mathematics (Zhang, 2015) . In a study examining whether computer administered tests helped math students' performance, it was shown that the control group (a paper-based test) provided better test support than the computer (Kingston, 2009) . One study suggested that students who played computer math games each day in school scored low on assessments compared to those that did not play math games (Kim and Chang, 2010) . Finally, an intervention combining school and home environments that used computers to extend learning activities showed no increases in student achievement in math and other subjects (Miller and McInerney, 1994) . General studies from the k-12 learning domain have shown overall effect sizes indicating that other studies produced very small effect sizes. Campuzano et al. (2009) 's work suggests that the overall effects of math software products produced non-significant effects for users when testing for three of four different softwares. The IES (2019)'s report on cognitive tutor found only small effects on general mathematics and geometry overall. Dynarski et al. (2007) 's findings indicate that none of the software products tested in the field produced significant effects on test scores. These products included Larson Algebra, Achieve Now, and iLearn. While a variety of conclusions have been reached about computer technology in the elementary math classroom, questions such as the discourse on using computers to teach young people have not been explored to a great degree. Likewise, the use of computers and their pragmatic effect have not been approached adequately with questioning at the national US level. It is known that in experimental settings, computers as pedagogical tools can produce mixed results, but it is not currently known how computers work on the ground, outside of experimental conditions, and whether computers are associated with increases in math performance on international tests. What is proposed is an examination of the practice-based use of computers in the fourth grade math classroom to fill this gap in the literature, along with an examination of the national discourse of using computers to teach math to young students. It is believed that this examination will be of significance to historians of education, researchers programming for young children for mathematics, and to those positioning Winograd and Flores (1987)'s theory in applied settings. The paper advances Winograd and Flores (1987) 's theory by filling in details from the 2009-2014 era of computing, performativity, and action. This is believed to be vital to understanding a computational trajectory towards artificial intelligence and human cognition. 1. What kind of talk is there on teachers use of computers in teaching math to elementary aged children? 2. Are there differences in group means in math knowledge on an international assessment relative to differences in group exposures in computer use to explore math concepts? 3. Are there differences in group means in applied math scores on an international assessment relative to differences in group exposures in computer use to practice math skills? Using Fig. 1 as a guide, the conceptual framework situates the study. Starting from top left to right, the conceptual framework places the research questions (top left box) over the data (center top box) and to the right the findings are revealed (right top box). Going from the top down to the far right center box, these findings are bound in relation to the theoretical frame adopted from Winograd and Flores (1987) and what is known about computer uses in education (lower right box), and the general topic (central box). The theoretical orientation comes from Winograd and Flores (1987) 's assumptions about language (listed in the top left circle) in Fig. 2 , that it is steeped in action and is performative in nature. For purposes here, sharing ideas about how to use computers among a particular sphere of computer users has the ability to change what we know about teaching mathematics from, say a 1960s view into a 2010s point of view. Performativity theory bridges this understanding for Winograd and Flores (1987) by their commitment to stating that everything exists within language. Language use and computer component creation is subject to interpreting the context around computer use, and creating cognition through attempts at generating, if only, small accounts of representations into a programming environment: this unto a trajectory in which artificial intelligence (AI) fully represents human understanding (Winograd and Flores, 1987) . The lower left circle represents Winograd and Flores (1987) 's views on computers and software, how we build them, and how we use them. While the ultimate thrust of their work builds to artificial intelligence (AI), the general theory can be used to understand non-AI software in its trajectory towards greater computer capacities. This view of thinking is regarded as performative every time a program is discussed, leading to writing or executing a representation of a pattern or a problem. However, what computers can do is dependent on the structures hidden within the machine itself that are able to perceive in an artificially intelligent way, find patterns through clustering methods, or run a simple program (Winograd and Flores, 1987) . Together, the two left hand circles come to what we can understand as computers operating in the world towards a trajectory of artificial intelligence (AI). This means computers' reflexive work with itself as a categorizing and predictive entity and its work with human cognition. This paper situates the trajectory of Winograd and Flores (1987) 's theory in the 2010s. The first study answers research question 1 (RQ1). For research question 1 of the study author explored the talk on using computers in elementary school mathematics teaching by way of exploratory text analytics. The University of Houston reviewed the IRB proposal submitted for this paper and deemed it exempt from review. The author accessed permission from JSTOR to work with the journal articles in Teaching Children Mathematics (TCM), which is a publication of the National Council of Teachers of Mathematics (NCTM). The study author reasoned that since it is sponsored by the NCTM, it is considered part of the national discourse on mathematics teaching in the US. N = 726 articles were obtained from the years 2009-2013, which was the closest The articles were processed in the R language where latent Dirichlet allocation was carried out to determine a topic model for articles with "computer" in them. The data used for the first study is publicly available through JSTOR through their Data for Research Scheme. The researcher used topic modeling, a form of content analysis (Krippendorff, 2019), to reduce data into groups that can be understood more readily. Topic modeling uses latent Dirichlet allocation (LDA) for fitting the topic model, treating each text as a collection of topics, and each topic as a body of words. The texts can then show repetition in content when they are considered as a corpus, which forms the basis for the analysis. LDA is an unsupervised method for classification used for this kind of grouping of documents (Silge, 2017) . In the LDA model, = 30 topics were discovered out of 149 documents with the term "computer" in them. Stopwords and punctuation were eliminated from the corpus. A correlated topic model was produced. All four estimators listed in Fig. 3 indicate = 30 for the topic modeling, to produce a set of topics without over or under estimating views into the corpus (Arun et al., 2010 , Cao et al., 2009 , Deveaud et al., 2014 , Griffiths and Steyvers, 2004 , Nikita, 2020 , Silge and Robinson, 2016 ). Fig. 3 shows the topic model estimator, while Fig. 4 shows the topic model with all 30 topic models. In this study the independent variable was the collection of TCM articles. The dependent variable was the composition of redundant words that formed a topic (Table 1) . The second study addressed research questions 2 and 3 (RQ2 and RQ3). The study author used the 2015 TIMSS data for the United States to test hypotheses on: (1) whether various exposures to computers for exploring math concepts would show mean group differences accord- ing to the math knowledge subtest on the TIMSS test, and (2) whether various exposures to computers for practicing math skills would result in mean group differences according to the math application subtest on the TIMSS test (IEA, 2019). Analysis for the second study relied on the R programming language, and used the TIMSS and PIRLS database to download data. Questions two and three used data-provided weights and jackknife arguments in their estimators (BIFIE et al., 2019) , to avoid bias (Bell et al., 2012) . The TIMSS 2015 data is publicly available to users. Questions two and three used the TOTWGT variable for weights, and the JKTIMSS2 jackknife variable set to estimate achievement subscores. Additionally, the author examined the following variables: ASM-KNO, ASMAPP, ATBM05CBM, and ATBM05CA. Table 1 shows the variables as independent and dependent variables. The study used a form of analysis known as inference from a stratified sample towards a population, a form of complex survey process (Lumley, 2010) . In this case the stratified sample reflected the population of 4th grade math students in the United states. A requirement of this kind of work is the use of sampling weights in order to calculate the results. Jackknife replications were used to estimate achievement score outcomes with standard errors (TIMSS, 2015b), which are reflected in the findings section. The first research question centered around the discourse of computer use in the classroom as a focus in the journal, TCM for the years 2009 to 2013. The modeling is viewed as capturing the practice knowledge and suggestions of experienced teachers and making them public in a sphere of pedagogical discourse. Below is a highlight of some of the topics dealing with technology in the corpus of journal articles. Fig. 5 , Topic 30 is an example of hollowing out actual use of computers in space and time for teachers; by speaking about students, math- ematics and the terms use, can, and used for student use, coupled with association of the terms resources and learning, readers of TCM see the horizon of possibility of using computers in their practice. Fig. 6 , Topic 16 suggests that spreadsheets can be used to graph functions, x and y values, and for graphing purposes. A key word in context (kwic) search was performed on the corpus, and it indicated that spreadsheets were used for their formula making abilities, for data collection with simple tallies, and as an overall analysis tool. A kwic search of Fig. 7 , Topic 8 revealed that virtual base ten blocks were used to enumerate, examine place value, and were supported as an alternative to concrete manipulatives. Additionally, in the corpus there were several observations of the term triangle and square. The discussions about the triangle focused on rotating the triangle, ideas of symmetry, and definitions of a triangle. Discussions about "squares" focused on the shape's properties, comparisons between triangles and squares, and the role of parallel sides, as noted by simple kwic searches. RQ2: Are there differences in group means in math knowledge on an international assessment relative to differences in group exposures in computer use to explore math concepts? An ANOVA was calculated from the dependent variable, ASMKNO, which measured math knowledge on the 2015 US TIMSS subscore, and the independent variable, ATBM05CA, or the grouping variable, which was one of four categories that teachers responded that their classes fell into (groups 1-4) for using computers or tablets for activities to explore math concepts (1: Every or almost every day; 2: Once or twice a week; 3: Once or twice a month; 4: Never or almost never). Table 2 shows the mean scores on the subtest measuring math knowledge. Group 1, the case where teachers indicated that computers were used every or almost every day, resulted in M = 540.94, M SE = 7.96; SD = 79.12, SD SE = 3.38. This score represented weighted cases of N = 286,761, according to the weighting variable used (TOTWGT). Comparatively speaking the total average US Math score for TIMSS in 2015 was 539 (TIMSS, 2015a) (see Table 3 ). The ANOVA results indicate a nonsignificant result (Table 4) , with an 2 of 0. Likewise, Cohen's d comparisons between groups indicated nonsignificant results (Table 4 ), indicating that there were no significant differences in math knowledge subtest score outcomes between group use of computers or tablets to explore math concepts. RQ3: Are there differences in group means in applied math scores on an international assessment relative to differences in group exposures in computer use to practice math skills? A separate ANOVA was calculated to answer RQ3. The question asked whether there were statistically significant mean differences in the dependent variable ASMAPP, measuring math application on the 2015 TIMSS subscore, by the independent variable, ATBM05CB, which was one of four categories that teachers responded that their classes fell into (groups 1-4) for using computers or tablets for activities to practice skills (1: Every or almost every day; 2: Once or twice a week; 3: Once or twice a month; 4: Never or almost never). Table 5 shows the mean scores on the subtest measuring applied math. Group 1, the case where teachers indicated that computers were used every or almost every day, resulted in M = 533.29, SE = 5.86; SD = 89.24, SE = 3.34. This score represented a weighted N of 610,010, according to the weighting variable, TOTWGT. The score can be put into loose perspective considering that the average US math score for TIMSS in 2015 was 539 (TIMSS, 2015a) (see Table 6 ). The ANOVA results show a nonsignificant result, indicating that there are no significant differences in means between groups. Further examination of Cohen's d p-values bears this out. Therefore, there were no significant differences in applied math subtest score outcomes between group use of computers or tablets to practice skills (Table 7) . The two studies in this manuscript come together synergistically as they tie together a major issue. On the one hand, there is a push to use computers with young people in the mathematics classroom. The arti-cles from TCM for the years 2009-2013 bear this out. Topics include encouraging the use of computers with associations between terms like resources and learning, and use and can for topic 30, and graph functions for topic 16 with words like functions, x, y, and graphing and finally topic 8 with its emphasis on triangle and square and virtual base ten blocks. However, on the other hand, the 4th grade representative US 2015 TIMSS scores indicate that no matter how many teachers use computers in the classroom, be it from 1-2 times in a week to almost never, there are no statistically different mean differences in achievement scores, and means hover around the average 2015 US TIMSS 4th grade mathematics score. This is a major issue represented by studies 1, 2, and 3 in this manuscript, if the goal is to use computers to increase math scores in TIMSS (and possibly other standardized) testing. The last sentence is a major assumption worth considering, as the ultimate goal might be mere exposure to computers in the math content area, in which case, somewhat flat TIMSS scores could be considered completely acceptable. Some may argue that at most exposure to computers to 1-2 times a week might not be enough to affect outcomes in math skills or knowledge. When one considers the scope and sequence of the weekly lesson, one to two days spent on computer use is generous and can make an impact on students learning concepts or practicing skills (Glendale Elementary School District, 2020). Or, one can also argue for the case of the hybrid lesson (viz. part book based, part computer based). Also at stake is how the lesson is created. The TIMSS questionnaires do not consider this. A full day's lesson on a computer or tablet that has been prepared and preempted by a previous lesson can make a difference. Still one must account for the lack of boost to scores that one might expect from the use of computers and tablets on the subscores. Some of the lack of score boosting might be due to the lack of resources. For the US 2015 TIMSS study, it was revealed that in approximately 22% of classes each student had a computer. Additionally, it was noted that about 81% of classes had computers (but not each student had one), and finally, it was noted that nearly 85% of schools had computers (but not each student had one, and they were not in classrooms). The case where only 22% of classes with each student having a computer might translate to being a situation where only a limited number of classrooms have math lessons designed for individual students. In the case of 81% of classes having computers, this does not say how many computers are in the classroom (the number may vary in each case). In the event that classrooms do not have a 1:1 student to computer ratio, students may have to share a computer to do schoolwork: this is not the most ideal situation for computer use in the content area. Finally, if classrooms fall within the category of not having the 1:1 student to computer ratio, and do not have computers in the classroom, students may have to travel to another room altogether (say, a computer lab) on a rotating basis to get computer instruction in the math content area. Each of these scenarios are possible ways of viewing computer access as a factor that plays into the decision to use computers to practice math skills or explore math concepts. One of the issues that this brings to light in the United States, is that 4th grade common core math standards do not require the use of technology to achieve their ends (Common Core, 2021). On another side of the spectrum lie the ISTE standards (ISTE, 2021), which are technology standards that could easily tie into the use of computers to elevate the curriculum into a 21st century form of usage. A few examples exist in the literature in which computer integration with the lower grades objectives is coupled with technology to great success (Bush, 2021 , Elizabeth Casey et al., 2018 . Care must be taken to provide professional development for teachers to help them integrate computers into the curriculum to make these couplings successful. However, when asked about whether there was professional development on integrating math information (which the study author believes would fall under the rubric of integrating math and computers, and is the closest TIMSS question to report on this kind of subject), roughly 41% of respondents stated they received this kind of professional development, whereas roughly 59% of respondents indicated that they had not. Formally interpreted, roughly less than half of teachers received professional development towards integrating math information; this could mean integrating many different kinds of information, such as working with concrete manipulatives and word problems, not just math objectives and computers. Finally, the lack of statistically significant mean differences in the ANOVA tests presents a continuity with the present use of computers with COVID-19 and its impact in the 2020-2021 school year (Bailey et al., 2021 , Dorn et al., 2020 , Engzell et al., 2021 , Sawchuk and Sparks, 2020 . This means that despite the use of computers as interventions in online teaching, the gains in teaching and learning in many cases did not see a boost in scores. In fact, losses occurred in math and losses were sharply divided between those families that could afford resources to enhance teaching online and those that could not afford resources to enhance online teaching (Strauss, 2020) . As a word of caution, the two comparisons are somewhat different, with not all things being equal in the comparison. On balance, other researchers have found results that show gains in math when computers are used in the curriculum. However, these findings are beyond the scope of the parameters of the paper. One study suggested that adaptive math showed potential to increase students' scores (but is dated at 2019, four years after the TIMSS study took place) (Herold, 2019b) , while another article showed that about 25% of math instruction occurs with the help of technology (again, published roughly 4 years after the TIMSS study took place) (Herold, 2019a) . In contention, and more tightly controlled and supporting the current study findings, however, are those of Carr (2012) , which found that in a pre-test post-test control group setting that 1:1 ipad work in 5th grade math did not boost scores significantly for the experimental group (again, somewhat outside of the date scope of the TIMSS study, but 3 years prior to it, and closer to the grade context). There are four major conclusions drawn from the two studies in this paper. First, from the two studies it is clear that there exists an impetus to include computers in the elementary math curriculum (as seen from the content analysis/topic modeling), yet, no matter how many computers are used in the 4th grade math curriculum for the 2015 survey year in the United States, achievement scores on math knowledge and applied math scores were about the same as the average math scores on the overall US 4th grade TIMSS scores where one would expect higher scores with the use of computers in the classroom. Second, the issue of including computers in the math content area in the elementary school years brings up several issues, particularly with common core standards, as they do not say anything on the use of computers to achieve their ends. This is over against the ISTE standards, which await to bring every student into the 21st century. The lack of discussion of computer use in the common core standards is important because it may explain why some teachers rarely use computers in the math content area, as seen in the grouping variables for the ANOVA analysis. Germane to this is the inequities in professional development opportunities. Reflecting the third issue is the lack of availability of computers during the survey year 2015 in the US, which throws light on some of the inequities in opportunities to use computers during math time for practice of skills and exploration of concepts. Finally, the results might be interpreted as providing a continuity with present student performances over the 2020-2021 school year due to COVID-19. Mario A. Martinez: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Data associated with this study has been deposited at https:// timssandpirls .bc .edu /timss2015 /international -database/ under the accession number ICPSR. The authors declare no conflict of interest. Supplementary content related to this article has been published online at [URL] . On finding the natural number of topics with latent Dirichlet allocation: some observations Achievement gaps in the wake of Covid-19 Effects of playing mathematics computer games on primary school students' multiplicative reasoning ability. Contemp Evaluation of the effectiveness of a tablet computer application (app) in helping students with visual impairments solve mathematics problems Use of design effects and sample weights in complex health survey data: a review of published articles using data from 3 commonly used adolescent health surveys Bifiesurvey: tools for survey statistics in educational assessment Software-based intervention with digital manipulatives to support student conceptual understandings of fractions Effectiveness of reading and mathematics software products: findings from two student corhorts A density-based method for adaptive lda model selection Does math achievement h'app'en when ipads and game-based learning are incorporated into fifth-grade mathematics instruction? Combined fluency and cognitive strategies instruction improves mathematics achievement in early elementary school An empirical investigation into student's mathematical word-based problem-solving process: a computerized approach Computer-assisted learning for mathematical problem solving Common Core, 2021. Common core state standards initiative Supporting conceptual understanding of the associative and distributive properties through digital gameplay Accurate and effective latent concept modeling for ad hoc information retrieval Covid-19 and learning loss-disparities grow and students need help Effectiveness of reading and mathematics software products: Findings from the first student cohort (ncee 2007-4005). Institute of Education Sciences Orchestrating the xo computer with digital and conventional resources to teach mathematics Lines, roamers, and squares: oh my! Using floor robots to enhance hispanic students' understanding of programming Learning loss due to school closures during the Covid-19 pandemic Educational theories and computer game design: lessons from an experiment in elementary mathematics education Web-based russian math curriculum shows positive results Part 1: How the internet has woven itself into american life Part 2: Americans' views about the role of the internet in their lives Supporting primary students' on-line learning in a virtual enrichment program Glendale Elementary School District, 2020. Mathematics pacing guide Development of the learning game tile war Finding scientific topics One-fourth of k-12 math instruction occurs digitally, survey suggests Tech research: 5 key lessons for educators Development and evaluation of multimedia whiteboard system for improving mathematical problem solving Timss 2015 international database downloads Wwc intervention report: a summary of findings from a systematic review of the evidence (cognitive tutor) Iste standards: students Alternative goal structures for computer game-based learning Computer games for the math achievement of diverse students Comparability of computer-and paper-administered multiple-choice tests for k-12 populations: a synthesis World publics welcome global trade -but not immigration Online discussion and self-regulated learning: effects of instructional methods on mathematical literacy Content Analysis: an Introduction to Its Methodology Complex Surveys: a Guide to Analysis Using R Effects on achievement of a home/school computer project Ldatuning: tuning of the latent Dirichlet allocation models parameters Technology trends The nature of discourse as students collaborate on a mathematics webquest How to enhance interest in mathematics by using game-based learning In-fuse-ing steam learning with spatial reasoning: distributed spatial sensemaking in school-based making activities Integrating Educational Technology into Teaching Using modified schema-based instruction with technology-based supports to teach data analysis A randomized trial of an elementary school mathematics software intervention: spatial-temporal math What research says about Covid-19 and math learning loss Text Mining with R: A Tidy Approach Tidytext: text mining and analysis using tidy data principles in R Americans and their gadgets Elementary students' mathematical explanations and attention to audience with screencasts Why Covid-19 will 'explode' existing academic achievement gaps TIMSS, 2015a. Mathematics for grades Process assessment in dynamic testing using electronic tangibles When a game supports prevocational math education but integrated reflection does not How artificial intelligence is transforming the world Understanding Computers and Cognition: A New Foundation for Design Understanding the relationships between interest in online math games and academic performance