key: cord-0689332-4meyc2lb
authors: Kingsbury, Ian
title: Online learning: How do brick and mortar schools stack up to virtual schools?
date: 2021-01-30
journal: Educ Inf Technol (Dordr)
DOI: 10.1007/s10639-021-10450-1
sha: a47b7d39ab04b66b1d0def569a1d0aa2591ed61d
doc_id: 689332
cord_uid: 4meyc2lb

The Covid-19 pandemic forced many American schools to hastily transition to online learning. I assess how the online learning experience of students enrolled in brick and mortar schools that transitioned to online learning in Spring 2020 compared to the experience of students who were already enrolled in virtual schools when the pandemic began. Absent formal assessments to quantify learning loss, such comparison can help contextualize the performance of brick and mortar schools in their transition to online learning, and perhaps inform how policy can promote higher-quality online schooling, a burgeoning policy concern amidst widespread school closures forecasted for the 2020–2021 academic year. I hypothesize that, owing to experience and expertise, virtual schools provided a higher quality education than did brick and mortar schools operating online. I test this hypothesize by administering surveys to parents of students enrolled in online schools. When applicable, parents also complete a survey about the online learning experience of siblings enrolled in brick and mortar schools that switched to online learning in Spring 2020. I compare survey outcomes across four constructs: active learning, communication, pedagogical efficacy, and classroom management. Overall, I observe that virtual schools earned substantially higher marks across the four constructs.

Thousands of American schools shuttered their doors during the second half of the 2019-2020 school year due to the Covid-19 pandemic. Most schools that closed their physical campus switched to online learning to conclude the school year. Brick and mortar schools uninitiated to online learning were tasked with adapting teaching practices to a different modality with almost no training or experience to call upon. While anecdotal accounts indicate that some schools achieved remarkable success in transitioning to distance modalities of learning, many others schools struggled to keep students and teachers engaged in the learning process.

There is great urgency in assessing the overall efficacy with which brick and mortar schools executed online learning. Many American schools are beginning the 2020-2021 academic year with virtual or blended models of schooling (i.e. partially in-person and partially online), while countless others scheduled to open will inevitably transition to virtual or blended learning as public health concerns mount.

Difficulties in transitioning to online learning elevate concerns about national competitiveness. A comparatively greater number of schools in other developed regions of the globe (e.g. Europe and East Asia) began the 2020-2021 school year in person, and appear poised to sustain in-person activities due to a lower incidence of Covid-19 among the general population and/or greater willingness to incur the risks associated with in-person schooling. If American brick and mortar schools struggle in their delivery of online education, the human capital stock of Generation Z and Generation Alpha could languish compared to other developed nations (Psacharopoulos et al. 2020) .

Difficulties in transitioning to online learning also elevate concerns about equity. Privileged families unsatisfied with the virtual learning experience delivered by their public schools paid for alternatives, including private schools or microschools/learning pods, an arrangement whereby families pool resources to allow students to learn together at home with the assistance of teachers to facilitate individualized learning. Others homeschooled their children, an option with limited direct cost but potentially high opportunity cost (i.e. loss of parental income).

For some families who pursued alternatives to their residentially assigned public school, the chief impetus was a concern that virtual education cannot replicate inperson schooling in terms of academic effectiveness or that it could not adequately meet the social-emotional needs of their children. For others, perhaps, the concern was not the modality of schooling itself but the quality of instruction offered by the school, as evidenced by strong demand for established virtual schools (Tanner 2020; Associated Press 2020) .

Though some scholars question the wisdom and limitations of online education, expanded utilization appears to be a foregone conclusion. In the short run, millions of families will utilize virtual education due to safety concerns surrounding Covid-19 or resource constraints which compel them to enroll in a school operating virtually, even if it is not their preference. In the long run, it is likely that a non-negligible number of families compelled to participate in virtual schooling will discover that it better meets the needs of their child (Schroeder 2020) , and they will enroll in virtual schools regardless of the broader public health situation. Indeed, a survey conducted in May 2020 found that 73% of parents would be willing to have their children take at least some high school courses online, a 17-percentage point increase from 2009 (Henderson et al. 2020 ).

The recency of Covid-19 and dearth of formal assessments administered since schools closed renders impossible a causal evaluation of learning loss incurred by students enrolled in brick and mortar schools that switched to online learning during the 2019-2020 school year. Nevertheless, given the volume of students who have experienced at least some online learning during the 2020-2021 school year, there is an urgent demand to understand as best as possible what went well and what did not go well after brick and mortar schools switched to online learning.

I offer preliminary evaluation of how schools handled the transition to online learning by administering surveys that facilitate comparison between the online learning experience of students enrolled in brick and mortar schools that switched to online learning during the 2019-2020 school year to students enrolled in virtual schools. While virtual schools have faced scrutiny for their performance, their years of institutional knowledge, expertise, and virtual learning infrastructure should in theory make them more adept than brick and mortar schools at delivering virtual instruction. So while this study does not weigh in-philosophically or empirically-about the overall effectiveness of online schooling compared to in-person schooling, it offers a first glimpse at how well brick and mortar schools handled virtual learning compared to established virtual schools.

I piloted surveys with four sets of parents with students in brick and mortar schools that switched to online learning, and one set of parents with a child enrolled in an established virtual school. The pilot was used to assess whether questions were clear, and whether they were all interpreted with the same meaning. The piloted surveys did not reveal the need for any revision in which questions were asked or how they were phrased.

One survey asks parents to reflect on the experience of their child enrolled in a virtual school powered by K12 Inc., a provider of online schooling and curricula. A second survey asks parents whether they have children who were enrolled in brick and mortar schools during the 2019-2020 academic school year and, when applicable, to reflect on their experience with virtual learning through that brick and mortar school. Parents were directed to disregard the brick and mortar school survey if the school never switched to virtual learning.

Anonymous, voluntary surveys were emailed to parents between July 30 -July 31, 2020. The survey was sent to all parents of students that were enrolled in a school affiliated with K12 Inc. as of July 29, 2020, with the exception of students enrolled in schools located in New Mexico and Louisiana. No surveys were sent to parents in those states due to state laws about survey administration. Parents were asked to evaluate K12 Inc.-affiliated schools if they had a child or children enrolled before May 1st. 1 Parents of students that enrolled May 1st or later were tasked to evaluate the virtual learning experience of the school from which their child transferred (i.e. the previous school they attended in the 2019-20 school year, not a K12 Inc. affiliated school). 99,826 surveys were deployed, of which 10,144 were answered (10.1% response rate). 61% of submitted surveys were completed on behalf of students enrolled in virtual schools. The remainder were completed on behalf of students enrolled in brick and mortar schools that switched to online instruction. Two thirds of respondents identified their child as white whereas 13% identified them as African American, 13% as Hispanic or Latino, and 2% as Asian. The demographic composition of the sample mirrors the national demographic composition of virtual school students (Molnar 2019) . Meanwhile, 24% of respondents reported that their children were between preschool and third grade (roughly 4 years old to 8 years old), 47% reported that their children were between fourth and eighth grade (8 to 14 years old), and 29% reported that their children were between ninth and twelfth grade (14 to 18 years old). According to internal enrollment data, the responses are representative of enrollment in schools affiliated with K12 Inc.

Ordinal questions prompt respondents to assess the degree to which they agree with a statement (1, Strongly disagree; 2, Disagree; 3, Somewhat disagree; 4, Neither agree nor disagree; 5, Somewhat agree; 6, Agree; 7, Strongly agree) or the frequency with which an event occurred (1, Never; 2, Once per month or less; 3, A couple times per month; 4, About once per week; 5, More than once per week). Respondents were also given the option of responding that they were uncertain of their response to a question. Uncertain responses are coded as missing data. Questions are assigned to one of four constructs. Those constructs, the questions that comprise them, and Cronbach's alpha coefficients are displayed in Table One . Overall, the constructs display a high level of internal consistency (Tables 1, 2, 3, 4, 5 and 6).

I hypothesize that virtual schools would outperform brick and mortar schools in their delivery of online education. The specific purpose of this study is to glean the magnitude of that advantage and assess relative strengths and weaknesses by surveying parents to assess efficacy within four constructs that are critical to successful virtual education, as revealed by an extensive literature review. Specifically, I assess differences in active learning (LaPointe and Gunawardena 2004; Nummenmaa and Nummenmaa 2010), communication (Dennen et al. 2007; Eom et al. 2006; Greenway and Vanourek 2006; Oliver et al. 2009; Weiner 2003) , pedagogical efficacy (Dipietro 2010; Sellnow-Richmond et al. 2020) , and classroom management (Stewart 2008; Dipietro et al. 2008) .

Overall, the survey results indicate that virtual schools significantly outperformed brick and mortar schools on each index and all items which comprise the index. Differences in outcomes on all items are statistically significant at the 99% confidence level.

Active learning entails encountering new information or ideas, engaging with them, and reflecting on what was learned (Fink 2003) . A meta-analysis of studies conducted on active learning indicates that it is associated with stronger learning outcomes compared to transmissionist approaches in which teachers impart knowledge which students are tasked with absorbing (Freeman et al. 2014) . Active learning is especially important for successful distance learning. Nummenmaa and Nummenmaa (2010) observe a link between evaluations of online courses and the degree of interactivity demanded by the course. That is, students have stronger appraisals of online courses which, per Moore's (1989) typology, leverage learner-content interaction, learner-instructor interaction, and learner-learner interaction.

Interaction, the process of reciprocal communication or exchange, improves learning through two non-mutually exclusive channels. First, transactional distance theory holds that the sense of obligation that learner and teacher feel toward one another is closely linked to the frequency and duration of their interactions. To that end, interaction can make the learner more invested in their own learning, thereby increasing effort. Second, the social-constructivist view of learning posits that interaction with peers, especially those of high intelligence and diverse backgrounds, is foundational to cognitive development.

Preliminary evidence suggests that active learning-which is achieved largely through the three types of interactions in Moore's typology-was deficient in many schools that switched to virtual instruction during the 2019-2020 school year. For example, media accounts indicate that some teachers simply assigned work packets for students to complete but were otherwise largely unengaged in the learning process (Kamenetz 2020) . The parents polled for this survey corroborate the notion that brick and mortar schools mostly neglected the active learning process. Indeed, certain results are both striking not only in their comparison to virtual schools but in absolute terms. For example, 69.4% of parents reflecting on the experience of their children in brick and mortar schools report that their children were never tasked with participating in group projects or team-based tasks. A similar percentage of parents of virtual school Overall, 39.5% of respondents with children enrolled in brick and mortar schools that switched to online learning report that their children never participated in any of the activities which comprise the active learning construct, whereas 2.1% of virtual school parents report that their children never participated in such activities. Meanwhile, 46.1% of respondents reported that their children in brick and mortar schools did not do participate in any of the four activities more than a couple times per month, whereas 4.5% of respondents make the same claim about children enrolled in virtual schools.

Note that I do not tabulate a composite index score for this construct, as the responses offered to the questions in the active learning construct are not quantitatively equal in their magnitude of difference from one another. I elected for precisely defined frequency responses rather than Likert-scale responses because Likert-scale responses are vague and difficult to interpret when they address event frequency. Quality of communication from instructors to students and parents is critical to successful online learning. Teachers must be clear in their expectations for student work and conduct during synchronous sessions and "must establish the purpose of assessment, the criteria being measured, and the intended outcomes before meaningful assessment methods can be achieved." (Gaytan and McEwen 2007, p. 118) . Quantity of communication matters, too. Teachers must respond to student inquiries in a timely fashion to overcome transactional distance and practical limitations imposed by the absence of face-to-face interaction. For that reason, some virtual schools require that teachers respond to all e-mails and phone calls within 24 h (Watson et al. 2009 ).

The virtual learning construct asks whether instructors answered phone, text, or email inquiries in a timely fashion, and whether instructors were accessible outside of classroom hours. It also gauges communication across key stages in the teaching process, including whether instructors set clear academic standards, whether course materials were easy to access, and whether they provided timely feedback on student work (Figs. 5, 6, 7, 8 and 9) .

All told, parents evaluating the clarity and frequency of communications from instructors to students and families give considerably higher marks to virtual school staff. The average communications score for virtual schools is 6.3, and the median is 6.4, indicating that parents typically fall in the range of either agreeing or strongly agreeing that communication was clear and frequent. Parents of brick and mortar students who received virtual instruction receive an average score of 4.1 and a median score of 4.2, scores that fall closer to neutrality than modest agreement. Unsurprisingly, there was also greater variation in how the parents of brick and mortar students evaluated communications. Whereas the virtual school communications index score has a standard deviation of .87 points, the brick and mortar index score has a standard deviation of 1.55 points. Consequently, virtual schools in the 25th percentile receive an overall score of 6, indicating agreement with clear and frequent communication. Meanwhile, brick and mortar schools in the 25th percentile earn a score of 3, indicating that parents "somewhat disagree" that communication was effective. 

In devising a conceptual model for virtual classroom management, Rufai et al. (2015) begin by stating, "The primary purpose of teaching is to impact knowledge. However, two factors can facilitate this objective. One is having a simplified and well explained course material and the other is ensuring its effective delivery. A well-managed classroom will guarantee effective delivery." (p. 27). While this perhaps overstates the importance of classroom management-indeed, a teacher can employ pedagogically unsound practices in an otherwise well-managed classroom-it highlights that 9 "Accessing course materials was easy and straightforward" content during Zoom sessions (Stevens 2020) . The construct also asks about pace of learning. Depending on design, distance learning can empower students to learn at their own pace, but to what degree this occurred in brick and mortar schools that closed down is unclear. Finally, the construct gauges whether attendance and participation were tracked, whether teachers had a clear learning plan for each week, and whether instructors demonstrated competency with virtual classroom software (Figs. 10, 11, 12, 13 and 14) . Parents give strong marks to virtual school teachers for their managing of a virtual classroom. On average, parents agree to strongly agree that classrooms were skillfully managed, and three quarters either agree or strongly agree with that premise. Parents are less bullish on how classroom management was handled in brick and mortar schools. On average, parents express neutrality to modest disagreement with the notion that online classrooms were well-managed. Unsurprisingly, while the superior assessments of classroom management in virtual schools is significant on all items within the construct, it is primarily driven by perceived competency with managing virtual classroom software, a phenomenon likely explained by pronounced asymmetries in software experience and expertise between virtual school teachers and brick and mortar teachers.

Ultimately, virtual schools are judged according to the same criterion upon which other schools are judged: Chiefly, to what extent did the school produce knowledge and skill acquisition? A 2004 meta-analysis analyzing the effects of web-delivered K-12 programs vis-à-vis academic outcomes concludes that, on average, virtual schools are indistinguishable from brick and mortar schools in the degree to which they produce gains in student achievement (Cavanaugh et al. 2004 Education-sponsored meta-analysis authored by Means et al. (2009) concludes that online learning is modestly more effective than in-person learning, but the study cautions that there are a limited number of studies from K-12 education, and only 5 qualify as experimental or quasi-experimental. Moreover, "In many of the studies showing an advantage for online learning, the online and classroom conditions differed in terms of time spent, curriculum and pedagogy…the studies in this meta-analysis do not demonstrate that online learning is superior as a medium." (Means et al. 2009, XVII) . Both meta-analyses illustrate significant variation in effect sizes among different programs, so meta-analytic averages can obfuscate the fact that virtual school programs sometimes perform significantly better and sometimes significantly worse than inperson programs. Miron and Urschel (2012) indicators indicate weak performance. However, Chingos (2013) criticizes their conclusions because they rely upon achievement data rather than growth data. Such data is more instructive about the characteristics and demographics of students enrolled in a school than it is about the quality of the school itself. Unfortunately, then, comparison between K12 Inc. affiliated virtual schools and brick and mortar schools cannot directly inform the degree to which student learning in the latter was impacted by the switch to virtual learning. Again, I emphasize that the purpose of this endeavor is to compare student experiences between the virtual schooling sector and the brick and mortar sector operating virtually.

To date, no assessment data has been collected subsequent to campus closures. Right now, then, the most instructive data comes from assessments and comparisons across sectors (brick and mortar versus virtual) of pedagogical efficacy. That is, to what extent did teachers succeed in promoting student cognitive development and knowledge acquisition? The learning efficacy construct assesses this question first and foremost by asking parents whether they feel their children learned a lot. It also inquires whether teachers covered new material-state guidance varied widely on this matter (Schwartz 2020)-whether teachers motivated students to care about their learning (Theobald 2006) , and whether instructional materials were well-suited for virtual learning (Figs. 15, 16, 17, 18 and 19) .

Assessments of the pedagogical efficacy of virtual schools almost mirrors the results for communication. That is, three quarters of parents either agree or strongly agree that teachers employed instructional practices that were conducive to student learning. Assessments from brick and mortar parents, however, appear unremarkable. On average, parents report somewhere between neutrality and modest disagreement with the premise that instructional practices were conducive to learning.

One item within the construct deserves particular attention. A question asks whether parents feel that their children learned a lot. The responses were more divergent according to school sector than any agree/disagree question in the survey. Whereas imagine, for example, that ambivalence toward responding to questions in a timely fashion tamped down on student enthusiasm for learning.

Researchers and policymakers worry that school closures could exacerbate longstanding racial and socioeconomic achievement gaps. I probe this concern by 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% comparing assessments of brick and mortar online schooling performance from parents according to their racial self-identification, and whether they completed a survey for a child enrolled in a private school. Private school enrollment is a rudimentary proxy for wealth, as enrollment costs can be prohibitive for many families. Overall, I observe mixed signals regarding equity concerns. Parents of African American and Hispanic students, whose achievement historically lags behind the achievement of White and Asian students in the United States (Startz 2020), give appreciably higher marks for the performance of brick and mortar schools than do white or Asian parents. 2 Indeed, African American parents give higher marks for communication, pedagogical efficacy, and classroom management. They also report more frequent active learning activities. Most tellingly, while 21.6% of white or Asian parents at least somewhat agreed that their children learned a lot while their brick and mortar school operated online, 33.2% of parents of Hispanic children and 39% of parents of Black children at least somewhat agreed that their children learned a lot. Difference of means tests indicate that all mean differences in construct score by racial group are statistically significant at the 99% confidence level.

Differences in scores by school sector offer mixed evidence as to whether school closures will exacerbate achievement gaps. Among students in the sample who attended a brick and mortar school, 2410 attended a traditional public school, 234 a private school, and 173 a charter school. Survey responses indicate that private school students received an appreciably better online education than did public school students, whether traditional public or charter school. Because private schools are unaffordable to many families, their relatively superior performance during school closures elevates concerns about the exacerbation of achievement gaps. On the other hand, parents gave higher marks to charter schools than to traditional public schools. Charter schools-publicly funded, privately run tuition-free schools that operate independently of the traditional public school system-serve a disproportionately high number of students of color and students who qualify for free or reduced-price lunch, so the superior performance of charter schools over traditional public schools arguably dampens concerns regarding a widened achievement gap (Rebarber and Zgainer 2014) .

All differences across sectors are statistically significant at the 99% confidence level except for communication between traditional public and charter school and classroom management between charter school and private school. Those differences are statistically significant at the 95% confidence level.

Policy-relevant questions remain about these findings. For one, to what extent do parental assessments of learning predict achievement outcomes? As mentioned, questionnaires currently represent the best hope for gauging student learning during the 2 African American and Hispanic parents also consistently provide stronger evaluations of virtual schools powered by K12 Inc., although the racial gap is appreciably smaller than differences in evaluations of the performance of brick and mortar schools operating virtually. Difference by racial group in evaluation of the classroom management construct is significant at the 95% confidence level, and the others at 99% confidence level.

pandemic, but it isn't clear to what extent such measures proxy for actual student learning.

Going further, bias is a concern whenever questionnaires are used as party of a study. Bias could manifest in two forms. First, nonresponse bias-systematic differences in who elects to participate in a survey versus who elects not to participate-is a plausible threat. One can imagine, for example, that individuals with more extreme opinions of their virtual school (positive or negative) feel more strongly about their opinion being counted than do those with moderate positions. Second, response bias could arise from the fact that, unlike in residentially assigned public schools, parents purposefully select and opt into virtual schools, so responses could be influenced by self-affirmation (i.e. parents believing that they made a good decision). However, concern about self-affirmation bias is somewhat alleviated through comparisons of outcomes across school sectors. Virtual schools significantly outperform brick and mortar private schools and public schools of choice (i.e. charter schools) on the survey. In other words, it does not appear that a bias toward self-affirmation can explain away the superior performance of virtual schools.

Looking ahead, it's unclear to what extent the performance of brick and mortar schools might improve in the 2020-'21 school year with regard to virtual teaching. On one hand, that many and perhaps most districts did not announce their reopening plans until July or August restricted the amount of time for which teachers and administrators could prepare for a virtual reopening. On the other hand, a few weeks of preparation for virtual learning is an improvement over the hasty closings that schools were forced to conduct in the Spring. Finally, it is unclear to what extent the experience of students enrolled in schools affiliated with K12 Inc. are representative of the virtual schooling sector.

Survey results indicate that students' experience with virtual learning in Spring 2020 varied markedly according to whether they were enrolled in brick and mortar schools or virtual schools. That outcome does not qualify as a surprise: Virtual schools would be expected to outperform brick and mortar counterparts that were forced to adapt to virtual learning with limited warning. The magnitude of difference, however, is jarring: Respondents were almost 6.5 times more likely to report that their child "learned a lot" in the Spring if they were enrolled in a virtual school. Moreover, the advantage in the performance of virtual schools in delivering online education is not easily attributable to their performance in any one facet of schooling. Rather, survey results indicate that virtual schools dramatically outperformed brick and mortar schools when it comes to promoting active learning, communicating effectively, managing a classroom, and providing high-quality instruction. The magnitude of difference was less among students of color, whose parents reported a substantially better experience with online instruction in brick and mortar schools than did white or Asian parents. The magnitude of difference was also less among charter school students compared to traditional public school students, and private school students compared to all public school students. Study limitations notwithstanding, the radically divergent results revealed in this study suggest that, in prioritizing student learning, states would be imprudent to cap virtual school enrollment. On the contrary, in the interest of student learning, states should seek to expand access to established virtual schools through the course of pandemic-related school closures.

N.C. education board won't expand virtual charter enrollment

Keeping lines of communication open during distance learning

The effects of distance education on K-12 student outcomes: A meta-analysis', Learning point associates

Questioning the quality of virtual schools: NEPC report on K12 uses flawed measures of school performance

Instructor-learner interaction in online courses: The relative perceived importance of particular instructor actions on performance and satisfaction

Virtual school pedagogy: The instructional practices of K-12 virtual school teachers

Best practices in teaching K-12 online: Lessons learned from Michigan virtual school teachers

The determinants of students' perceived learning outcomes and satisfaction in university online education: An empirical investigation

Creating significant learning experiences: An integrated approach to designing college courses

Active learning increases student performance in science, engineering, and mathematics

Effective online instructional and assessment strategies

The virtual revolution: Understanding online schools

Amid pandemic, support soars for online learning, parent poll shows

Communication barriers in distance education

Building competence in classroom management and discipline

Survey shows big remote learning gaps for low-income and special needs children', National Public Radio

Developing, testing and refining of a model to understand the relationship between peer interaction and learning outcomes in computer-mediated conferencing

Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies', US Department of education

Understanding and improving full-time virtual schools: A study of student characteristics, school finance, and school performance in schools operated by K12 Inc

Virtual schools in the U.S. 2019. National Education Policy Center

Education and Information Technologies

University students' emotions, interest and activities in a webbased learning environment

What are secondary students' expectations for teachers in virtual school environments? Distance Education

The covid-19 cost of school closures', Brookings institution

Survey of America's charter schools

A conceptual model for virtual classroom management

What if some kids are better off at home?', The New York times

Teach new content or review familiar material? A tough call during coronavirus closures

Student perceptions of teaching effectiveness and learning achievement: A comparative examination of online and hybrid course delivery format

Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors

The achievement gap in education: Racial segregation versus segregation by poverty', Brookings institution

One more challenge during virtual learning: Lewd zoom sessions

Classroom management in the online environment

Two online charter schools in Utah see massive jumps in enrollment during pandemic', The salt Lake tribune

Increasing student motivation: Strategies for middle and high school teachers

Keeping pace with K-12 online learning: An annual review of state-level policy and practice', Evergreen education group

Key ingredients to online learning: Adolescent students study in cyberspace