key: cord-0955721-4vel6lmh
authors: Hachey, Alyse C.; Conway, Katherine M.; Wladis, Claire; Karim, Shirsti
title: Post-secondary online learning in the U.S.: an integrative review of the literature on undergraduate student characteristics
date: 2022-05-10
journal: J Comput High Educ
DOI: 10.1007/s12528-022-09319-0
sha: fee825611320ac8dad978e4e59da62a1d39503ed
doc_id: 955721
cord_uid: 4vel6lmh

Even prior to the COVID-19 pandemic, online learning had become a fundamental part of post-secondary education. At the same time, empirical evidence from the last decade documents higher dropout online in comparison to face-to-face courses for some students. Thus, while online learning may provide students access to post-secondary education, concerns about academic momentum and degree attainment dominate the higher education online learning landscape. Because course completion is often used as a measure of effectiveness, there is a strong need for institutions to be able to predict the potential persistence of online students to direct efforts towards ameliorating dropout. Yet currently, a widely tested and validated archetypical predictive model of retention and success does not exist for undergraduate online learning. This integrative review of the literature examines evidence gathered over the last decade, organizing and summarizing major findings, to help identify potential undergraduate student characteristics for inclusion in such a model. The body of literature collected in this review suggests ten factors for consideration.

Even before the global pandemic necessitated a radical shift in course delivery in the spring of 2020, online learning had become fundamental to postsecondary education. In the U.S., online learning has increased at a substantial rate over the last decade, and this is expected to continue: the 7% annual growth rate for online enrollments stands in marked contrast to the decline found in general post-secondary enrollments (Allen & Seaman, 2016) . Pre COVID-19, between 28 and 36% of all post-secondary students in the U.S. enrolled in at least one online course annually (Allen & Seaman, 2016; Lederman, 2019; NCES, 2013) and online learning was a critical part of two-thirds of post-secondary institution's long-term strategy . Initially led by community colleges (Pearson, 2011; Shea & Bidjerano, 2014) , online learning is now a reality for virtually all undergraduate institutions. Yet, because of the potentially high rates and costs of online attrition, there remains a need for institutions to be able to predict the potential persistence of online undergraduate students to foster inclusion and direct efforts towards ameliorating dropout (Hachey et al., 2013; Lee & Choi, 2011) . Hence, enrollment and retention research-and the identification of influential factors to include in preenrollment predictive models-is critical.

There is support that students can learn as much online as they do face-to-face via comparison of course-level factors. An oft-cited meta-analysis of 232 online learning studies (Bernard et al., 2004) concluded no significant difference between student outcomes online versus face-to-face. After controlling for G.P.A, Driscoll et al. (2012) found grade difference of online students versus their face-to-face counterparts were statistically insignificant. Numerous other studies and meta-analyses support that there are no positive or negative effects on outcomes online versus face-toface as measured by exam scores and aggregate course grades-the "no significant difference" claim (Ashby et al., 2011; Bowen et al., 2014; Brown, 2012; Cavanaugh & Jacquemin, 2015; Dell et al., 2010; Enriquez, 2010; Fish & Kang, 2014; Holmes & Reid, 2017; Jones & Long, 2013; Lack, 2013; Lyke & Frank, 2012; McCutcheon et al., 2015; McDonough et al., 2014; Meder, 2013; Mollenkopf et al., 2017; Nemetz et al., 2017; Rich & Dereshiwsky, 2011; Schmidt, 2012; Stack, 2015; Stocker, 2018; Tallent-Runnels et al., 2006; Wagner et al., 2011; Wu, 2015; Wuellner, 2013; Ya Ni, 2013; Zhao et al., 2005) . Thus, collectively, there is a general consensus in the literature of little to no difference in student performance between instructional mediums for courses where both mediums are available.

Nonetheless, in an early review by Zhao et al. (2005) and supported by more recent findings, although aggregated data show no significant difference in outcomes between online and face-to-face learning, there continue to be some differences reported within studies, often connected to student characteristics. Several studies report higher exam scores or final grades in online courses compared to face-to-face courses (Holbert, 2017; Jorczak & Dupuis, 2014; Soffer & Nachmias, 2018) . A meta-analysis of 176 studies from the Department of Education (2010) suggests that online learning outcomes are superior to those found in face-to-face courses. However, Jaggars and Bailey (2010) question these findings, as the only studies in the meta-analysis that were fully online and semester long (seven in total) did not find a difference between online vs. face-to-face course outcomes. Atchley et al. (2013) found differences in student outcomes between online vs. face-to-face courses, with online students earning more A, D and F grades, and Buchanan and Palmer (2017) report that median learning outcomes were similar but online assessments had larger standard deviations and higher F grades; yet, for both of these studies, differences did not necessarily mean better or worse overall outcomes between modalities. Fendler et al. (2018) contend that while no significant difference may exist between instructional modes at the aggregate level, the choice between online and face-to-face course modes may matter at the individual level. A meta-review by Nguyen (2015) contends that characteristics such as gender, race/ethnicity, and ability moderate the learning outcomes for online learning. In line with this, Figlio et al. (2013) report that students with specific characteristics (Latinx, men, low achievers) do poorly online in comparison to face-to-face. Xu and Jaggars (2011b) report lower grades for online English and Math "gatekeeper" courses and further (2013, 2014) identify lower grades in online versus face-to-face course sections for specific populations (i.e., men, Black Students, younger students and students with lower G.P.A.s). Tanyel and Griffin (2014) , in a 10-year longitudinal study, found final grades for students in face-to-face sections were significantly higher than online sections of the same course. Bettinger et al (2017) report that online learning produced lower grades on average and was more likely to negatively impact future enrollment than in-person courses. Similar findings of lower grades and negative impact on later enrollment are reported for both baccalaureate and community college students across a variety of disciplines (Alpert et al., 2016; Burns et al., 2013; Faidley, 2018; Francis et al., 2019; Garman, 2012; Hart et al., 2018; Keramidas, 2012) . It is important to note that none of these studies compared outcomes within students, rather they compared outcomes in online and face-to-face courses by different students. While the inferences we can make about which characteristics put students at highest risk online from these particular studies are limited, they do suggest that a consideration of student characteristics maybe important when investigating outcomes.

Course completion (i.e., successful finish of a course and final grade earned for which a student receives credit from the institution) is often used to measure the effectiveness of online courses (Boston & Ice, 2011) . Attrition rates among online learners fluctuate between 40 and 80% (Smith, 2010) and research shows that online attrition may be 7-20 percentage points higher than the attrition found in face-toface courses ( Anderson & Kim, 2006; Boston & Ice, 2011; Garman, 2012; Hachey et al., 2013; Jaggars et al., 2013; Johnson & Mejia, 2014; Kinder, 2013; Morris & Finnegan, 2009; Muse, 2003; Nora & Snyder, 2009; Patterson & McFadden, 2009; Smith, 2016; Smith & Ferguson, 2005; Tanyel & Griffin, 2014) . Online dropout rates have been linked by some researchers to academic non-success (Angelino et al., 2007; Atchely et al., 2013; Carr, 2000; Diaz, 2002; Gregory, 2016; Jaggars & Bailey, 2010; Rovai, & Downey, 2010; Tyler-Smith, 2006; Wojciechowski & Palmer, 2005) . In contrast to most of the literature, Moore and Fetzner (2009) present a review of six higher education institutions with 80% or greater online retention rates, Ashby et al. (2011) report better completion online, and others have found no differences in the completion rates between online and face-to-face students (Soffer & Nachman, 2018; Wilson & Allen, 2018) . Despite the few noted exceptions, overwhelmingly, aggregate findings in the literature have demonstrated that students enrolled in online courses are more likely to drop the course than those enrolled in a face-to-face course. Yet, one major limitation of this research is that studies have largely not addressed differences in the characteristics of students who enroll in online courses (versus face-to-face courses).

At least 74% of all U.S. undergraduate students have been found to have one (or more) non-traditional characteristic (U.S. Department of Education, 2015) . [Nontraditional/underserved often includes students with any of the following characteristics: minorities; low-income; rural/inner-city; first generation; academically unprepared; the most common definition of non-traditional student characteristics is: delayed enrollment (age > 24); no high school diploma; part-time enrollment; financially independent; have dependents; single parent status; working full-time while enrolled (NCES, 1996 (NCES, , 2002 ]. Online courses are often believed to increase college access, particularly for non-traditional and/or underserved students (Epper & Garn, 2003; Jaggars & Bailey, 2010; Johnson & Mejia, 2014; Parsad et al., 2008; Picciano et al., 2010) . Many students who take online courses do so because they need the flexibility that these courses offer due to a wide range of life challenges that make it difficult to attend face-to-face courses (Brown, 2012; Burnette, 2015; Daymont & Blau, 2011; Fox, 2017; Jaggars, 2014; Jaggars & Bailey, 2010; Jaggars, 2013; Lei & Gupta, 2010; Pontes et al., 2010; Skopek & Schuhmann, 2008 ). Yet, there is little evidence to support the claim that online courses increase access (Jaggars, 2011) , although improved access is often assumed .

Moreover, it is still unclear how online offerings impact college persistence, as results have been very mixed (Jaggars, 2011; Shea & Bidjerano, 2014; . In some large-scale multi-institution studies (Jaggars & Xu, 2010; Xu & Jaggars, 2011a , 2011b , results suggest that online coursework may hinder subsequent semester re-enrollment and students who took a higher proportion of online courses were less likely to graduate or transfer to a senior college. Xu and Jaggars (2013) found that for community college students, students who were men, younger, Black, or had lower G.P.A.'s, struggled in online courses. Similar findings for community college students are reported by Huntington-Klein et al. (2017) . Shea and Bidjerano (2019) report that minority community college students with higher online loads were more likely to drop out of college than non-minority students. Additionally, Smith (2016) , using multi-institution data and controlling for student self-selection bias, found results for 4-year colleges that confirm Jaggars and Xu's earlier findings of an overall negative impact of online enrollments on course and college retention.

However, these initial studies have not been confirmed in other recent research on academic momentum and degree completion which attempt to control for the student characteristics which often differ between online and face-to-face learners or that include nuanced analyses on online course taking and graduation attainment. Johnson and Mejia (2014) , assessing all community colleges in the California system and controlling for demographics and G.P.A., found that in the short term, student outcomes were worse in online courses versus face-to-face courses. Yet, they also found that students who took at least some online courses were more likely than those who took only face-to-face courses to earn an associate degree or to transfer to a 4-year institution. Bidjerano (2014, 2016) , using national data (rather than the state-wide datasets that other large-scale studies utilize) and controlling for student characteristics, also found that students (even those less academically prepared) who had taken some of their early courses online, attained degrees at significantly higher rates than those who did not. More recently, Shea and Bidjerano (2018) report that community college students who enrolled in online course loads up to 40% had better chances of completing their college degree (although it is important to note that beyond 40% online enrollment, the enhanced rate of degree completion was shown to begin to rapidly decline). They also found, on average, online course completion significantly increased the odds of completing a community college degree and transfer to a 4-year institution; more specifically, online course taking was shown to improve the degree completion/transfer of low G.P.A. part-time students relative to their full-time counterparts (Shea & Bidjerano, 2017 , 2019 . Also, using national data, Sublett (2019a) reports that community college students who engaged in online learning obtained their BA degrees faster than their peers who did not. James et al. (2016) in a large scale, multi-institutional study including community colleges and 4-year institutions, report students taking a mix of online and faceto-face courses had higher retention and passing grades than students attending fully online or fully face-to-face, but differences were explained by student characteristics rather than delivery mode and thus, showed little evidence that the act of taking online courses itself placed students at greater risk of dropout. Similarly, , using data from a large, urban, east coast, multi-institution university system, report that it cannot be assumed that students who withdraw after taking online courses do so because of the outcomes of the online course; rather, other student characteristics (which drive the students to take online courses in the first place) may be significant in determining college persistence. Recent findings, which all attempt to control for student characteristics, suggest that restricting access to online courses could reduce academic momentum and student progress toward a degree. However, given the conflicting results, more research is needed (particularly accounting for student characteristics) before firm conclusions can be drawn about how enrollment in online courses impacts college persistence and degree completion.

This review initially arose from the question "What factors are important to include in any archetypical pre-enrollment predictive model of undergraduate online learning enrollment, retention and success? To date, a widely empirically tested, archetypical predictive model of retention and success does not exist for undergraduate online learning. Traditional models of persistence and retention (i.e., Bean & Metzner, 1985; Nora & Cabrera, 1996; Tinto, 1975 Tinto, , 1986 Tinto, , 1993 Tinto, , 2016 may not be applicable to online learning because these models emphasize the role of social integration, a process very different for the online student (Boston & Ice, 2011; Hachey et al., 2013; Rovai, 2002 Rovai, , 2003 .

Scholars have posited online retention models, although none have been completely validated. Kember (1995) proposed a two-prong model of persistence for students in online courses based on how well students socially integrate school with their lives and the number of external demands they face. Kember's own research found that there are likely other factors not included in the model (it did not account for 20% of the variance in student persistence). Further, Kember's model has been empirically challenged, with Woodley et al. (2001) finding little internal consistency in the sub-scales for key constructs in the model. Rovai (2003) proposed a model of online student retention that synthesized traditional departure models with research on the skills and needs of online students; it included student characteristics, skills, and external factors as the major inputs that lead to internal factors affecting student persistence. Studies have validated and extended some of the external and internal factors identified in Rovai's model (Lehman & Conceição, 2013; Packham et al., 2004) , although other specific sub-factors that were identified have not been supported (Lee et al., 2013) . More critically, both Kember and Rovai's models are old and do not reflect recent knowledge about online learning, particularly related to modern social media phenomena (Lint, 2013) . Park (2007) proposed a model of online dropout based on Rovai's model and a review of the online learning literature at that time. Efforts to validate Park's model (Park & Choi, 2009 ) used a small sample and did not test all components. Money and Dean (2019) presented a model with major categories of antecedents and processes that they suggest may directly or indirectly impact online outcomes. Aligning with factors in Park's model, they placed individual antecedents in seven broad categories and included broad categories of student internal processes (cognitive, affective, behavioral, and social). This model has yet to be validated and while the broad categories do reflect the literature, the model does not seem to account for the totality of potential predictor factors.

Research on students who succeed in the online environment has identified internal affective qualities such as self-discipline, independence, and motivation (Dabbagh, 2007; Johnson & Mejia, 2014; Stevens & Switzer, 2006) or skills such as time management and computer literacy (Dutton et al., 2002; Johnson & Galy, 2013) , which (although not detailed) would likely fall under the internal process category in Money and Dean's model. However, these internal qualities and skills may be no less applicable to success in the face-to-face environment than in the online environment or they may be less predictive than other factors (Kauffman, 2015; Park & Choi, 2009) . Further, it is not well understood how such qualities and skills can be identified pre-enrollment; the effectiveness of screening tools is often assumed, but surveys developed to test student readiness for online learning are often costly, time consuming and may not be as effective in predicting student retention and success in online courses as other measures (Wladis & Samuels, 2016) .

A cursory examination of early reviews devoted to online learning points to two broad conclusions. First, there are a plethora of variables that have been investigated as impacting retention and success in online learning; as Tyler-Smith (2006) contends, the reasons for attrition in online courses are numerous and complex. While many of the factors reported as impacting the online student's decision to drop out likely contribute to online attrition, these variables may not easily be obtained preenrollment, nor may they account for most of the variability in retention and success. Second, discussions of early online learning research reveal a "lack of rigor" in research design (e.g., Jaggars & Bailey, 2010; Lack, 2013; Wu, 2015) . This probably stems from the lack of controlled experimental designs. For example, in an oftcited meta-analysis focused on learning outcomes (Means et al., 2010) , the authors focus on 176 articles in their study, but a critique of this work (Jaggars & Bailey, 2010) notes that only seven of the studies included experimental/quasi-experimental designs and reflect conditions in typical postsecondary education. This may provide a partial explanation for why there is little consensus for which factors are significant and lead to persistence for online students and further, why there is currently no archetypical predictive model.

The ideal approach to analyzing the effects of the online environment on attrition would be one in which students were randomly assigned to either an online or a face-to-face section of a course. Some studies have attempted to do this with mixed results (i.e., Alpert et al., 2016; Arias et al., 2018; Figlio et al., 2013; Stack, 2015) , though most randomized studies typically compare hybrid courses to face-toface courses (see Bowen et al., 2014; Vernadakis et al., 2012) . This is problematic because most online courses at colleges are fully online [80% or more online] rather than hybrid [30-79% online] (Instructional Technology Council, 2010; Lutzer et al., 2007) , and because hybrid students seem to be comparable to face-to-face students in terms of demographics, course persistence and success (Jaggars & Xu, 2010; Xu & Jaggars, 2011a) .

The dearth of large-scale randomized studies is likely due to the difficulties of recruiting students to participate in a study in which they could be randomly placed in a fully online or a face-to-face course. This is because a critical aspect of online learning is that students choose to enroll in the online modality (Fendler et al., 2018) . Lack (2013) and Paulson and McCormick (2020) note that the differences in pre-existing background characteristics of students taking online versus face-to face courses are not trivial; the characteristics that drive students to enroll in online courses are likely correlated to course outcomes (Muljana & Luo, 2019; Nguyen, 2015; Smith, 2016; Wuellner, 2013) . It has been posited that it is the characteristics of students who select online classes, rather than the instructional methodology itself, which impacts dropout (Jaggars & Bailey, 2010; McDonough et al., 2014; Wladis et al., 2015a) .

A review of the literature from 1999 to 2009 conducted by Lee and Choi (2011) identified 69 possible factors from 35 empirical studies that may impact the online student's decision to drop-out. However, Lee and Choi deliberately exclude findings related to student demographics (such as age, gender, or ethnicity), citing that the literature at the time offered inconsistent results. They are not incorrect; much of the research on the impact of student characteristics on online retention and success is conflicting (Jones, 2010 ). Yet, because findings are mixed does not mean that student characteristics should be completely ignored, nor does it mean that findings and related theorizing cannot be informative. This connects back to the earlier "lack of rigor" contention; because students opt into online courses and our lack of knowledge of what matters, many of the empirical studies on retention and success lack sufficient controls (Cavanaugh & Jacquemin, 2015; Lack, 2013; Wu, 2015) . This clarified for us that an integrative review of the literature focused on the question of "What student characteristics lead to online enrollment, retention, success and later college persistence?" is needed, both to foster better controls in future research and to help clarify potential factors for inclusion in an archetypical pre-enrollment predictive model of undergraduate online learning enrollment, retention and success.

Our goal was to be as inclusive as possible, providing a comprehensive view of the undergraduate literature for the last decade (January 2010-March 2021). In line with the methodology employed by Wallace (2003) and the guidelines for an integrative review as outlined by de Souza et al. (2010) and the PRISMA statement (Page et al., 2021) , we present a wide-ranging overview that includes findings from previous reviews and research summaries, and any empirical research that addresses student factors in online learning that affect enrollment, retention, success and/or college persistence at institutions that also offer face-to-face courses in the U.S. (this review excludes research conducted at online-only institutions). The reasons for the inclusion of these types of work are two-fold. First, previous reviews and policy reports offer conclusions that are often used by researchers and policy makers as the basis for current interventions and research directions, and therefore, deserve to be noted. Second, we include all research studies that may offer insight, regardless of sample size or methodology or type. We do note multi-institutional or national datasets present findings that are likely more generalizable; therefore, these results may be more critical to the development of any archetypical predictive model (See the Appendix, Tables 5, 6).

To conduct the search (see Fig. 1 ), relevant articles on undergraduate students enrolled in online learning in the U.S. were first located through a search of publicly available literature published between January 2010 and March 2021. Using key words, searches were conducted in three educational online databases: ERIC, Education Full Text and PsychINFO; these databases were chosen because they include adult education and they are large and commonly searched educational databases devoted to peer-reviewed literature. Outcomes. Our search of the three online educational databases initially gathered 682 articles. These articles were then screened to remove duplicates and articles that fell outside the scope of the search criteria; screening of the initial collection of articles based on the described criteria reduced the total educational database gathered works to 17. For the initial screening, articles had to meet the following criteria: published 2010 or later; focus is on online learning/online courses, with online learning being classified as taking place over the Internet (i.e., exclude articles on self-paced video-only; hybrid or blended courses); focus is on typical, semester long, for-credit undergraduate courses at institutions that offer both online and face-to-face courses/programs (i.e., exclude articles on graduate populations and/ Other Sources: Articles identified through ancestral/snowball review of initial screened database articles and works cited in select meta-reviews on online learning; manual search in Google Scholar and of selected journals during search time frame: N = 82

Articles with titles, abstracts and keywords screened:

Articles discarded as duplicate/ not relevant to search parameters: n = 665

Full-text articles assessed for eligibility using inclusion and exclusion criteria:

Full-text articles excluded because did not meet criteria/duplicates (excluding noted exceptions):

Final Literature Sample included: n = 84 Fig. 1 Flowchart outlining the search protocol adopted in this integrative review or MOOCS and/or about online-only institutions); focus is on outcomes as measured by grade and/or completion and/or future enrollment (exclude articles reporting only perceived learning, satisfaction or engagement by faculty/students and/or only student affective factors/motivation data and/or only self-reported data); specifically reports the impact of student demographics/characteristics on enrollment, course success, course retention or future online persistence; enrollment takes place in the United States and the work is published in English. Adopting Wallace's (2003) comprehensive methodology, our search was supplemented by an ancestral/snowball review of the initial articles found and also works cited in several meta-reviews and summaries on research in online learning (i.e., Hart, 2012; Jaggars et al., 2013; Lee & Choi, 2011; Money & Dean, 2019; Muljana & Luo, 2019; Sublett, 2019b; Wu, 2015) . Additional articles were gathered through manual searches in Google Scholar of the first five pages brought up in our searches [with the key words: "enrollment", "attrition" in online learning]. Gathered articles were screened by our original criteria, and then we also applied an ancestral/ snowball review to these articles. Finally, abstracts were manually reviewed based on our initial search criteria for articles published from January 2010 to March 2021 in the following journals (which focus on/often include the topic of online learn-

[Note: The Journal of Online Learning and Teaching merged with Online Learning, so an independent search of the former was conducted from 2010-2015]. As the search process progressed, duplications were removed. Note: although pre-dating 2010, we have chosen to include the meta-reviews of Bernard et al., (2004) , Tallent-Runnels et al. (2006) and Zhao et al. (2005) due to the applicability of the findings and to provide critical historical context. This inclusion and exclusion process brought the number of articles detailed in the results section of this review to 83 (See study summaries in the Appendix, Tables 2, 3, 4, 5, 6).

Except for emergency conditions such as pandemic driven necessity, students select to enroll in online courses (versus face-to-face courses). The research suggests that students who tend to enroll in online courses are likely on average different from those who enroll in face-to-face courses in terms of academic preparation/G.P.A., ethnicity/English language skills, gender, age, family/employment, part-time enrollment, non-traditional student characteristics and other characteristics such as socioeconomic status, prior online experience & credit accumulation. The literature on each of these factors is discussed and we share what is known about their relationship with undergraduate student enrollment and success in online courses.

Several large-scale and multi-institution studies suggest that undergraduate students who enroll in online classes generally have higher G.P.A.s. than those who only take face-to-face courses (Ary & Brune, 2011; Jaggars & Xu, 2010; Xu & Jaggars, 2011a*) . In more recent multi-institution studies, Johnson and Mejia (2014*) found that online course enrollment is more prevalent among more academically prepared students and Cavanaugh and Jacquemin (2015) report online students having higher G.P.A.s despite similar re-enrollment credit accumulation. On the other hand, some single course studies (Driscoll et al., 2012; Faidley, 2018; Helms, 2014; Johnson & Palmer, 2015) found that students who chose online courses had lower G.P.A.s and performed more poorly than face-to-face peers. Additionally, both a 10-year longitudinal study and a study using national data report that online students may be no more prepared or even less academically prepared than students who only take faceto-face courses (Shea & Bidjerano, 2014; Tanyel & Griffin, 2014) .

Academic preparation (usually measured by G.P.A., SAT score and/or class rank), has been found to be a mediating variable affecting online grades and dropout rates (Bettinger et al., 2017; Figlio et al., 2013; Gregory, 2016; Harrell & Bower, 2011; Ice et al., 2012; Johnson & Mejia, 2014; Lee & Choi, 2011; Lint, 2013; Ryabov, 2012; Xu & Jaggars, 2011a*) , although it is unclear the extent to which this is any different in online courses than in face-to-face courses. Remedial placement (a strong indicator of academic preparedness) has been found to negatively impact online success as measured by lower grades and higher dropout (Jaggars & Xu, 2010; Xu & Jaggars, 2013 . Recently, Francis et al., (2019*) reported that online students in developmental math courses 1 had worse outcomes than comparable face-to-face students, including lower grades and lower pass rates. Research also suggests that taking online developmental courses may negatively impact future online enrollment (Xu & Jaggars, 2011b*) . Using multi-institutional data, Johnson and Mejia (2014*) found that students who were less academically prepared performed worse in online courses. Along the same line, Cavanaugh and Jacquemin (2015) report that students with higher G.P.A.s perform better in online courses compared to a face-to-face format. Overall, findings substantiate earlier assertions by Lee and Choi (2011) , who in their review of the literature from 1999 to 2009, cite eight empirical studies that indicated that students with higher levels of academic preparation were less likely to drop-out of online courses.

Research has shown that G.P.A was a highly significant predictor of grades, for both online and face-to-face formats (Burns et al., 2013; Fendler et al., 2018; Wilson & Allen, 2018) . Still, we note that there are nuances in the research. Figlio et al. (2013) found no significant difference between online and face-to face course formats among students with higher prior G.P.A.s. However, among those with lower G.P.A.s, students in the online course scored significantly lower on in-class exams than did those in the face-to-face section. G.P.A. has been noted as having a significant relationship with online course dropout (Harrell & Bower, 2011; Helms, 2014) . Cochran et al. (2014) found that students with lower G.P.A.s were more likely to withdraw from online courses than those with higher G.P.A.s. Similarly, report that the weakest G.P.A. was associated with higher risk online, though the effect size in this study was small and students with G.P.A.s in the 2.0-3.0 range did not have significantly different risk online than students with higher or lower grades. Additionally, Hachey et al., (2013*) found that students in the 2.0-3.5 G.P.A. range had the highest proportional difference in attrition between online and face-to-face courses. For students with no prior online course experience, G.P.A. was a good predictor of future online course outcomes; but (corroborating earlier research) for students with previous online course experience, prior online course outcomes were a better predictor of future online course grades and retention than G.P.A. (Hachey et al., 2014 (Hachey et al., , 2015 . While the literature indicates that G.P.A. can be a predictor of online course retention, there is no evidence that G.P.A. is any better at predicting student outcomes in online courses than in face-to-face courses.

Available data generally seems to suggest that there is a proportional disparity in online enrollment by ethnicity, but the relationship between ethnicity and online success is less clear. Online learners are less likely than face-to-face learners to be Latinx or Black (Ashby et al., 2011; Cavanaugh & Jacquemin, 2015; Conway et al., 2011; Kaupp, 2012; Ortagus, 2017; Xu & Xu, 2019*) . Jaggars and Xu (2010*) and Xu and Jaggars (2011a*, 2011b*), assessed students who took online courses at community and technical colleges in Virginia and Washington State, and reported that online students tended to be White. Similarly, Johnson and Mejia (2014*) report that online enrollments were proportionally lower for Black and Latinx California community college students than White students, with Latinx students lagging all other student groups in online enrollment. Within the New York SUNY system, Shea and Bidjerano (2017*) report that White community college students were much more likely to take both online and faceto-face courses than to be face-to-face only students. Using national data, Shea and Bidjerano (2014) found that the odds of taking an online course were lower for Black students in comparison to White Students. Specific to STEM online courses, Wladis et al. (2015b) found that Latinx and Black STEM majors were significantly less likely to take online courses than their White student peers. Further, using national data, results indicate that Black and Latino men students are particularly proportionally underrepresented in STEM online courses (Wladis et al., 2015c*) . Additionally, ESL and immigrant status have been suggested as potentially having a negative effect on enrollment in online courses, as some research has shown that online students tend to be native English speakers (Xu & Jaggars, 2011a*) .

Research regarding the impact of ethnicity on online success is mixed, with most studies failing to differentiate between overall relationships which may exist both online and face-to-face, versus those that may be specific to the online environment. Figlio et al. (2013) randomly assigned students in a single course and report that the face-to-face format produced more favorable outcomes than the online format for students who were lower-achieving, men, and Latinx (we note this study tested one narrow definition of online learning among a small sample from a single class at a single university). Several studies using statewide data from California community colleges found that Latinx and Black students in online classes had higher withdrawal rates and lower grades than their peers in face-to-face classes (Johnson & Mejia, 2014; Kaupp, 2012*) ; however, we note that online and face-to-face students may not be comparable on key characteristics. Ice et al. (2012) report race and ethnicity as a factor in online retention and success. Krajewski (2015*) found that non-White students were 2.4 times less likely to complete an online course than White Students and further, White students were 2.5 times more likely to earn a C or above in comparison to non-White students. Similarly, Xu and Jaggars (2013*) using multi-institutional datasets, found that Black students had poorer performance (i.e., lower grades, higher withdrawal) in comparison to White students in online courses. In the other direction, Yeboah and Smith (2016) found a positive relationship for minority students (defined as African American, Hispanic, Asian, Native American or Pacific Islander, including international students) between the number of online courses taken and academic performance. The results found in these studies may simply be reflecting general trends, and not ones that are specific to the online environment.

In contrast, Xu and Jaggars (2014*) report that for students of different ethnicities, although all types of students were more likely to drop out from an online course than a face-to-face course, the size of this difference did not significantly vary across ethnic groups. Similarly, Strang (2017) found that there was no interaction between ethnicity and course medium, suggesting that gaps in success rates between Latinx and White students were no greater online than face-to-face. Shea and Bidjerano (2017*), analyzing community college student data from the New York SUNY system, found that minority status was not a significant predictor of differences in G.P.A. between face-to-face and online courses. Lei and Gupta (2010) posited that the online environment can build a bridge between academia and students from diverse cultures which can lead to better academic outcomes. Supporting this, found that foreign-born students (with English as a second language) in a large, urban, U.S. university system were at less risk of dropout online than native-born students.

There is strong evidence that women (who enroll in higher education in greater numbers than men) are also represented to an even greater degree in the online environment. Numerous studies, which confirm a plethora of earlier research, indicate that a larger proportion of women students enroll in online courses (Ashby et al., 2011; Cavanaugh, & Jacquemin, 2015; Faidley, 2018; Helms, 2014; James et al., 2016; Pao, 2016; Ryabov, 2012; Streich, 2014*; Wilson & Allen, 2018 . Within the New York SUNY system, Shea and Bidjerano (2017*) report that women community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. Jaggars and Xu (2010*), Johnson and Mejia (2014*), Smith (2016) and Jaggars (2011a*, 2011b*) , all using different multi-institution state datasets, report that online courses are significantly more popular among women students in comparison to men. Shea and Bidjerano (2014) , using national data, confirm that there is an overrepresentation of women among students who take online courses. Also using national data, Wladis et al. (2015b Wladis et al. ( *, 2015c found that women STEM majors are significantly more likely than men STEM majors to take online courses, even when academic preparation, socioeconomic status (SES), citizenship and English-as-second-language status was controlled.

Research on the relationship between gender and online course retention and success is inconclusive. Ice et al. (2012) report gender as a factor in online retention and success. Wagner et al. (2011) and Gregory (2016*) found interaction of gender and course delivery method, with lower average grades for men online than in face-toface courses. Using multi-institutional datasets, Xu and Jaggars (2013*) and Johnson and Mejia (2014*) found that women community college students had better online course persistence and grades than men students. Cochran et al. (2014) found that men were more likely to withdraw from online courses than women. Other studies report no relationship between gender and online course grades or completion rates (Burns et al., 2013; James et al., 2016; Krajewski, 2015; Park & Choi, 2009; Strang, 2017) . Shea and Bidjerano (2017*) , analyzing community college student data from the New York SUNY system, report gender was not a significant predictor of differences in G.P.A. between face-to-face and online courses. Specific to STEM online courses, one study found that women students did significantly worse online than would be expected based on their comparable face-to-face STEM outcomes [and yet, still no worse than men students] (Wladis et al., 2015a*). Palacios and Wood (2016*) found that community college men had higher success face-toface in comparison to men online, even when accounting for ethnicity. While some studies cite no differences and others have found women outperform men, we found only one study (Fendler et al., 2018) that reports men obtaining better grades than women in online courses (at the 10% significance level); this study reported no gender differences observed in face-to-face courses.

Research suggests that many students who take online courses do so because they need the flexibility that these courses offer due to life challenges that make it difficult to attend face-to-face courses (Jaggars & Bailey, 2010; Lei & Gupta, 2010; Pontes et al., 2010) . Studies have found that online learners are more likely to be older (> 24), married with families, employed and with other responsibilities (Ashby et al., 2011*; Cavanaugh & Jacquemin, 2015; Driscoll et al., 2012; Faidley, 2018; Jaggars & Xu, 2010*; Johnson & Mejia, 2014*; Ortagus, 2017; Pao, 2016; Rich & Dereshiwsky, 2011; Smith, 2016; Tanyel & Griffin, 2014; Wilson & Allen, 2018; Xu & Jaggars, 2011a*) . Within the New York SUNY system, Shea and Bidjerano (2017*) report that older community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. The current literature overwhelmingly confirms an early research review by Tallent-Runnels et al. (2006) , which puts the average age of online students above the traditional college age range of 18-24 years.

Most large-scale studies indicate that on average students in online courses are older (> 25) and that older students tend to be more successful in the online environment than younger students, both in terms of grades and completion rates (Garman, 2012; Ice et al., 2012; Kinder, 2013*) . Older students (> 25) have been reported to adapt more readily to online courses than younger students, and to benefit from higher levels of critical thinking, responsibility, and maturity (Kinder, 2013; Wuellner, 2013; Xu & Jaggars, 2013 . Johnson and Mejia (2014*) report that older students (> 24) did better than younger students in online courses, with a 4.6 percentage point achievement gap. In another multi-institutional study, James et al. (2016) report that among students taking only online courses, older students had better retention. Krajewski (2015*) reports that for every year of age, students were 1.1 times more likely to complete an online course. Xu and Jaggars (2014*) found that older online students did better than younger online students (but not to comparable face-to-face younger students). Specific to STEM online courses, Wladis et al., (2015a*) found that older students did significantly better online than would be expected based on their outcomes in comparable face-to-face STEM courses. Park and Choi (2009) , in a small-scale study, did not find age differences between those who dropped out and those who were retained in online courses. Similarly, Burns et al. (2013) and Strang (2017) found no impact of age on online performance. In a few exceptions to general trends, Francis et al., (2019*) found that older students had lower grades than both face-to-face peers and younger students in either delivery modality and Johnson and Galy (2013) report lower online grades for older Hispanic students in comparison to younger Hispanic students. Studies which attempted to compare trends by age online versus face-to-face seem to suggest that age is a factor. Yet, as with other characteristics, it is often difficult to determine the extent to which this relationship is specific to the online environment, or simply reflective of overall academic trends by age. 2

While most studies seem to point to greater success for older students, the mixed findings may be due to the added complexity of many older students lives: they are more likely to work, be married, have children and have other responsibilities that compete with time to devote to academics and impact decisions to dropout (Jaggars et al., 2013*; Lee & Choi, 2011; Smith, 2016) . Students with the additional complexities of work, family commitments, and other life responsibilities have been shown to have significantly greater preference for the flexibility and convenience of online courses (Brown, 2012; Pontes et al., 2010) . Enrollment in online courses helps these students better manage their busy schedules and provides better work/ life balance (Helms, 2014; Jaggars, 2013*; .

Nonetheless, time commitments outside of school also seem to impact their online academic performance: found that students with insufficient time for their academics (i.e., those with "time poverty" due to family and work responsibilities) may be more likely to drop an online course than a face-toface class, all else being equal. Additionally, they found evidence that the number of credits/courses taken (increasing time poverty) may be a significant predictor of differential online versus face-to-face course retention, with students taking more credits/courses at higher risk of online dropout. They further found that parents of young children (under age six) were particularly at risk of online dropout; in contrast, Gregory (2016*) did not find an impact for having a dependent child (but being married made it more likely to earn an "A" online). Park and Choi (2009) found no difference in completion results with adult learners taking online courses related to their work. There is almost no research investigating the extent to which increased work and family commitments may simultaneously relate to online course enrollment and outcomes in online versus face-to-face courses, despite the extensive research showing that online students are significantly more likely to have work and family commitments. Thus, this is a significant area of research where more research is needed.

Several studies have found that students who enrolled in online classes were employed for more hours than their face-to-face counterparts (Driscoll et al., 2012; Xu & Jaggars, 2011a , 2011b and as a result are more likely to attend part-time, which may impact enrollment and persistence. Part-time enrollment, for example, may be a weak proxy variable for time poverty, which may significantly relate to both online enrollment and course outcomes. Johnson and Mejia (2014*) report that part-time students do significantly worse in online courses. Similarly, Krajewski (2015*) found that full-time students were 2.1 times more likely to complete an online course than part-time students. Research suggests that without adequate support (e.g., childcare; financial aid to reduce work hours), the flexibility that online courses offer may not be enough to compensate for the time demands of non-academic responsibilities . Still, for students juggling school, family, and work obligations, the ability to maintain a full-time course load by taking some online courses each term may outweigh any theoretical risks of higher attrition online in terms of actual degree attainment (Johnson & Mejia, 2014*) . Perhaps in support of this, within the New York SUNY system, Shea and Bidjerano (2017*) found that full-time community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. 3

As noted earlier, there are seven characteristics typically considered non-traditional: delayed enrollment (> age 24); no high school diploma; part-time enrollment; financially independent; have dependents; single parent status; working full-time while enrolled (NCES, 1996 (NCES, , 2002 . While some of these characteristics were addressed above, studies have often looked at these characteristics as a group, as there is evidence that online learners are more likely to possess non-traditional student characteristics (Pao, 2016; Pontes et al., 2010; Rovai & Downey, 2010; Xu & Jaggars, 2013) . Jaggars and Xu (2010*), Xu and Jaggars (2011a*, 2011b*) and Streich (2014*), using multi-institutional state data, report that being financially independent, having dependents and/or being employed was associated with online course taking. Shea and Bidjerano (2014) , using national data, report an overrepresentation of online students in comparison to face-to-face students in six out of the seven categories of non-traditional characteristics. Wladis et al. (2015b Wladis et al. ( *, 2015c , also using national data, found that having non-traditional student characteristics strongly increased the likelihood of enrolling in a STEM course online, more so than any other characteristic, with online enrollment probability increasing steeply as the number of non-traditional factors increased. Further, they found that the impact of non-traditional factors on online enrollment was significantly stronger for STEM than non-STEM majors. Wladis et al. (2015b*) contend that as non-traditional students are more likely to be women and non-White, that non-traditional characteristics may serve as a mediating variable for differences in online enrollment by gender and ethnicity. Further, military students, also shown to often have non-traditional characteristics, have overwhelmingly moved to online learning modalities, yet there is little specific research on this population (Ford & Vignare, 2015) [Outside our search parameters due to self-report methodology, we do note the only found research related to military students (Wright, 2015 ; See the Appendix, Table 7) which suggests a connection between military duty, online enrollment, and dropout.]

There is concern about the online course performance of underprepared and/or traditionally underserved students who are already at higher risk of course dropout (Jaggars & Bailey, 2010*) . Evidence that non-traditional students tend to enroll in online courses, combined with research reporting that non-traditional students are more likely to be non-White and women, as well as significantly more vulnerable to negative academic outcomes (NCES, 1996 (NCES, , 2002 suggests that non-traditional characteristics may be a mediating variable for differences in online outcomes by gender and ethnicity (Wladis et al., 2015c*) . Yet, data on the effect of all non-traditional characteristics on online enrollment and persistence to date is inconsistent across the literature (Wladis et al., 2015c*) or generally missing. The high proportion of non-traditional students who enroll online compared to face-to-face students should make us cautious about generalizing inferences based on simple comparisons of online versus face-to-face students, as these two populations are significantly different; imposing norms developed with traditional students on non-traditional students may lead to invalid inferences.

Other characteristics that warrant further exploration include socioeconomic status (SES), prior online course taking experience, and credit accumulation. SES has been posited as an important mediating variable when considering online course enrollment and ethnicity (Wladis et al., 2015b*) . Ortagus (2017) found increasing participation of low-income students in comparison to middle/high income students in online courses, though low-income students were also less likely to enroll completely online. Shea and Bidjerano (2017*) found that community college students within the New York SUNY system who were Pell Grant recipients were much more likely to take both online and face-to-face courses than to be face-to-face only students. Additionally, several large-scale and national data research studies have reported that in comparison to face-to-face enrollment, students enrolling online are more likely to have applied for, or received, financial aid (Jaggars & Xu, 2010; Shea & Bidjerano, 2014; Smith, 2016; Streich, 2014; Xu & Jaggars, 2011a*) .

Krajewski (2015*) did not find financial aid was related to online completion but did report that Pell grant recipients were 2.7 times less likely to achieve a C or better in an online course. Similarly, Gregory (2016*) reports that students ineligible for Pell Grants were more likely to earn As, Bs and C's in online courses than those who were eligible. However, Burns et al. (2013) report that Pell Grant eligibility was not a significant factor in online grades. Cochran et al. (2014) found that students with loans were more likely to withdraw from online courses than those without loans. and James et al. (2016) report that while household income and Pell Grant eligible were strongly correlated with course and college outcomes generally, it was not relevant to the online environment specifically. This does not eliminate SES as a factor, yet it may simply be that lower-income students need significant support regardless of course modality.

Both prior online experience and college experience generally may have an impact on online success. Several studies report that students with a greater number of completed higher education credits had better performance in terms of grades and/or retention online (Cochran et al., 2014; Fendler et al., 2018; Fetzner, 2013*; Wilson & Allen, 2018) , though Burns et al. (2013) did not find an effect for class status. Krajewski (2015*) found that students in their first term were 2.6 times less likely to complete an online course than students who completed at least one semester. We note it is difficult to determine whether these patterns are specific to the online medium, or just reflective of a relationship between course outcomes and college level.

Previous online course experience may particularly have a significant impact on future online success (Burns et al., 2013; Lint, 2013*) . Wang et al. (2013) report that online experience directly effects self-regulation, which impacts course performance. Hachey et al., (2012*) report that knowing a student's prior online course success explains 13.2% of the variation found in retention and 24.8% of the variation found in online success, over and above GPA alone, making prior experience a significant predictor variable, which is more specific than prior academic outcomes more generally. Students with no previous online course experience (including new freshman) or those with prior unsuccessful online experience (i.e., course withdrawal or low grade) have been found to be at greatest risk (Cochran et al., 2014; Goomas & Clayton, 2013*; Hachey et al., 2012 Hachey et al., *, 2014 Ice et al., 2012; Jaggars, 2011*) , suggesting that this may be an important factor to consider in future models.

In applying Wallace (2003) and de Souza's et al. (2010) integrative methodology, we attempted to cast a wide net, conducting a comprehensive review of the literature in the U.S. for the past decade related to student characteristics which may impact the decision to enroll and subsequently succeed in online learning. To this end, we employed multiple search techniques. As all-encompassing as our method tried to be, it still cannot be certain that all relevant literature has been gathered within our defined search parameters. Further, aspects of the search criteria utilized in our protocol in their own way serve as limitations; by definition, setting exclusions means that some data is lost. We note that while this review covered a large time span, it does not include articles prior to 2010 barring a few exceptions, so all knowledge gained in previous decades are not accounted for in this study. And, while we selected databases and additional meta-reviews and journals to manually search that were likely to contain articles relevant to this review, there may have been other databases and journals that were not included that could have contributed to our findings. Additionally, we acknowledge that our search criteria of excluding studies outside the U.S. or not written in English likely filtered out studies pertinent to online student characteristics and in this way, does not allow generalizability of our findings to global contexts. Finally, our findings may only be applied to undergraduate students at institutions that offer both online and face-to-face courses, thus excluding graduate and online-only sectors in higher education. That said, the multiple approaches utilized for searching and the protocol adopted did provide a relatively large number of articles gathered and discussed in this review and thus, serves to shed some light on the online learning literature in the U.S. that focuses on undergraduate student characteristics and the potential relationships to enrollment and outcomes.

The results of this integrative review, though in many ways inconclusive, are also informative. We identified ten student characteristics which most past research suggests is relevant in describing online undergraduate students (G.P.A./academic preparation; ethnicity; ESL; gender; non-traditional status; age; family responsibilities; employment; student level; and SES). Thus, this review provides more systematic evidence that certain student characteristics are a factor significantly differentiating those who enroll online versus those who enroll face-to-face. Further, while Money and Dean (2019) provide several likely broad categories of antecedents in their model of student differences for learning in online education, this review enlarges their limited sub-category discussion of demographic attributes that are likely to have an impact on outcomes, expanding to a more comprehensive set of categories.

To accurately assess whether a factor puts an undergraduate student at greater risk in the online environment, it is essential to analyze the interaction between that factor and course medium, while simultaneously controlling for the characteristics of students who select online courses . However, clear evidence of the potential impact of the characteristics of undergraduate students who select online courses has yet to be determined; the research for many of the variables influencing outcomes is indeed mixed (See Table 1 ). Further, most of the studies found have limited potential for more generalized inferences (e.g., because they only considered patterns in online courses and direct face-to-face controls; because they compared the same courses taken by different students without thoroughly controlling for all of the factors that correlate with student selection into online courses). That said, there seems to be evidence that the ten student variables identified are sufficiently correlated with both online enrollment and academic outcomes such that they cannot completely be ignored in any online enrollment/retention model. Thus, this review supports previous assertions that several student characteristics have emerged which need further empirical testing, both regarding their impact and the predictive power these factors may have in terms of identifying students at highest risk online (Hachey et al., 2014; Street, 2010) . Table 1 Overall summary description of findings related to identified student characteristics a While the evidence from the studies found do seem to suggest a relationship between enrollment/outcomes and each of the characteristics identified (and thus, that they should be considered in online enrollment/retention models), we note that the literature remains unclear regarding how specific the relationship for each factor is to the online modality versus a trend that holds for courses more generally regardless of modality Characteristic Enrollment Outcomes a Academic preparation and G.P.A Relationship suggested; evidence mixed with some indication of higher GPA/Preparation = more enrollment Relationship suggested; strong support that higher GPA/Prepara- The characteristics we identify above, plus the prior online course experience factor, combined with the additional factors (69 additional factors, including items related to course design and institutional support) identified in the review by Lee and Choi (2011) offer insight into the creation of an archetypical predictive model of retention and success for undergraduate online learning. However, before such a model can be constructed, further work is needed. First, more large-scale, well controlled empirical research is needed to narrow down which student characteristics truly do impact outcomes in the online course modality and specifically, in what direction and to what extent. Many of the supports or interventions which are relevant to all students (e.g., supporting low-income students, students with lower levels of college preparation, minoritized students, etc.) remain equally relevant to online students, as this literature review demonstrates. What is less clear is which specific student characteristics may be most relevant to predicting differential outcomes online versus face-to-face. While some of the studies in this review have attempted to investigate this question, they are in the minority, and many of them suffer from critical limitations of the groups to which inferences can be generalized.

In the absence of large-scale randomized control trials (RCTs) and clearer evidence, this study suggests that future observational studies should be careful to systematically measure and control for all of the student-level demographic variables identified in this review which have been shown to differ between online and faceto-face students. While many of the characteristics of undergraduate students who enroll in online courses are readily available through routine and/or automatized institutional research data collections, others (e.g., work and family commitments) are less readily available, but could become readily available if they were automatically included in routine data collection efforts (e.g., colleges could ask these questions on routine admissions surveys, or federal forms such as the FAFSA could collect more specific information about ages and numbers of children, for example).

In addition, both relevant student characteristics and those possible factors identified by Lee and Choi (2011) need to be further assessed to find out which of the multitude are most predictive of academic outcomes. An archetypical model with 79+ variables is overwhelming and likely of little practical use to advisors and policy makers at undergraduate institutions; such a model need not include every possible variable but only those that capture the most variability in outcomes to be accurate enough to be predictive of those at highest risk. Finally, effective and targeted interventions need to be developed and empirically tested to support those undergraduate students identified as being at most risk of drop-out in the online environment. Because of the critical role that online courses may be playing in providing students access to college who might otherwise not have been able to take particular courses face-to-face, it is essential that we consider ways to provide all students with the supports that they need to succeed in this environment. Until this work is accomplished, there is likely to be little reduction in online attrition.

See Tables 2, 3 , 4, 5, 6 and 7. Wide variability in effect sizes on all measures suggests mixed outcomes for online courses; literature weak in terms of design features-more than half of the codable study features were missing; distinction between synchronous and asynchronous forms of online moderates effect sizes for achievement and attitudes Ford and Vignare (2015) Reviews Creation of the Predictive Analytics Reporting (PAR) Framework; identified and defined 33 variables that impact retention, progression, and completion; number of degree hours completed highly associated with a student's likelihood of remaining enrolled or graduating; race next most highly associated variable if the student does not have any degree hours; used statistical analyses including descriptive statistics, CHAID analysis, regression analysis, and group differences and found that CHAID analysis was useful/ beneficial for the stakeholders because they could produce data outputs that are "more visually appealing and relatively easy for non-statisticians to understand." Jaggars (2011) Working paper-review examining the impact of online learning on low-income and underprepared college students CC students taking online courses more likely to withdraw; online courses may discourage students from reenrolling and persisting; completers, both online and face-to-face (F2F) have similar endof-semester learning outcomes; more rigorous studies tended to find negative effects for online learning; studies with few controls and allowed for self-selection of modality had no effect for online learning; online course completion lower due to: technical difficulties, social distance, lack of structure and student supports and cost barriers Jaggars and Bailey (2010) MA of seven fully online semester-long college courses found in the Means et al. (2010) Yielded little evidence suggesting online or hybrid learning, on average, is more or less effective than F2F learning. Most studies included have mixed results: on some measures, students in the online or hybrid did better, but on others did worse, relative to students in the F2F format-or else, on some measures onlineor hybrid-format students did significantly better or worse than students in F2F, but on other measures no significant difference between groups [1996] [1997] [1998] [1999] [2000] [2001] [2002] [2003] [2004] [2005] [2006] [2007] [2008] of 1000+ empirical studies of online learning; 45 studies had rigorous design and adequate effect size and contrasted online to F2F and measured student learning outcomes Fifty independent effects identified that could be subjected to meta-analysis; found that, on average, students in online learning conditions performed modestly better than those receiving F2F instruction

Money and Dean (2019) Review of online learning; utilized a content-based literature review methodology to produce new model of online retention Literature did not coherently/consistently identify factors impacting online learning; developed retention model citing cognition; prior knowledge and experience; personal traits; motivation; familiarity with and preference for online learning; demographic attributes; and learning styles Nguyen (2015) MA of studies found on nosignificantdifference.org comparing online versus F2F 92% of all online education studies find online education is at least as effective, if not better, than F2F; author notes the lack of rigorous methodology of earlier studies and selection issues related to voluntary submittal to the site reviewed Picciano et al. (2010) MA of 6 years of national studies of online learning in American Education to examine the macrolevel impact In higher education, primarily 4-year private liberal arts schools, ignore or don't prioritize online education; issues regarding the quality of online learning and level of effect required to teach online courses at all levels of education, suggesting more developmental work needs to be done; also examined K-12 Rovai and Downey (2010) Review of literature and U.S. Securities and Exchange Commission (SEC) filings Discusses major factors in online learning (e.g. planning; marketing & recruitment; financial management; quality assurance; student retention and faculty development); states online learning attracts a higher percentage of non-traditional students and details related risk factors Shachar and Neumann (2003) MA of 86 experimental and quasi-experimental studies (1990) (1991) (1992) (1993) (1994) (1995) (1996) (1997) (1998) (1999) (2000) (2001) (2002) that met inclusion criteria Found in 2/3 of the cases, students taking courses online outperformed their student counterparts enrolled in F2F courses Tallent-Runnels et al. (2006 ) Review (1993 of common educational databases, identified 76 studies focusing on distance education

Mainly descriptive and exploratory studies; online students are non-traditional and Anglo American; synchronous communication facilitated in-depth communication (but not more than F2F classes); students liked self-pacing; online learning outcomes same as in F2F courses; students with prior computer experience were more satisfied with online courses (2015) MA of twelve studies published between 2013 and 2014 which compared at least one F2F section to at least one hybrid or fully online section Three studies employing randomization or quasi-experimental strategies found students in online or hybrid courses performed slightly worse to no differently than peers in F2F courses; descriptive studies incorporating control variables found online and hybrid courses associated with lower learning outcomes; six studies employing strictly observational analyses found students in online and hybrid formats performed no worse and, in some cases, better than F2F sections Xu and Xu (2019) Review of online learning utilizing author's research, IPEDS data, National Center for Education Statistics [NCES] data and summaries of a selection of studies from the online literature Online education can benefit some CC students; however, students learn less well in online courses compared to similar students in F2F classes-particularly at 2-year and nonselective institutions; online learning can exacerbate education inequality, since online courses are substantially more prevalent at nonselective institutions that disproportionately enroll students from underrepresented groups and lower socioeconomic backgrounds Zhao et al. (2005) Review of 51 journal articles which compared F12F versus online outcomes Aggregated data show no significant difference in outcomes between online and F2F courses, but remarkable difference across studies; effective online education needs interaction by live human instructors and the right mix of technology; online may be more appropriate for specific content and student groups Strongest predictor of online withdrawal is academic experience; seniors less likely to withdraw than non-seniors; students who previously withdrew from online courses more likely to withdraw in the current term; students with G.P.A.'s > 3.0 less likely to withdraw online Students scored significantly higher on final grades and had higher G.P.A.s in F2F section than online; women outperformed men in both modalities; older students (> 25 years) scored significantly higher than younger students in online courses; women enrolled more online than F2F and in comparison to men Online students failed, withdrew and earned lower grades than F2F students; no impact of gender, ethnic/racial status or generation status; older online learners had lower grades and pass rates that F2F older learners and also younger learners in either modality; F2F and online students did not show differential patterns of motivational change Attrition; Grades Age, race/ethnicity, academic load and first-term were predictive of completion, Pell recipient was not; age, race/ethnicity, and Pell recipient were significant predictors of grade of "C" or better; no gender effects found; full-time students more likely to complete than part-time students but academic load did not impact grade for completers; first-term students less likely to complete online but first-term did not impact grade for completers Park and Choi (2009) Online students more likely to withdraw than students faceto-face (F2F); online students more likely to make and 'A' or 'F' with more 'B', 'C', or 'D' grades for F2F students; nontraditional students more likely to earn an 'A' in both modalities; no significant differences online by gender for completion but women out performed men online in terms of grades; online students had higher ACT scores and correlated to higher grades; part-time students more likely to withdraw than full-time students in both modalities; students with more credits hours had higher grades in both modalities; Pell Grant-eligible students less likely to earn an "A" grade in an online course than non-Pell Granteligible students; no impact of first-generation status in either modality; married online students more likely to earn an "A" grade than single students; no impact of dependent child status in either modality students with Pell grants retained at higher rates regardless of modality; more women students enrolled online yet no significant difference by gender for modality; older students had greater online enrollment and retention than younger students in only online courses (the reverse was found for F2F, with younger students outperforming older students) Online students more likely than their F2F peers to fail or withdraw; online students received more grades of A than F2F courses; older students less likely to withdraw from, and more likely to receive higher grades, in online or hybrid courses Online enrollment negatively impacted course retention and performance; English enrollments had higher negative impact over math enrollments; online students more likely to be fourth

year students, older, women, and file as independent for FAFSA; online students also have lower high school G.P.A.s and SAT scores Online students more likely to work more hours and have better academic preparation, but also more likely to fail or withdraw compared to F2F students; students who took online courses in earlier terms less likely to reenroll; students who took a higher number of courses online less likely to transfer to 4-year institutions; students were equally likely to complete hybrid and F2F courses Students who are men, younger, Black, and/or have lower GPAs struggled to adapt to the online environment; older students adapted more readily to online courses than younger students did; online learning effects varied across subject areas Xu and Jaggars (2014) n = over 40,000 students in 500,000 courses at WA's 34 community and technical colleges QE design, retrospective IR data analysis Attrition; Grades All student groups suffered decrements in performance online compared to F2F, those with the strongest declines were men, younger students, Black students, and students with lower G.P.A.s; suggests there may be an achievement gap online versus F2F in some academic areas Less prepared students who took online courses early o in their time at college were more likely to attain a degree compared to students who did not do; more community college (CC) students who took courses online completed a credential 4 years after the fact compared to students who did not do; women are overrepresented in online courses Enroll-ment by student character-istics Hispanic students significantly less likely to enroll online, with Black and Hispanic men students particularly underrepresented; women significantly more likely to enroll online, as were students with non-traditional student characteristics; at CCs, ethnicity was a stronger predictor than non-traditional characteristics, whereas at 4-year colleges, the reverse was true:

each additional non-traditional risk factor increased the likelihood of online enrollment by two and five percentage points at 2-year and 4-year colleges, respectively 

Class differences: Online education in the United States

Online report card: Tracking online education in the United States

A randomized assessment of online learning

Increasing the success of minority students in science and technology (No. 310736)

Strategies to engage online students and reduce attrition rates

Online vs. face-to-face: A comparison of student outcomes with random assignment

A comparison of student learning outcomes in traditional and online personal finance courses

Comparing student success between developmental math courses offered online, blended, and face-to-face

Comparison of course completion and student performance through online and traditional courses. The International Review of Research in Open and Distance Learning

A conceptual model of nontraditional undergraduate student attrition

How does distance education compare with classroom instruction? A meta-analysis of the empirical literature

Virtual classrooms: How online college courses affect student success

Assessing retention in online learning: An administrative perspective

Interactive learning online at public universities: Evidence from a six-campus randomized trial

Online learning: A comparison of web-based and land-based courses

Role immersion in a history course: Online versus face-to-face in reacting to the past

Negotiating the mine field: Strategies for effective online education administrative leadership in higher education institutions

A Longitudinal comparison of course delivery modes of an introductory information systems course and their impact on a subsequent information systems course

As distance education comes of age, the challenge is keeping the students

A large sample comparison of grade based student learning outcomes in online vs face-to-face courses

The role of student characteristics in predicting retention in online courses

Minority student access in the online environment

The online learner: Characteristics and pedagogical implications

Deciding between traditional and online formats: Exploring the role of learning advantages, flexibility and compensatory adaptation

Comparing student achievement in online and face-to-face class formats

Integrative review: What is it? How to do it?

Online drop rates revisited

Can online courses deliver in-class results? A comparison of student performance and satisfaction in an online versus a faceto-face introductory sociology course

How do online students differ from lecture students?

Assessing the effectiveness of synchronous content delivery in an online introductory circuits analysis course

Virtual college and university consortia: A national study. Instructional Technology Council

Comparison of learning outcomes from online and face-to-face accounting courses. Electronic Theses and Dissertations

No significant difference-unless you are a jumper

What do unsuccessful online students want us to know

Is it live or is it Internet? Experimental estimates of the effects of online instruction on student learning

Learning outcomes in a stress management course: Online versus faceto-face

The evolving military learner population: A Review of the literature

What motivates community college students to enroll online and why it matters. Insights on equity and outcomes, 19. Office of Community College Research and Leadership

The costs of online learning: Examining differences in motivation and academic outcomes in online and face-to-face community college developmental mathematics courses

Student success in face-to-face and online sections of biology courses at a community college in east Tennessee. (3515024). Electronic Theses and Dissertations. Paper 1408

New-to-college "academic transformation" distance learning: A paradox

Community college student success in online versus equivalent face-to-face courses. Electronic Theses and Dissertations. Paper 3007

Is the second time the charm? Investigating trends in online re-enrollment, retention and success

Balancing retention and access in online courses: Restricting enrollment… Is it worth the cost?

Do prior online course outcomes provide more information than GPA alone in predicting subsequent online course grades and retention? An observational study at an urban community college

Prior online course experience and GPA as predictors of subsequent online STEM course outcomes. The Internet and Higher Education

Student characteristics that predict persistence in community college online courses

Factors associated with student persistence in an online program of study: A review of the literature

Online course-taking and student outcomes in California community colleges. Education Finance and Policy

Comparing student performance in online and face-to-face delivery modalities

Comparison of traditional face-to-face and online student performance in two online-delivered engineering technical electives. Paper presented at the American Society for Engineering Conference

A comparison study of on-campus and online learning outcomes for a research methods course. The Journal of Counselor Preparation and Supervision

2010 distance education survey results: Trends in eLearning: Tracking the impact of eLearning at community colleges. (No. 66). Instructional Technology Council

Effectiveness of fully online courses for college students: A response to a department of education meta-analysis. Teachers College Columbia University

Online learning: Does it help low-income and underprepared students? Community College Research Center

Choosing between online and face-to-face courses: Communitycollege student voices

Choosing between online and face-to-face courses: Community college student voices

What we know about online course outcomes. Research Review. Teachers College Columbia University, Community College Research Center

Online learning in the Virginia community college system. Teachers College Columbia University, Community College Research Center

Retention, progression and the taking of online courses

The use of E-learning tools for improving Hispanic students' academic performance

Online learning and student outcomes in California's community colleges

Comparing student assessments and perceptions of online and face-to-face versions of an introductory linguistics course

Exploring common characteristics among community college students: Comparing online and traditional student success

Learning equity between online and on-site mathematics courses

Differences in classroom versus online exam performance due to asynchronous discussion

A review of predictive factors of student success in and satisfaction with online learning

Online penalty: The impact of online instruction on the Latino-White achievement gap

Open learning courses for adults: A model of student progress

Are undergraduates ready for online learning? A comparison of online and face-to-face sections of a course

Completion rates in West Virginia community and technical colleges

Current status of research on online learning in postsecondary education. Ithaka S+R

Online enrollment grows, but pace slows. Inside Higher Ed

A review of online course dropout research: Implications for practice and future research

Discriminating factors between completers of and dropouts from online learning courses

Motivating and retaining online students: Research-based strategies that work

College distance education courses: Evaluating benefits and costs from institutional, faculty and student perspectives

Academic persistence of online students in higher education impacted by student progress factors and social media

Statistical abstract of undergraduate programs in the mathematical sciences in the United States: Fall 2005 CBMS survey

Comparison of student learning outcomes in online and traditional classroom environments in a psychology course

A systematic review evaluating of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education

Evaluation of evidence based practices in online learning: A meta-analysis and review of online learning studies. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development

Counselor education delivery modalities: Do they affect student learning outcomes (Doctoral dissertation)

Does Online Learning deliver? A comparison of student teacher outcomes from candidates in face-to-face and online program pathways

Incorporating student population differences for effective online education: A content-based review and integrative model

The road to retention: A closer look at institutions that achieve high course completion rates

Best practices in predicting and encouraging student persistence and achievement online

Factors contributing to student retention in online learning and recommended strategies for improvement: A systematic literature review

The web-based community college student: An examination of factors that lead to success and risk

The condition of education at a glance: Distance education in post-secondary institutions

Comparative effectiveness and student choice for online and face-to-face classwork

The effectiveness of online learning: Beyond no significant difference and future horizons

The role of perceptions in prejudice and discrimination and the adjustment of minority students to college

Technology and higher education: The impact of e-learning approaches on student academic achievement, perceptions and persistence

From the periphery to prominence: An examination of the changing profile of online students in American higher education. Internet and Higher Education

E-learning and retention: Key factors influencing student withdrawal

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews

Is online learning the silver bullet for men of color? An institutional-level analysis of the California community college system

Nontraditional student risk factors and gender as predictors for enrollment in college distance education (Doctoral dissertation

Factors related to learner dropout in online learning. Paper presented at the International Research Conference in the Americas of the Academy of Human Resource Development

Factors influencing adult learners' decision to drop out or persist in online learning

Distance education at degree-granting postsecondary institutions

Attrition in online and campus degree programs

Reassessing disparities in online learner student engagement in higher education

Community college student survey: Summary of results

Educational transformation through online learning: To be or not to be

Variables related to undergraduate students' preference for distance education classes

Assessing the comparative effectiveness of teaching undergraduate intermediate accounting in the online classroom format

Building sense of community at a distance

In search of higher persistence rates in distance education online programs

Why some distance education programs fail while others succeed in a global environment. The Internet and Higher Education

The effect of time online on grades in online sociology courses

The rush to online: Comparing students' learning outcomes in online and face-toface accounting courses

Differences between traditional and distance education academic performances: A meta-analytic approach

Does online learning impede degree completion? A national study of community college students

A national study of differences between online and classroom-only community college students in time to first associate degree attainment, transfer, and dropout

Online learning in community colleges of the State University of New York: Initial results on differences between classroom-only and online learners

Online course enrollment in community college and degree completion: The tipping point. The International Review of Research in Open and Distributed Learning

Effects of online course load on degree completion, transfer, and dropout among community college students

E-learning technologies: A comparative study of adult learners enrolled on blended and online campuses engaging in a virtual classroom (Doctoral dissertation)

Examining the effects of college courses on student outcomes using a joint nearest neighbour matching procedure on a state-wide university system

Student attrition in mathematics e-learning

Effectiveness of learning in online academic courses compared with face-to-face courses in higher education

Learning outcomes in an online vs traditional course

Differences between online and traditional students: A study of motivational orientation, self-efficacy, and attitudes

Beyond engagement analytics: Which online mixed-data factors predict student learning outcomes? Education and Information Technologies

Factors influencing a learner's decision to drop-out or persist in higher education distance learning

Transitioning from on-campus to online in a master of science nursing program: A comparative study of academic success

Examining distance education coursetaking and time-to-completion among community college students

What do we know about online coursetaking, persistence, transfer, and degree completion among community college students?

Teaching courses online: A review of the research

A ten-year comparison of outcomes and persistence rates in online versus face-to-face courses

Dropout from higher education: A theoretical synthesis of recent research

Theories of student departure revisited

Leaving college: Rethinking the causes and cures of student attrition

From retention to persistence. Inside Higher Ed

Early attrition among online E-learners: A review of factors that contribute to dropout, withdrawal and non-completion rates of adult learners undertaking E-learning programs

Demographic and enrollment characteristics of nontraditional undergraduates

A comparison of student satisfaction between traditional and blended technology course offerings in physical education

A longitudinal comparison of online versus traditional instruction

Online learning in higher education: A review of research on interactions among teachers and students

Students' characteristics, self-regulated learning, technology self-efficacy, and course outcomes in online learning

Success rates of online versus traditional college students

The online STEM Classroom-Who succeeds? An exploration of the impact of ethnicity, gender, and non-traditional student characteristics in the community college context

Which STEM majors enroll in online courses, and why should we care? the impact of ethnicity, gender, and non-traditional student characteristics

The representation of minority, female, and non-traditional STEM majors in the online environment at community colleges: A nationally representative study

Assessing readiness for online education -Research models for identifying students at risk

Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions

Individual student characteristics: Can any be predictors of successes in online classes?

Student progress in distance education: Kember revisited

Identifying successful online adult learners. Walden Dissertations and Doctoral Studies. 1430

Online learning in post-secondary education: A review of the literature 2013-2014. ITHAKA S+R

Student learning and instructor investment in online and face-to-face natural resources courses

Online and hybrid course enrollment and performance in Washington state community and technical colleges

The effectiveness of distance education across Virginia's community colleges: Evidence from introductory college-level Math and English courses

Adaptability to online learning: Differences across types of students and academic subject areas

Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas

The promises and limits of online higher education: Understanding how distance education affects access, cost, and quality

Comparing the effectiveness of classroom and online learning: Teaching research methods

Relationships between minority students' online learning experiences and academic performance

What makes the difference? A practical analysis of research on the effectiveness of distance education

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.