Distance Education Vol. 29, No. 3, November 2008, 231–252 ISSN 0158-7919 print/ISSN 1475-0198 online © 2008 Open and Distance Learning Association of Australia, Inc. DOI: 10.1080/01587910802395771 http://www.informaworld.com Learner and instructor identified success factors in distance education Michael P. Menchacaa* and Teklu Abate Bekeleb aDepartment of Educational Technology, University of Hawaii, Honolulu, USA; bInstitute for Educational Research, University of Oslo, Oslo, Norway Taylor and FrancisCDIE_A_339744.sgm10.1080/01587910802395771 (Received 23 May 2008; final version received 6 August 2008) Distance Education0158-7919 (print)/1475-0198 (online)Research Article2008Open and Distance Learning Association of Australia, Inc.293000000November 2008MichaelMenchacamikepm@hawaii.edu To better understand their perceptions of optimal tools and strategies for success, this research analyzed the experiences of learners and instructors in an online distance education environment. A qualitative constant comparative analysis methodology supported by an appropriate conceptual framework guided the study. Data were collected over multiple years and from multiple stakeholders. The study identified the following significant conclusions: the availability of multiple tools added flexibility to the learning environment; technology tools should appeal to multiple learning styles; collaboration, reflection, and building a learning community were important strategies supported by multiple tools; and participant satisfaction, appropriate prerequisite skills, and faculty and administrative involvement ensured programmatic success. According to this study, optimal distance education environments should address factors identified in the conceptual framework. Keywords: distance education; online learning environments; qualitative analysis Introduction Traditionally, distance education utilized one or more tools to service learning and teaching across long distances (Groff, 1996). Some of these tools included written materials sent via post, cable television, satellite, phone conferencing, or perhaps two-way video conferenc- ing. Early research tended to focus on student outcomes, student attitudes, and overall student satisfaction (Phipps, Wellman, & Merisotis, 1998). To address the need for expanded research, early researchers argued distance education was being transformed to include the utilization of interactive, communications-based technologies (Dede, 1996, 1998; Van Dusen, 1997; Watson, 1997). Some researchers noted institutions were increas- ingly relying on new technologies to deliver not only courses but also entire programs (Blumenstyk, 1999; Dede, 1998; Drucker, 1992), whereas others posited that institutions incorporating modern technologies for delivery would continue to grow (Dede, 1998; Drucker, 1992; Dyson, 1997; Peters, 1997). Indeed, online courses and programs, many relying on interactive tools, have proliferated (Allen & Seaman, 2004, 2005; Dede, 2004; Dede, Whitehouse, & Brown-L’Bahy, 2002; Fletcher, 2004). In 2004, Fletcher noted, ‘distance learning programs are offered by two-thirds of colleges and universities, and the percentage of institutions offering accredited degrees through distance learning has increased to about 55%’ (p. 2). Distance education and related research has continued to proliferate. Research has focused on how students and faculty members experience or perceive distance education environments as meeting their expectations (Bekele & Menchaca, in press). Although some *Corresponding author. Email: mikepm@hawaii.edu 232 M.P. Menchaca and T.A. Bekele research has examined the prerequisites and conditions required for optimal use in distance learning, most has focused on the effect of such learning environments on student achieve- ment (Bekele & Menchaca, in press). Studies have considered prior computer literacy (Erlich, Erlich-Philip, & Gal-Ezer, 2005); prior experience with technologies (Shih, Muñoz, & Sanchez, 2006; Yan, 2006); and perceived prerequisites for optimal use (Ostlund, 2008). These studies have typically utilized Likert scales, rankings, and multiple-choice questions for quantitative data collection and analysis. In addition, most have focused on measuring student achievement in some quantitative format, such as grades, test scores, and other outcome measures. Less frequent has been the inclusion of open-ended questions, inter- views, or focus group research. Thus, how learners perceive e-learning environments and how such perceptions affect learning is less clear (Chen & Macredie, 2002; Gilbert, Morton, & Rowley, 2007; Imel, 2002; Moore & Aspden, 2004). In this regard, ‘more in-depth qualitative interviews and case studies could provide richer insights into student e-learning behaviour, and answers to “why” and “how” questions’ (Gilbert et al., 2007, p. 561). This study was conducted to contribute to the existing body of knowledge by qualitatively analyzing student and faculty member experiences in an online Master in Educational Technology program. Purpose and questions To better inform instructional design, practice, and further research, this study explored factors needed for success in an online learning environment (OLE) as viewed by students, faculty, and administrators. It also examined the specific form of technologies optimally used for teaching and learning. The study identified instructional and learning approaches preferred in the learning environment. Overall, it examined the experience and perception of individuals participating in an OLE where only 25% of instructional time was devoted to face-to-face (f2f) meetings, by addressing the following questions: (1) What prerequisite knowledge, skills, or combination of both were required of participants for success? (2) Which technology or form of technology was optimally used for instruction and learning? (3) What learning, instructional, or combination of approaches were required or preferred for success? Literature review Relevant literature was reviewed to (a) identify success factors in OLEs, (b) identify major methodological and theoretical issues embedded in previous research, and (c) better inform the study. Table 1 presents a summary of the literature. The review indicated several limitations in previous research on success factors in OLEs. First, overall knowledge of success factors was fragmented and inconclusive partly because research on the area was relatively recent with few longitudinal analyses (Bekele, 2008; Bekele & Menchaca, in press). Particularly, how learners perceive OLEs and how these environments affect learning were inconclusive (Chen & Macredie, 2002; Gilbert et al., 2007; Imel, 2002; Moore & Aspden, 2004; Romero, du Boulay, Cox, Lutz, & Bryant, 2007). The dynamic and complex nature of OLEs seemed to explain part of this phenomenon. Second, most studies were quantitatively driven, possibly limiting depth and breadth as far as learner experiences and perceptions were concerned (Bekele, 2008; Bekele & Menchaca, in press; Gilbert et al., 2007). Distance Education 233 Table 1. Summary of success factors in online learning. Study Success factors Abel (2005) Motivation, measurements and expectations, student and faculty support, and delivery format Baker and Schihl (2005) Instructional support for staff (training on technology) and administrative support Bekele (2008) Human, technology, pedagogic, course, and leadership factors Bekele and Menchaca (in press) Factors related to technology, media, content, method of learning, and support services Carr-Chellman and Duchastel (2000) Study guides, projects/assignments, examples online, course communications through asynchronous and synchronous tools, and interactive skill building Erlich et al. (2005) Prior computer literacy and applications courses Gilbert et al. (2007) Theory–practice matches, several subject themes, social interactions, and support services Howell and Wilcken (2005) Student support (instructor and administrative based) Kung-Ming and Khoon-Seng (2005) Synchronous and asynchronous interaction Lammintakanen and Rissanen (2005) Student motivation; asynchronous and synchronous tools; experience; students’ time; support from peers and instructors; teachers’ motivation, attitude to technology, and ICT skill; and student need assessment Martz and Reddy (2005) Technology use, fairness, classroom interaction, and course content Naidu (2005) Pedagogy and the design and development process in general Novitzki (2005) Course quality, course pedagogy, various online interaction methods, and high level of interaction Ostlund (2008) Structure, autonomy, dialogue, and social presence Pituch and Lee (2006) System characteristics such as functionality, interactivity, response, self-efficacy, and Internet experience Romero et al. (2007) Learner confidence, prior operational and conceptual knowledge, teacher presence and involvement, communication between teachers and learners, and the cultural issues relating to managing change, motivation and technology platform Salter (2005) Perception of staff about technology, experience with technology, workload, and institutional factors (value of f2f, limited or no reward for online learning, logistics) Shih et al. (2006) Prior Internet-related experience Soong et al. (2001) Human factors (instructor’s time and motivation), technical competency (of instructor and students), mindset about learning (students and instructor), high collaboration, IT infrastructure and technical support Weaver (2008) Relevant learning resources, timely feedback, and interaction with teachers, administrative support, experience in WebCT Yan (2006) Earlier experience of using computer network systems 234 M.P. Menchaca and T.A. Bekele Third, many studies primarily examined student experiences with OLEs. These studies included faculty experiences and roles only from the student point of view. However, the experience of instructors and administrators is equally important. Inclusion of their perceptions of and experiences with OLEs should improve our understanding of critical success factors. Fourth, individual studies considered limited and varied success measures. Some studies measured satisfaction (Martz & Reddy, 2005); high student retention and course completion (Howell & Wilcken, 2005); high usage, enjoyment, perceived helpfulness, and higher learning (Soong, Chan, Chua, & Loh, 2001); student outcomes, student satisfaction, enrolment growth, faculty satisfaction, return on investment, and number of courses or sections (Abel, 2005); and achievement, student and faculty satisfaction, interaction, participation, and motivation (Bekele & Menchaca, in press). Overall, most of these indicators were related to benefits students and institutions were getting from the OLEs. However, each study examined only limited outcomes. Last and most important, our holistic understanding of successful OLEs is incomplete partly because each study considered only a limited number of success factors. A host of factors at technology, content, method, and support levels are involved in and impact OLEs (see Table 1). A more comprehensive approach would better inform instructional design, practice, and further research. Success factors from the literature could roughly be grouped into five interdependent categories: technology-related, user characteristics, course-related, learning approach, and support services. There was an evolving and often reciprocal influence between and among these categories. These categories helped inform how success factors were related to the human and non-human resources available to the OLEs. Overlooking one or more of the categories could unfavorably affect other categories and thus overall success measures. Technology-related factors These factors were linked to the capability or quality of hardware and software available to OLEs. Some studies (Romero et al., 2007; Soong et al., 2001) indicated general technologic infrastructure and platform were crucial for success. These researchers concluded careful consideration was required when designing, implementing, and updating technologies. In short, infrastructure was significant. Other researchers (Bekele & Menchaca, in press; Carr- Chellman & Duchastel, 2000; Lammintakanen & Rissanen, 2005; Martz & Reddy, 2005; Novitzki, 2005; Pituch & Lee, 2006; Soong et al., 2001) indicated the use of multiple technologies in different contexts was crucial for success. These researchers concluded participants should have access to many different types of technologic tools: synchronous, asynchronous, and multimedia based. In short, tools were significant. Taken together, that both infrastructure and tools were crucial for success reflected a highly contextual nature for OLEs. Generally, the technology-related category of success factors emphasized how technologies influenced online learning. This category reflected a narrow yet public view of the role of educational technology in learning. User characteristics Student and instructor roles, perceptions, and competencies in the OLEs also impacted learning. Some studies (Lammintakanen & Rissanen, 2005; Pituch & Lee, 2006; Salter, 2005; Shih et al., 2006; Weaver, 2008; Yan, 2006) indicated experience in using technology was crucial for success. The more experienced the students and instructors were, the better Distance Education 235 their success. The implication here was that novices would unlikely succeed before they acquired some level of experience with the OLEs. Other studies (Romero et al., 2007; Erlich et al., 2005; Lammintakanen & Rissanen, 2005; Soong et al., 2001) indicated the importance of skill with technologies. Some studies (Abel, 2005; Lammintakanen & Rissanen, 2005; Romero et al., 2007; Soong et al., 2001) linked levels of student motivation to success. Additional success factors included learner confidence (Romero et al., 2007), attitude to technologies (Lammintakanen & Rissanen, 2005), and learning view (Soong et al., 2001). These human factors would impact the level of learner interaction with technology and instructional content. Course-related factors These factors generally referred to quality issues in designing and offering online courses. Clear expectations (Abel, 2005) and structure (Ostlund, 2008), for instance, were among the factors that impacted success. Other studies (Bekele & Menchaca, in press; Carr-Chellman & Duchastel, 2000; Gilbert et al., 2007; Martz & Reddy, 2005; Novitzki, 2005; Weaver, 2008) have identified the importance of quality online courses that adopted appropriate learning views. Learning approach The learning approach adopted in the OLEs also impacted success. Such factors included general design process and pedagogy (Naidu, 2005) as well as online collaboration and inter- action (Bekele & Menchaca, in press; Carr-Chellman & Duchastel, 2000; Kung-Ming & Khoon-Seng, 2005; Martz & Reddy, 2005; Novitzki, 2005; Ostlund, 2008; Soong et al., 2001; Weaver, 2008). According to this category of studies, process-oriented and social learning (Benigno & Trentin, 2000; Harasim, 1996; Thornburg, 2000) was crucial for success. Support services The level of administrative support available to students and instructors was also crucial. Administrative, technical support, and logistics (Gilbert et al., 2007; Salter, 2005; Soong et al., 2001; Weaver, 2008) was reported. Student and faculty support services also impacted success (Abel, 2005; Baker & Schihl, 2005; Bekele & Menchaca, in press; Howell & Wilcken, 2005; Lammintakanen & Rissanen, 2005). The specific factors included helpdesks, support teaching staff, technical training, faculty professional development opportunities, and update of the technologic pool. In sum, the five categories of success factors were not reported in any single study. Rather, each study focused on a limited number of factors. Thus, each category should not be considered as an alternative explanation for success. Some individual studies also reported factors that belonged to different categories. Although all categories taken together might holistically explain success, current literature could address only limited success factors. Generally, success factors were linked to the systematic use of human and non- human resources available to learning. The latest definition of educational technology seemed congruent with this logic (AECT, 2008). Bekele (2008) also categorized success factors into human, technology, course, pedagogy, and leadership, which are described in the conceptual framework section below. Generally, our limited knowledge of success factors could be improved by conducting more in-depth qualitative interviews and case studies (Bekele, 2008; Bekele & Menchaca, 236 M.P. Menchaca and T.A. Bekele in press; Gilbert et al., 2007). Thus, this empirical study was aimed at contributing to the existing body of knowledge by (a) using a qualitative methodology, (b) including both student and faculty perceptions and experiences, (c) examining multiple factors at various levels, (d) considering several success measures, and (e) using a comprehensive conceptual framework. Conceptual framework Two possible reasons for challenges in research could be the absence of theoretic frameworks or the reliance on frameworks that do not consider multiple determinants (Chen, Gillenson, & Sherrell, 2002). The literature reviewed indicated that several factors impacted OLEs although each independent study considered only a limited number of factors at limited levels. Bekele (2008) has developed a more comprehensive model based on a review of 82 studies published in major educational technology journals between 1995 and 2006. The studies generally investigated impacts of modern learning technologies on student motivation, satisfaction, achievement, and critical thinking and problem-solving skills. The model was further supported by the findings of two empirical studies conducted in an unstudied setting, Ethiopia. Although recent and not yet validated or falsified across various learning contexts, the model conceptually supported the current study because (a) it considered multiple factors at various levels, (b) it was developed based on the findings of over 80 contemporary studies on OLEs, and (c) it explicitly and simultaneously stated both success measures and success factors (see Figure 1). Figure 1. Model of success and success factors in Internet-supported learning environments (Bekele, 2008, p. 57). ���������� ���� � ������������ ���������� ��� ������ �������� ���������� ����� �� ���� ��� ���� ������ ������� ��������� ���� � ��������� ��� �� ���� ��� �������� ����� �� ������������� ������� ����� �� ����������� ��� ������������� ! ��� ��" ����� ���� ���� � #�������"� ��������� � �� ���� ������" � ���� �������� ���� $��� ����� ��# ������ ����� ����� ������" � � % ��� ��"�� ��� ����� ���� � ��# ����� ���� �� ��� ��� � � ��� &��������� �������" ���' (��'���"� ���' #�������"� ���' )��' � �������"� ���� �� �������" ������� ����� �� * �������" �� ����� * ���� �� �� �� ��� * $�"��� �������" * ����� � �� �� �� ��� * �� �������� � * �������� � * +� � � �� ��� ��� �� ���� � ��� ������"���,� ��� -���� � ��� �� �� ��� �������.�� � +�������� ����� "���� ����� ����� � ���� �� ��� ��" �������"��" �������� Figure 1. Model of success and success factors in Internet-supported learning environments (Bekele, 2008, p. 57). Distance Education 237 According to Bekele’s model (2008), success in the OLEs was a function of a compli- cated interplay of human, technologic, course, pedagogic, and leadership factors, abbrevi- ated as HF, TF, CF, PF, and LF, respectively. Thus, overlooking one or more of these factors would unfavorably affect success measures such as learning outcomes, higher order learning (e.g., critical thinking, problem solving, and metacognition), student and faculty satisfaction, sustainability and scalability of online learning projects, and rate of return from investment on such projects. The HF, TF, CF, and PF would impact success directly, whereas LP would affect it only indirectly by substantially impacting HF, TF, CF, and PF. Factors are described below. HF referred to student and instructor understandings and perceptions as well as their competencies related to the OLEs. It was surmised that higher levels of motivation, infor- mation and communications technology (ICT) competency, attitude, and experience in the OLEs would result in higher success. Student and instructor views of technology and tech- nology’s role in knowledge and learning would also impact success. For example, if partic- ipants viewed knowledge as something to be acquired and defended, they would not be as actively involved in learning. Similarly, if the role of technology in learning were limited merely to carrying information, students would be unlikely to exploit the potential of tech- nology optimally. TF were linked to the attributes of educational technology. If users had dependable access to an ample variety of technologies in different contexts, then that should have a positive impact in the how, when, where, and even what of learning. Moreover, online collaboration, interaction, and communication and other pedagogic elements were possible only when one had multiple tools at hand. PF primarily referred to the how of learning and instruction in OLEs. Theoretically, success should be measured higher if learning were student focused, collaborative, problem based, and process oriented. Flexibility in approach was at least as important as course content. CF were linked to the critical elements needed in instructional design. Examples included course organization, relevance to student need, clear goals and expectations, flexibility, and other quality elements. Bekele’s model (2008) proposed that CF were among the crucial factors impacting success measures in the OLEs given appropriate technologic leadership. LF denoted the role played by the administration related to technology leadership. It was proposed that LF substantially impacted all the other four factors directly and success measures only indirectly. Technology leadership provided all the logistics required in the successful implementation of online learning. Within this framework, there existed a complex relationship among HF, TF, CF, and PF, including symbiotic relationships between HF and PF, HF and CF, HF and TF, CF and PF, and CF and TF. However, the focus of this study was limited to investigating only factors impacting optimal OLEs. Thus, the relation of the success factors to success measures was the interest. It was proposed that HF, TF, CF, PF, and LF would reflect the success factors in the California State University (CSU) system’s OLE, which was examined using a qual- itative research methodology. Research methodology Because one of the faculty members was also a participant in the program studied, the study employed participatory action research (PAR) as a method for implementation, data collection, and analysis. The PAR model requires a systematic method for observing and 238 M.P. Menchaca and T.A. Bekele then critically evaluating the implementation of some action within a participatory commu- nity in order to improve some aspect, problem, or challenge within that community (Stringer, 2007). The community in this study included faculty and students participating in a hybrid master’s program in educational technology. The challenge was to design an optimal delivery methodology for the program. Results would then be used to improve the learning experience within the community in an iterative design process requiring regular re-analysis. The PAR methodology utilized an empowerment evaluation (Fetterman, Kaftarian, & Wandersman, 1996) design for data collection and analysis. The design required triangula- tion of data from multiple stakeholders. The methodology was open-ended and easily customizable for the program studied. In addition, the methodology addressed areas of concern identified in the review of relevant literature. Some researchers have debated the validity of empowerment evaluation methodology. Fetterman and Wandersman (2007) have responded to such critics by enhancing clarity and specificity of the design. As part of the design, an outside author was added to ensure reliability and limit bias. The study was conducted at CSU, Sacramento, one of the 23 campuses that comprise the CSU system. The mission of all campuses within the CSU system is to serve the state of California by producing a conscientious, democratic society and workforce. The population studied included faculty, students, and administrators participating in the first five cohorts of an online master’s program in educational technology, one of the first online master’s programs at any campus in the CSU system. The student population consisted of K-12 master teachers, corporate trainers, school administrators, independent consultants, and technology support personnel and managers. Regarding the seven faculty member participants, four had experience with online teaching, having taught two years or more; two were novice online teachers, having taught no more than one year; and one was just joining the online instruction team. Thus, about half the faculty members were experienced with online learning and half were novices. No data were collected for this study from the faculty member who was also the researcher for this study. Overall, there were 72 students and six teaching faculty from whom data were collected. Two administrators were also consulted for the study but, because of limited participation, those data were not included in the results. Participants were asked to voluntarily participate in a survey and some were then randomly selected for follow-up focus group meetings regarding their experience in the program. Data were collected from five multiple cohorts as well as program instructors over three years. Participants had a range of skills in online learning from none to significant with the average having been exposed to one or two courses. Based on the framework provided by Bekele (2008), a series of qualitative prompts were created for an initial survey. The authors collaborated with experts in the field to design, test, and implement appropriate prompts. Noticeably absent from the prompts were any references to the framework’s factors and measures. Collaborators indicated prompts, while clear, should be as general as possible to minimize influence on participant responses. The following categories were considered clear yet broad enough to elicit diverse responses in a qualitative venue: (a) technologies, (b) optimal learning environments, (c) components for success, (d) strategies, (e) prerequisite knowledge, and (f) learning or teaching experiences. The final questions were: (1) Which technology would have been most difficult to do without? Why? (2) Describe the optimal online learning environment. (3) What components are critical to the success of online learning? Distance Education 239 (4) What strategies help ensure student success in online learning? (5) What prerequisite knowledge should students possess before they begin learning online? (6) Please describe your own experience learning (or teaching) online. What were your successes and failures? A preliminary analysis of survey responses indicated that the data lacked sufficient depth. Two focus groups were then added. The same questions were asked in the focus groups. Although focus group sessions did not create any new coded categories, responses added depth and richness to the study. One focus group consisted of eight participants (two participants randomly selected from each cohort except for the first). The second group consisted of five of the participating faculty (the newest member of the team declined to participate stating: ‘I don’t understand enough about the issues yet’). According to Stewart, Shamdasani, and Rook (2006), for focus groups, ‘the ideal range seems to be 8 to 12 people’ (p. 82). Discussion from smaller groups could be ‘narrow and biased’ (p. 82). Thus, sufficient group size was met for this study for the students but not for the faculty. Analysis methodology Coding of the data was conducted by hand using the constant comparative method (Maykut & Morehouse, 1994). The study employed a CSU statistics support consultant to assist in the coding process and data analysis and to ensure inter-rater reliability, or ‘agreement among raters’ (Vogt, 1993, p. 114). According to the procedure outlined by Maykut and Morehouse, the constant comparative coding proceeded in the fashion described below for both the open-ended and focus group analysis. First, focus group and survey responses were combined and transcribed or copied from other documents and consolidated by question into a spreadsheet in column format. Next, multiple researchers independently read all transcripts and organized the data according to phrase segments or chunks into single cells in the spreadsheet (a phrase segment or chunk of information was determined to be a part of a sentence, a complete sentence, or several contiguous sentences expressing one idea). Next, researchers compared the revised transcripts and resolved any discrepancies between the two transcripts to create a single transcript that could then be coded. The researchers collaboratively coded the transcripts using inductive category coding and simultaneous comparing of units of meaning across categories in which each phrase segment was given a code to indicate its appropriate theme. Each emergent theme was then compared to previous ones to determine whether it would receive a previous code or be assigned a new, unique coded theme (Menchaca & Hoffman, 2007). Once all segments were coded, patterns, themes, and relationships were identified and refined. Then, data were integrated and counted, patterns analyzed, categories created, and representative quotations from salient responses were extracted from transcripts to be included in the manuscript. When preliminary analysis of the data was performed for each question, significant overlap between not only questions targeted toward technology tools but also between all questions was noted. For example, responding to the question about how the program might have differed without access to tools, one student stated: ‘by communicating electronically, the individuals that did not participate verbally when we were face-to-face do so on line. This enabled me to understand those individuals more so than when we were in face [sic].’ Although responding to a tools question, the student in reality related 240 M.P. Menchaca and T.A. Bekele how important it was to integrate tools that appealed to multiple learning styles. That is, the interaction of tool and strategy was significant. Lincoln and Guba (1985) noted that such overlap of data was common in qualitative analysis. Talley (1998) proposed a solu- tion when faced with overlapping data: ‘all of the questions were analyzed together for their common elements rather than being analyzed individually’ (p. 110). Since analysis was augmented by such a strategy, all questions were coded and analyzed holistically (Talley, 1998). Coding of data Table 2 presents a synopsis of qualitative data. Coded responses were separated into three areas: technology tools, pedagogic strategies, and programmatic issues. These areas were subdivided into categories thematically. With regard to technology tools, the following categories were coded: multiple tools, technical proficiency, asynchronous tools, and synchronous tools. Technical proficiency was further subdivided into none, basic, moderate, and advanced levels. With regard to pedagogic strategies, the following were coded: situated learning, face-to-face, change, and faculty import. For programmatic issues, the following were coded: overall experience, enrolment, and program difficulty. For illustrative purposes, Table 2 indicates the percentage a certain category was coded in comparison to all other categories both for students and faculty. However, no inference about the relative significance of these themes should be drawn from these percentage rankings. According to Stewart et al. (2006), it is not typical to relate coded responses to individual respondents in focus groups. Respondents often speak simultaneously and it would be difficult to attribute responses to unique individuals even though such responses might be clear. Also, the number of respondents in focus groups is large compared to Table 2. Qualitatively coded categories as a percent of overall coded responses. Theme Student % Faculty % Technology tools 49.6 56.5 Multiple tools 37.0 32.6 Technical proficiency 10.6 13.0 • Basic 47.3 66.7 • Moderate 34.5 0.0 • None 14.5 33.3 • Advanced 3.6 0.0 Asynchronous tools 1.2 0.0 Synchronous tools 0.8 10.9 Pedagogic strategies 33.8 36.9 Situated learning 13.2 15.2 Face-to-face 9.8 8.7 Change 7.3 4.3 Faculty import 3.5 8.7 Programmatic issues 16.7 6.5 Overall experience 8.6 4.3 Enrolment 6.9 n/a Program difficulty 1.2 2.2 Note: Some totals slightly more or slightly less than 100% due to rounding errors. Distance Education 241 interviews. Thus, it was not possible to count the number of times a particular individual mentioned a particular theme although it was possible to count the number of times that theme was mentioned compared to all other themes. For that reason, while illustrative, that a particular category was mentioned more compared to others did not necessarily indicate greater significance. Findings During data collection, there were five cohorts of students in the program. Since the program had been in existence for over three years, concern regarding a novelty effect (Phipps, 1999), where students were motivated and satisfied with online delivery only because the methods were new, was addressed. When coded, 11 major categories divided into three areas were identified (Table 3). Technology tools When coded, responses regarding tools appeared nearly half the time for students (49.6%) and over half the time for faculty (56.5%). Multiple tools By far, the largest coded category for both student and faculty data was multiple tools. Included in this category were responses indicating multiple interactive tools were crucial in creating successful OLEs. References to multiple tools were coded 37% among student and 32.6% among faculty member responses. Data revealed technology tools assisted students in commenting on each other’s work. Student comments included ‘[tools provided] the opportunity for critical feedback from peers’ and ‘we posted much of our work for others to look at and for us to reflect upon and give feedback.’ Table 3. Coded categories defined. Technology tools Multiple tools Respondents indicated multiple tools were important for success Technical proficiency Respondents indicated whether students should possess some prerequisite technology skills before starting the program Asynchronous tools Respondents indicated asynchronous tools were most important Synchronous tools Respondents indicated synchronous tools were most important Pedagogic strategies Situated learning Cooperative learning in a social context was a key success factor Face-to-face Face-to-face was needed for the success of the program Change Students must be willing to embrace change Faculty import Faculty were important to the program’s success Programmatic issues Overall experience Overall experience in the program was positive Enrolment Students enroll in the program only because it is available online Program difficulty Students and faculty experienced difficulty in the program 242 M.P. Menchaca and T.A. Bekele Similar to commenting, data also revealed technology tools assisted students in discuss- ing concepts and ideas taught in the program. Two student comments were ‘[technology tools have] allowed us to hold relevant and meaningful course discussions without having to be in the same room to do it’ and ‘the discussions of the materials were invaluable. I was able to “digest” the thoughts of others at my own pace and as a result I saw a wider set of interpretations.’ Data also indicated technology tools allowed students to actively participate in the program. Student comments included ‘by communicating electronically, the individuals that did not participate verbally when we were face-to-face do so on line’ and ‘[without tools] it would be less interesting, less applicable to my real world situation, and much less actively engaging.’ Similar to allowing students to actively participate, data indicated technology tools decreased student isolation. Student comments indicated that without electronic communication ‘I would still feel isolated from others who enjoy integrating technology’ and ‘I would feel isolated and unable to contribute to the course discussion.’ The data also indicated technologies helped students access, process, and understand the content required for the program. Student comments included ‘WWW [World Wide Web] information was the most important information … without the knowledge for internet publication, navigation, and how to digest the massive amounts of information … this class could not survive’ and the ‘syllabus online with questions and reading in advance and then online discussions “forced” me to prepare at a higher level that I have done for classes in the past.’ One faculty member stated: ‘I think the web-based agendas are particularly helpful. Everyone knows ahead of time what to expect, what to prepare, and what resources to use.’ Finally, students and faculty consistently indicated that the interaction of many tech- nology tools was the key to a successful OLE. In short, one tool was not more important than another. Student comments describing their view of an optimal OLE included: ● ‘Mixture of real time communications (chat) and reflective communications (bulletin board or forum). Reading assigned in advance. Clear agendas and instructions posted well in advance.’ ● ‘Multiple means of communication (email, web, listserv, forum, chat, f2f).’ ● ‘Optimal OLE is a combination of real time chat, forum posting with time for reflection, and some f2f interaction.’ ● ‘Combination of f2f, chat and forum postings.’ ● ‘The OLE must be supported by group interaction and some f2f meetings to make connections with other students. It relies heavily on interactive chat, delayed posting assignments, the use of electronic mail.’ Faculty member comments included: ● ‘Multiple tools interacting in many ways, appealing to the different style of learning of different students.’ ● ‘Democratic access to interaction, i.e., being able to have a voice, to engage in activ- ities, to reflect, and to give, receive, and integrate feedback. These elements critical to online success are no different from quality learning elements f2f. Therefore, tools must be chosen to enable same such democratic access.’ ● ‘Online learning must have a combination of f2f (25–40%) with synchronous and asynchronous environments. Web-based agendas with plenty of scaffolding for Distance Education 243 short-term and long-term assignments. Students in OLE need to access a location where they can meet in real time without the instructor.’ Overall, there was little indication any particular tool was considered unimportant for online learning. Moreover, many tools were considered very important or ranked nearly equally in comparison to other tools. This was especially true for synchronous and asynchronous communication, which were similarly rated by students and nearly equally rated by faculty. Students and faculty consistently described the importance of multiple tools to support the OLE even when asked to describe a specific technology they could least do without. Technical proficiency Responses regarding the level of technical proficiency a student should possess before entering the program were coded 10.6% for students and 13% for faculty. Technical profi- ciency was further subdivided by skill level. For students, requiring no technical skill at all was coded 14.5%, basic skills 47.3%, moderate skills 34.5%, and advanced skills 3.6%. By far, students indicated either basic or moderate skills as needed for success in the program. Typical comments given by students regarding proficiency included ‘basic typing skills,’ ‘basic Internet familiarity,’ and ‘moderate familiarity with computer.’ Two students indicated more advanced level of proficiency including ‘HTML, use of Internet Explorer … familiarity with computer internal and external devices, a good knowledge of MS word and it’s [sic] parts, XL [sic] and access [sic].’ For faculty, requiring no technical skills was coded 33.3% and basic skills 66.7%. Inter- estingly, faculty did not deem it necessary for students to possess more than a rudimentary or basic understanding of the computer. One faculty member indicated ‘teaching experiences are more important.’ Overall, data indicated student proficiency before entering the program should be at or below basic skills with 61.8% for students and 100% for faculty members. Asynchronous tools Responses indicating asynchronous communication in the form of discussion groups as the single most important tool were coded 1.2% among students. While the percentage was small, it was compared to all concepts coded, including those not related to tool usage. In addition, none of the qualitative questions specifically mentioned asynchronous discussions. In short, several students related without prompting that discussion group software was the most important tool available. Typical comments indicated the importance of ‘delayed communication media’ and ‘reflective communications (bulletin board or forum).’ Synchronous tools Responses indicating synchronous communication as the single most important tool were coded 0.8% among students and 10.9% among faculty. Once again, without prompting, students and faculty members mentioned synchronous discussion as most critical for a successful learning environment. A typical comment from a student was ‘TI [TappedIn] was key.’ A comment from one faculty member was ‘small group discussions in synchronous environments have been successful.’ In general, faculty mentioned the value of synchronous chat more than students. Faculty seemed to be more comfortable in a synchronous environment which required students to 244 M.P. Menchaca and T.A. Bekele be present at specified times. Student coding of asynchronous and synchronous concepts differed only by 0.4%. Pedagogic strategies Pedagogic strategies were divided into the following categories: situated learning, face-to- face, change, and faculty import. When coded, concepts regarding strategies appeared in the responses one-third of the time for students (33.8%) and just over one-third of the time for faculty (36.9%). Situated learning Responses indicating the importance of community-based learning were coded 13.2% among students and 15.2% among faculty members. Both students and faculty indicated a strong belief in the value of group work and collaboration for the success of the OLE. Their typical words or phrases included ‘collaboration,’ ‘teamwork,’ ‘group work,’ ‘small groups,’ and ‘cooperative.’ Student responses included ‘the cooperative groups online ensured success and involvement,’ ‘the ability to communicate and collaborate within this course drives the curriculum and the experience,’ ‘group activities to foster interaction with students,’ and ‘I was able to “digest” the thoughts of others at my own pace and as a result I saw a wider set of interpretations.’ Faculty member responses included ‘group facilitates inter student dialogue’ and ‘the most successful experiences have been in small group discussions.’ In addition, faculty and student responses supported the idea that a strong sense of community was important to success. Typical phrases included ‘community,’ ‘community- building,’ ‘team-building,’ ‘community of learners,’ ‘interaction,’ ‘social learning,’ ‘multiple expertise,’ and ‘social interaction.’ One student, when asked to identify critical success factors for online learning stated: ‘community, community, community.’ Other student responses included ‘the environment should foster a feeling of community and cama- raderie,’ ‘the cohesiveness of the group is extremely important to the entire learning envi- ronment,’ ‘a community of learners is essential to success,’ ‘the knowledge that learning is a social process and that technology is a tool to allow that social connection,’ ‘an optimal environment would include a community of people sharing similar interests,’ and ‘the opti- mal online learning environment would be one that is accepting of all learners regardless of expertise.’ One faculty member stated: ‘the basis for collegiality, the environment necessary for constructivism (which is deeply grounded in trust), and the protocols for collaboration would not have been established to transform them online without community.’ Face-to-face (f2f) A significant number of responses indicated the importance of f2f interaction for success in online learning. Moreover, respondents were not prompted to rate the importance of f2f meetings. They were asked only general questions such as ‘Describe the optimal learning environment’ or ‘What components are critical to the success of online learning?’ One student responded: I think the initial bootcamp [orientation] was something that helped build an understanding and appreciation for the work ahead. The f2f was more crucial than I originally thought due to the need to connect and get greater clarity. It also helped build ties that I know helped later. Distance Education 245 Overall, responses indicating f2f as important occurred in about one out of every 10 unique codings for students (9.8%) and nearly as often for faculty (8.7%). Faculty responses included ‘developing community of learners through initial f2f sessions,’ ‘community building technology that was established f2f,’ ‘opportunities for f2f,’ and ‘community needs to be established early in the program.’ Change Responses indicating that students should be motivated and not resistant to change were coded 7.3% among students and 4.3% among faculty. Student comments included ‘must tolerate AC [ambiguous clarity],’ ‘open-communication, flexibility, and a commitment to achieve and excel,’ ‘team work and cooperation by all members as well as participation by all members at all meetings,’ ‘you have helped me to become a change agent,’ and ‘try new things, communicate with classmates, be extremely self-motivated.’ Two faculty members stated students must possess ‘a tolerance for ambiguity.’ Data indicated change was significant. Faculty import Student responses underscored the importance faculty played in the OLE. The influence of faculty members was coded 3.5% among students and 8.7% among faculty. Most commen- tary was focused on the importance of faculty providing prompt and reflective feedback. Other comments indicated the role faculty members played in supporting students as well as their ability to create a viable learning community. Student comments included ‘immediate feedback is crucial,’ ‘having immediate feed- back has been very helpful when working on subject matter that is difficult to comprehend,’ ‘good, prompt feedback from the instructor,’ ‘support from instructors,’ ‘professors … are able to create the intimacy of an online community,’ and ‘by far the most critical is the competence level of the instructors … the instructor needs to know how the tools function in a learning environment before bringing students to use them.’ Instructor comments included ‘I felt that the students needed the comfort of having access to the instructor, even if it is limited by the online environment,’ ‘tools were constantly used for asking clarification from instructor,’ and ‘students need access to instructor.’ Additionally, one faculty member indicated: ‘an ideal online learning environ- ment requires more than one instructor, each with areas of expertise to share.’ Programmatic issues Programmatic issues were divided into three categories: overall experience, enrolment, and program difficulty. Overall experience Responses were coded 8.6% for students and 4.3% for faculty. Typical student words included ‘satisfied,’ ‘enjoyed,’ ‘valuable,’ ‘learned a lot,’ ‘wonderful,’ ‘best experience,’ ‘exhilarating,’ ‘loved it,’ ‘meaningful,’ ‘fulfilling,’ ‘positive,’ ‘motivating,’ and ‘effective.’ Statements included ‘I have truly enjoyed the total experience,’ ‘this is my first time using on-line learning, and its great!,’ ‘this is the best learning experience I have ever had and I have 2 masters degrees, a Ph.D. and I teach a class online,’ ‘powerful, effective, flexible, 246 M.P. Menchaca and T.A. Bekele and a big gas savings,’ and ‘my own experience as co-developer of the iMET [the acronym for the online program] format was the most self-motivating and rewarding learning experiences I have had in my post-secondary educational experience – absolute.’ Faculty keywords included ‘satisfied,’ ‘wonderful,’ and ‘successful.’ Comments included ‘iMET has been a wonderful experience, both from a teaching and a learning perspective’ and ‘I really think that all aspects were successful.’ Enrolment Student responses, coded at 6.9%, indicated that if the program was not available online, they would not have enrolled. Responses included ‘I would not have enrolled in a Master of Educational Technology course if technology was not going to be used,’ ‘I would not have enrolled in the program,’ and ‘without access to the electronic communication, the course would not have been a possibility for me.’ From a programmatic standpoint, this data indicated the program was appealing to a student population that would not otherwise enroll in a traditional format, due to location, convenience, professional obligations, or some other reason. Program difficulty Responses indicating some level of difficulty with the program were coded at 1.2% for students and 2.2% for faculty. Difficulties included problems accessing the synchronous environment, lack of adequate feedback, and difficulty working in groups. With regards to the synchronous environment, one student stated: ‘I didn’t like TappedIn most of the time. It was difficult to keep people on task. What could be done in 10 minutes f2f could take 30 minutes or more in TappedIn.’ Two other students stated respectively: ‘tapped-in [sic] was challenging at times’ and ‘My biggest stress with TappedIn was the ever-present possibility of losing my connection.’ With regards to feedback, one student stated: ‘Quicker and more in depth feedback would have been appreciated.’ Another student said: ‘I needed more feedback.’ Finally, one student mentioned difficulty working in groups. I found at times that I became very frustrated with some of the group assignments because some of the participants were not doing their part. Another problem was the distance between the members made it difficult to put together assignment to the quality [sic]. Among faculty, one respondent stated: ‘Not successful was any activity without making explicit specific goals and criteria for achieving goals; inadequate tools; lack of protocol for interaction.’ This faculty member noted that there were difficulties when students did not have a clear understanding of assignments, when they did not have adequate resources, and when it was not clear how they should collaborate online. Discussion This study examined the experience and perception of students and faculty in the OLE in the CSU system exploring factors needed for success. Participants were asked about optimal technologies used, preferred learning approaches, and prerequisite knowledge or skill needed for online learning. Data were collected using surveys and focus group interviews. A constant comparative methodology was employed for data coding and analysis. The major findings are discussed below. Distance Education 247 Through qualitative analysis, the study identified multiple categories divided into technologic, pedagogic, and programmatic areas. With regard to technologic tools, faculty and students alike identified the importance of using multiple tools appealing to diverse learning styles. The use of multiple tools coded at the highest percentage. The availability of technology-based tools, such as online collaboration, electronic communication, and web publishing, provided students the strongest opportunity to participate and collaborate with each other. Student and faculty data indicated students would more actively participate in discussions and meetings than if these tools were unavailable. Flexibility in the learning environment was also provided by the availability of multiple tools. Most respondents viewed the usage of synchronous and asynchronous tools as equally important, each appeal- ing to a particular learning style, such as real-time discussion, collaboration, and reflection. This finding corroborated other research. Several studies (Bekele & Menchaca, in press; Carr-Chellman & Duchastel, 2000; Kung-Ming & Khoon-Seng, 2005; Lammintakanen & Rissanen, 2005; Soong et al., 2001) indicated the importance of using multiple technologies for success in OLE. Some studies (Bekele & Menchaca, in press; Carr-Chellman & Duchastel, 2000; Kung-Ming & Khoon-Seng, 2005; Lammintakanen & Rissanen, 2005; Novitzki, 2005) specifically recognized the enormous potential of asynchronous tools in supporting learning and related communication. Additionally, some studies indicated synchronous tools were considered vital for success in learning (Bekele & Menchaca, in press; Carr-Chellman & Duchastel, 2000; Kung-Ming & Khoon-Seng, 2005; Lammintakanen & Rissanen, 2005; Soong et al., 2001). The conceptual framework of the study also noted the importance of appropriate technologies for specific contexts. Asynchronous and synchronous technologies were two of the technology factors mentioned in the conceptual framework. Generally, no technology was perceived as more important than another in all contexts; technology use was context dependent. The study also revealed that some level of ICT competency was required for successful OLE. Students and faculty needed to acquire basic computer and Internet skills. Some level of experience with OLE was also perceived as important. This finding corroborated previous studies indicating some level of ICT proficiency, experience, or both was required for online learning success (Erlich et al., 2005; Lammintakanen & Rissanen, 2005; Pituch & Lee, 2006; Salter, 2005; Shih et al., 2006; Soong et al., 2001; Weaver, 2008; Yan, 2006). In addition, skill and experience were among the human factor category of the conceptual framework underpinning this study. The implication was that novices would require more time and experience before becoming satisfied with OLE. However, pedagogic factors were also important. The pedagogic area received the second highest number of responses after the techno- logic area. A significant pedagogic category was the situated nature of learning. Learning was considered a social, problem-based and collaborative phenomenon. Faculty and students repeatedly mentioned the importance of collaborative discussion and reflection supported by multiple tools. Through collaborative tools, students were more likely to comment on each other’s work, providing critical feedback and suggestions for modifica- tion. Literature also indicated that cooperative learning in a social environment was crucial to success in online learning (Benigno & Trentin, 2000; Blumenstyk, 1999; Dede, 2004; Harasim, 1996; Thornburg, 2000). The conceptual framework maintained that collaborative and process-oriented learning was required for success. This is in line with the latest view of learning and educational technology (AECT, 2008). Additionally, the inclusion of some f2f interaction, especially for the purpose of building community, was also found significant. Students and faculty strongly endorsed the inclusion of f2f time in the online environment, especially at the onset of the program. The f2f retreat 248 M.P. Menchaca and T.A. Bekele and renewal meetings created a sustainable community where students did not feel isolated. This was due in large part to the active community-building students repeatedly stated sustained their ability to interact and collaborate with peers during long periods of working online. Research indicated many institutions used f2f meetings to supplement online delivery (Abel, 2005; Bekele & Menchaca, in press). Additionally, the conceptual frame- work considered f2f interaction as one of the pedagogic factors required for success. Since most research has focused on other factors, the role or advantage of f2f in online learning is generally unknown. This study recognizes the importance of f2f, at least during the start and completion of a course or program. Attitudinal change in participants was an additional pedagogic category. Participants needed to embrace change and develop favorable attitudes toward technology. This corrob- orated previous research (Lammintakanen & Rissanen, 2005) as well as the conceptual framework, which included a positive attitude toward technology and OLE as one of the human factors. The role of faculty was a final pedagogic-related category required for successful online learning. Participants indicated that faculty did play a major role in organizing learning, providing feedback, and in monitoring the online process. This role of faculty was found in several studies (Abel, 2005; Baker & Schihl, 2005; Bekele & Menchaca, in press; Gilbert et al., 2007; Howell & Wilcken, 2005; Lammintakanen & Rissanen, 2005; Naidu, 2005; Salter, 2005; Soong et al., 2001; Weaver, 2008). The role of faculty was also reflected in the human, course, pedagogic, and technologic factors of the conceptual framework. The framework indicated the importance of faculty for (a) supporting student motivation; (b) optimally utilizing appropriate technologies; (c) choosing relevant learning approaches; and (d) designing, offering, and monitoring online courses. With regard to the programmatic area, one finding was that students would not have enrolled in the program had it been offered in a traditional format. Thus, the CSU system was able to attract students who would not otherwise have matriculated. In this way, OLE reached the otherwise unreachable audience by expanding educational access. This was a major motive behind the launch of distance education globally. The online program was convenient for participants as most were non-traditional students. Regard- less of space and time constraints, students could attend online courses while being at work or with family. Still, students did face challenges and difficulties in the OLE. These challenges were related to clarity of course materials, course organization, group dynamics, limited level of feedback, and technical difficulties. Such challenges might be addressed by increasing faculty and administrative involvement, as indicated under course and leadership factors of the framework. Programmatically, students and faculty were satisfied overall with their online experi- ence. This is important as research indicated satisfaction was directly related to achieve- ment and negatively related to dropout rates (British Columbia College and Institute, 2003), a significant problem in the OLE. Thus, successful OLEs were likely to support better student performance, if not also learning, while improving student satisfaction. For this study, satisfaction positively influenced the sustainability and scalability of the online program. Since students were satisfied with the learning environment, they were more likely to enroll in additional online courses. In addition, anecdotal evidence indicated new students were attracted via word of mouth. Student demand definitely influenced CSU’s decision to sustain and scale up existing OLEs. Current research and the conceptual framework corroborated these findings. A recent review of educational technology research analyzing studies published between 1995 and 2007 indicated OLE influenced student Distance Education 249 motivation and satisfaction in various contexts (Bekele, 2008). In short, student satisfaction was one of the major success measures in the CSU system and was supported by literature and the conceptual framework of this study. In summary, the success factors found in the CSU online program corresponded to factors stipulated in the conceptual framework. All five categories of factors were evident although with only a limited number of factors within each category. For instance, technol- ogy skills, attitude, experience, and instructor roles reflected human factors. The use of multiple tools, asynchronous and synchronous, for different contexts as well as access and dependability reflected technologic factors. Pedagogic factors included the collaborative, situated nature of learning and f2f interaction. Clarity of goals, course organization, and quality of projects/assignments were the major course factors identified. Leadership factors included the importance of training, support, technical provision, and technology logistics. Generally, participants perceived technologic, pedagogic, human, course, and leadership factors were required for successful online learning. None of these factors was considered more important than others; rather, each factor impacted success depending on context. A complex interplay among these factors was required for successful OLEs. Implications for practice This study was an important step to better understand the experience of learners and instructors in OLE. Although data were collected from multiple cohorts and faculty, the study was limited to a single program at the graduate level. However, initial results were promising. Optimal distance delivery should address the factors identified in the study and those defined by the conceptual framework. The factors should be considered holistically when implementing quality distance delivery. That participants responded qualitatively without prompting indicated substantial importance for all factors. Specifically, successful OLEs should: ● integrate multiple tools for different contexts; ● promote a positive attitude toward technology and OLE; ● incorporate a social and situated learning environment; ● include some level of f2f interaction; ● involve and rely on faculty at many levels; ● help participants develop appropriate skills, experience, and training; and ● provide sustained administrative support. Implications for research Although the study revealed interesting findings about student and faculty experiences, a better and more definitive understanding of specific tools and factors, perhaps identifying the best of the best that exist for certain contexts, is still needed. According to the study, factors at human, pedagogic, technologic, course, and leadership levels all impact online learning experiences. Consequently, both quantitative and qualitative longitudinal studies employing comprehensive conceptual frameworks are required to better understand how these factors operate in complex OLE. The quality and nature of online collaboration and interaction as well as group dynamics should be examined further. Contemporary society requires higher order skills such as critical reflection, metacognition, and problem solving from graduates. Thus, how and to what extent such skills are supported through distance education should also be examined further. 250 M.P. Menchaca and T.A. Bekele Notes on contributors Michael Menchaca is an assistant professor in the University of Hawaii’s Department of Educational Technology. He directs the online Master in Educational Technology program. He conducts research on online learning, social justice with technology, and integrating technology into teaching and learning. Teklu Abate Bekele is a doctoral candidate at the Institute for Educational Research, the University of Oslo, Norway. He conducts research on the impacts of Internet-supported learning environments on student motivation, satisfaction, achievement, and critical thinking and problem-solving skills in higher education. References Abel, R. (2005). Achieving success in Internet-supported learning in higher education: Case studies illuminate success factors, challenges, and future directions. Lake Mary, FL: Alliance for Higher Education Competitiveness. Retrieved April 10, 2006, from http://www.a-hec.org/ research/study%5Freports/IsL0205/TOC.html AECT. (2008). Definition. In A. Januszewski & M. Molenda (Eds.), Educational technology: A definition with commentary (pp. 1–14). New York: Lawrence Erlbaum Associates. Allen, I.E., & Seaman, J. (2004). Entering the mainstream: The quality and extent of online educa- tion in the United States, 2003 and 2004. Sloan Consortium: A Consortium of Institutions and Organizations Committed to Quality Online Education. Retrieved April 10, 2006, from http:// www.sloan-c.org/resources/survey.asp Allen, I.E., & Seaman, J. (2005). Growing by degrees: Online education in the United States, 2005. Sloan Consortium: A Consortium of Institutions and Organizations Committed to Quality Online Education. Retrieved April 10, 2006, from http://www.sloan-c.org/resources/survey.asp Baker, J.D., & Schihl, R.J. (2005). Faculty support systems. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 2, pp. 936–940). Hershey, PA: Idea Group Reference. Bekele, T.A. (2008). Impact of technology-supported learning environments in higher education: Issues in and for research. Unpublished doctoral dissertation, University of Oslo, Norway. Bekele, T.A., & Menchaca, M. (in press). Research on Internet-supported learning: A review. Quarterly Review of Distance Education. Benigno, V., & Trentin, G. (2000). The evaluation of online courses. Journal of Computer Assisted Learning, 16(3), 259–270. Blumenstyk, G. (1999, June 18). The growth of unext.com. Chronicle of Higher Education. Retrieved July 23, 2008, from http://chronicle.com/chronicle/v45/4541guide.htm British Columbia College and Institute. (2003). Understanding student satisfaction. Issue Paper, 3(1), 1–4. Retrieved July 23, 2008, from http://outcomes.bcstats.gov.bc.ca/Publications/ collegereports/issue_satisfaction.pdf Carr-Chellman, A.A., & Duchastel, P. (2000). The ideal online course. British Journal of Educational Technology, 31(3), 229–241. Chen, L.D., Gillenson, M.L., & Sherrell, D.L. (2002). Enticing online consumers: An extended technology acceptance perspective. Information and Management, 39(8), 705–719. Chen, S., & Macredie, R. (2002). Cognitive styles and hypermedia navigation: Development of learning model. Journal of the American Society for Information Science and Technology, 53(1), 3–15. Dede, C. (1996). The evolution of distance education: Emerging technologies and distributed learning. The American Journal of Distance Education, 10(2), 4–36. Dede, C. (Ed.). (1998). Learning with technology: 1998 ACSD yearbook. Alexandria, VA: Association for Supervision and Curriculum Development. Dede, C. (2004). Enabling distributed learning communities via emerging technologies – Part One. THE Journal, 32(2), 12–17. Retrieved July 23, 2008, from http://www.thejournal.com/articles/ 16909 Dede, C., Whitehouse, P., & Brown-L’Bahy, T. (2002). Designing and studying learning experiences that use multiple interactive media to bridge distance and time. In C. Vrasidas & G. Glass (Eds.), Distance education and distributed learning: Current perspectives on applied information technologies (pp. 1–30). Greenwich, CT: Information Age Publishing. Distance Education 251 Drucker, P. (1992). Managing for the future: The 1990’s and beyond. New York: Dutton. Dyson, E. (1997). Release 2.0: A design for living in the digital age. New York: Broadway Books. Erlich, Z., Erlich-Philip, I., & Gal-Ezer, J. (2005). Skills required for participating in CMC courses: An empirical study. Computers & Education, 44(4), 477–487. Fetterman, D., Kaftarian, S., & Wandersman, A. (1996). Empowerment evaluation: Knowledge and tools for self-assessment and accountability. Thousand Oaks, CA: Sage. Fetterman, D., & Wandersman, A. (2007). Empowerment evaluation: Yesterday, today, and tomorrow. American Journal of Evaluation, 28(2), 179–198. Fletcher, G. (2004). The future of e-learning. THE Journal, 32(2), 2–3. Retrieved July 23, 2008, from http://www.thejournal.com/articles/16899 Gilbert, J., Morton, S., & Rowley, J. (2007). e-Learning: The student experience. British Journal of Educational Technology, 38(4), 560–573. Groff, W.H. (1996). Creating and sustaining learning communities in the digital era. Fort Lauderdale-Davie, FL: Nova Southeastern University. Retrieved July 23, 2008, from ERIC Document Reproduction Service No. ED396188. Harasim, L. (1996). Online education: The future. In T. Harrison & T. Stephen (Eds.), Computer networking and scholarly communication in the 21st century university (pp. 203–214). Cambridge, MA: MIT Press. Howell, S.L., & Wilcken, W. (2005). Student support services. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 4, pp. 1687–1692). Hershey, PA: Idea Group Reference. Imel, S. (2002). E-learning: Trends and issues alert No. 40. Washington, DC: Office of Educational Research and Improvement. Retrieved July 23, 2008, from ERIC Document Reproduction Service No. ED469265. Kung-Ming, T., & Khoon-Seng, S. (2005). Asynchronous vs. synchronous interaction. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 1, pp. 104–112). Hershey, PA: Idea Group Reference. Lammintakanen, J., & Rissanen, S. (2005). Online learning experiences of university students. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclo- pedia of distance education (Vol. 3, pp. 1370–1374). Hershey, PA: Idea Group Reference. Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage. Martz, W.M., Jr., & Reddy, V.K. (2005). Success in distance education. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 3, pp. 1440–1445). Hershey, PA: Idea Group Reference. Maykut, P., & Morehouse, R. (1994). Beginning qualitative research: A philosophical and practical guide. Washington, DC: Falmer Press. Menchaca, M., & Hoffman, E. (2007). Designing multiple-tool learning environments in the Pacific Rim. In P. Potiyaraj (Ed.), Proceedings of Distance Learning International Conference 2008 (pp. 10–17). Bangkok, Thailand: Association of Pacific Rim Universities. Moore, K., & Aspden, L. (2004). Coping adapting, evolving: The student experience of e-learning. Update, 3(4), 22–24. Naidu, S. (2005). Evaluating distance education and e-learning. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 2, pp. 857–864). Hershey, PA: Idea Group Reference. Novitzki, J.E. (2005). Necessities for effective asynchronous learning. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 3, pp. 1325–1331). Hershey, PA: Idea Group Reference. Ostlund, B. (2008). Prerequisites for interactive learning in distance education: Perspectives from Swedish students. Australasian Journal of Educational Technology, 24(1), 42–56. Peters, T. (1997). The circle of innovation: You can’t shrink your way to greatness. New York: Knopf. Phipps, R. (1999). What’s the difference? A review of contemporary research on the effectiveness of distance learning in higher education. Washington, DC: Institute for Higher Education Policy. Phipps, R., Wellman, J., & Merisotis, J. (1998). Assuring quality in distance learning: A preliminary review. Washington, DC: Council for Higher Education Accreditation. Pituch, K.A., & Lee, Y. (2006). The influence of system characteristics on e-learning use. Computers & Education, 47(2), 222–244. 252 M.P. Menchaca and T.A. Bekele Romero, P., du Boulay, B., Cox, R., Lutz, R., & Bryant, S. (2007). Debugging strategies and tactics in a multi-representation software environment. International Journal of Human–Computer Studies, 65(12), 992–1009. Salter, G. (2005). Factors affecting the adoption of educational technology. In C. Howard, J. Boettcher, L. Justice, K. Schenk, P.L. Rogers, & G.A. Berg (Eds.), Encyclopedia of distance education (Vol. 2, pp. 922–929). Hershey, PA: Idea Group Reference. Shih, P., Muñoz, D., & Sanchez, F. (2006). The effect of previous experience with information and communication technologies on performance in a Web-based learning program. Computers in Human Behavior, 22(6), 962–970. Soong, M.H.B., Chan, H.C., Chua, B.C., & Loh, K.F. (2001). Critical success factors for online course resources. Computers & Education, 36(2), 101–120. Stewart, D., Shamdasani, P.N., & Rook, D. (2006). Focus groups: Theory and practice (2nd ed.). London: Sage. Stringer, E.T. (2007). Action research. Thousand Oaks, CA: Sage. Talley, S. (1998). The use of online technology tools and teacher professional development. Unpublished doctoral dissertation, Pepperdine University, Malibu, CA. Thornburg, D. (2000). Campfires in cyberspace: Primordial metaphors for learning in the 21st century. Retrieved June 1, 2005, from http://www.tcpd.org/thornburg/handouts/Campfires.pdf Van Dusen, G. (1997). The virtual campus: Technology and reform in higher education (ASHE– ERIC Higher Education Report, Vol. 25, No. 5). Washington, DC: US Department of Education. Vogt, W. (1993). Dictionary of statistics and methodology: A nontechnical guide for the social sciences. Newbury Park, CA: Sage. Watson, D. (1997). Information technology: Policy and practice. In J. Dillon & M. Maguire (Eds.), Becoming a teacher (pp. 199–210). Buckingham: Open University Press. Weaver, D. (2008). Academic and student use of a learning management system: Implications for quality. Australasian Journal of Educational Technology, 24(1), 30–41. Yan, Z. (2006). Different experiences, different effects: A longitudinal study of learning a computer program in a network environment. Computers in Human Behavior, 22(3), 364–380.