Curtin University of Technology Science and Mathematics Education Centre Development and Validation of an Instrument for Assessing Distance Education Learning Environments in Higher Education: The Distance Education Learning Environments Survey (DELES) Scott L. Walker This thesis is presented for the Degree of Doctor of Science Education of Curtin University of Technology November 2003 ii This thesis contains no material which has been accepted for the award of any other degree or diploma in any university. To the best of my knowledge and belief this thesis contains no material previously published by any other person except where due acknowledgment has been made. Signature: Date: November 13, 2003 iii ABSTRACT Globally, as distance education has become firmly embedded as a part of the higher education landscape, governments and institutions are calling for meaningful research on distance education. This study involved designing, developing and validating a learning environment survey instrument for use in distance education-delivered courses in post-secondary education. Specifically it involved merging two distinctive areas of study: psychosocial learning environments research and distance education research. The unique social structure of asynchronous distance education learning environments requires a unique and economical instrument for measuring the perceptions of distance education course participants. The research followed a three- stage instrument-development process of identifying salient scales, developing survey items, and field testing and analysing data using item analysis and validation procedures. This was followed by an investigation into the associations between the psychosocial learning environment and students’ enjoyment of distance education. The results yielded a new six-scale, 34-item Web-based learning environment instrument suitable for use in a number of asynchronous post-secondary distance education environments. The new instrument, the Distance Education Learning Environment Survey (DELES) assesses Instructor Support, Student Interaction and Collaboration, Personal Relevance, Authentic Learning, Active Learning, and Student Autonomy. Analyses of data obtained from 680 subjects supported the factorial validity and internal consistency reliability. The results also indicated statistically significant associations between the distance education learning environment and student enjoyment of distance education. iv ACKNOWLEDGEMENTS The undertaking and successful completion of a project of this nature is only possible with the assistance of several people. I would like to thank the following people who have been involved in this process. I would like to express thanks and appreciation to my principal supervisor, Professor Barry J. Fraser, without whom the completion of this project would have never been possible. Despite the 16,800 km (10,439 mi.) distance between San Marcos, Texas and Perth, Western Australia, the world became a smaller place with the prompt attention, guidance, and editorial direction I received. I would also like to extend a ‘thank you’ to the administrative staff at the Curtin University of Technology, Science and Mathematics Education Centre. Without the aid of folks like Petrina du Plessis and Rosalie Wood, I would never have been able to handle the more mundane administrative parts of my educational process from such a distance. Also having a significant hand in this undertaking was Jacquelyn Alexander, PhD, formerly the dean of my school at Our Lady of the Lake University (OLLU). Dr. Alexander manipulated the higher education system in mysterious ways to keep me employed as I worked toward this goal. When I had the fleeting notion 10 years ago to aim toward a doctorate, unbeknownst to him, my step-father pointed the direction that my mother, unbeknownst to her, had laid as a foundation. Thanks go to Jackie Elliott, EdD, and Althea Walker Elliott, for the examples they set. Finally, I must thank my wife, Kelly, and my children, Chloe and Quinton, for the role they have played in allowing me substantial time to think and write and work when I could have been with them instead. Without their understanding, none of this would have been possible. v TABLE OF CONTENTS Abstract iii Acknowledgements iv List of Tables x List of Figures xi Chapter 1 – Background and Rationale 1 1.1 Introduction: Distance Education in Universities 1 1.2 Distance Education Terms and Usage 4 1.3 Learning Environments 4 1.4 Aims of this Study 6 1.5 Significance of this Study 6 1.5.1 Significance in Distance Education 7 1.5.2 Significance in Higher Education 8 1.5.3 Significance in the Field of Learning Environments 9 1.5.4 Summary of the Discussion of Significance 10 1.6 Overview of Other Thesis Chapters 11 Chapter 2 – Literature Review 12 2.1 Introduction 12 2.1.1 Terminology 13 2.1.2 Background to Learning Environments Research 14 2.1.3 Need to Describe Learning Environments in 15 Today’s Rapidly-Expanding Higher Education Market 2.2 Evolution of Psychosocial Environments Research 18 2.2.1 History of Psychosocial Environment Research 18 2.2.2 Environmental Press 19 2.2.3 Contemporary Research on Psychosocial 21 Environments 2.2.4 Evolution of Learning Environments Research 23 2.2.5 The Influence of Learning Environments on 25 Student Outcomes vi 2.2.6 Learning Environment Instruments 26 2.2.6.1 Science Laboratory Environment 28 Inventory (SLEI) 2.2.6.2 Constructivist Learning Environment 29 Survey (CLES) 2.2.6.3 What is Happening in this Classroom 30 (WIHIC) Questionnaire 2.2.6.4 Questionnaire of Teacher Interaction 32 (QTI) 2.2.6.5 Other Contemporary Learning 33 Environment Instruments and Their Uses 2.2.7 Categories of Learning Environments Research 34 2.2.7.1 Differences Between Students’ and 35 Teachers’ Perceptions of the Same Class 2.2.7.2 Cross-Cultural Learning Environments 37 Research 2.2.7.3 Associations Between Student Outcomes 39 and Learning Environments 2.3 Distance Education Research 43 2.3.1 Distance Education Learning Environments in 46 Distance Education Research 2.4 Distance Education Learning Environments Research 48 2.4.1 Technology-Oriented Learning Environments 49 Research 2.4.2 The First Look at Distance Education Learning 50 Environments 2.4.3 Subsequent Distance Education Learning 50 Environments Research 2.4.4 Emerging Distance Education Learning 51 Environments Research 2.5 Student Satisfaction 52 2.6 Chapter Summary 55 vii Chapter 3 – Research Methods 57 3.1 Introduction 57 3.2 Categories of Analysis 57 3.3 Data Collection 60 3.4 Overview of the Stages in the Development of the 62 Distance Education Learning Environments Survey (DELES) 3.4.1 Overview of Stage 1 – Identification and 62 Development of Salient Scales 3.4.2 Overview of Stage 2 – Writing Individual Items 63 3.4.3 Overview of Stage 3 – Field Testing and Analyses 63 3.5 Stage 1 – Identification and Development of Salient Scales 63 3.5.1 Review of Scales from Previously-Developed 63 Instruments and Development of New Scales 3.5.2 Strategies for Developing New Scales and 64 Adapting Previous Scales 3.5.3 Review of and Consistency with the Literature 66 Related to Distance Education Learning Environments 3.5.3.1 Relationship Dimension 66 3.5.3.2 Personal Development Dimension 67 3.5.3.3 System Maintenance and Change 68 Dimension 3.5.4 Preliminary Scales 68 3.5.5 Review of Preliminary Scales by a Panel of Experts 68 3.5.6 Outcomes of Stage 1 71 3.6 Stage 2 – Writing Individual Items 71 3.6.1 Consideration of Negatively-Worded or 72 Reverse-Scored Items 3.6.2 Validating Items 73 3.6.2.1 Expert Panel’s Comments 73 3.6.2.2 Development of the Web-Based Form 74 3.6.2.3 Pilot Test 74 3.6.3 Final Version of the DELES 75 3.6.4 Scale of Student Enjoyment 78 viii 3.7 Stage 3 – Field Testing and Analysis 78 3.7.1 Field Testing 79 3.7.2 Field Test Sample 79 3.7.3 Item Analysis and Factor Analysis 80 3.8 Chapter Summary 80 Chapter 4 – Data Analysis and Results 83 4.1 Introduction 83 4.2 Reliability and Validity of the DELES 83 4.2.1 Validity 84 4.2.2 Reliability 89 4.2.3 Limitations of the Data Collected 90 4.3 Final Version of the DELES 91 4.4 Associations Between Student Enjoyment and Learning 91 Environment Factors 4.4.1 Associations Between the DELES Scales and 91 Students’ Enjoyment of Distance Education 4.4.2 Discussion of the Strongest 94 Enjoyment-Environment Association 4.4.3 Discussion of the Weakest 94 Enjoyment-Environment Association 4.4.4 Consistency with Previous Research on 95 Associations Between Attitudes and Learning Environments 4.4.5 Limitations of Analysis of Enjoyment-Learning 97 Environment Associations 4.5 Conclusion 97 Chapter 5 – Conclusions, Implications, and Recommendations 100 5.1 Introduction 100 5.2 Overview of this Thesis 100 ix 5.3 Major Findings 102 5.3.1 Development and Validation of a New 102 Learning Environment Instrument for Post-Secondary Distance Education 5.3.2 Investigation of Associations Between Enjoyment 103 of Distance Education and Six Psychosocial Learning Environment Scales 5.4 Distinctive Contributions of My Study 104 5.5 Constraints and Limitations 106 5.6 Recommendations for Future Research 107 5.7 Concluding Comments 108 References 110 Appendix A – Scale and Item Review Panellists 137 Scale Review Panellists 137 Item Review Panellists 138 Appendix B – Final Revised Version of the Distance Education 140 Learning Environment Survey (DELES) Appendix C – Satisfaction Scale of Enjoyment of Distance 143 Education x LIST OF TABLES Table 3.1 Frequency of DELES field test responses by level of study 60 being undertaken Table 3.2 Preliminary set of DELES scales before review by a panel of 69 distance education experts Table 3.3 Learning environment scales remaining after review and 70 reduction by expert panel Table 3.4 Scale descriptions and sample items after pilot test 76 Table 3.5 Items in the Enjoyment scale 78 Table 4.1 Factor loadings for a refined 34-item version of the DELES 86 Table 4.2 Scale reliability using Cronbach’s alpha coefficient 89 for DELES and Enjoyment scales Table 4.3 Listing of items in the refined version of each Distance 92 Education Learning Environment Survey (DELES) scale Table 4.4 Associations between six DELES scales and student 93 enjoyment using simple correlation and multiple regression analyses xi LIST OF FIGURES Figure 2.1 Diagram of the relationship between the two distinctive 13 areas of research in social ecology and distance education Figure 2.2 Classification of press in terms of positive and negative 20 characteristics Figure 2.3 Classification of press in terms of beta and alpha 21 characteristics Figure 3.1 Categories and levels of analysis 58 1 Chapter 1 BACKGROUND AND RATIONALE 1.1 INTRODUCTION: DISTANCE EDUCATION IN UNIVERSITIES Distance education has become a firmly embedded part of the higher education landscape over the last decade. Networked digital communication has facilitated an explosive growth in this relatively new method of reaching learning populations to the point that the higher education trend to produce distance education units and programs has been referred to as a “land rush” (Molenda & Harris, 2001, p. 6) to get online. Molenda and Harris note that 60 percent of all United States colleges and universities have Internet-based courses. Meanwhile, the United States Army currently has over 15,000 soldier-students enrolled in its University Access Online and plans to increase enrolments to 80,000 by the end of 2005 (Lorenzo, 2002). At the same time, Australian universities are experiencing converging influences driving them “towards online, for profit education on a global scale” initiated by a “desperate need to improve income to compensate for the lack of public funding” (Gururajan, 2002, ¶ 2). This issue is of sufficient consequence that it prompted former Deakin University Vice-Chancellor, Malcolm Skilbeck, to pose the question “Does the university have a future?” (Skilbeck, 2001, p. 61). While change is inevitable in higher education, regardless of the forces driving it, quality in education must remain high, lest universities lose their status as degree- granting bodies and become little more than market players in the global corporate milieu. Fifth-generation distance education, which is defined by the convergence of telecommunications and computing technologies available today (Taylor, 2001), has come to a point where higher education chances falling prey to “the commoditization of the educational function of the university, transforming courses into courseware, the activity of instruction itself into commercially viable proprietary products that can be owned and bought and sold in the market” (Noble, 1998, ¶ 6). In consideration of this statement, universities must consider not only how distance education is presented in terms of its value in the market, but also in terms of its value to the learner (Morgan & McKenzie, 2003). 2 Many distance education classes and seminars are modelled after a traditional face- to-face, instructor-centred, deductive perspective on teaching (Diaz, 2000; Palloff & Pratt, 1999)—the same instructional model used by the ancient Greeks. In this classical mode of education, an instructor is positioned as the expert oracle espousing knowledge to a group of students who produce products to be evaluated by the expert. Discussion between the expert instructor and the student is often couched in terms of the value of the products that students produce in relation to the expert’s opinion. In the light of the fissure between how education is classically delivered, what we know about how people learn (Palloff & Pratt, 1999), and trends in how distance education can become commoditized, the question at hand becomes: “What leads to successful teaching and learning in distance education?” When teaching and learning leaves the four walls of a classroom and enters an electronic environment, perhaps a new education paradigm must be adopted in order to facilitate student success. Successful students lend credence to what makes higher education unique. “In this new e-learning environment there is no viable option for the university to do as it has always done. To do so will be to become more and more marginalised” (Spender, 2001, p. 25). Distance education is more than the simple transfer of an existing instructivist’s verbal lecture to an electronic textual/image/audio environment which is enclosed within the pre-packaged structure of an off-the-shelf Internet-based course administration application. A critical component in any distance education environment is the human factor (Morgan & McKenzie, 2003). The role of people and the interaction between people in the distance education environment is essential to the development of a high functioning distance education class (Palloff & Pratt, 1999). “Future development in online education will require universities to re- evaluate the pedagogies of the campus learning environment” (Commonwealth Department of Education, Science & Training, 2002, ¶ 52). Computer-mediated distance education classes have a distinctive social structure, unlike those found in a face-to-face class. This social structure has a strong impact on students’ learning and students’ satisfaction with the class and the method by 3 which the class is presented in the digital world (Swan, 2001). There is a firm need for a social connection and a sense of presence in electronically-mediated distance education (Palloff & Pratt, 1999). Advantageous to distance education is the fact that distance and social class play a limited role in student interaction. Social interaction is limited only by time and access. In the asynchronous distance education setting, participants can become a part of a social milieu with anyone who has access to a computer. “Connections are made through sharing of ideas and thoughts” (Pallof & Pratt, 1999, p. 15). Furthermore, it is through personal relationships and interactions between participants that learning is developed. Because the psychosocial environment of the distance education classroom is quite different from that of a face-to-face or place-based class, it must be cultivated and developed in order to become an effective medium for education. Diaz (2000) suggests that the current focus on distance education research is wholly framed within a dichotomy between instructor-centred and student-centred learning perspectives that leads to a black-and-white view of instruction that supports passive learning and a strong dependency on students conforming to a dependent (passive) style of learning. The area of study distinctively missing from the body of research related to distance education involves the character of the learning environment and what types of distance learning environments are successful (Diaz & Cartnal, 1999; Harnish & Reeves, 2000; Lane, n.d.). Nevertheless, emerging literature suggests that this trend is shifting as more and more components of distance education learning environments are being considered in recent and emerging research (Graham & Scarborough, 2001; Murphy & Cifuentes, 2001; Swan, 2001; Tu & Corry, 2001). My study was designed to consider the character of the post-secondary distance education environment in terms of what researchers and practitioners find influential and to seek associations between the psychosocial environment and the attitudes of students. To begin, this chapter defines terms as they are used in this thesis (Section 1.2). Then it briefly introduces the notion of learning environments (Section 1.3), and this is followed by the aims of this study (Section 1.4). The significance of this study (Section 1.5) is described in terms of learning environment research in distance education, learning environments in general, distance education research, and research in higher education. I then conclude the chapter (Section 1.6) with an 4 outline of the remainder of this thesis—providing readers with a comprehensive overview of the work at hand. 1.2 DISTANCE EDUCATION TERMS AND USAGE The term ‘distance education’ is used to define instructional delivery that does not restrict students to being physically present in the same location as an instructor (Steiner, 1995). For clarification, it should be noted that multiple modes of distance education are available. These modes can be categorised as those that are synchronous, requiring students and their instructor to gather at some place, or places, at a given time, and those that are asynchronous, for which students and their instructor are not confined to time and place. Asynchronous distance education includes, but is not limited to, delivery of instruction using the postal system to mail text-based materials, audio or video cassettes or CD-ROMs, computer-based conferencing, Web-based bulletin boards, or e-mail (Leach & Walker, 2000). Meanwhile, synchronous forms of distance education include those delivered by Web-based chats, Internet relay chats, multiple-user domains (MUDs), satellite television broadcasts, radio broadcasts, audio-conferences, and two-way videoconferences, which are occasionally referred to as interactive television (Steiner, 1995). This study focuses on education delivered by any means of electronic, computer-mediated asynchronous Internet communication, irrespective of the arrangement, software, or equipment used. Given the above distinctions, much of the literature does not distinguish between the delivery modes used in distance education; rather, authors interchange terms such as ‘distance learning’, ‘distance education’, ‘open learning’, and ‘distributed learning’. Therefore, the intentions of the authors and reported results are occasionally difficult to interpret when reviewing the associated body of literature. 1.3 LEARNING ENVIRONMENTS Learning environments studies capture education participants’ perspectives of the psychosocial environments in which they participate on a day-to-day basis. As participants deeply involved in classroom environments, students and instructors 5 have unique viewpoints of what goes on in that environment (Fraser, 1998b). Capturing and contrasting student and instructor perspectives, which often differ, and considering those perspectives in relation to observations of external investigators is the essence of learning environments research. Herbert Walberg and Rudolf Moos, in independent research, developed and formalized learning environments research. Walberg developed the Learning Environment Inventory (LEI) (Walberg & Anderson, 1968) to capture student perspectives of their learning environments in classes in which Harvard Project Physics curriculum materials were being used. Moos (1974, 1976) conceptualised universal social climate dimensions omnipresent in social organizations such as health institutions, correctional institutions, and schools. He later developed the Classroom Environment Scale (CES) to capture teacher behaviour, teacher-student interaction, and student-student interaction in schools (Moos, 1979) using his social climate dimensions as a framework. These early learning environments instruments established a historical benchmark on the face of learning environments research that continues to expand today with considerations of associations between student learning outcomes and learning environments, cross-cultural/cross-national studies, evaluation of educational innovations (Fraser, 2002a), and multi-level organization analyses (e.g., classroom studies, school studies, studies of students as they move to higher grade levels, personal student perceptions and shared student perceptions) (Fraser, 1998b). An evolving learning environments research trend involves study of the fast- changing distance education learning environment. Few studies have been conducted exclusively on distance education learning environments, although numerous studies have focused on other areas of distance education. This study supports the merging of two previously distinctive fields of study, distance education research and learning environments research, which began as a realisation for the need of this sub- discipline only a decade ago (Jegede, 1992). Chapter 2 (Section 2.2) expands on this introduction to learning environments research by presenting the evolution of this area of study, the influence that learning environments have on student outcomes, and a detailed outline of numerous learning 6 environment instruments that have been developed over the last three decades to aid investigators in measuring various aspects of learning environments. 1.4 AIMS OF THIS STUDY This study essentially involved the development and validation of a new learning environment instrument for distance education at the post-secondary level. While numerous learning environment instruments exist, only three are available specifically for this level and method of education (Chang & Fisher, 2001a; Jegede, Fraser, & Fisher, 1998; Taylor & Maor, 2000). The main purpose of this study was to facilitate the investigation of psychosocial aspects of post-secondary asynchronous distance education that support social and inductive perspectives of learning by developing and validating a new learning environment instrument. A secondary aim was to investigate associations between student satisfaction and the nature of the distance education learning environment in order to identify what psychosocial characteristics influence the affective aspects of distance education students. 1.5 SIGNIFICANCE OF THIS STUDY Three learning environment instruments are currently available for higher education distance education—the Distance and Open Learning Environment Survey (DOLES), the Constructivist On-Line Learning Environment Survey (COLLES), and the Web Based Learning Environment Instrument (WEBLEI). The DOLES focuses on Internet-delivered education by means of five core scales (Student Cohesiveness, Teacher Support, Personal Involvement and Flexibility, Task Orientation, and Material Environment) and two optional scales (Communications Technology Resources and Study Centre Environment) (Jegede, Fraser, & Fisher, 1998). The COLLES considers the quality of online learning environments from a social constructivist perspective with six scales (Relevance, Reflection, Interactivity, Tutor Support, Peer Support, and Interpretation) (Taylor & Maor, 2000). The WEBLEI focuses exclusively on Web-based learning (Chang, 1999; Chang & Fisher, 2001a) 7 by means of four scales (Emancipatory Activities, Co-participatory Activities, Qualia, and Information Structure and Design Activities). The availability of the new instrument resulting from my study, the Distance Education Learning Environments Survey (DELES), will enable practitioners and researchers to examine educational learning environments in tertiary education settings in a different way. The DELES has the potential to provide opportunities to collect information as outlined below: • Learning environment instruments are economical in that they do not require trained observers and time-consuming coding (Jegede, Fraser, & Fisher, 1998). And, unlike the DOLES and the WEBLEI, the DELES can easily be employed by practitioners, evaluators, and researchers alike because it is in the form of a Web-based instrument—available to anyone with a Web browser. Further, because it is a Web-based instrument, data need not be transcribed from paper response forms. This offers fewer opportunities for errors in data collection. • The learning environment can be assessed in terms of beta press, that is, from the perceptions of the learner, rather than exclusively in terms of alpha press, or that which is assessed by a detached third party (Murray, 1938). This offers the dual advantage of characterising a setting through the eyes of both the observer and of the participants, thus collecting information that an observer could overlook or deem insignificant (Fraser, 1998a). • Student and instructor perceptions can be contrasted with those of external evaluators or observers by means of this instrument. • The DELES assesses the new psychosocial dimensions of Student Interaction and Collaboration, Authentic Learning, Active Learning, and Student Autonomy, which are not included in existing learning environment instruments. 1.5.1 Significance in distance education In the context of the larger research setting, in an era of an increasing call for national and international distance education, researchers and evaluators will be able 8 to use the learning environment instrument developed in my study to complement distance education research that currently focuses primarily on student achievement, attitudes and behaviours (Harnar, Brown, & Mayall, 2000), student participation (Grasinger, 1999), the role of technology in a distance education environment (Jamieson, 1999), the ‘no significant difference’ between traditional vs. distance education argument (Lane, n.d.), and the ‘how-to’ reports of instructional methods found predominantly in conference proceedings (Squire & Johnson, 2000). 1.5.2 Significance in higher education What is most significant about this study in terms of higher education is that e- learning, a corporate term for electronic learning, has developed to the point that education as a consumer-based commodity (Connell, 2001) is creating new demands upon universities to become “market actors” (Traub, 2000, p. 50) in what is predicted to be a US$23 billion market in the United States alone by 2004 (Kercher, 2002). During this period of “corporatization” (Molenda & Harris, 2001, p. 3) of universities and colleges—in which higher education institutions are increasingly operating like businesses—these institutions are experiencing pressure to become part of the profit-driven market (Tait, 2003), but they must preserve that which gives them status as unique institutions of higher learning (Rose Communications, 2001). Trends also point toward diminishing instructor autonomy and governance in terms of instructional direction (Molenda & Harris, 2001). Post-secondary distance education, which is not grounded in high-quality teaching and learning theory, with respect to social and psychological determinants and effects of the environment in which they are presented, stands to become diminished, appearing less of a scholarly pursuit and more “work-for-hire” oriented (Traub, 2000, p. 5), thus blurring the lines between university status and business profit making. Finally, as pressure increases on higher education instructors to teach in some form of distance model, those with several years of experience teaching in face-to-face classrooms will need to develop new models and methods if they are to continue to provide the high-quality teaching demanded by a global education market (Slay, 1998). The DELES, once applied, is likely to assist instructors in improving their 9 teaching, which aids in maintaining the historical significance of the university, leading to improvements in post-secondary distance education as a whole. 1.5.3 Significance in the field of learning environments In terms of other learning environment instruments that are already established, and those that are emerging, the DELES will lend itself to the “bricolage of theories, methods, and prior histories” (Tobin, 1998, p. 130) that shape how we view learning environments by adding a distance education learning environment instrument. Higher education institutions are beginning to see distance education as more than an add-on to the ways in which teaching and learning have traditionally been conducted. They are beginning to consider distance education as a genuine alternative form of education with its own audience, purposes, and patterns (Clinton, 2002). My study addresses this phenomenon by approaching it as a new direction for learning environments research. Today aspects of the ever-growing body of learning environment research looks deep into traditional education environments. However, studies in distance teaching and learning are relatively new in learning environment research. Modifications of learning environment questionnaires suited for use in cross-cultural applications, the expansion of instruments suited for use at different education levels (Fraser, 2002a), and the growth of learning environment research specific to ever-expanding subject areas (e.g., Goh & Tobin, 1999; Teh & Fraser, 1994; Zandvliet, 2002) are examples of the depth of learning environment studies. Meanwhile, however, other aspects of learning environment research must focus on altogether new ways of teaching and learning that are embedded in new learning environments such as asynchronous distance education. The DELES, while perhaps suitable in some aspects for traditional learning environments, was developed with new ways of teaching and learning in mind. While the steps in the development and validation of the DELES followed precedent set during the development of previous learning environment instruments, little of this instrument was adapted from traditional instruments themselves. The vast majority of the DELES is derived exclusively from considerations of fifth-generation (Taylor, 2001) distance teaching and learning, which is a relatively recent line of learning environment research. 10 1.5.4 Summary of the Discussion of Significance These three significant areas of contribution—toward distance education, higher education, and learning environments research—offer the education community a new and unique perspective on distance education learning environments in higher education through a combination of topics that have yet to be considered in this way. While there is an expectation that information technology will significantly contribute to teaching and learning (Cuban, 2001; Mioduser, Nachmias, Lahav, & Oren, 2000), it is only what goes on in a technology-rich learning environment, in this case a post-secondary distance education environment, that will ever truly make a difference in how people learn (Cookson, 2002). Technology and telecommunications in education, in and of themselves, do not improve education. It is how they are used that make the largest contributions to education (Harris, 1998). Distance education has been characterized as taking us “one step ahead for the technology, and two steps back for the pedagogy” (Mioduser et al., 2000, p. 73). Conceivably, we are experiencing what has been termed a cultural lag (Ogburn, 1964)—the difference between innovation and the way in which a culture reacts to it. For instance, at one point in history, automobiles were viewed as horseless carriages and the internal combustion engine was seen as a way to replace the horse rather than as a way to provide a whole new means of transportation. Steam engines were placed in sailing vessels, not to provide a primary source of locomotion, but to serve as a back-up to sail power when there was no wind (Tinker, 2000). Likewise, in some cases, distance education has been viewed as a technological extension of the traditional college or university classroom rather than as a completely new way of educating (Cookson, 2002). Given that there is a complex process of cultural maturation that must be achieved with any new way of doing things, there too must be a new ways of researching and asking questions about these new ways of doing things. Despite the newness of electronically-delivered distance education in the overall history of education, researchers are already calling for revisions in distance- education models (Diaz, 2000; Mioduser et al., 2000; Tait, 2003; Taylor & Maor, 11 2000). If nothing else, this study will contribute to distance education by looking at it from a teaching- and learning-centred perspective, a psychosocial perspective to be more precise, and not from a technocentric perspective, nor a perspective bound to traditional educational models. 1.6 OVERVIEW OF OTHER THESIS CHAPTERS Chapter 2 reviews literature related to learning environment research as a body of knowledge, how learning environments can be studied, a brief history of learning environment research instruments, and distance education learning environments. Chapter 3 details the methods that I used in the design and development of a learning environment survey instrument intended exclusively for distance education in higher education. This includes a report of three stages in the development and validation of the learning environment survey instrument: (1) identification and development of salient scales; (2) writing individual items; and (3) field testing the instrument and analysing the resulting data. Chapter 4 is a presentation of the results of analyses of the data utilized to validate the instrument and to investigate associations between the learning environment and student enjoyment of distance education. Chapter 5 outlines the conclusions and limitations related to the instrument validation, implications of the research, and recommendations for future research related to the Distance Education Learning Environment Survey. 12 Chapter 2 LITERATURE REVIEW 2.1 INTRODUCTION This chapter provides a review of literature in four areas related to my present study: the evolution of psychosocial environment research; distance education research in general; distance education psychosocial learning environments; and student satisfaction. The major section (Section 2.2) concerning the evolution of learning environments research includes fundamental theories behind the study of psychosocial environments and the work that ultimately led to the study of environments in terms of education. This is followed by a presentation and description of an ever-increasing body of survey instruments that are used in learning environments assessing, as well as a discussion of several lines of learning environments research. Then, to begin linking learning environments and distance education, the next section (Section 2.3) outlines distance education research in general terms. Following on, the next section (Section 2.4) brings the two disciplines of learning environments and distance education together. It begins with a description of research on fragmented portions of distance education learning environments. This is followed by a discussion of learning environments research that takes a somewhat more holistic view of the distance education learning environment as a combined subset of distance education research and psychosocial learning environment research aimed at asking and answering questions of social ecology in terms of Internet-based distance education. The final section (Section 2.5) delves into aspects of student satisfaction, especially in terms of post-secondary education, as my study involved assessment and investigation of student satisfaction. I have selected these topics to present in this chapter because they represent the merging of two distinctively different research topics—social ecology and distance education. I start from the broadest of categories in each and move toward a specific niche suitable for contributing to the greater body of knowledge related to tertiary education. Figure 2.1 presents this convergence in which learning environment research is a sub-field of psychosocial environment research within the broader scope of social ecology. Internet-based distance education falls under the umbrella of 13 distance education in general, and narrowly-focused and incremental studies in Internet-based distance education more specifically. These two divergent research areas come together in this chapter and loosely follow this converging path culminating in the ultimate topic at hand in my study, that of post-secondary distance education learning environments. Figure 2.1. Diagram of the relationship between the two distinctive areas of research in social ecology and distance education There is a strong tradition in learning environments research to study associations between students’ perception of their psychosocial environment and affective learning outcomes such as attitude, self-esteem, and satisfaction (Fraser, 2002a; Wong, Young, & Fraser, 1997). Therefore, a final section in this chapter introduces the concept of student satisfaction in distance education, an area where any linkages between satisfaction and psychosocial environments have gone unstudied until now. 2.1.1 Terminology The term learning environment carries with it a variety of meanings. It has been used to indicate a type of learning task (Tynjälä, 1999), to denote virtual spaces found in computer applications and on the Internet (Fulkerth, 2002; Gibbs, 1999), and to refer to the classroom psychosocial environment (Henderson, Fisher, & Fraser, 2000). A 14 quick review of today’s educational psychology textbooks reveals chapters on learning environments that focus on deviant behaviour management, classroom management in terms of rules and discipline, student motivation, instructional methods, the physical layout of desks and chairs, and even the colour that the classroom is painted (Chesebro & McCroskey, 2002; Slavin, 2000; Snowman & Biehler, 2003; Woolfolk, 2001). In this study, the concept of environment, unless otherwise noted, refers exclusively to the psychosocial environment. Again though, even when the environment is overtly framed exclusively in these terms, literature reveals differing terminology to explain the concept of environment. In Owens’ (2001) textbook on organisational behaviour in educational settings, he notes the use of terms synonymous with environment that include an organisation’s atmosphere, personality, tone, or ethos. Moos used the term “social climate” and “personality of the environment” (1979, p. vii), although he notes that he was criticized for attributing personality to an environment. For the sake of consistency, I will use the term psychosocial environment throughout this study when writing generally on the topic of environment. I will use the term learning environment when referring to the psychosocial environment in education. For further clarity, I should note that this study is ultimately concerned with learning environments in asynchronous distance education. 2.1.2 Background to learning environments research Learning environments research, just over three decades old, is firmly established (Fraser, 1998a; Goh & Khine, 2002; Tobin & Fraser, 1998) among a variety of educational research and evaluation methods dominated by the assessment of students’ academic achievement (Fraser, 1998b). While quantitative measures of classroom effectiveness are often based on "narrow testable, standardized, superficial, and easily forgotten outcomes", other areas of schooling are less emphasized (Kyle, 1997, p. 851) and a complete image of the process of education is not formed within the research. In the early 1960s, Bloom pointed to measurements of educational environments as decisive components for prediction and successful learning manipulation (Anderson & Walberg, 1974). Since then, numerous studies 15 have demonstrated that students' perceptions of their educational environments can be measured with survey instruments, with their assessments being valid predictors of learning (Anderson & Walberg, 1974; Fraser, 1997, 1998a, 1998b, 2002b; Moos, 1979). Thus, evaluation turns away from individual student achievement and toward the effectiveness of the environment of the learning organization (Walberg, 1974). Moreover, variables within learning environments themselves can be manipulated to achieve different affective and cognitive learning outcomes (Anderson & Walberg, 1974). In many studies of associations between learning environments and learning outcomes, learning environments have dimensions that have consistently been identified as determinants of learning (Fraser, 1986; Khine, 2002). Learning environments which students perceive as affirmative, favourable, and fulfilling tend to lead toward increased student achievement (Chang & Fisher, 2001a). Recently, Asian researchers have adapted and cross-validated Western-originated learning environment instruments (Fraser, 2002a), demonstrating the universality of this perspective in educational research. Cross-national studies utilizing samples from multiple countries have also recently been conducted, allowing researchers to consider those perspectives with which one becomes familiar in his/her own culture to be exposed and made overt and questionable in the context of a different culture (Aldridge, Fraser, & Huang, 1999; Fraser, 2002a; Zandvliet, 1999). 2.1.3 Need to describe learning environments in today’s rapidly-expanding higher education market There are increasingly strong indicators of the need to accommodate tertiary education students in a globalised economy in order to create, distribute, and exploit knowledge for international competitive advantages (Commonwealth Department of Education, Science & Training, 2000; Hinde, 2000; Organisation for Economic Co- operation & Development, 2000; Salmi, 2000; Wagner, 1998). Many universities, large and small alike, are marketing globally (Hinde, 2000; Salmi, 2000) and developing borderless programming. Australia has become a leader in education exporting (Hanley, 2002; Hinde, 2000) due in part to late-1980s government shifts in policy that allowed full-cost tuition to be charged to international students, coupled with reductions in public funding for universities. This policy shift launched 16 Australian universities into an aggressive pursuit of overseas students, not for cultural awareness or foreign policy goals, but for market-driven pursuit of foreign income. This shift toward looking outward has led Australia’s strong distance education tradition to become one of the fastest-growing segments in Australian tertiary education (Marginson, 2002). At the opposite end of the global education market spectrum are individual universities that are not part of education exportation on a mega-scale within a national movement. Rather, they are taking advantage of micro-scale opportunities for self-sufficiency. For example, in 2000, the 121-year old University of the Incarnate Word, a private 4000-student Catholic University in the United States, opened a campus in Guangzhou, China (University of the Incarnate Word [UIW], 2002). Around the same time, this small university implemented their Universe Online program, a degree-offering distance education program (UIW, n.d.), in part to take advantage of revenue advantages of PricewaterhouseCoopers’ US$453 million eArmyU program (Lorenzo, 2002). These two actions demonstrate the same trend toward self-sufficiency as any education exporting university in Australia, yet on an individual and necessarily smaller scale. The same need for self-sufficiency on an individual, university-by-university level has been listed as one of the reasons why Hispanic Caribbean institutions in Puerto Rico, the Dominican Republic, and Cuba have taken initiatives to become involved in the global market (Canino, 2002). Marginalized countries are seeing greater tertiary education impacts spawned from the global marketplace too. Institutions such as Great Britain’s Open University and India’s Indira Gandhi National Open University, among others, have traditionally supported education in developing regions with print-based media and radio. However, these media typically lack social interaction. Advances in technology and the ability of some countries to leap frog in technology and telecommunications have created openings for newer, digital-based distance education programs to step in (Eastmond, 2000). Large-scale government-sponsored programs, such as AusAID’s Virtual Colombo Plan, which is funded as a portion of AusAID’s AU$280 million education-sector budget (Commonwealth of Australia, 2002), is set up to support substantial education exporting by means of distance education through virtual 17 universities, such as the proposed Curtin African Virtual University (Curtin University of Technology, 2002). However, despite opportunities for the expansion of borderless education in a global market, the quality of university education must remain high, lest universities become little more than knowledge-production houses feeding the global consumption of education (Traub, 2000). Traub asks, “Who will bother to come to the university’s defense once it starts behaving like every other institution in the [Wall Street] culture?” (2000, ¶ 5). Marginson (2002) suggests that Australian student-staff ratios resulting from the “commercialization of foreign education” have already led to “downward pressures in quality” (p. 24). Regarding the exportation of education to developing regions, Eastmond (2002) notes that globalised distance education is “not a panacea that will solve the world’s educational problems; in fact, if not done right, distance education will exacerbate poor quality instruction and contribute to education problems in developing countries” (Conclusion, ¶ 1). The question of what is the right way to do distance education still remans. From the perspective of asking this question, assurances of quality in education move to the forefront and must be addressed (Olsen, 2000). Learning environment research can provide some of these assurances by means of addressing what factors shape effective learning environments. Many of these factors are beginning to be considered in distance education research, yet they are often considered in isolation. In 1976, Moos wrote that “the growth of new institutional environments has increased the need for accurate descriptions of these environments” (p. 351). He went on to postulate, in reference to the events of the late 1970s, that: …currently available descriptions of social environments are inadequate. The environment is usually described as it is seen by a small and unrepresentative sample of the people in it… In addition, no ‘feel’ of how the environment actually functions is provided. (p. 351) Moos’ statements regarding ‘new environments’ and ‘currently available descriptions’ are no less pertinent today than they were in the ‘70s in regard to distance education in our contemporary globalised world order. Perhaps knowledge about distance education research and psychosocial distance education learning 18 environments will be increased somewhat by this study, which is little more than the tip of the proverbial iceberg in terms of learning environments in distance education. 2.2 EVOLUTION OF PSYCHOSOCIAL ENVIRONMENTS RESEARCH 2.2.1 History of psychosocial environment research The concept of psychosocial environment and the enduring features and processes that give social milieus their strength and vulnerability (Moos, 2002), have multidisciplinary roots extending back to the mid-1930s to the psychological field theory of Kurt Lewin and the personality theory of Henry Murray (Moos, 1976). Lewin formulated his psychological field theory from his experience with the physical theory of electromagnetic fields, combined with Gestalt psychology, to conclude that the environment influences the person and that the person influences the environment (Moos, 1976; Murray, 1938). Lewin defined his idea with the representative formula of B=f(P,E), whereby B represents behaviour, f is function, P is the person, and E is the person's environment (Lewin, 1936). Lewin noted that "every scientific psychology must take into account whole situations, i.e., the state of both person and environment" (1936, p. 12). Thus, determinants of B are describable by composite measures of P and E (Stern, 1974). Lewin's purpose for this definition was to conceptualise human behaviour with new strategies in psychological research in which functional relationships and states of interaction are emphasized over those of correlation of disjointed responses derived from isolated stimuli—the prevailing psychological trend of the time (Stern, 1974). It is perhaps noteworthy to add here that, despite 12 references to Lewin’s work in Murray’s (1938) Explorations in Personality, Murray stated regarding Lewin’s theory that, “If we were concerned with the individual merely as a unit in a field of social forces, then perhaps he might be treated as physicists treat a body: his behaviour might be represented by an arrow” (p. 73). Murray postulates that it is “dynamically pertinent” (1938, p. 120) to analyse an environment in terms of the attributes of what press is applied, rather than solely in the isolation of the present. 19 2.2.2 Environmental press Murray developed a theory that included not only the notion of one’s internal need, or personal determinants of behaviour, but also the concept of environmental press, which represents external determinants of behaviour (Moos, 1976). Press, in these terms, is the directional influence that the environment has on one’s behaviour. Press has a directional tendency with properties not obtainable by the sum of the parts of the environment or situation (Murray, 1938). As diagrammed in Figure 2.2, an environment’s press has positive and negative quantifiable aspects in terms of its ability to benefit or harm. Likewise, the press of an environment has a qualitative aspect relating to the type of effect that it has on a person. Both positive (enjoyable) and negative (not enjoyable) press can be mobile or immobile. Mobile press impacts on persons who are passive in a particular setting and can be further defined as docile or autonomous press—docile when the press is regulated by the person and autonomous when regulated by the environment. Immobile press has no impact on persons unless the person alters the press. Thus, in terms of one’s external psychosocial environment, components or characteristics of an environment, if manipulated, result in a different environment and can have an impact on those existing within that environment (Murray, 1938). Murray further defined press as being either alpha press (the actual press as far as it can be determined by an observer) and beta press (a person’s interpretation of the environment based upon his/her apperception). Beta press is what determines behaviour (Murray, 1938). The difference between the alpha and the beta press is referred to by Murray as “delusion” (1938, p. 122). What is important here has less to do with delusion and more to do with what a person does in the light of his/her perceptions in the research which I address. When a person recognizes the direction of the environment, that is, what the environment is doing to the person, s/he has pressive perception, which inevitably leads to some type of change in behaviour. Furthering this concept, when a person draws upon experience to predict what the environment could do to him/her, it is referred to as pressive apperception—leading to an unconscious reaction to the potential of the environment (Murray, 1938). Contemporary learning environment studies are 20 concerned more with the beta press in lieu of, or in addition to, the alpha press, rather than differences between the two. Pre ss Positive (enjoyable) Neg ative (not enjoyable) Mo b ile (af f ects person if pers on is pas s ive) Imm ob ile Auton o mou s (ac tiv ity is initiated by the env ironment) Do c ile (ac tiv ity is regulated by the pers on) Mo b ile Imm ob ile Auton o mou s Figure 2.2. Classification of press in terms of positive and negative characteristics Stern, Stein, and Bloom (1956) expanded Murray’s concept of press and further rationalized that people view the environment in terms of: their own highly-selective subjective perceptions that result from their experiences gained in previous situations—the private beta press; and in terms of their shared view as a member of a group immersed in a given environment—the common beta press as diagrammed in Figure 2.3 The common beta press “usually reflects some of the means by which the group maintains its orientation to reality” (Stern, Stein, & Bloom, 1956, p. 37). The common beta press and the private beta press can be different from one another and also have the potential to differ from the view of the researcher (alpha press) who is not a part of the environment. 21 Pre ss Be ta (pa rticipa nt) Alp ha (ob se r ve r) Pr ivate (pers on's perc eption) Co mm on (shared perc eption) Figure 2.3. Classification of press in terms of beta and alpha characteristics 2.2.3 Contemporary research on psychosocial environments In the 1970s, Rudolf Moos attributed an increased awareness and action related to the natural environment to an upsurge of interest in human environments. He noted that human ecologists became concerned with the way in which people grow and adapt in their environs, and that psychologists and sociologists became more concerned with creating environments that lead to the maximization of human functioning and competency (Moos, 1976). Moos had been studying environments for over a decade when, in The Human Context, he put forward “five different, yet related, conceptions of how the [psychosocial] environment works” (1976, p. 29). These conceptions are: 1) the perspective of evolution and human ecology, with environments being limiting on the actions of people; 2) the perspective of social Darwinism, with environments choosing, or favouring, people with stronger characteristics; 3) that environments motivate and challenge individuals, facilitating individual and social growth in terms of the development of civilizations; 4) a social ecological approach, with individuals seeking information about environments in order to select those with the greatest probability for success; and 5) individuals seeking to increase their control over environments in order to increase individual freedom, akin to modern philosophy in regard to our natural environment (Oelschlaeger, 1991). The integration of these five concepts led to the development of the perspective that Moos termed “a social ecological approach” (1976, p. 28) 22 designed to help us to comprehend the influence of the psychosocial environment from the viewpoint of the individual and to enhance our psychosocial environments to enrich the quality of life. From this conceptual perspective, Moos described what have long stood as environmental system domains in social ecology that can infinitely depict different environments in terms of three dimensions: 1) the Relationship Dimension, 2) the Personal Growth Dimension, and 3) the System Maintenance and Change Dimension (Moos, 1974, 2002). Through subsequent work, Moos demonstrated the enduring quality of these dimensions in terms of family, work, school, health, military, prison and community social contexts (Moos, 1976, 1979, 2002). The Relationship Dimension distinguishes the nature and strength of personal relationships. This is the “extent to which individuals are involved in the environment and the extent to which they tend to support and help each other” (Moos, 1974, p. 19). Aspects encompassed by this dimension include: cohesion, expression, support, affiliation, and involvement. The Personal Development Dimension is characterized by personal growth and self-enhancement opportunities offered by the environment. Aspects related to this dimension include: independence, achievement, task orientation, self-discovery, anger, aggression, competition, autonomy, and personal status (Moos, 1974). The System Maintenance and Change Dimension considers the degree of control of the environment, the orderliness, clarity in expectations, and responsiveness to change. Aspects characterizing this dimension include: organization, control, order, clarity, innovation, physical comfort, and influence (Moos, 1974). Moos suggested too that the System Maintenance and Change Dimension is well suited for making cross- cultural comparisons of social organizations. It is through the framework of these dimensions that investigators can begin to characterize and integrate the impacts that social environments have on individuals and groups. Environments, especially when couched in terms of Murray’s concept of temporal tendencies, tend to preserve the individual characteristics that are compatible with their prevailing aspects (Moos, 2002). The settings in which we find ourselves, in families, schools, work, etc. are “ubiquitous in everyday life, are highly salient for the people who live and work in them, and exemplify how individuals 23 construct and select life contexts that profoundly influence their morale and behavior” (Moos, 1996, ¶ 6). When participants in an environment are offered information about their environment, opportunities for adaptation to the environment can affect the participants’ expectations of the social milieu. Further, given information about the social climate of an environment, participants have potential opportunities to alter their environment positively to promote productivity within it. Likewise, when stakeholders participating in an environment are supplied with information on what is an ideal environment, they can use that information to shape their own environment toward the goal of making it an ideal environment. “Practical applications of the concept of social climate”, such as in school settings, “make it one of the most exciting and potentially useful ways of characterizing environments” (Moos, 1976, p. 352). Research into psychosocial environments must be ongoing due to the nature of ever- changing environments. “We have not reached the goal Murray (1938) espoused more than 60 years ago: A [sic] common taxonomy of individuals’ needs and environmental press that enables us to identify the presence of a person-environment match” (Moos, 2002, Enigma III section, ¶ 2). This is due, in part, to the fact that research on psychosocial environments faces the dilemma that environments themselves “are likely to have only evanescent effects because they are superseded by the demands of new environments” (Moos, 1996, ¶ 17). The study for which this review of the literature provides support involves one of these new environments— the distance learning environment. 2.2.4 Evolution of learning environments research Following the progression of research in psychosocial human environments generally, I now consider its evolution into the specific domain of educational environments. Hartshorne and May (1928) and Newcomb (1929) similarly noted that students’ behaviour could be altered by the environment in which they were immersed. Hartshorne and May verified that personality traits were poorly correlated to students’ tendency to participate in deceitful behaviour, such a cheating on examinations, given the opportunity in differing situations. Newcomb noted that students’ talkativeness during lunch periods was a highly stable trait; however, the 24 same trait did not carry over to other situations. These early studies demonstrated that investigators must consider the environment in which behaviour takes place in order to predict individual student actions, because students’ values change according to the expectations of the setting (Moos, 1979). Nearly three decades after these early studies, Pace and Stern (1958) recognized and investigated the association of major fields of study with social climates in institution-wide college and university settings, in part by developing and implementing the 30-scale, 300 true-false item College Characteristics Index (CCI). Pace (1962, 1967) measured the five social climate scales of practicality, community, awareness, propriety, and scholarship through the development and implementation of the College and University Environment Scale (CUES). He searched for associations between the degree by which the environment impacts intellectual capacity and academic competition. However, these studies were conducted at a broad level, involving analysing whole institutions. While important in the evolution of studies in learning environments, the scope needed to be more narrowly focused. Walberg and Moos, independent of one another, began considering psychosocial environments and their influence on student outcomes in the late 1960s and early 1970s. Their work can be considered the “starting points for contemporary classroom environment research” (Fraser, 1990, p. 201) that “took off in the 1970s” (Tobin, 2000, p. 223). Methods of studying learning environments during that time, and perhaps still today, can be distinguished as conforming to three forms: 1) elaborate coding schemes for teacher and student activities, 2) “global observation scales,” and 3) “perceptual indexes” (Moos, 1979, p. 138). Self-report perceptual indexes focusing on classroom environments included peer judgement and teacher nomination in the Classroom Climate Inventory (CCI); the Learning Structure Questionnaire (LSQ) based on dimensions of teacher-centeredness, class- centeredness, and self-directed dimensions; Walberg’s widely-used Learning Environment Inventory (LEI) focusing on cohesiveness, friction, speed, and disorganization (Anderson & Walberg, 1974; Fraser, 1986); and the Classroom Environment Scale (CES) developed by Trickett and Moos that considers teacher behaviour, teacher-student interaction and student-student interaction (Moos, 1979). 25 Until the introduction of the CES, though, perceptual indexes “lacked the guidance of theoretical or conceptual frameworks producing isolated findings that are difficult to organize into a coherent body of knowledge about classroom functioning” (Moos, 1979, p. 138). The CES, and the numerous instruments that followed, defined the classroom environment in terms of the shared perceptions of the participants (common beta press), rather than those from outside observers’ views alone (alpha press). Students, with their distinctive frame of reference generated from spending numerous hours as learners, have a large interest in what is going on around them in their educational environments "and their reactions to and perceptions of school experiences are significant" (Fraser, 1998b, p. 527) given that environments, like people, take on distinctive personalities (Insel & Moos, 1974; Kiritz & Moos, 1974). Moreover, students have the advantage of familiarity with differing learning environments and have distinctive impressions of classroom environments (Moos, 1979). This point of shared perceptions, coupled with the framework of Moos’ (1974) universal environment dimensions of Relationship, Personal Relevance, and System Maintenance and Change investigated by means of a perceptual index, led to a solid theoretical structure for considering psychosocial environments in educational settings. 2.2.5 The influence of learning environments on student outcomes Walberg led classroom environment research further by posing two important questions. “Do the educational means, that is, the manipulation of the environment, justify the ends?” “If so, to what extent or with what degree of productivity?” (Walberg, 1981, p. 81). Walberg suggested that, where B=f(P, E) was adopted in psychological settings to explain behaviour, researchers in education had, unconditionally perhaps, adopted L=f(A, T) where learning (L) is a function of an individual’s aptitude (A) and the instructional treatment (T). He suggested though that the factors that are not manipulated in the natural setting of education, those being factors of the environment, carry more weight than deliberate instruction. Therefore, he presented the formula L=f(A, T, E) to include the learning environment (E) (Walberg, 1981). This supposition has been supported by others in terms of 26 student assessment being incomplete without consideration of the classroom context (Englemann, Granzin, & Severson, 1979; Ysseldyke & Elliott, 1999). In Walberg’s same theory of educational productivity work, he postulated that psychosocial characteristics in classrooms provided valid indicators of factors of student achievement, and perhaps even goals of their own accord to balance academic measurement by means of test scores alone (1981). Others have followed suit in agreeing that there is an association between psychosocial characteristics of classrooms and student learning achievements and viewpoints (Fraser, 1998a) (see Subsection 2.2.7.3). Moos (1979) cited demonstrations of this linkage by several researchers, including Bennett, Epstein and McPartland, Kennedy, Forman and McKinney, Fraser, O’Reilly, Soloman and Kendall, and Stallings. Moos also made the point that, given that there are real, observable outcomes that can be manipulated through environments, teachers can alter the social environment within their classrooms to influence outcomes. By following the four steps of 1) systematic learning environment assessment, 2) feedback to teachers with practical issues stressed, 3) implementation of positive changes, and 4) reassessing the classroom environment in a perpetual loop, social climate changes could be made (Fraser, Sinclair, & Ledbetter, 2001; Moos, 1979). This early extensive body of research on learning environments demonstrated that instructors, cognizant of the learning environments that they support, could utilize classroom environments studies to discover differences between their perceptions and those of their students and then attempt to make improvements in the actual classroom environment, based upon the preferences of students, to lead to increased productivity and achievement. Later, as we are about to see in Section 2.2.6 researchers developed highly-economical, easy-to-implement, easy-to-score inventories such as the My Class Inventory (MCI) to aid teachers in their own assessment of what is going on in their classrooms (Fraser, 1998b). 2.2.6 Learning environment instruments Although students must be able to demonstrate measurable content skills, education consists of more than curricular content and quantifiable student outcomes. The 27 quality of the environment in which students learn plays a paramount role in those things that we desire for education (Fraser, 1986, 2002b). While classroom learning environments can be altered with the intention of enhancing student outcomes, as previously noted (see Section 2.2.5), education practitioners and researchers must have a means by which they can measure the learning environment before they can enact any changes in that environment that will lead to improving the effectiveness of education. Qualitative observation, inquiry, ethnography, student and teacher interviews, and case studies, among other qualitative and subjective forms of assessment and evaluation, have commonly been used by researchers to gather information on educational environments. However, in order to bridge the gap between the third-party observer/researcher’s views and the students’ and teachers’ own perceptions of what goes in on their environments, a less subjective, qualitative, and economical means of measuring the learning environment exists through the use of learning environment survey instruments. This alternative research method is based on validated, efficient, and broadly relevant questionnaires that students and teachers complete for researchers’ gathering of perceptions of learning environments from the stakeholders’ perspectives (Fraser, 1998a). There has been a “prolific development of questionnaires” (Tobin, 2000, p. 223) in this field and investigators are able to select salient scales and the items within them for the purposes of their own studies without having to independently construct new instruments. Learning environments research instruments have been utilised in multiple countries, at multiple educational levels, and in any number of educational subject areas (Fraser, 2002b). These instruments have been utilised by “hundreds of researchers, thousands of teachers, and millions of students around the world” (Fraser, 2002b, p. vii). Early instruments used in the measurement of psychosocial environments in education include the Learning Environment Inventory (LEI), the My Class Inventory (MCI), the Class Activities Questionnaire (CAQ) (Steele, House, & Kering, 1971), and the Classroom Environment Scale (CES) (Moos & Trickett, 1974). The LEI, patterned after Hemphill's Group Dimensions Description Questionnaire, was established in the 1960s (Anderson & Walberg, 1974; Fraser, 1998a). It assumes that the students, as well as the teacher, are determinants of the 28 learning environment (Steele, House, & Kering, 1971). The MCI is a simplified version of the LEI, adapted for use with younger children aged 6-12 years. The CAQ was constructed to measure Bloom's six-level taxonomy (Anderson & Walberg, 1974) consisting of: knowledge, comprehension, application, analysis, synthesis, and evaluation. Unlike the LEI, MCI, and CAQ that considered only the students’ perceptions of the actual environment, the CES was designed for use in secondary classrooms with several forms of the instrument: the student expected form for use in a new class, the student preferred (“ideal”) form, the student actual form, the teacher preferred, and the teacher actual (Moos & Trickett, 1974). Meanwhile, the College and University Classroom Environment Inventory (CUCEI) focused exclusively upon perspectives at the post-secondary level with multiple forms (Fraser, Treagust, & Dennis, 1986). Instruments that are more contemporary than those briefly reviewed also are numerous and ever growing. The following subsections describe several influential instruments (Fraser 2002a) that include the Science Laboratory Environment Instrument (SLEI) (Fraser, Giddings, & McRobbie, 1992), the Constructivist Learning Environment Survey (CLES) (Taylor, Fraser, & Fisher, 1997), the What is Happening in this Classroom (WIHIC) questionnaire (Aldridge, Fraser, & Huang, 1999), and the Questionnaire on Teacher Interaction (QTI) (Wubbels, 1993). These subsections are then followed by a brief presentation of some other instruments. 2.2.6.1 Science Laboratory Environment Inventory (SLEI) The SLEI is geared toward the science laboratory classroom experiences of secondary and post-secondary students for whom the science laboratory is a separate class (Fraser, Giddings, & McRobbie, 1992). The SLEI, cross-nationally field tested with 5,557 students in 269 classes in six countries (Fraser, Giddings, & McRobbie, 1995), has 35 items equally divided amongst five scales of Student Cohesiveness, Open-endedness, Integration, Rule Clarity, and Material Environment. The frequency response alternatives are Almost Never, Seldom, Sometimes, Often, and Very Often, and approximately half of the items are reverse scored. The SLEI introduced the concept of a personal version to learning environment instruments to augment the class version and to aid in differentiating between within-class 29 subgroups, such as boys and girls (Fraser, Giddings, & McRobbie, 1992). Using the SLEI, Harrison, Fisher, and Henderson (1997) found they were able to differentiate between students’ perceptions in biology, chemistry and physics in three ways. Students perceived physics as more open-ended than biology or chemistry classes; chemistry classes had greater rule clarity; and physics and chemistry were more integrated than biology. The SLEI has been reported as being cross-validated in its English form in Singapore (Quek, Wong, & Fraser, 2001; Wong & Waldrip, 1996; Wong, Young, & Fraser, 1997) and Brunei Darussalam (Riah & Fraser, 1998a). A Korean-language form has also been validated from responses of 439 Korean students (Lee & Fraser, 2001; Lee & Kim, 2002) adding to the strength of this important instrument through rigorous testing and validation in multiple languages and cultures. 2.2.6.2 Constructivist Learning Environment Survey (CLES) The CLES, aimed toward secondary students, aids teachers and investigators in examining the utilisation of constructivist teaching methods and teachers’ epistemological assumptions (Fraser, 2002b). The original form of the CLES “was based largely on a psychosocial view of constructivist reform that focused on students as co-constructors of knowledge” (Taylor, Fraser, & Fisher, 1997, p. 293), yet did not consider the cultural context of the classroom under examination. Considering that the very culture in which a class exists has a strong influence on psychosocial environment perceptions and learning, the CLES was redesigned in an attempt to capture this critical element (Taylor, Fraser, & Fisher, 1997). The redesigned CLES contains the five scales of Personal Relevance, Uncertainty, Critical Voice, Shared Control, and Student Negotiation. It has 30 items, with six items per scale. The response alternatives for each item consist of Almost Always, Often, Sometimes, Seldom, and Almost Never. Departing from traditional learning environment instruments and setting a precedent for new instruments, the number of negatively-worded items was reduced to only one (Fraser, 1998b) to minimize the conceptual complexity of the instrument. 30 Like the SLEI, the CLES too has been validated in a Korean version (Lee & Kim, 2002) and an English-language version has been developed in Singapore—the General Paper CLES (GPCLES) —which adds a scale of Political Awareness and Ethic of Care (Fraser, 2002b). The five-factor CLES has likewise been translated into Chinese for use in Taiwan and demonstrates scale reliability when evaluated in an Australian-Chinese cross-national study (Aldridge, Fraser, Taylor, & Chen, 2000). It has been translated into Korean and, when administered to 1083 students in 24 classes, it replicated the original five-scale factor structure (Kim, Fisher, & Fraser, 1999). The CLES has also been found to be useful in an evaluation of systemic reform in Texas (Dryden & Fraser, 1998). 2.2.6.3 What is Happening in this Classroom (WIHIC) Questionnaire The WIHIC questionnaire, perhaps one of the most widely adopted and modified learning environment instruments (Fraser, 2002a), focuses on secondary classrooms and is designed to bring economy to the field by combining the most relevant scales from existing questionnaires (Aldridge, Fraser, & Huang, 1999) to meet the contemporary concerns of education today by considering such classroom dimensions as equity and constructivism (Zandvliet, 1999). Like the SLEI, the WIHIC too has a personal version designed to elicit students’ perceptions of their individual classroom roles, and a class version aimed at examining students’ perceptions of the classroom as a whole (Aldridge & Fraser, 1997). The WIHIC is a seven-scale instrument with eight items each in the scales of Student Cohesiveness, Teacher Support, Involvement, Investigation, Task Orientation, Cooperation, and Equity. The WIHIC’s response choices are Almost Never, Seldom, Sometimes, Often, and Very Often, with no reverse-scored items. Evidence of the widespread use of the WIHIC includes its treatment in South Africa where it has been modified for a unique practical application. Student teachers conducting their teaching practice at a distance implement the instrument, modified for their primary school mathematics classes and certain nuances of South African culture, as a means to aid the student teachers’ university supervisors in assessing their classroom teaching practices because the instructors themselves cannot be present to observe (Ntuli, 2001; Ntuli, Aldridge, & Fraser, 2003). The WIHIC has 31 also been employed and cross-validated in Brunei Darussalam (Khine, 2002; Riah & Fraser, 1998b) and in Singapore (Chionh & Fraser, 1998; Fraser & Chionh, 2000), both in English, while it has also been translated and validated in Chinese (Aldridge & Fraser, 1997; Aldridge, Fraser, Huang, 1999), Korean, (Kim, Fisher, & Fraser, 1999), and Indonesian (Margianti, 2002). It has also been used recently in India for the first time to investigate the perceptions of students in science classes (Koul & Fisher, 2002) and to investigate differences in mathematics classroom environments in Australia, Canada, and England (Dorman, Adams, & Ferguson, 2001). Further, with the aid of the WIHIC, a recent Canadian study noted differences between the perceptions of male and female students in mathematics and science classes that use laptop computers (Raaflaub & Fraser, 2002, 2003). Additional evidence of the widespread adoption of the WIHIC is apparent in a Canadian-Australian study that used the WIHIC and additional scales related to ergonomics, workspace, computer, visual, and spatial environments, to consider students’ perceptions of the psychosocial learning environment within the physical environment of computerized classrooms (Zandvliet, 2002). Attesting further to the robust nature of the WIHIC, it has been adapted as the foundation for the development of other instruments. For example, the WIHIC was used as the base for the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI). Six of the TROFLEI’s nine scales are based upon the WIHIC (Fisher, Aldridge, Fraser, & Wood, 2001). The WIHIC was also used as the basis of the Chinese Language Classroom Environment Inventory (CLCEI) in a Singapore secondary schools Chinese language study context. The CLCEI differs from the Taiwanese Chinese version of the WIHIC noted above due to differences between the way the Chinese language is used in Singapore and Taiwan (Chua, Wong, & Chen, 2001). Modifications have also been made so that the WIHIC could be used to investigate parent perceptions in conjunction with student perceptions in a primary school setting (Allen & Fraser, 2002) and to investigate classroom learning environments associated with a mentoring program for primary school science teachers (Pickett & Fraser, 2002). 32 2.2.6.4 Questionnaire of Teacher Interaction (QTI) The QTI, originally developed in the Netherlands, began with 77 items related to the nature and quality of the interpersonal relationships between students and their mathematics and science teachers (Wubbels, 1993; Wubbels & Levy, 1993). It has since been reduced to a 64-item United States version and thereafter a 48-item Australian version (Goh & Fraser, 1998; Scott & Fisher, 2001). The QTI, unlike many other learning environment instruments, relies upon Leary’s (1957) two- dimensional theoretical model and uses a circumplex ‘map’ to graph results by plotting influence (dominance-submission) along a vertical axis and proximity (cooperation-opposition) along a horizontal axis (Wubbels, Créton, & Hooymayers, 1992). The QTI contains scales that assess students’ perceptions of eight aspects of behaviour, namely, Leadership, Helping/Friendly, Understanding, Student Responsibility/Friendly, Uncertain, Dissatisfied, Admonishing, and Strict Behaviour. The response scales range from Never to Always. As with the SLEI, the CLES, and the WIHIC, the QTI has been widely used and modified to fit specific circumstances. In terms of differences in cultural discernment, the QTI has been used to examine teacher behaviour perceptions of Asian-American and Hispanic-American students (den Brok, Levy, Rodriguez, & Wubbels, 2002) and, in an altogether different adaptation, Australian students’ perceptions of teacher behaviour were examined in terms of the use of laptop computers in their classrooms (Stolarchuk & Fisher, 1998). Among these and other modifications, the QTI was tailored to assess teacher-principal interactions with the same eight scales with the Principal Interaction Questionnaire (Cresswell & Fisher, 1997). The QTI was used in Australia to examine the perceptions of senior secondary biology students (Fisher, Henderson, Fraser, 1995), confirming its validity and reliability in yet another setting. Fisher, Goh, Wong, and Rickards (1996) conducted a cross-national study of secondary science students and their teachers in Australia and Singapore using the QTI. And, it has also been adapted and used in a large-scale study in Singapore to investigate student-teacher interactions in primary school mathematics classes (Goh & Fraser, 1996, 1998, 2000). It has also been used in a 33 variety of translated versions, including a Malay version (Scott & Fisher, 2001), a Korean version (Kim, Fisher, Fraser, 2000), and an Indonesian version (Soerjaningsih, Fraser, & Aldridge, 2001). The Indonesian version was used to describe and compare post-secondary Management and Computer Science course students’ perceptions of the interactions between the students and their instructors (Soerjaningsih, Fraser, & Aldridge, 2001), further demonstrating the widespread applications of the QTI. 2.2.6.5 Other learning environment instruments and their uses While the SLEI, CLES, WIHIC, and QTI are among the more influential contemporary instruments (Fraser, 1998b), there is no shortage of other instruments that have a specific focus or that have been modified or adapted from previous instruments to serve a particular purpose. Among these are the Geography Classroom Environment Inventory (GCEI), a four-scale questionnaire intended for assessing and investigating computer-aided learning classroom environments (Teh & Fraser, 1994). The New Classroom Environment Instrument (NCEI) is a nine- scale inventory applied in classes using mathematical computer modelling (Newhouse, 1994). The Computer Laboratory Environment Instrument (CLEI) has foundations in the SLEI (Newby & Fisher, 1997) and the Computer-Facilitated Learning (CFL) environments instrument was developed for use in technology-rich university courses (Bain, McNaught, Mills, & Lueckenhausen, 1998). The Constructivist Multimedia Learning Environment Survey (CMLES) was developed specifically to evaluate constructivist-oriented learning environments that make use of interactive multi-media in teacher professional development (Maor, 1999). Meanwhile, the original College and University Classroom Environment Inventory (CUCEI), developed in 1986, was enhanced in response to the notion that students who were more involved in classroom activities might have more favourable perceptions of the classroom environment than those with less involvement (Nair & Fisher, 2001). The above review attests to the robust nature of learning environment instruments. Meanwhile, several distinctive lines of research have been commonly conducted with the aid of these instruments. These are presented next. 34 2.2.7 Categories of learning environments research As learning environments research continues to mature, investigators are using established and well-validated instruments and their subsequent modifications in broadening and deepening the questions assessed in relation to person-environment interaction. Researchers are looking toward other disciplines as well to forge truly cross-disciplinary research that influences studies in learning environments. For example, McRobbie and Thomas (2001) used the WIHIC to go beyond simple characterization of classroom learning environments to include a study of participants’ perceptions when the environment underwent intentional change. Fisher and Waldrip (2002) have integrated traditional learning environment dimensions with dimensions of culture extracted from anthropology, sociology, and management to measure classroom factors related to cultural sensitivity with the Cultural Learning Environment Questionnaire (CLEQ). Even further, learning environments research has reached beyond interdisciplinary considerations and into established cross-cultural studies. For example, Zandvliet (2002) integrated workplace ergonomic aspects in the technology-rich classroom with psychosocial considerations of Canadian and Australian students through a unique combination of the use of the What is Happening in this Classroom (WIHIC), the Computerized Classroom Ergonomic Inventory (CCEI), and a student satisfaction scale borrowed from the classic Test of Science Related Attitudes (TOSRA) (Fraser, 1981). These studies and others, when considered holistically, tend to fall into distinctive categories. As recognized by Fraser (2002a), there are six common learning environment research categories that focus on 1) student outcome-environment associations, 2) evaluation of educational innovations, 3) differences between teachers’ and students’ perceptions of the same classroom, 4) determinants of classroom environments, 5) utilisation of qualitative research methods, and 6) cross- national/cross-cultural studies. Nevertheless, other lines of research certainly exist, yet in less frequently researched categories. For example, Fraser (1998b) noted trends related to the measurement of school-level environments, studies on links between non-educational environments and educational environments, studies related to the transition from primary school to high school, applications in teacher education, and utilization of learning environment instruments for teacher 35 assessment. Nonetheless, the strongest line of study from any of the categories above appears to be investigating outcome-environment associations (Fraser, 2002a). As with previous learning environment studies, my study also examined associations between student outcomes and the psychosocial learning environment, but specifically for distance education environments (see Chapter 4, Section 4.4). Therefore, I thoroughly review the literature related to this line of study in Subsection 2.2.7.3 below. Likewise, given the unique ability of distance education to be able to easily reach across cultures and political boundaries, I have included an expanded subsection on cross-national/cross-cultural aspects of learning environment research (see Subsection 2.2.7.2). Finally, while additional data continues to be acquired, the Distance Education Learning Environment Survey (DELES), which I preliminarily validated in my study, has concurrently been used to investigate differences between students’ and teachers’ perceptions of the same class (Walker, 2001b). Subsection 2.2.7.1 briefly introduces this line of study. Discussion of the remaining categories of learning environments research, namely, evaluation of educational innovations, determinants of classroom environments, and the use of qualitative research methods becomes too lengthy to discuss in this review. However, an in-depth accounting of these three lines of research can be found in Fraser (1998a, 1998b) and with a focus on the Asian context in Fraser (2002a). 2.2.7.1 Differences between students’ and teachers’ perceptions of the same class A highly utilitarian line of learning environments research focuses on 1) differences between students’ perceptions of their classroom environments and their instructors’ perceptions of the same environments, and 2) students’ preferred environment in contrast to the actual classroom environment (Fisher & Fraser, 1983; Fraser, 2002a). Often this line of study involves administering a given learning environment instrument to measure students’ preferred environment with a preferred version of the instrument. This is then followed up by measuring their perceptions of the actual environment using an actual version of the same instrument. At the same time, the instructor responds to an instructor version of the instrument, thus producing three sets of data related to perceptions of the same class. Each version of the instrument contains the same items, but they are worded slightly different in each version to 36 capture students’ perceptions of the preferred and actual environment. The instructor version has the same items, yet it is modified to capture the instructor’s perceptions. See Chapter 3, Section 3.2 for further discussion of measurement of perceptions. A common pattern that has been replicated in a variety of studies is that students prefer a more positive environment than their classroom environments actually give them (Fraser, 2002a; Fraser, Giddings, & McRobbie, 1992; Margianti, Fraser, & Aldridge, 2001). This same pattern has occurred in distance education learning environment research as measured with the Distance and Open Learning Environment Survey (DOLES) (Walker, 2001a) and the Distance Education Learning Environment Survey (DELES) (Walker, 2002b). On the other hand, results of studies comparing instructors’ perceptions of their classroom learning environment to those of their students has indicated that instructors have a tendency to see their classes “through rose-coloured glasses” (Fraser, Giddings, & McRobbie, 1992, p. 6; Fraser, Sinclair, & Ledbetter, 2001; Giddings & Fraser, 1990; Walker, 2002a). Feedback from these types of practical applications of classroom participants’ perceptions can be used as a foundation for dialogue and improvement of classroom environments (Fraser, Sinclair, & Ledbetter, 2001; Yarrow, Millwater, & Fraser, 1997). The National Key Centre for School Science and Mathematics at Curtin University of Technology has produced a series of What Research Says to the Science and Mathematics Teacher publications describing how learning environments instruments can be used to aid education practitioners in discerning differences between their perceptions of their classroom environment and those perceptions that their students hold (e.g., Fraser, 1989; Fraser, Giddings, & McRobbie, 1992; Wubbels, 1993). The rationale behind identifying these differences is grounded in findings that suggest that “the classroom environment that is most conducive to high quality learning is one where there is congruence between student perceptions of actual and preferred environments” (Yarrow, Millwater, & Fraser, 1997, p. 70). When instructors use learning environment instruments to measure students’ perceptions against their own, they can graph the differences and easily recognize discrepancies. Instructors interested in improving their classroom learning environment can make adjustments in their teaching that address these discrepancies 37 as a means to enhance learning in their classrooms (Fraser & Fisher, 1986; Yarrow, Millwater, & Fraser, 1997). In an asynchronous distance education learning environment, this line of research can be especially useful given that instructors and students might not actually ever meet. Thus, instructors rarely receive subtle contextual cues from students for what is really going on in their distance education classes (Swan, 2001). Some studies have indicated perceptual differences between distance education instructors’ views of their asynchronous learning environments and the views of their students whereas, with face-to-face studies, certain environment characteristics were viewed as more positive by the instructor than by the students (Walker, 2001b, 2002a). In this vein of study, action research using distance-education oriented learning environment instruments to measure the participants’ perceptions of their class climate stands to be very useful in improving the psychosocial nature of distance education learning environments. 2.2.7.2 Cross-cultural learning environments research In what seems to be a natural expansion of learning environments research, there has been a trend toward cross-cultural studies and internationalization of this research genre, most notably and well documented in Asia (Gopinathan, 2002). While a comprehensive look at the history and depth of cross-cultural learning environments research is too broad for presentation here, it is worthwhile to make note of some of these efforts. Walberg, Singh, and Rasher (1977) translated the LEI into Hindi for a large-scale study of Indian students, establishing the validity of utilizing learning environment instruments that have been developed in Western countries in foreign contexts. Later, in the 1980s, in the micro-state of Brunei Darussalam, investigators began considering learning environments in conjunction with students’ attitudes toward science. This was followed by applications of the Individualised Classroom Environment Questionnaire (ICEQ), Science Laboratory Environment Inventory (SLEI), and modifications of the WIHIC and the QTI designed to address local cultural contexts (Khine, 2002). Indonesia has been another beneficiary of cross- 38 cultural work in learning environments through the emergence of modified and translated versions of the ICEQ, the LEI, and the Classroom Environment Scale (CES). These have been followed by adaptations of the WIHIC and the College and University Classroom Environment Inventory (CUCEI) (Margianti, 2002). Learning environments research in Korea emerged in the early 1990s in terms of the environment in science classes. Adaptations and translations of components of the ICEQ, the SLEI, and the CES have been carried out in research into student perceptions of new curricula. The CES was translated and modified for cultural context differences to study students’ perceptions of science classes at different education levels, among other studies in Korea (Lee & Kim, 2002). Goh (2002) reports the emergence in the early 1990s of learning environments research at all educational levels and across disciplines in Singapore. Learning environments research in Singapore consists of studies using the ICEQ, portions of the QTI and MCI, and the Chemistry Laboratory Environment Inventory (CLEI). Establishing a second generation cross-cultural transfer of learning environment instruments between two Asian countries is a version of the WIHIC, developed in Mandarin for students in Taiwan (Yang, Huang & Aldridge, 2002), that has been adapted to investigate students’ perceptions of their Chinese Language class environment for students in Singapore—the Chinese Language Classroom Environment Inventory (CLCEI) (Goh, 2002). Aldridge and Fraser (1997) conducted a large-scale, quantitative-qualitative, cross- national study of secondary students’ perceptions of their science classes using the WIHIC. In this study, the WIHIC was administered to 1879 students in Taiwan and 1081 students in Australia. For deeper insight into the differences in between the students’ perceptions the quantitative study was followed by qualitative analyses that focused on the scales that had the largest differences between the Taiwanese students and the Australian students. This study ultimately found that students’ perceptions of their learning environments are influenced by socio-cultural factors, noting that caution must be exercised in using Western-developed instruments in cross-national investigations. 39 Perhaps the most widely-adopted learning environment instrument across cultures has been the WIHIC. The WIHIC has been validated and used in Australia, Brunei Darussalam, Canada, Indonesia, Korea, Singapore, Taiwan, and the United States (Fraser, 2002a; Yang, Huang, & Aldridge, 2002). Further, adding to evidence of the global implications of how entrenched learning environments research is becoming, there is evidence of a language of ‘ownership’ among investigators when they refer to certain learning environment instruments. For example, Lee and Kim (2002) make reference to the “Korean version of the CLES” (p. 179) implying its specificity to the Korean language and culture. Similarly, Margianti (2002) makes note of the use of the “Indonesian version of the What Is Happening In This Class?” questionnaire (p. 157). This concept of unique ownership, coupled with new instruments emerging out of various cultural contexts (i.e., Geography Classroom Environment Inventory from Singapore) (Teh & Fraser, 1994), is perhaps an indication of the depth to which learning environments research’s roots have grown since Moos’ early work in the 1970s. 2.2.7.3 Associations between student outcomes and learning environments Research involving learning environment instruments has a strong tradition of considering associations between perceptions of the psychosocial characteristics of classrooms and students’ cognitive and affective learning outcomes (Fraser, 1998a, 1998b, 2002a). Beyond the variance in learning outcomes that can be attributed to individual student characteristics lies that which can be attributed to students’ perceptions of the environment in which they learn. Learning environments research has consistently demonstrated, across nations, languages, cultures, subject matter, and education levels, that there are associations between classroom environment perceptions and student outcomes (Fraser, 1998a, 1998b, 2002a). Often affective student outcomes are considered as integral parts of studies of educational environments. While the term ‘affective’ could have different meanings to different individuals who consider the use of the term, perhaps Klopfer’s (as cited in Fraser, 1977) six affective categories, that set the stage for the oft-used Test of Science Related Attitudes (TOSRA) (Fraser, 1981), can be used as a guide in determining what is meant by ‘affective outcomes’. Klopfer lists students’ 1) attitude 40 toward the subject matter, 2) attitude toward inquiry, 3) adoption of attitudes similar to the subject at hand, 4) enjoyment of the learning experience, 5) interest in the subject matter at hand, apart from the learning experience, and 6) interest in the subject of study as a career as affective categories. Additional affective considerations have included students’ satisfaction in the classroom, which is to educational outcomes what job satisfaction is to workplace productivity (Zandvliet, 1999), and efficacy which has been measured by a scale of Student Academic Efficacy using such items as “I am good at this subject”, or “I am an intelligent student” (Fisher, Aldridge, Fraser, & Wood, 2001). Regardless of which affective outcomes one considers, their influences can be measured by their associations with learning environment variables of interest to the researcher (Fraser, 1977). Commonly, learning environment research includes investigation of associations between an affective scale and the psychosocial scales at hand. Conventional multiple regression analyses are regularly conducted in order to investigate these associations but, on occasion, the Hierarchical Linear Model is used (e.g., Wong, Young, & Fraser, 1997) to takes the hierarchical character of the learning environment into account (Fraser, 2002a). The remainder of this subsection presents previous studies for which associations were investigated between psychosocial learning environments and student outcomes. These studies are reviewed in terms of investigations between 1) learning environments and affective outcomes, 2) learning environments, cognitive achievement and affective outcomes, and 3) variations using multiple learning environment instruments and specific outcome instruments. Koul and Fisher (2002) conducted the first investigation in India that considered student attitudes and psychosocial environment perceptions. They used the WIHIC and a scale of students’ attitude toward their science lessons with 1,021 grade 9 and 10 students, finding positive associations between students’ attitude toward science and the WIHIC scales. Majeed, Fraser, and Aldridge (2001, 2002) conducted an analysis of student satisfaction and three scales of a version of the My Class Inventory (MCI) designed for use in mathematics classes in Brunei Darussalam. Using a version of the WIHIC modified for use in South African primary-school 41 mathematics classes, Ntuli, Aldridge, and Fraser (2003) found positive associations between the seven WIHIC scales and student satisfaction in classes taught by student teachers. Likewise, in the first post-secondary learning environments study in Singapore, Myint and Goh (2001) discovered positive and statistically significant associations between CUCEI scales and student attitudinal scales of Difficulty and Speed. Further evidence of positive associations can be found in Zandvliet’s (2002) study that utilized the WIHIC and a scale of student satisfaction. In this study, Zandvliet reported a positive association between five WIHIC scales and the scale of satisfaction that he utilized. Associations between psychosocial scales and both cognitive and affective outcomes have also been investigated and reported in numerous cases. Students’ responses to the QTI scales, Computer Science and Management students’ achievement scores, and their responses to affective scales of leisure interest in computers and attitudes toward the Internet were investigated in Indonesia (Soerjaningsih, Fraser, & Aldridge, 2001). This study utilized the newly-developed Test of Computer-Related Attitudes (TOCRA) that, in its original form, contained three scales modified from the Test of Science-Related Attitudes (TOSRA) and one new scale concerning attitudes toward the Internet. After factor analysis though, only two of the TOCRA scales held up and these two remaining scales resulted in unexpectedly mixed associations, suggesting that perhaps the 422-student sample was insufficiently large. An Indonesian version of the WIHIC, one scale assessing attitude toward lectures (modified from the TOSRA), and cognitive achievement were included in a study of outcome-learning environment associations by Margianti, Fraser, and Aldridge (2001) using responses from a 2,498-student, post-secondary sample. The seven WIHIC scales were found to be positively associated with the cognitive achievement scale and the student attitude scale. Previously, Fisher and Stolarchuk (1998) had used the Science Classroom Environment Survey (SCES), the Test of Enquiry Skills (TOES), a measure of cognitive achievement, and a five-item scale of students’ enjoyment of science lessons to investigate environment-outcome associations in Australian grade 8 and 9 classrooms using laptop computers. The results were mixed, demonstrating minimal associations that possibly indicate that consideration should be given to how laptops are utilised in mathematics classrooms at this 42 educational level. Similarly, using the same sample, Stolarchuk and Fisher (1998) used the QTI, the TOES, and an attitude scale to investigate associations between student outcomes and teacher-student interpersonal relationships. They found that, in laptop-using classrooms, teacher-student relationships have positive associations with students’ affective and cognitive outcomes. Yet other studies have adapted multiple instruments to investigate specific outcome- environment associations. For example, Fisher, Aldridge, Fraser, and Wood (2001) adapted scales from multiple, widely-used general classroom instruments and individually-developed scales to initially develop the Technology-Rich Outcomes- Focused Learning Environment Inventory (TROFLEI) with 24 items in three scales. The TROFLEI has since been revised and now contains 76 items in 10 scales (Aldridge, Fraser, Fisher, Trinidad, & Wood, 2003). In their most recent study, Aldridge et al. (2003) measured students’ attitudes toward their subject matter, attitudes toward computer usage, academic efficacy, and student achievement in conjunction with their perceptions of the psychosocial classroom environment. For a sample of 1035 students, they determined that there were several statistically significant positive associations between the three attitude scales and the psychosocial scales of their newly-developed TROFLEI. For a subsample of 386 students, they determined that, of the ten psychosocial scales, there were positive and significant associations with student achievement for six scales. Likewise, in secondary chemistry classes in Singapore, Quek, Wong, and Fraser (2001) investigated associations between the science laboratory classroom environment, teacher-student interactions, and student attitude using the Chemistry Laboratory Inventory (CLEI), the QTI, and the 30-item Questionnaire on Chemistry- Related Attitudes (QOCRA). They found a number of significant positive associations between the CLEI and the three QOCRA scales of Attitude to Scientific Inquiry in Chemistry, Adoption of Scientific Attitudes in Chemistry, and Enjoyment of Chemistry Lessons. The also found statistically significant associations between six of the eight QTI scales and one QOCRA scale, namely, Enjoyment of Chemistry Lessons. However, only one of these associations, that between Enjoyment and Helping/Friendly teacher behaviour, continued to be significant when all QTI scales were mutually controlled. 43 The variety of recent studies involving perceptions of psychosocial characteristics of classrooms and students’ cognitive and affective learning outcomes has been well documented. Walberg’s (1981) assertion that psychosocial characteristics in classrooms provide valid predictors of student outcomes (see Section 2.2.5) has undoubtedly been supported through these and other studies of associations between learning environments and student outcomes. Nevertheless, Moos’ (2002) supposition of the enigma that plagues psychosocial environment research still holds true. By the very nature of changing environments, their study must continue to grow and change along with the evanescent effects of new environments. This is to say that, in all of the learning environment studies outlined here, with the exception of the Distance and Open Learning Environment Scale (DOLES), no instrument deals with the asynchronous distance education environment—a relatively new learning environment. While Section 2.4 below specifically discusses distance education learning environments, it is at this point that I should identify the notion that no distance education learning environment studies found in my review of the literature overtly considers associations between the learning environment and student outcomes. Perhaps my study will be the first. However, before too much headway is made toward the treatment of distance education learning environments themselves, we must start at the beginning and consider research in distance education in order to set the stage appropriately. 2.3 DISTANCE EDUCATION RESEARCH Although distance education evolved in the early 1700s in the form of postal- delivered correspondence (Jayroe, 1998), recent advances in and proliferation of technology and telecommunications have created possibilities that stretch the boundaries of post-secondary distance education (Harnar, Brown, & Mayall, 2000). Developments in distance education have changed how we communicate and learn (Leh, 1999) and will continue to do so as growing numbers of students become distance learners and a growing number of instructors become distance educators. Distance education, in general terms, relies upon a combination of technologies spanning the spectrum from print correspondence to high-bandwidth synchronous 44 videoconferencing. Although a variety of models exist for distance education using various available tele-media (Leach & Walker, 2000), rapid changes in technologies do not in themselves create effective distance education opportunities. Few commercial, large-scale, technology-driven, virtual learning environments are designed with a systemic view of learning grounded in learning theory (Spector, Wasson, & Davidson, 1999). Commercial virtual learning environments focus primarily on course administration rather than on learner relationships, personal development, or expectations and control and little, if any, instructional input is integrated into distance education course design (Cook, 2000). Put succinctly, teaching and learning do not improve because of improved distance education technology or the use of distance education technology (Cookson, 2002; Jamieson, 1999). Learning is likely to improve when instruction is grounded in practical learning theory. Currently, distance education research is narrow and is not keeping pace with the growth of distance education implementation around the world (Boling & Robinson, 1999). While a plethora of literature on the distance education phenomenon is available, original empirical research on distance education is still limited (Merisotis & Olsen, 2000; Olsen & Wisher, 2002). Distance education evaluation is concentrated primarily on 1) student outcomes (achievement, grades, test scores), 2) attitudes of students and instructors, and 3) satisfaction of students and instructors (Diaz & Cartnal, 1999; Harnar, Brown, & Mayall, 2000; Institute for Higher Education Policy [IHEP], 1999; Lane, n.d.; Olsen, 2000). Murphy and Cifuentes (2001), citing multiple reviews of literature, report criticisms of the rigor of distance education research, noting that, in some cases, less than three-quarters of distance education literature focuses on learning—most focuses on technology and the role of the instructor. They also report that as little as one-third of the distance education literature is research based, while the remaining two-thirds is either theory or anecdote. And, while they see a shift away from telecommunication theory and toward teaching and learning, there is still limited research investigating the “psychological and communications space” in distance education (p. 286). 45 Further, postulated in the context of distance education system evaluation, Harnish and Reeves (2000) discovered the emergence of distance education evaluation primarily in terms of: 1) Training (programming skills, barriers, availability, identification of needs, costs); 2) Implementation (administration, costs, fees course credits, institutional ownership, priority for use, integration, coordination); 3) System Usage (information collection, electronic data collection, accuracy); 4) Communication (information sharing around internal, local, and regional issues of concern regarding distance education); and 5) Support (fiscal, staff, faculty, instructional, administrative resource allocation). While traditional student outcome assessment and student attitudes are important, as are technical issues, system implementation components, and even positive-leaning anecdotal description, we must be able to understand students’ learning environment needs in order to create the most advantageous learning experiences in distance education (Howland & Moore, 2002). Mioduser, Nachmias, Lahav, and Oren (2000) identified and described a taxonomy of web-based learning environments as a “practical tool for describing the complexity of the educational kaleidoscope that has been generated by the Web” (p. 57). Their taxonomy consists of 100 variables categorised into four dimensions that can be considered for research purposes: 1) the descriptive dimension; 2) the pedagogical dimension; 3) the knowledge dimension; and 4) the communication dimension. Nevertheless, while it is necessary to pick apart, categorise, and develop distance education schema based on hindsight and content analyses, and to develop new categories of distance education nomenclature for the purpose of study, theoretical implications and practice, what remains conspicuously missing are studies of components related to psychosocial learning environments (Kreijns, Kirschner, & Jochems, 2002). My study focused not on a new conceptualization of learning environments in distance education, but rather on those ideas grounded in established learning environments research based on time- honoured theoretical principles. 46 2.3.1 Distance education learning environments in distance education research Despite criticisms of distance education research, independent distance education psychosocial learning environment factors, considered in terms of Moos’ (1974) social organization dimensions (Relationship, Personal Relevance, System Maintenance and Change), are being studied in what has been deemed fifth- generation distance education (Taylor, 2001). However, when categorised in terms of Moos’ three dimensions, most distance education literature focuses on those components found in the Relationship Dimension, such as collaboration, interaction, and instructor support. Oren, Mioduser, and Machmias (2002) remark on the importance of considering social climate in distance education and follow up by summarising five studies of social climate in Web-based environments. However, they focus primarily on group and interpersonal interaction, excluding, or at least not considering, other social-psychological factors. Further, the research method of the studies that they summarise consists primarily of counting and categorising messages from online classes, yielding little insight into personal relevance or system maintenance and change. Diaz (2000) calls for analysis of the quality of student-student and student-instructor interaction so that certain aspects of these types of relationships can be altered in order for improvements to be made that can influence distance education practices. This call follows Slay’s (1998) proposed theoretical framework for distance education teaching that outlines a need for considerations of student-student and student-instructor interactions, as well as control and structure within the distance education learning environment, thus taking into account two of Moos’ three social organization dimensions. Regarding the System Maintenance and Change Dimension, Cox, Preston, and Cox (1999), using empirical survey research, identified student locus of control as being correlated strongly with motivation in distance education students. Similarly, Wang and Newlin (2002) present student locus of control as having a moderate correlation with student performance in distance education. They consider locus of control to be a global trait—an attribute that is durable and constant across time and environment—and link this control to instructor support and peer-to-peer interaction. In their qualitative study, Melland and Volden (2001) made note that clarity in online class syllabi plays an important role in influencing student security. 47 Youngblood, Trede, and DeCorpo (2001), studying pilot distance education classes, revealed that over 80% of their study population indicated that the instructor’s role in establishing an organized environment with clear expectations contributed to the success of an online class. Certainly numerous other investigations related to distance education environment system maintenance and change have been produced. However, there is an exceedingly strong focus on interaction—a factor in the Relationship Dimension. O’Reilly and Newton (2001) suggest that instructors and students alike must adapt to new forms of interaction offered in online learning environments. They reported in one student survey that they conducted that 82% of the responses strongly favoured peer-to-peer interaction. They indicate that mutual support, social cohesion, motivation, and confidence are all learning by-products of interaction. Lattu (2000) offers that student-instructor interaction in online distance education provides a “dialogic contact between learner and material” (p. 1) whereby the learner receives evaluative feedback, as opposed to simple one-way instruction dominated by the instructor. Swan (2001) identifies and labels three types of interaction in online education, namely, interaction with 1) content, 2) instructors, and 3) classmates. She observes that interaction with content does not equal learning. Design of instruction is a strong component of how students learn by means of their interactions with content, thus indicating that distance education must be more than posting content materials. She further remarks that the psychological distance between students and the instructor is reduced by immediacy of instructors (e.g., instructors’ use of humour, self-disclosure, praise, etc.). Finally, she notes that peer interaction leads to successful discussion and that instructors place high value on student-student interaction. Considerations of interaction in distance education inevitably lead to the contemplation of community development and collaboration derived through interaction. However, interaction can be limited to simple online communication, with or without a learning purpose, that does not lead to collaboration. Collaboration, the antithesis of competition, suggests that students cooperate as members of a group through consensus (Walker & Resta, 2002). Online community, in terms of distance education, refers to a group’s bond formed by means of communication and social 48 interaction and involves all members sensing that they belong and that their contributions are appreciated and respected (Lefoe, Gunn, & Hedberg, 2001; Tu & Corry, 2001). While an extensive description of literature related to distance education interaction, collaboration, and community is too detailed to provide here, it is noteworthy to point out that some studies are considering online learning environments as communities in terms of how participants see themselves (Tu & Corry, 2001) in relation to the social structure of that community. Graham and Scarborough (2001) connect collaborative learning skills and increases in student- student communication to students’ personal growth and development—a component of the Personal Relevance Dimension. Others consider social presence and self-disclosure in terms of group process and online community development as it relates to the success of collaboration in online learning (Cifuentes & Murphy, 2000; Gunawardena, Nolla, Wilson, & Lopez-Islas, 2001; Kollock & Smith, 1999). Components of these studies in community and collaboration cross over between the Relationship Dimension (e.g., involvement, affiliation, peer cohesion) and some relevant aspects of the Personal Development Dimension (e.g., competition or lack thereof, personal growth). However, for the most part, research that takes a holistic view of the distance education psychosocial learning environment remains limited (Teh, 1999). Perhaps, similar to learning environments research of the 1960s and 1970s, distance education learning environments research today lacks “the guidance of theoretical or conceptual frameworks producing isolated findings that are difficult to organize into a coherent body of knowledge” (Moos, 1979, p. 138) about distance education class functioning. 2.4 DISTANCE EDUCATION LEARNING ENVIRONMENTS RESEARCH While there are several learning environments studies related to classroom use of computers and technology, researchers have documented that there is limited research on psychosocial perceptions of Web-based learning environments (Jegede, Fraser, & Fisher, 1998; Taylor & Maor, 2000; Teh, 1999). This section introduces the brief history of distance education learning environments research, noting the first such investigation and those that are recent and emerging. It also includes a look into the scales, based on Moos’ social organization dimensions, which are evolving in distance education learning environments research. 49 2.4.1 Technology-oriented learning environments research Learning environments research has been conducted and associated survey instruments have been developed that relate to computer uses in classrooms or laboratories, telecomputing, and computer-mediated communication. Briefly, examples of technology-related learning environments instruments include the Geography Classroom Environment Inventory (GCEI) that investigates gender inequities, among other factors, in computer assisted learning in Singapore (Teh & Fraser, 1994), the Constructivist Multimedia Learning Environment Survey (Maor, 1999), the Computer Classroom Environment Inventory (CCEI), the Computer Laboratory Environment Inventory (CLEI) (Newby & Fisher, 1997), and the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI) (Fisher, Aldridge, Fraser, & Wood, 2001). Related research includes studies of perspectives of computer-mediated learning environments specific to teacher education (Admiraal, Lockhorst, Wubbels, Korthagen, & Veen, 1998; Goh & Tobin, 1999), computer-facilitated learning environments in higher education (Bain, McNaught, Mills, & Lueckenhausen, 1998), collaborative distance learning environment design (Spector, Wasson, & Davidson, 1999), the function and useability of virtual learning environment software (Britain & Liber, 1999), and students’ perceptions of their learning environments in mathematics and science classes that use laptops (Raaflaub & Fraser, 2002, 2003). Nevertheless, technology-oriented learning environments research, while closely associated with today’s technology-oriented distance education environments, does not fully capture salient characteristics of distance education despite both having technological features as a part of the educational milieu. Additionally, while telecomputing studies and computer-mediated communication studies are relative to distance education, they do not organize distance education learning environments research into a consistent psychosocial framework. Research specifically on distance education learning environments must be developed and conducted on its own terms. 50 2.4.2 The first look at distance education learning environments In 1995, the development of the Distance and Open Learning Environment Scale (DOLES) (Jegede, Fraser, & Fisher, 1995) was a pioneering investigation bringing learning environments research and distance education research together into one cohesive body of study. And, like early distance education research, it too had aspects focusing on technology and interaction. The DOLES considered participants’ perspectives of salient scales of the environment primarily in distance education science classes originating from Queensland and Western Australian universities. The DOLES is a paper-based instrument initially validated using 660 student responses to the five core scales of 1) student cohesiveness, 2) teacher support, 3) personal involvement and flexibility, 4) task orientation and material environment, and 5) home environment. Optional scales are student centre environment, validated using 464 responses, and technology resources, validated with 169 responses (Jegede, Fraser, & Fisher, 1998). 2.4.3 Subsequent distance education learning environments research The previously-mentioned Geography Classroom Environment Inventory (GCEI), validated with 348 responses form computer-assisted learning classrooms in Singapore (Teh & Fraser, 1994), was later applied in undergraduate-level distance education geography classes. Teh (1999) considered internal consistency and discrimant validity of 92 responses to the paper-based version for asynchronous distance education students in Singapore. The GCEI consists of the four scales of 1) gender equity, 2) investigation, 3) innovation, and 4) resource adequacy. The initiation of geography distance education learning environment research is important in this case due to the scarcity of a combination of geography education research and distance education research in Singapore (Teh, 1999) and elsewhere. The Constructivist On-Line Learning Environment Survey (COLLES) was developed from its three-scale predecessor, the Constructivist Virtual Learning Environment Survey (CVLES) (Taylor & Maor, 1998), to measure aspects of the quality of online learning environments from a social constructivist perspective in an 51 effort to ensure that “technological determinism doesn’t overshadow sound educational judgement” (Taylor & Maor, 2000, Conclusion section, ¶ 1). The COLLES, arranged in the six scales of 1) relevance, 2) reflection, 3) interactivity, 4) tutor support, 5) peer support, and 6) interpretation, has been applied to support social constructivism epistemologies of teaching and learning during the construction and utilisation of Web-based software in online education (Dougiamas & Taylor, 2002). The results of the COLLES, in triangulation with other class assessment methods, has led to “significant and possibly radical” changes in the way in which online discussions are conducted in an Internet-based postgraduate class in Western Australia (Dougiamas & Taylor, 2002, p. 8). Another recent distance education learning environment instrument is the Web Based Learning Environment Instrument (WEBLEI) that considers Web-based learning effectiveness in terms of a cycle that includes access to materials, interaction, students’ perceptions of the environment, and students’ determinations of what they have learned (Chang & Fisher, 2001a). These factors are summarised in the four scales of 1) emancipatory activities (viz., convenience, efficiency, autonomy), 2) co-participatory activities (viz., flexibility, reflection, interaction, feedback, collaboration), 3) information structure and design activities (e.g., clear objectives, planned activities, appropriate content, material design and layout, logical structure), and 4) qualia, a scale of attitude (viz., enjoyment, confidence accomplishment, success, frustration, tedium). The WEBLEI was piloted and initially validated from responses using 334 postsecondary students enrolled in a class that could be taken either in a hybrid fashion (partially online, partially face-to- face) or taken 100% online. Just over 73% of the responses were from students taking the class online (Chang & Fisher, 2001b). 2.4.4 Emerging distance education learning environments research The Online Learning Environment Survey (OLLES) is currently undergoing development in New Zealand. The OLLES considers the eight scales of 1) reflective thinking, 2) information design and appeal, 3) order and organization, 4) active learning, 5) affective support, 6) student cohesiveness and affiliation, 7) computer anxiety and competence, and 8) material environment and rule clarity (J. Clayton, 52 personal communication, May 5, 2003). These scales are nearly equally spread across Moos’ three social organization dimensions, addressing gaps in many general distance education studies previously mentioned that go without a strong theoretical framework. The DOLES, GCEI, COLLES, WEBLEI, and OLLES, with their differing variations and foci, are leading to promising knowledge development in terms of distance education learning environments. Perhaps these instruments will be looked upon as benchmarks from which other research on psychosocial learning environments in distance education will grow. However, as previously noted, no distance education learning environment instruments have considered environment-outcome associations. Because student satisfaction can presumably lead to increased student outcomes in the same way that job satisfaction can lead to increased worker productivity (Zandvliet, 1999), my study investigated associations between distance education psychosocial characteristics and student satisfaction. The next section considers what the literature has to say about the importance of student satisfaction. 2.5 STUDENT SATISFACTION As previously noted in Subsection 2.2.7.3, there is a strong tradition of researchers investigating environment-outcome associations. Similarly, my study also examined these associations. As noted in Chapter 1, Section 1.1, one of the design aspects of this study was to seek associations between the psychosocial learning environment in distance education and student attitudes, specifically students’ enjoyment of distance education. Studies of student and instructor satisfaction in distance education have been well documented (Jegede & Kirkwood, 1994; McAllister, 2001; Simonson, Smaldino, Albright, & Zvacek, 2003). However, Felix (2001) notes that students who prefer working in distance education settings often seek them out and are the ones who are most successful in them. He also points out that there are few studies that tie instructional strategies to effective changes in student attitudes. Seeking to discover relationships between positive student attitudes and the psychosocial environment could possibly lead to favourable changes in distance education environments. Yet, 53 no studies in distance education learning environments were found that involve attempts to investigate these associations. Student satisfaction is used consistently in post-secondary education to measure how effectively a program or institution delivers what students expect, need, and want, and it is associated with student achievement (Kuh, 2001a, 2001b). Student satisfaction is also a key indicator of educational quality (National Survey of Student Engagement, 2000) and can be measured for strength of relation to learning environment characteristics. With the addition of an attitudinal scale to a study of psychosocial learning environments, the relationship between learners’ attitudes and their perceptions of the psychosocial learning environment can be investigated (Fraser, 1981). What is unique in relating student satisfaction to the psychosocial learning environment in distance education is that there is a discrepancy in the results of studies in the literature related to student satisfaction and distance education, primarily where student-to-instructor involvement occurs. Cannon et al. (2001) indicate dissatisfaction by students when surveyed with items related to direct instructor feedback. Eastmond and Kim (2000) note that, when the Korean corporate-operated, degree-granting Cyber Multi Campus outsourced its distance education classes to instructors who were subject-matter experts, yet not experienced educators, the satisfaction of students was lower than when instructors versed in instructional design taught the classes. Moreover, learners were infrequently satisfied with the classes when the instructors’ role was simply content development and not interaction with students. In an accredited MBA marketing distance education environment, Ponzurick, France, and Logar (2000) reported a high level of dissatisfaction with presentation, participation, and activities in a distance education environment when compared to a face-to-face learning environment. However, in contrast to these studies, LaRose and Whitten (2000) discovered that, in an online environment, learner attitudes and immediacy in the design of the class are comparable to those of a traditional classroom environment and that collaborative activities led to comparable levels of student satisfaction and achievement when compared to those of a face-to-face environment. This disagreement in the results 54 found in the literature regarding student satisfaction in online learning environments needs further consideration. When classified in terms of Kirkpatrick’s four-level framework of evaluation within a distance environment (Walker, 1998), the researcher gains a subjective measure of the learners’ reactions to the materials, instructor, instructional method, and environment by employing measures of satisfaction. Human resources training development programs regularly use any number of formal approaches to identify what Kirkpatrick (Phillips, 1991) referred to as level-one reaction evaluation information related to training. The following are well-established examples of satisfaction survey instruments used in corporate training for seeking level-one data: Reaction Outcomes (Bell System Approach), Reaction Evaluation (CIRO Approach), Training Satisfaction (Saratoga Institute Approach), and Reaction (IBM Approach) (Phillips, 1991). Following level-one reaction evaluation is the second of Kirkpatrick’s four levels, participant learning. The third level is that of the performance of the learner, and the fourth level is that of results or outcomes produced from learning (Phillips, 1991). This fourth level is typically not measured in educational settings, but is necessarily measured in terms of performance outcomes in task-oriented corporate training situations. The research at hand in this study incorporates Kirkpatrick’s level-one measure of student satisfaction in terms of Kirkpatrick’s framework by means of a measure of student satisfaction through enjoyment of distance education. Further research could incorporate Kirkpatrick’s level-two learning (cognitive achievement) as a measure in addition to satisfaction as has been done in learning environments research before (see Subsection 2.2.7.3). Rather than relying on corporate training instruments, I modified the scale of Enjoyment of Science Learning Experiences from the Test of Science Related Attitudes that was originally developed to measure the science-related attitudes of secondary school students (Fraser, 1981). This scale consists of eight items (see Appendix C), including “Distance education is stimulating” and “I enjoy studying by distance”. In my study these eight items had the same five frequency response choices as the psychosocial environment items, namely, Always, Often, Sometimes, Seldom, and Never. 55 2.6 CHAPTER SUMMARY The term learning environments carries with it several connotations in today’s world of digital communications for teaching and learning. This study uses the term strictly in the sense of the psychosocial learning environment. Learning environments research is solidly grounded in early 19th century research that includes the psychological field theory of Lewin (1936), personality theory of Murray (1938), and education behaviour research of Hartshorne and May (1928) and Newcomb (1929). These theoretical underpinnings have been built upon and enhanced by Pace and Stern (1958) and strongly influenced the conceptual framework of Moos (1976, 1979) who continues to expand on social ecology concepts today. Further expansions were undertaken by Walberg (1974, 1981) alongside a plethora of work by Fraser (1986, 1998a, 1998b) who began contributions in the mid-1970s. Thus well-established, learning environments research has expanded even further with strong evidence of its universality through research in numerous Asian countries (Goh & Khine, 2002). Heavily utilised learning environment instruments that I have reviewed in this chapter include the Constructivist Learning Environment Survey (CLES) (Taylor, Fraser, & Fisher, 1997), the Questionnaire on Teacher Interaction (QTI) (Wubbels, 1993), the Science Laboratory Environment Inventory (SLEI) (Fraser, Giddings, & McRobbie, 1992), and the What is Happening in this Class (WIHIC) questionnaire (Fraser, Fisher, & McRobbie, 1996). These are only four among the numerous instruments and modifications of instruments that exist today. One of the lines of research that is traditionally generated from using learning environment instruments is the investigation of environment-outcome associations. This chapter presented three aspects of this specific research direction, which include using environment instruments and affective scales, using environment instruments and scales of cognitive achievement alongside affective scales, and utilising multiple learning environment instruments and specific outcome instruments such as the Questionnaire on Chemistry-Related Attitudes (QOCRA). Research in distance education in general has demonstrated that technology expansions are difficult to keep up with and, despite our current role in fifth- 56 generation distance education (Taylor, 2001), we are still trying to solidify future directions in distance education. Often, though, the look forward is based upon looking backward at literature laden with anecdote and nominal empirical rigor and, in many cases, considered in terms of traditional methods of educating by means of instructor-centred, didactic education models, and the assessment of the achievements of students by means of examinations or non-validated surveys. While social climate in distance education is not the ultimate research topic, it does have its role because changes in learning environments have strong impacts on learning outcomes. The perceived shortfalls in distance education research in terms of well- founded psychosocial conceptualizations are not for the lack of trying. Studies have recently demonstrated movement toward an increasing consideration of psychosocial components, yet they remain disjointed without a guiding theoretical framework. My review of learning environments research literature reveals that there are few instruments that focus exclusively on learning environments for postsecondary asynchronous distance education, despite the number of studies on distance education as a whole. However, this situation is rapidly changing following the advent of the Distance and Open Learning Environment Scale at a time of increases in telecomputing use in education. Teh (1999), Taylor and Maor (2000), Chang and Fisher (2001a, 2001b), and Clayton (in press) are in the midst of contributing to the ever-growing body of knowledge related to psychosocial learning environments in distance education. Others are certainly contributing to this growing body of knowledge, but a review of any literature related in some way to distance education is a moving target in the fast-paced adoption of distance education by postsecondary institutions and the rapid expansion of ubiquitous telecommunications and technology. In fact, despite the relative infancy of distance education research, pundits are already calling for epistemological reform in the way in which distance education is conducted (Taylor & Maor, 2000) and we have barely scratched the surface in regard to distance education research or in collectively figuring out in which direction we are headed. 57 Chapter 3 RESEARCH METHODS 3.1 INTRODUCTION This chapter describes a three-stage approach used to develop, implement, and validate a questionnaire—the Distance Education Learning Environments Survey (DELES)—which is a new learning environments instrument for international, post- secondary distance education settings. The design and development was guided by past learning environments research and practice, consistency with previously- developed learning environment instruments, and relevance to distance education learning environments. This chapter also describes the types of analysis for which the instrument was designed and the means by which the data were collected (see Section 3.2). Section 3.3 describes the target population and how the data were collected using the DELES. It also presents an overview of a three-stage, multi-step approach used to develop the instrument (see Section 3.4) where Stage 1 was identification and development of salient scales, Stage 2 was writing individual items, and Stage 3 was field testing and analysis. Section 3.5 describes in detail how the scales were developed and reviewed, while Section 3.6 describes in detail how the items were developed and reviewed. Ultimately, the DELES, which was face validated by a panel of distance education practitioners and researchers, was field tested with nearly 700 respondents (see Section 3.7). 3.2 CATEGORIES OF ANALYSIS The development of a learning environment survey instrument requires a choice from multiple and interrelated layers of categories of analysis (see Figure 3.1). These varying categories, in narrowing order, are outlined and explained further below under the categories of: education level, environment level, form, measurement of perception, and psychosocial dimension: Education Level – This research focuses on post-secondary education learning environments—those of college and university programs. 58 Environment Level – The literature distinguishes between two learning environment levels, that of the classroom and that of an entire school (Fraser, 1998b). My research focuses on classroom-level environments. Edu ca tio n L ev el P ri m ary S eco nd a ry T erti ary Enviro nm en t L ev el S ch o ol Cla ss F orm P erso na l Fo rm (Pri va te Pre ss) Cla ss Fo rm (Co n sen sua l P re ss) Me a su re me n t o f Perc ep tio n S tu d en t Actu al S tu d en t P re fe rre d Psych oso cia l Dimen sio n Rel a tio n shi p Dim e nsi o ns P erso na l Deve l op me n t Dim e nsi o ns S yste m M ai n ten a nce a n d S yste m Cha n ge Dim e nsi o ns In stru cto r Figure 3.1. Categories and levels of analysis Form – The assessment of a learning environment from the participants’ point of view (beta press) (see Murray, 1938; Chapter 2, Section 2.1.1) can involve either private beta press (perceptions that individual students have of an environment) or consensual beta press (a shared perception that members of a group have about an environment) (Stern, Stein, & Bloom, 1956; see also Fraser, 1998b; McRobbie, Fisher, & Wong, 1998). Because these two forms of beta press can differ from one another, researchers are justified in distinguishing between the two in their development of survey instruments (Fraser, 1998a, 1998b; McRobbie, Fisher, & Wong, 1998). Further, Personal Forms of learning environment instruments assess a student’s perception of his or her role in the environment of a classroom (private beta press), while Class Forms assess a student’s perception of the classroom environment as a whole (consensual beta press). Personal Forms have proven more valid, especially when within-classroom subgroups of individual students are under study (McRobbie, Fisher, & Wong, 1998). My study focused on the Personal Form (private press). 59 Measurement of Perceptions – Learning environment perceptions can be measured in three ways: 1) Actual environment, for which students give their perception of how the environment currently is. 2) Preferred environment focuses on the ideal learning environment that students prefer. 3) Instructors’ perceptions of the actual environment, which can be compared to the perceptions of the students. Preferred and Instructor versions in learning environment survey instruments are variations on the Actual version. A statement in an Actual version of a survey might read “Activities are carefully planned in this subject”, whereas the same statement in the Preferred version might read “Activities should be carefully planned...” Likewise, the Instructor’s version might read “I carefully plan activities…” This study was conducted using an Actual survey version that can be modified to address the preferred environment of students and instructors’ perceptions of the actual environment. Psychosocial Dimensions – Moos (1974) conceptualised three overarching dimensions characterising and discriminating among subunits within social organizations. These dimensions are the Relationship Dimension, the Personal Development Dimension, and the System Maintenance and Change Dimension (see Chapter 2, Section 2.2.3). The Relationship Dimension distinguishes the nature and strength of personal relationships. The Personal Development Dimension is characterized by personal growth and self-enhancement opportunities offered by the environment. And, the System Maintenance and Change Dimension considers the degree of control of the environment, the orderliness, clarity in expectations, and responsiveness to change (Moos, 1974). One of the primary tasks involved in this study was to adequately cover each of these three dimensions with the scales. 60 3.3 DATA COLLECTION The target population for this study was higher education students in distance education-delivered classes in public and private universities and colleges. The field test version of the instrument was announced by means of e-mail discussion groups and direct e-mail, which led to a wide set of responses. The survey sample was a nonprobability sample of convenience drawn from voluntary participants enrolled in post-secondary distance education classes. The field test version of the survey was available as a Hypertext Mark-up Language (HTML) form on the World Wide Web. The sample of respondents consisted of 680 post-secondary students enrolled in distance education classes in Australia, Canada, Mexico, New Zealand, Philippines, Puerto Rico, Saudi Arabia, Scotland, Senegal, Singapore, South Africa, United Arab Emirates, United States. However, the majority (83.7%) of the responses were from students in the United States. To be clear, this study was not intended to compare responses from students in different countries, it is intended to gain insight on distance education learning environments in general. Regarding education levels, the majority of the respondents (72.6%) were graduate-level students, while 27.5% were undergraduate students as indicated in Table 3.1. Table 3.1. Frequency of DELES field test responses by level of study being undertaken Level of Study Frequency Percent Doctoral 130 19.1 Masters 364 53.5 Undergraduate 186 27.4 Total 680 100.0 Prior to field testing, the Web-based form was pilot tested by distance education students (see details in Section 3.6.2.3) and then revised, based upon e-mail feedback from the respondents. Web-based surveys have grown in popularity in post-secondary education with the increasing amount of information available on the Web, increasing competition for a more diverse student population, and more demand for flexible education settings 61 (Ballantyne, 2000). Advantages of using the Web for surveys include easy-to- complete forms and flexible time and place to respond. Advantages to researchers occur in reduced opportunities for the introduction of errors due to data transfer from paper forms and a reduction in the time that it takes to transfer data. It is also more economical because paper-based forms do not have to be distributed and returned. Further, there is evidence of improvements in the quality of responses due to a reduction of the number of respondents making double responses to single items and in making no responses to items (Cummings & Ballantyne, 1999). On the other hand, Web-based surveys are subject to the same errors as paper-based surveys and these error possibilities must be addressed: 1) coverage errors, 2) sampling errors, 3) measurement errors, and 4) nonresponse errors (Dillman & Bowker, 2001). Coverage errors occur as a result of not having a known nonzero chance of being included in a survey, which is the case in Web-based surveys administered to the general public. Such occurrences have become increasingly common with broadcast news-media Web-based opinion polls (Dillman & Bowker, 2001). However, when all of the members of a population, for instance post- secondary distance education students as surveyed for the DELES, have access to computers and the Internet, coverage errors are less of a concern (Dillman & Bowker, 2001). Sampling errors that result from surveying only a sample of a selected population, rather than an entire population, are likely in this initial study using the DELES. This type of error does not indicate that the sample has been improperly extracted or that the data were improperly analysed or that such an error can be attributed to the design of my study. It is simply an acknowledgement that sampling errors are a natural part of survey research and should not affect the interpretation of the results (Huck, 2004). Measurement errors that result from poorly-worded items and poor instrument layout have been addressed in the development of the DELES at multiple points that include items being modified from previously-validated learning environment instruments (see Section 3.6), content validation by a panel of experts (see Subsection 3.6.2.1), and a pilot test administered to distance education students (see Subsection 3.6.2.3). Nonresponse errors, that occur from respondents not selecting a response to a given item, were eliminated by means of the technical design of the Web-based instrument. In the 62 DELES pilot test and field test, if a participant did not respond to an item, she or he was prompted to go back to select a response to that particular item before submitting the survey. Thus, the four major sources of survey error have been addressed in my study and attempts have been made to reduce them during data collection using the DELES. 3.4 OVERVIEW OF THE STAGES IN THE DEVELOPMENT OF THE DISTANCE EDUCATION LEARNING ENVIRONMENTS SURVEY (DELES) The development of the DELES used a common three-stage approach following Fraser (1986) and Jegede, Fraser, and Fisher (1998) for developing learning environments instruments. Stage 1 included identification of salient scales within Moos’ (1974) three social organization dimensions of Relationship, Personal Development, and System Maintenance and Change. Stage 2 involved writing individual items within the scales. Stage 3 involved field testing items followed by item analysis and validation procedures. Below, in Sections 3.4.1, 3.4.2, and 3.4.3, are brief descriptions of the steps involved in each stage. The remainder of the chapter, Sections 3.5, 3.6, and 3.7, describes each stage in detail. 3.4.1 Overview of Stage 1 – Identification and development of salient scales Stage 1 consisted of four steps that led to the identification and development of salient scales. The first step involved reviewing previously-developed learning environment instruments (Fraser, 1986, 1998a, 1998b). The second step within this stage included reviewing the literature related to psychosocial learning environments in distance education. This crucial step sought to identify key components that researchers and practitioners consider as important in high-quality distance education learning environments. The third step was to classify new scales using Moos’ three psychosocial dimensions in order to ensure adequate coverage of these dimensions. Finally, the fourth step was to develop a set of preliminary scales to distribute for review by a panel of experts. 63 3.4.2 Overview of Stage 2 – Writing individual items Stage 2 involved three steps. Step one was a consideration of negatively-worded or reverse-scored items. Step two involved both adapting items used in previously- validated learning environment questionnaires and developing new items for the new scales identified in Stage 1. Step three involved subjecting the entire set of items to face validation by a panel of experts. It also involved pilot testing reviewed items with one distance education class for Web-form useability and layout and determination of how best to process the digital data efficiently from the Web-based forms. 3.4.3 Overview of Stage 3 – Field testing and analyses Stage 3 required two steps. Step one included field testing the draft instrument with a large sample from the target population in order to collect sufficient responses to utilize in the statistical analysis. Step two involved factor analysis, aimed at identifying items whose removal would enhance the instrument’s factor structure, and internal consistency reliability analysis to determine the extent to which items within a scale measure the same construct as other items within that scale. 3.5 STAGE 1 – IDENTIFICATION AND DEVELOPMENT OF SALIENT SCALES 3.5.1 Review of scales from previously-developed instruments and development of new scales As discussed by Fisher and Fraser (1990), new inventories require scales that identify important aspects of the environment under study. I followed the precedent of modifying and adapting scales from previously-validated and tested instruments (Fish & Dane, 2000; Fisher, Rickards, & Fraser, 1996; Maor, 1999; Newby & Fisher, 1997), as well as creating new scales tailored to specific learning environments. For this study, I reviewed a variety of previously-developed instruments (see Chapter 2, Subsection 2.2.6) and adapted appropriate scales from 64 previous inventories including the Constructivist Learning Environment Survey (Taylor, Fraser, & Fisher, 1997), the Constructivist On-Line Learning Environment Survey (Taylor & Maor, 2000) and the Distance and Open Learning Environment Scale (Jegede, Fraser, & Fisher, 1998). I also created scales based on literature pertaining to the characteristics of high-quality distance education learning environments. As outlined in detail later, a panel of distance education experts subjectively validated the appropriateness of the scales selected for this study—the scales modified from previously-developed scales and those scales developed new for this instrument. 3.5.2 Strategies for developing new scales and adapting previous scales The selection of previously developed scales and the development of new scales was guided by Hase and Goldberg’s (1967) National Science Foundation-supported study on strategies for the development of personality inventory scales. Their research purports that there are three primary strategies for developing and validating such inventories: 1) intuitive, 2) internal, and 3) external. Within the intuitive strategy falls the intuitive-rational and the intuitive-theoretical strategies for scale development. The categorisation of the scales developed for my study fall under the intuitive-rational development category following Fraser (1986). This categorisation is termed as such due to the intuitive understanding of the subject matter by the researcher. Validity within this categorisation of scales in the case of this study was contingent upon the subjective opinion of the researcher, comments from an expert panel, and results of the pilot test which involved readability comments. While no specific psychological theory was followed under this category of scale development, I relied upon guidance from literature pertaining to learning environments and distance education, as well as that literature specific to distance education learning environment research (see Section 3.5.3). The scales were then refined through factor and item analyses and by selecting items that contributed to high internal consistency of their a priori scale, as well as to consistency with the instrument’s a priori scale structure. 65 In addition to the intuitive-rational strategy, the intuitive-theoretical strategy, while still relying on filtering by means of high internal consistency, is based upon a given psychological theory. An example of a learning environments instrument using the intuitive-theoretical strategy is the Questionnaire on Teacher Interaction (QTI) which derives scales based upon Leary’s (1957) clinical psychology model (Wubbels, Créton, & Hooymayers, 1992). Within Hase and Goldberg’s internal development strategy falls the factor- analytical strategy. Factor analysis can be used for exploring the structure of a pool of items, or to confirm and refine the a priori structure of an instrument. In this study, survey responses were subjected to factor analyses for confirmation and refining of the a priori (intuitive-rational) structure rather than for exploration and the creation of a new structure. Within the external development strategy falls the empirical group discriminative strategy. In developing scales from this perspective, the researcher administers the instrument to persons falling on opposite poles of the personality or other trait being measured. Items within these scales are examined from the point of view of discriminating between the two groups at some level of statistical significance. Those items with significant discriminating power remain in the scale. Hase and Goldberg found that, when constructing scales using these strategies, there were little differences in the outcomes and that they were “equivalent in their validity across 13 diverse criteria” (1967, p. 242). They went on to state that “dogmatic assertions of the superiority of one strategy over another are premature” (pp. 242-243). The development of my instrument used a combination of the intuitive-rational strategy and the internal factor-analytical strategy. The intuitive-rational strategy, while subjective, is based on my experience in instructional design for distance education and training and my work as a distance education instructor since 1996. It is also based on my review of the literature. In order not to rest my entire study on my subjective opinion alone, I utilised input from a panel of volunteer distance 66 education researchers and practitioners selected through my review of the literature. In a pilot study, I also utilised the knowledge of graduate-level teachers who were immersed as students in the distance education environment in order to obtain feedback on item clarity within the scales used for assessing high-quality distance education psychosocial environments. Complementing the qualitative experiential input in the intuitive-rational strategy used in this study was the quantitative internal strategy that was used to refine scales and remove faulty items. Item and factor analyses were used to identify items whose removal improved either scale internal consistency or fit to the a priori structure of the instrument. 3.5.3 Review of and consistency with the literature related to distance education learning environments Literature related to distance education is prolific. However, when filtered for indications of its relevance to distance education learning environments, the number of sources drops in number to a level that allows for the salient characteristics of distance education learning environments to be identified and categorised under Moos’ (1974) three dimensions of Relationship, Personal Development, and System Maintenance and Change. The following sections describe views of distance education used in the development of new scales and adaptation of previous scales that are consistent with these dimensions. 3.5.3.1 Relationship dimension Clearly, over one-third of the scales in this study represent characteristics of distance education that tend to fall within the Relationship Dimension (Moos, 1974), which involves individuals being active in their learning environment and engaging with one another. Peer collaboration, review, interaction, exchange, contribution, and community development are keywords continually identified as indicators of high-quality distance education environments that promote thinking and learning (Butler, 2001; Chickering & Ehrmann, 1996; Fahy, 2003; Frederick, Cannon, Umble, Steckler, & Shay, 2001; Golbeck & Sinagra, 2000; Hartley, 2000; Ho & 67 Tabata, 2001; Levine, 2003; Morihara, 2001; Nader, 2001; Oren, Mioduser, & Machmias, 2002; Owen, 2000; Park-Kim, 2001; Zhu & McKnight, 2001). Likewise, the opportunity for interaction between the student and the instructor is continually represented in the literature as a leading indicator of a high-quality distance education environment (Chickering & Ehrmann, 1996; Ho & Tabata, 2001; Morgan & McKenzie, 2003; Morihara, 2001; Park-Kim, 2001; Zhu & McKnight, 2001). Sinagra and Golbeck (2000) summarized the strength of the need for relationship- oriented interaction in distance education by stating that, “from a Piagetian, constructivist perspective, the symmetrical nature of peer relationships presents an ideal context for promoting the development of thinking” (p. 22). Of related importance to this dimension is that asynchronous interactions—student- student, student-instructor, and student-class—are not temporally constrained. Students have the luxury of more time to do much deeper thinking. Also, they are able to devote substantially more time to creating comments or responses in their distance education environments than in traditional face-to-face classroom scenarios. Yet, all individuals involved in the learning process must contribute to the activities of the class in order to benefit themselves and for others in the environment to benefit (Seagreen & Stick, 1999). 3.5.3.2 Personal development dimension The opportunities offered by the distance-education learning environment for self- enhancement and development of self-worth, personal development, independence, intellectual pursuit, and academic achievement make up Moos’ (1974) dimension of Personal Development. Fewer references to characteristics of this dimension were found in the literature directly related to distance education. However, key indicators include such aspects of learning as authentic learning, problem solving, active learning, student reflection, and scaffolded activities (Ballard, 2003; Chickering & Ehrmann, 1996; Hartley, 2000; Ho & Tabata, 2001; Markel, 1999; Merrill, 2001; Morihara, 2001; Owen, 2000). 3.5.3.3 System maintenance and change dimension 68 System Maintenance and Change (Moos, 1974) is the extent to which the distance education environment is orderly, organized, and clear in expectation, student control is possible, and the environment is student focused, responsive to change, and orderly. In this dimension, motivational goals are mastery goals, locus of control is student-oriented, and effort and outcome are seen as interdependent (Cox, Preston, & Cox, 1999; Jegede, Taplin, Fan, Chan, & Yum, 1999). Students are provided with overviews of assignments and advanced organizers (Park-Kim, 2001), there is room for student decision-making (Brookfield, 2003) and activity initiation, and expectations are clearly set forth by the instructor (Owen, 2000). Murphy et al. (2003) discovered, in their content analysis investigation of one distance education class, that graduate teaching assistants were involved in these types of activities for 62.5% of the time in which they interacted with students. 3.5.4 Preliminary scales Based on the literature previously cited, a set of 14 preliminary scales were created to address Moos’ three social organization dimensions. Three of the scales were adapted from the previously-developed Constructivist Learning Environment Survey (CLES) (Taylor, Fraser, & Fisher, 1997), one from the Constructivist On-Line Learning Environment Survey (COLLES) (Taylor & Maor, 2000), and four from the Distance and Open Learning Environment Survey (DOLES) (Jegede, Fraser, & Fisher, 1998). Six new scales identified as being characteristic of high-quality distance education were developed based on converging evidence from the literature noted in Section 3.5.3. The entire set of 14 scales (see Table 3.2) was prepared and delivered to a third-party panel for scrutiny. 3.5.5 Review of preliminary scales by a panel of experts One of the desirable design features of learning environment survey instruments is that they are economical (Fraser, 1986). After the preliminary list of scales in Table 3.1 was developed, they required reduction in the number of items in order to generate an economical survey instrument. 69 Table 3.2. Preliminary set of DELES scales before review by a panel of distance education experts Moos’ Dimension Scales Identified for this Study Scale Description Student Negotiation Extent to which there are “opportunities for students to explain and justify to other students, listen to other students, reflect on viability of others’ ideas…” (Taylor, Fraser, & Fisher, 1997) Student Cohesiveness “Extent to which students are in contact, communicate with each other…” (Jegede, Fraser, & Fisher, 1998, p. 6) Instructor Support “…extent to which the teacher is approachable and responds quickly with feedback…” (Jegede, Fraser, & Fisher, 1998, p. 7) Student Interaction & Collaboration Extent to which students have opportunities to interact with one another, exchange information and engage in collaboration Relationship – Individuals are involved in the environment and support/help each other; personal relationships between inhabitants; involvement, affiliation, support, assistance, peer cohesion Instructor Interaction Extent to which student have the opportunity to interact with the instructor Personal Relevance “Connection between students’ out-of- school experiences” and their classroom experiences (Taylor, Fraser, & Fisher, 1997) Authentic Learning Extent to which students have the opportunity to solve real-world problems that are authentic Active Learning Extent to which students have the opportunity take an active role in their learning Personal Development – Opportunity afforded by the environment for self- enhancement and development of self-esteem; personal growth and development; autonomy, independence, intellectuality, academic achievement, competition Reflective Thinking Extent to which students have the opportunity to reflect upon their own thinking and learning (Taylor & Maor, 2000) Shared Control Extent to which students are invited to share in teacher control of the learning environment, including design and management of learning activities and the determination of assessment criteria (Taylor, Fraser, & Fisher, 1997) Personal Involvement & Flexibility “…extent to which students can make choices about their learning” (Jegede, Fraser, & Fisher, 1998, p. 8) Task Orientation & Material Development “The extent to which the distance education materials provide clear expectations and are well-organised and clear” (Jegede, Fraser, & Fisher, 1998, p. 8) Student Autonomy Extent to which students have opportunities to initiate ideas and make their own learning decisions, and the locus of control is student oriented System Maintenance & Change – The environment is orderly, clear in expectation, control, responsive to change, order, organization, clarity, control Order & Organization Extent to which materials and/or the instructor provide clear expectations, topics and student activities are well organized 70 To achieve the goal of reducing the number of scales, and to reduce subjectivity in that reduction, the seven new scales developed from the literature review and the seven scales adapted from previous learning environment surveys, along with their descriptions, were presented to a panel of experts following precedent in previous scale developments (Fish & Dane, 2000; Jegede, Fraser, & Fisher, 1998). A 14-person panel of distance education researchers and practitioners was assembled after being identified through literature review and a call for participation via 10 international distance education-specific e-mail discussion groups (see Appendix A). The panel was asked to rate each preliminary scale related to its appropriateness, relevance, and suitability for post-secondary distance education. The number of preliminary scales was then reduced based on the panel’s rankings and comments. This reduction resulted in the retention of six scales assumed key to the study of post-secondary distance education (see Table 3.3). Table 3.3. Learning environment scales remaining after review and reduction by expert panel Moos’ Dimension Scale Scale Description Instructor Support “…extent to which the teacher is approachable and responds quickly with feedback…” (Jegede, Fraser, & Fisher, 1998, p. 7) Relationship – Individuals are involved in the environment and support/help each other; personal relationships between inhabitants; involvement, affiliation, support, assistance, peer cohesion Student Interaction & Collaboration Extent to which students have opportunities to interact with one another, exchange information and engage in collaboration Personal Relevance “Connection between students’ out-of-school experiences” and their classroom experiences (Taylor, Fraser, & Fisher, 1997) Authentic Learning Extent to which students have the opportunity to solve real- world problems that are authentic Personal Development – Opportunity afforded by the environment for self-enhancement and development of self-esteem; personal growth and development; autonomy, independence, intellectuality, academic achievement, competition Active Learning Extent to which students have the opportunity take an active role in their learning System Maintenance & Change – The environment is orderly, clear in expectation, control, responsive to change, order, organization, clarity, control Student Autonomy Extent to which students have opportunities to initiate ideas and make their own learning decisions, and the locus of control is student oriented 71 3.5.6 Outcomes of Stage 1 Stage 1, involving identification and development of salient scales for inclusion in the DELES, was completed by reviewing literature pertinent to distance education, identifying relevant psychosocial factors found to be converging in the literature, and then converting those factors into 14 scales, with five representing Moos’ (1974) Relationship Dimension, four representing the Personal Development Dimension, and five related to the System Maintenance and Change Dimension. These 14 scales were submitted to and reviewed by a panel of distance education researchers and practitioners. This resulted in a reduced set of six scales, with two representing the Relationship Dimension, three representing the Personal Development Dimension, and one representing the System Maintenance and Change Dimension. 3.6 STAGE 2 – WRITING INDIVIDUAL ITEMS Once salient scales had been identified, the next step in the development of this survey was to write a set of items to measure each scale. In writing items, consideration was given to maximizing the conceptual similarity between the items within a given scale and avoiding items that could measure any scale other than their intended scale (Fraser, 1986). For the scales of Instructor Support, adapted from the DOLES, and Personal Relevance, adapted from the CLES, items from those instruments were tailored for use in this distance education instrument. The items for the remaining scales were created new specifically for this study. The subsections below outline the method used to develop individual items, beginning with a discussion of the use of negatively-worded items. Next, I outline how the items were assessed for face validity and readability, and I give a description of how items were modified to reduce ambiguity. This is followed by a listing of the items after face validation by a panel of experts and a pilot test of the survey using a small sample of distance education graduate students who were asked to comment on item clarity and page design. 3.6.1 Consideration of negatively-worded or reverse-scored items 72 Reverse-scored items are found in several earlier learning environment instruments such as the My Class Inventory (MCI) (Fraser, 1989), the Science Laboratory Environment Inventory (SLEI) (Fraser, Giddings, & McRobbie, 1992), and the School-Level Environment Questionnaire (SLEQ) (Fisher & Fraser, 1990). However, negatively-worded items or reverse-scored items have questionable utility in guarding against passive responses (i.e., those responses marked without the respondent making a conscious choice) (Barnette, 2000). In terms of response accuracy and internal consistency, studies have revealed higher reliability when all items are worded positively (Chamberlain & Cummings, 1984; Schreisheim, Eisenbach, & Hill, 1991; Schriesheim & Hill, 1981). Negatively-worded items, when mixed with positively-worded items, have been found using factor analyses to solicit differing response patterns (Benson, 1987; Knight, Chisholm, Marsh, & Godfrey, 1988). Barnette (2000) concludes that mixing positive and negative item is not a recommended procedure and that negatively-worded items cannot be considered direct opposites of positively worded counterparts. Further, Taylor, Fraser, and Fisher (1997) explicitly noted, in their remarks about negatively-worded items in the Constructivist Learning Environment Scale (CLES), that students became confused due to the complexity in item structure in relation to the response categories. Therefore, they opted to diverge from the tradition in which learning environment instruments use both positively- and negatively-worded items. Instead, they minimized the use of negatively-worded items in the final version of the CLES to include only one negatively-worded item in the 30-item instrument (Fraser, 1998b). Meanwhile, the What Is Happening In This Class? (WIHIC) questionnaire, that combines modified versions of salient scales from a number of learning environment instruments (Fraser, 1998a, 2002a; Zandvliet, 1999, 2002), was also designed to have no negatively-worded items. Aldridge et al. (2003) chose to use positively- worded items only in their development of the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI) “in order to minimise confusion to students” (p. 8). 73 Because the use of reverse-scored and negatively-worded items can reduce the reliability and validity of instruments and scales, I utilized neither negatively- worded nor reverse-scored items in the DELES. 3.6.2 Validating items Fifty-five items were initially developed and distributed to a panel of distance education practitioners for their comments on each individual item’s suitability, face validity, readability, and freedom from ambiguity (Fraser, 1986; Jegede, Fraser, & Fisher, 1998). The following subsections outline this procedure in more detail. Subsection 3.6.2.1 discusses comments made by a panel of experts, Subsection 3.6.2.2 describes the development of the survey as a Web-based form, Subsection 3.6.2.3 discusses a pilot test of the initial Web-based instrument, and Section 3.6.3 summarizes the results with a table containing the initial instrument. 3.6.2.1 Expert panel’s comments Fifty-five items were modified from existing instruments or were newly developed to fit within the six scales in the first draft of the DELES. These items and their corresponding scale descriptions were submitted to an eight-person panel of volunteer post-secondary distance education practitioners and researchers (see Appendix A) who had previously participated in reviewing and commenting on the face validity of the scales. The panel was asked to respond by e-mail with their comments regarding 1) the suitability and appropriateness of each individual item to its scale, 2) item ambiguity, and also to give 3) additional comments as needed. Seven panel members responded and their comments led to the elimination of seven items, reducing the questionnaire from 55 to 48 items, rewording of some items for clarity, readability, and brevity, and the transfer of one item to a more appropriate scale. This second draft of the items was then prepared for the Web-based form as described in the next subsection. 3.6.2.2 Development of the Web-based form After alterations were made to the instrument’s items based on the panel’s comments, the instrument was converted to a survey that included instructions and a 74 five-point frequency response scale of Never, Seldom, Sometimes, Often, and Always. This frequency response scale follows precedent set in previous instruments such as the Science Laboratory Environment Instrument (SLEI) (Fraser, Giddings, & McRobbie, 1992), the What Is Happening In this Classroom (WIHIC) questionnaire (Aldridge, Fraser, & Huang, 1999), and the Constructivist Learning Environment Survey (CLES) (Taylor, Fraser, & Fisher, 1997). However, what is unique about the DELES is that, unlike most learning environment instruments, it is Web-based. I created the initial version of the DELES for the Web using the online application WWW Survey Assistant (Schmidt, 1996), a Web-based survey editor that generates a Hyper-Text Mark-up Language (HTML) form document and Practical Extraction and Report Language (PERL) Common Gateway Interface (CGI) script for survey developers, without developers having to know how to write a computer program. The Web-based survey instructions asked respondents to select a response using a ‘radio button’ mark in a Web form for each item and then to click a ‘submit’ button to complete and send the form—both of which are common actions for even novice Web users. One of the advantages of this method of gathering data is that, due to the nature of this Web form, respondents were required to make a response for each item, because they were not able to submit the survey otherwise. This guaranteed that 100% of the responses received would be useable. Additionally, the use of a Web-based form allowed students from anywhere in the world, and at any time of the day or night, to complete the survey. It also eliminated errors that can be introduced in data transfer from paper-based questionnaires to statistical analysis software. 3.6.2.3 Pilot test The initial form of the survey was administered to a graduate-level distance education class (N=12) via the Web. Students were asked to respond to survey items and then comment via e-mail regarding the survey’s readability and ambiguity, if any, overall Web page set-up, and useability of the Web page. Because the students were dispersed across a wide geographic area in a true distance education class, e- mail was selected as the most economical means by which to gather the students’ comments. All 12 students completed the survey and responded with comments via 75 e-mail. After the pilot test, minor wording changes were made to reduce item ambiguity on two items. The pilot test data were successfully retrieved from the Web-based database and rendered in a case-wise format in the data analysis software application SPSS version 11.01. This pilot study served three purposes: 1) to test the layout and technical workings of the Web-based instrument, 2) to identify the ease or difficulty in retrieving the raw data for analysis, and 3) to utilize distance education students to obtain feedback on item clarity and Web-page useability. The pilot study was important because the ease of navigating through Web-based survey instruments and the simplicity of the page formatting are of “paramount importance” to the responses that one receives in Web surveys (Bowker & Dillman, 2000, p. 18). 3.6.3 Final version of the DELES Two items were altered based on respondents’ comments after the pilot version of the DELES was administered. Item 5 was changed from “In this class…the instructor adequately addresses my class content questions” to “In this class…the instructor adequately addresses my questions”, Item 30 in Table 3.4 previously read: “In this class…I am given ill-defined assignments.” While the term ‘ill defined’ is perhaps common in the educator’s lexicon, it is not common for a more general audience. Therefore, the item was altered to read: “In this class…I am given assignments with more than one solution.” After these changes were made, an new Web version of the survey was created for field testing, thus completing Stage 2— writing individual items. Table 3.4 presents the DELES dimensions, scale names, scale descriptions, and items as used in the field-test version. 76 Table 3.4. Scale descriptions and sample items after pilot test Dimension Scale Scale Description Items Instructor Support “…extent to which the teacher is approachable and responds quickly with feedback…” (Jegede, Fraser, & Fisher, 1998, p. 7) In this class… 1. If I have an inquiry, the instructor finds time to respond. 2. The instructor helps me to identify problem areas in my study. 3. The instructor responds promptly to my questions. 4. The instructor gives me valuable feedback on my assignments. 5. The instructor adequately addresses my questions. 6. The instructor treats me with respect. 7. The instructor encourages my participation. 8. It is easy to contact the instructor. 9. The instructor provides me with positive and negative feedback on my work. Relationship Student Interaction & Collaboration Extent to which students have opportunities to interact with one another, exchange information and engage in collaboration In this class… 10. I work with others. 11. I relate my work to other’s work. 12. I share information with other students. 13. I discuss my ideas with other students. 14. I collaborate with other students in the class. 15. Group work is a part of my activities. 16. I share my assignments with others. Personal Relevance “Connection between students’ out-of-school experiences” and their classroom experiences (Taylor, Fraser, Fisher, 1997) In this class… 17. I can relate what I learn to my life outside of university. 18. I am able to pursue topics that interest me. 19. I can connect my studies to my activities outside class. 20. I apply my everyday experiences in class. 21. I link class work to my life outside university. 22. I learn things about the world outside university. 23. I apply my out-of-class experience. Personal Development Authentic Learning Extent to which students have the opportunity to solve real-world problems that are authentic In this class… 24. I study real cases related to the class. 25. I use real facts in class activities. 26. I communicate with practitioners of the topic I am studying. 27. I work on assignments that deal with real-world information. 28. I work with real examples. 29. I enter the real world of the topic of study. 30. I am given assignments with more than one solution. continued 77 Active Learning Extent to which students have the opportunity take an active role in their learning In this class… 31. I take an active role in my learning. 32. I contribute to my own learning. 33. I explain my thinking. 34. I explore my own strategies for learning. 35. I seek my own answers. 36. I solve my own problems. 37. The instruction involves me in finding my own meaning related to what we study. 38. I am encouraged to think for myself. System Maintenance & Change Student Autonomy Extent students have opportunities to initiate ideas and make their own learning decisions, and the locus of control is student oriented In this class… 39. I incorporate my ideas in the class. 40. My ideas are valued. 41. I am motivated by various opportunities to be successful. 42. I make decisions about my learning. 43. I work during times I find convenient. 44. I am in control of my learning. 45. I play an important role in my learning. 46. I contribute to the class. 47. I give input. 48. I approach learning in my own way. Response choices are: Always, Often, Sometimes, Seldom, and Never. © 2003, Scott Walker 78 3.6.4 Scale of student enjoyment In addition to the scales and items listed in Table 3.4, an attitudinal scale of Enjoyment of Distance Education was included in the field test in order to gather data for investigating any associations between the psychosocial scales of the DELES and the attitudinal scale of Enjoyment (see Appendix C). Subsection 2.2.7.3 of Chapter 2 reviews previous studies that have sought to identify outcome- environment associations. The scale of Enjoyment of Distance Education consists of eight items that were modified from the Test of Science-Related Attitudes (TOSRA)(Fraser, 1981). The Enjoyment scale used the same Always, Often, Sometimes, Seldom, and Never response categories as those of the DELES to facilitate continuity in responses instead of using the TOSRA’s original response options of Strongly Agree, Agree, Not Sure, Disagree, and Strongly Disagree. Table 3.4 contains a listing of the items in the Enjoyment scale. Table 3.5. Items in the Enjoyment scale Scale Items Enjoyment of Distance Education • Distance education is stimulating. • I prefer distance education. • Distance education is exciting. • Distance education is worth my time. • I enjoy studying by distance. • I look forward to learning by distance. • I would enjoy my education more if all my classes were by distance. • I am satisfied with this class. Response choices are: Always, Often, Sometimes, Seldom, and Never. 3.7 STAGE 3 – FIELD TESTING AND ANALYSIS The third stage of developing a learning environment instrument involves conducting a field test with the target population, followed by statistical analysis of the item responses in terms of internal consistency and validity (Fraser, 1986; Jegede, Fraser, & Fisher, 1998). The purposes of these analyses are to refine the instrument and to provide evidence of the overall reliability and validity of the refined scales. This section describes the field-testing, including a description of how 79 the field test was conducted and the sample of students who responded. This is followed by a brief description of the data analyses methods, which are described more fully in Chapter 4. 3.7.1 Field testing Because this instrument was located on the World Wide Web, it was completed by respondents through a Web browser. Therefore, just as time and place are not important in asynchronous distance education, neither were they important when responding to this instrument. In October 2001, an e-mail call for participation was made during the Teaching Online in Higher Education Conference to all 227 online conference participants asking those with post-secondary distance education students to ask their students to respond to the DELES field test on the Web. In November 2001, another call for participation was made to Curtin University of Technology – Science and Mathematics Education Centre staff who were teaching via distance that semester, requesting they ask their distance education students to respond to the Web-based instrument. These two calls for participation resulted in 106 useable responses. In February 2002, another e-mail call for participation was made to a number of distance-education oriented e-mail discussion groups and to conference participant e-mail lists. Also, through direct e-mail to college and university instructors who were known to be teaching online during that semester, I asked them to ask their post-secondary distance education students to respond to the Web-based DELES. The 2002 calls for participation resulted in 574 useable responses. The 2001 and 2002 responses combined resulted in 680 sets of replies useable for analysis. 3.7.2 Field test sample Responses were collected from 186 undergraduate students, 364 Masters-level graduate students, and 130 Doctoral-level graduate students. The majority of the responses came from students studying in the United States, totalling 569, while 42 responses were from Australia, 12 from Canada, and 48 from New Zealand. The remaining responses came from students in nine separate countries and were included in the response set. However, unlike traditional learning environments 80 research, this sample does not focus on a known class or school population, which offers some limitation to this method of field testing. 3.7.3 Item analysis and factor analysis Item and factor analyses were conducted to serve two purposes: to refine the DELES scales; and to provide evidence regarding reliability and validity of the refined scales. Data were analysed for internal consistency reliability using the SPSS 11.01 software to calculate Cronbach’s alpha (α) coefficient to measure internal consistency in terms of intercorrelations among items. Specifically, this is a measure of the extent to which items within a scale measure the same construct as the other items in the same scale. Those items that were not highly correlated with their respective scales were removed and data were reanalysed until all the items with the lowest item-scale correlations were removed and the alpha coefficient was maximized. The construct validity of the a priori scales that were developed using the intuitive- rational strategy were investigated using principal component factor analysis. This method of analysis is used to ascertain the fundamental structure of a relatively large set of variables (Garson, 2001) and to refine the scales where necessary rather than to develop new scales. Essentially, this establishes whether items within a scale measure the construct assessed by that scale and not the construct assessed by any other scale. A factor loading of 0.50 was used as the criterion for retaining items based on precedent (Fish & Dane, 2000). This analysis led to the elimination of items. The analyses of the refined data set provided evidence to support the overall reliability and factorial validity of the refined scales. The results of these analyses are presented in Chapter 4. 3.8 CHAPTER SUMMARY 81 The Distance Education Learning Environment Survey (DELES) was developed to assess characteristics of tertiary distance education environments from students’ personal perspectives, based upon an overarching classical structure of Moos’ (1974) three social organization dimensions. The method by which the DELES was developed followed a three-stage approach modified from Fraser (1986) and Jegede, Fraser, and Fisher (1998), that included Stage 1, the identification of salient scales, Stage 2, modifying items from previous learning environment instrument and developing new items for newly-developed scales, and Stage 3, field testing the instrument and conducting item analysis and validation analyses. In Stage 1, eight scales were modified from three previous learning environment instruments—the CLES (Taylor, Fraser, & Fisher, 1997), the COLLES (Taylor & Maor, 2000), and the DOLES (Jegede, Fraser, & Fisher, 1998)—while six scales were newly developed based on converging literature. These 14 scales were reduced to the six most salient scales by means of review and comment by a panel of distance education experts and practitioners. One of the remaining scales was adapted from the DOLES and one was adapted from the CLES. Four scales were newly developed specifically for the DELES. Stage 2 required writing individual items for the four new scales and modifying items from the CLES and the DOLES scales to fit the distance education learning environment. Fifty-five items were subjected to comments from the panel of experts (N=8) and were subsequently modified and reduced to 48 items. These 48 items were formatted for use in a Web-based instrument using the response categories of Never, Seldom, Sometimes, Often, and Always, and then subjected to a Web-based pilot test that sought comments about the useability of the Web form from 12 graduate-level university students enrolled in a distance education course. Stage 3, as described in this chapter, was a field test of the initial Web-based DELES which involved 680 responses during a two-semester data acquisition time frame. The resulting data were analysed for internal consistency reliability using Cronbach’s alpha coefficient to determine intercorrelation among items. The 82 construct validity of the a priori scales was investigated using principal component factor analysis to ascertain the structure of the scales and to refine the scales. Chapter 4 presents and describes in detail the analysis of the field test data to provide information about scale reliability and validity. It also reports the results of analyses of associations between the six learning environment scales and an attitudinal scale assessing student enjoyment of distance education. 83 Chapter 4 DATA ANALYSIS AND RESULTS 4.1 INTRODUCTION Chapter 3 described the field testing component of Stage 3 of the development and validation of the Distance Education Learning Environment Survey (DELES), which involved gathering 680 unique and useable responses from post-secondary distance education students through a Web-based survey instrument. This chapter goes on to describe the conclusion of the three-stage approach to developing a valid learning environment instrument by, first, detailing the steps in the factor analysis and item analysis that I used to determine which items should be eliminated to improve the instrument’s reliability and validity and, second, reporting evidence of the overall reliability and validity of the refined version. The statistical analyses were conducted using the SPSS version 11.01. The item and scale analyses resulted in a final six- scale, 34-item version of the DELES that initially began with 48 items in six scales. This chapter also outlines the methods and results of analyses used for identifying associations between the six psychosocial learning environment scales and student satisfaction. The satisfaction scale was comprised of items related to enjoyment of distance education. Associations between the psychosocial scales and the satisfaction scale were investigated using simple correlations and multiple regression analyses modelled on previously-conducted studies of associations between student satisfaction and psychosocial environment scales (e.g., Fraser, 1998b; Zandvliet, 2002; Chapter 2, Section 2.5). 4.2 RELIABILITY AND VALIDITY OF THE DELES As outlined in detail in Chapter 3, the development of the Distance Education Learning Environments Survey (DELES) utilized the intuitive-rational strategy in which only items with high internal consistency remain in the final instrument. It also relies upon the internal strategy (Hase & Goldberg, 1967), whereby only those items with high factor loadings on their own scales and low loadings on other scales 84 are kept in the final instrument. This section describes the methods by which the DELES was refined and its validity and reliability were determined. The aim of these analyses was to establish the DELES as a reliable and valid learning environment instrument based on data obtained from the sample of 680 distance education students enrolled in post-secondary classes as described in detail in Chapter 3, Sections 3.3 and 3.7.2. The sample included undergraduate students, Masters-level graduate students, and doctoral-level graduate students. The majority of those in the sample population were studying at institutions in the United States, while some responses were from Australia and New Zealand, and a small number of respondents were studying in a variety of other countries. 4.2.1 Validity Validity was considered in terms of content or ‘face’ validity and also in terms of construct validity during the development of the DELES. Chapter 3, Section 3.5.5 addressed verification of content validity of the scales through a review by a 14- person panel (see Appendix A) of distance education researchers and practitioners. Section 3.6.2 addressed the content validity of individual items through an item review by an eight-person panel of distance education researchers and practitioners (see Appendix A). Construct validity, on the other hand, was investigated as described below using principal component factor analysis with varimax rotation and Kaiser normalization. The aim of factor analysis is to ascertain the fundamental structure of a comparatively large set of variables (Garson, 2001). This method of analysis is consistent with the intuitive-rational method of learning environment instrument development (Fraser, 1986) and has recently been used (Dorman, in press) to determine if items load on a priori scales. In essence, in my study, factor analysis provides information about whether items within a given scale are measuring that scale and no other scale. Only those items with a factor loading of at least 0.50 (Fish & Dane, 2000) with their own scale, and less than 0.50 with all other scales, were kept in the refined instrument. 85 With the advent of easy-to-use computer programs for analysing data, factor analysis has gained ground as a statistical tool (George & Mallery, 2001). Cattell brought attention to the use of factor analyses in the 1940s by reducing 4,500 personality attribute names down to 200 questions that measured 16 personality traits in an instrument that is in its fifth version today—the Sixteen Personality Factor Questionnaire (16PF) (Cattell & Cattell, 1995). Nevertheless, despite the ease with which one can conduct the statistical operations of a factor analysis today, dealing with “indeterminacies are logical, not statistical, in nature” (Kim & Mueller, 1978, p. 46). Therefore, the determination of which items to discard were based upon background knowledge of the subject matter, supported by statistical analysis, as outlined in Hase and Goldberg’s (1967) intuitive-rational strategy for developing instruments of this nature. Table 4.1 presents the rotated component matrix based on individual items. Fourteen (14) ‘faulty’ items of the original 48 items were identified and removed, leaving only 34 items suitable to remain in the instrument. The faulty items that were removed either loaded below the 0.50 threshold in their own scale or above 0.50 on scales other than their own. Six scales were originally developed for the DELES field test and, after factor analysis, the same six scales remained: Student Interaction and Collaboration, Instructor Support, Personal Relevance, Authentic Learning, Student Autonomy, and Active Learning. In the Student Interaction and Collaboration scale, seven items were originally developed and only one was lost. The Instructor Support scale was developed with nine items and factor analysis eliminated one item. In the original seven-item scale of Personal Relevance, no items were lost. However, in the scale of Authentic Learning, three items were lost of the original eight. The Student Autonomy scale was originally developed with 10 items, but lost five. Finally, the scale of Active Learning was conceived with seven items, yet lost four in factor analysis. Overall a six-scale instrument with 34 items was left after factor analysis. Exploratory analyses consisted, in part, of considering factor loadings both with and without a scale of satisfaction included. The satisfaction scale was included in my study so that I could investigate relationships between student satisfaction with 86 Table 4.1. Factor loadings for a refined 34-item version of the DELES Factor Loading Item Student Interaction & Collaboration Instructor Support Personal Relevance Authentic Learning Student Autonomy Active Learning INSTSUP1 .69 INSTSUP2 .73 INSTSUP3 .83 INSTSUP4 .84 INSTSUP5 .80 INSTSUP6 .63 INSTSUP7 .62 INSTSUP8 .76 INTER9 .90 INTER10 .83 INTER11 .85 INTER12 .86 INTER13 .90 INTER14 .87 RELEV15 .75 RELEV16 .69 RELEV17 .81 RELEV18 .78 RELEV19 .83 RELEV20 .55 RELEV21 .70 AUTH22 .61 AUTH23 .77 AUTH24 .80 AUTH25 .84 AUTH26 .69 ACT27 .63 ACT28 .79 ACT29 .75 AUTON30 .65 AUTON31 .65 AUTON32 .81 AUTON33 .75 AUTON34 .61 % variance 14.31 14.10 13.88 10.35 8.50 6.01 Factor loadings smaller than 0.50 have been omitted. INSTSUP = Instructor Support, INTER = Student Interaction & Collaboration, RELEV = Personal Relevance, AUTH = Authentic Learning, ACT = Active Learning, and AUTON = Student Autonomy. N=680. distance education and students’ perceptions of the psychosocial learning environment (see Chapter 2, Subsection 2.2.7). Because factor analysis uses a correlation matrix of data from a sample population, investigators can conclude that the outcome will be different when differing sample data are used. Therefore, investigators use a statistical test with which to examine the adequacy of a particular sample (Kim & Mueller, 1978). 87 In this study, I relied upon the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy that served to evaluate the appropriateness of using factor analysis based on my sample data. The KMO uses interpretive adjectives to rate the adequacy of samples. A measure of 0.90 and greater is marvellous, 0.80 to 0.89 is meritorious, 0.70 to 0.79 is middling, 0.60 to 0.69 is mediocre, scores of 0.50 to 0.59 are miserable, and those falling below 0.50 are unacceptable (George & Mallery, 2001). The KMO calculation for this study determined a distribution of adequate values for conducting factor analysis to be 0.91 when the satisfaction scale was considered. When the scale of satisfaction was not considered in the factor analysis the KMO was also 0.91. Analyses from both perspectives led to the same faulty items. Because the scale of satisfaction was adapted, with permission (B. J. Fraser, personal communication, June 15, 2001), from the Test of Science-Related Attitudes and has been validated and used since at least 1981 (Fraser, 1981), it was determined that the final factor analysis and presentation of data would not include the scale of satisfaction as recommended by Majeed, Fraser, and Aldridge (2002). Further exploratory analysis was conducted by comparing the simple structures of a normalized orthogonal (varimax) rotation to that of a normalized oblique (oblimin) rotation (Fish & Dane, 2000). Again, this led to identification of the same faulty items. Only the orthogonal rotation findings from the final analysis are presented in Table 4.1. The bottom row of Table 4.1 shows the percentage of variance for each factor. The scale of Student Interaction and Collaboration accounted for the highest proportion of variance among the items—the eigenvalue divided by the number of variables—at 14.31%. This was followed closely by the Instructor Support scale which explained 14.10% of the variance. The Personal Relevance scale explained 13.88% of the variance, while the Authentic Learning scale explained 10.35%. These four scales account for slightly more than half (52.64%) of the variance in the items within the instrument. To a lesser degree, the scale of Student Autonomy explained 8.50% of the variance while the Active Learning scale accounted for only 6.01% of the variance among the variables. The cumulative variance explained by all six DELES scales was 67.15%. While nearly 33% of the variance was unaccounted for, one must assume that either the unaccounted variance is due to measurement error or that 88 the true variance is exclusive to the individual items (Jaeger, 1993). Nevertheless, reducing 34 items to six planned scales offers economy of description of what is being measured. The first factor identified was Student Interaction and Collaboration. All a priori items loaded exclusively on this factor with loadings ranging from 0.83 to 0.90, except for one item which was lost due to its low factor loading. The items within this scale also accounted for the maximum proportion of variance at 14.31%. The items in the second factor, Instructor Support, loaded exclusively on this factor, with the loadings ranging from 0.62 to 0.84. Again, one item was lost due to its loading falling below the 0.50 threshold. The third identified factor was that of Personal Relevance with its a priori items loading entirely on this scale and with factor loadings ranging from 0.55 to 0.83. No items were lost from this scale due to low factor loadings. The fourth factor was identified as Authentic Learning. The items that were kept had factor loadings ranging from 0.61 to 0.84. Three of these items were lost due to factor loadings being below 0.50. The fifth identified factor was Student Autonomy. Four of the original items loaded greater than 0.50 within other factors, and one item loaded below the threshold on its own scale. The remaining items loaded between 0.61 and 0.81. The sixth identified factor was that of Active Learning. Three items in this scale were lost because they loaded too high on other scales. One item within Active Learning had a loading with its own scale that did not reach the threshold and was subsequently removed. The remaining three items associated with this factor had loadings that ranged between 0.63 and 0.79. When considering discriminant validity through factor analysis, each scale should measure a unique dimension not measured by another scale. Discriminant validity was improved by removing any item whose factor loading either fell below the 0.50 threshold within its a priori assigned scale or was above 0.50 with and of the other five scales. Seven items were lost due to low loadings with their own scales, while another seven were lost due to high loadings with scales other than their a priori scales. Further research replicating this study should be conducted in the future with larger and different samples to demonstrate the replication of similar results, because any 89 given sample will never perfectly reflect its population. While deviations might be reduced with increases in sample size, deviations will still occur (Kim & Mueller, 1978) even with larger samples. Therefore, it will be important to replicate the factor analysis procedures each time the DELES is used in order to demonstrate that the factor structure presented here is not peculiar to my sample. 4.2.2 Reliability One interpretation of reliability, synonymous with consistency, dependability, and accuracy (Kerlinger, 1986), involves whether or not an instrument is consistent internally. Cronbach’s alpha (α) coefficient is designed to measure the internal consistency of a scale in terms of item intercorrelations. That is, to what extents do items within a scale measure the same construct as the other items in the scale? The alpha coefficient has a range of -1 to 1, with 0 indicating no internal consistency, 1 indicating perfect internal consistency, and -1 indicating perfect inconsistency (George & Mallery, 2001). In the development of the DELES, each scale was assessed for internal consistency. Table 4.2 presents the alpha reliability for each refined DELES scale, as well as for the separate satisfaction scale, for the sample of 680 students. Table 4.2. Scale reliability using Cronbach’s alpha coefficient for DELES and Enjoyment scales Scale Number of Items α Reliability Instructor Support 8 .87 Student Interaction & Collaboration 6 .94 Personal Relevance 7 .92 Authentic Learning 5 .89 Active Learning 3 .75 Student Autonomy 5 .79 Enjoyment 8 .95 N=680 Low internal consistency can result from any number of sources. For example, items about which survey participants have no opinion or insufficient information, or those that require too accurate a response, can lead to data that are not reliable (de Vaus, 2001). Yet, high reliability is not necessarily an assurance of high-quality results or 90 validity. However there can be no high-quality results or validity without reliability (Kerlinger, 1986). The internal consistency reliability (coefficient alpha) ranged from 0.75 to 0.94 for the six DELES scales. Using a generally applied ‘rule-of-thumb’ this range is considered acceptable to excellent (George & Mallery, 2001), since the closer the alpha is to 1, the greater the internal consistency of the items. The alpha for the scales of Student Interaction and Collaboration (0.94) and Personal Relevance (0.92) are considered ‘excellent’, while the scales of Authentic Learning (0.89) and Instructor Support (0.87) are considered ‘good’. The remaining DELES scales of Student Autonomy (0.79) and Active Learning (0.75) are deemed ‘acceptable’. Likewise, the attitude scale of Enjoyment had an alpha of 0.95, which can be considered ‘excellent’ using this rule-of-thumb. 4.2.3 Limitations of the data collected In presenting validation and reliability results, it must be acknowledged that this study’s procedures do not completely conform to those followed in many past learning environment instrument developments and validations. In many cases, one would also report the factor analysis and alpha reliability data for class means, as well as the results of the analysis of each scale’s ability to differentiate between classrooms, as in previous investigations (Fraser, 1998b; Wubbels, 1993; Zandvliet, 2002). In this preliminary validation, data were not captured in a way that allowed identification of each student’s class membership. The data were only collected with the individual student as the unit of analysis. Class data are often captured and calculated in order to investigate the degrees of difference or similarity between two units of statistical analysis, that of the individual student and that of the class mean (Fraser, 1986), which provides much richer data interpretation. Likewise, insufficient data were available to distinguish reliability between countries. Further data collection with a larger, more-defined international sample is desirable to allow these analyses to be conducted in an ongoing effort to supplement the validation information on the DELES. 91 4.3 FINAL VERSION OF THE DELES To recapitulate, factor analysis was conducted on 680 responses of a preliminary version of the DELES. This resulted in the removal of 14 of 48 a priori learning environment items that either loaded below a 0.50 threshold in their own scale or loaded at greater than 0.50 on a scale other than their own. The construct validity of the DELES was supported using principal component factor analysis, whereas the internal consistency reliability of the refined DELES scales was reported using Cronbach’s alpha coefficient. Table 4.3 shows the wording of the 34 items that remained in the DELES after this preliminary validation. The complete DELES, including the five-point response categories of Always, Often, Sometimes, Seldom, and Never, is shown in Appendix B. 4.4 ASSOCIATIONS BETWEEN STUDENT ENJOYMENT AND LEARNING ENVIRONMENT FACTORS In previous learning environment research, investigations of associations between student outcomes and the nature of the classroom environment are common (Fraser, 1986, 2002a; Fraser, Giddings, & McRobbie, 1992) (see Chapter 2, Subsection 2.2.7.3 for a detailed review of past studies of the influence of learning environments on student outcomes). This section describes the analyses and results for associations between the dependent variable of students’ enjoyment of distance education and the independent variables consisting of scales of the Distance Education Learning Environment Survey (DELES). This section also elaborates on the strongest and weakest associations, offering speculative explanations. This is followed by a statement regarding the limitations of this part of my research and possible future opportunities for enhancing the use of the DELES when considering student outcomes in terms of their distance education learning environments. 4.4.1 Associations between the DELES scales and students’ enjoyment of distance education Associations between the attitudinal dependent variable and the independent variables of the psychosocial learning environment were explored for this study. 92 This section describes the associations between students’ enjoyment of distance education using an eight-item scale of Enjoyment (see Table 3.4 and Appendix B) modified from the Test of Science-Related Attitudes (TOSRA) (Fraser, 1981) and the six refined DELES learning environment scales previously described. Analyses were conducted using simple correlation and multilinear regression analysis using the individual student as the unit of analysis. Table 4.4 presents the results. Table 4.3. Listing of items in the refined version of each Distance Education Learning Environment Survey (DELES) scale Scale Items Instructor Support In this class… 1. If I have an inquiry, the instructor finds time to respond. 2. The instructor helps me identify problem areas in my study. 3. The instructor responds promptly to my questions. 4. The instructor gives me valuable feedback on my assignments. 5. The instructor adequately addresses my questions. 6. The instructor encourages my participation. 7. It is easy to contact the instructor. 8. The instructor provides me with positive and negative feedback on my work. Student Interaction & Collaboration In this class… 9. I work with others. 10. I relate my work to other’s work. 11. I share information with other students. 12. I discuss my ideas with other students. 13. I collaborate with other students in the class. 14. Group work is a part of my activities. Personal Relevance In this class… 15. I can relate what I learn to my life outside of university. 16. I am able to pursue topics that interest me. 17. I can connect my studies to my activities outside of class. 18. I apply my everyday experiences in class. 19. I link class work to my life outside of university. 20. I learn things about the world outside of university. 21. I apply my out-of-class experience. Authentic Learning In this class… 22. I study real cases related to the class. 23. I use real facts in class activities. 24. I work on assignments that deal with real-world information. 25. I work with real examples. 26. I enter the real world of the topic of study. Active Learning In this class… 27. I explore my own strategies for learning. 28. I seek my own answers. 29. I solve my own problems. Student Autonomy In this class… 30. I make decisions about my learning. 31. I work during times that I find convenient. 32. I am in control of my learning. 33. I play an important role in my learning. 34. I approach learning in my own way. Response choices are: Always, Often, Sometimes, Seldom, and Never. 93 Table 4.4. Associations between six DELES scales and student enjoyment using simple correlation and multiple regression analyses Psychosocial Scale r β Instructor Support .25** .12** Student Interaction and Collaboration .23** .14** Personal Relevance .31** .23** Authentic Learning .28** .16** Active Learning .12** .00 Student Autonomy .24** .11** Multiple correlation (R) R2 .46** .21** N=680 individuals, **p<.01 In Table 4.4, the simple correlation (r) represents the bivariate relationship between Enjoyment and each psychosocial learning environment scale. The standardized regression coefficient (β) represents the association between the Enjoyment scale and each learning environment scale when all other learning environment scales are mutually controlled. The coefficient of multiple correlation (R) indicates the multivariate relationship between the Enjoyment scale and the set of six learning environment scales, with 1.0 being a perfect correlation. The coefficient of multiple determination (R2) denotes the proportion of variance in Enjoyment accounted for by the set of six learning environment scales. Table 4.4 shows that the simple correlation between Enjoyment and the DELES scales ranges from 0.12 to 0.31. All simple correlations are positive and statistically significant (p<0.01) for all six scales. The multiple correlation between Enjoyment and the set of DELES scales (R) is 0.46 and is statistically significant. The proportion of variance explained (R2) is 0.21. In order to ascertain which DELES scales are independently related to Enjoyment when all other DELES scales are mutually controlled, the standardized regression coefficients in the last column of Table 4.4 were examined. This table shows that all DELES scales, with the exception of Active Learning, are independently and positively significantly related to Enjoyment. All significant simple correlations and regression weights are positive, suggesting that a higher level on the DELES scales is associated with greater enjoyment of distance education. 94 4.4.2 Discussion of the strongest Enjoyment-environment association Table 4.4.1 demonstrates that the strongest association between Enjoyment and a DELES scale occurred for Personal Relevance for both simple correlation and multiple regression analyses. Personal Relevance in this study consists of such notions as students’ ability to relate class content to personal and professional activities outside of the university setting. The Personal Relevance scale also considers individual students’ opportunities for being able to bring out-of-class experiences into the class setting. Perhaps this relatively strong association can be interpreted as one that we should expect in post-secondary education—students enjoy a distance education class more when they can relate the subject matter of that class to their personal lives. Otherwise, why would students take the class? A brief consideration of the level of study being undertaken by the students who responded to the field test of the DELES, from which these data were analysed, might offer further insight into the strength of this Enjoyment/Personal Relevance association. It could be argued that, given the fact that 72.6% (see Chapter 3, Table 3.1) of the respondents were graduate students, and graduate students tend to be more conscientious in terms of their time and personal reasons for advancing their education, these students enjoy more that which has significance to their lives outside class. Of course, this argument is speculative and requires more careful study in the future. 4.4.3 Discussion of the weakest Enjoyment-environment association In Table 4.4, the weakest Enjoyment/environment association for both the simple correlation and multiple regression analyses is represented by the scale of Active Learning. Active learning exemplifies the extent to which students take part in their learning by solving their own problems, exploring their own learning strategies, and seeking their own answers. While active learning was identified as a key indicator of high-quality distance education in the literature (see Chapter 3, Section 3.5.3.2) related to personal development, this scale had a weak association with students’ enjoyment of distance education. Given further exploration, the popular notion that 95 distance education, by virtue of the medium through which it is conducted, replaces passive learning through “active exploratory and inquiry-based learning” (Wilson, 2002, p. 638) could be held up as common, speculative, anecdotal distance education mythology. It could also be argued in this case that post-secondary students are more comfortable with their “habits of passive learning” (Sutdliff, Cogdell, Hansell, & McAteer, 1999, p. 53) that are common in many educational settings, thus resulting in this lack of association between students enjoying distance education and learning in active ways. A potential approach to exploring this issue further would be to develop an additional scale related to Passive Learning and to explore how that scale is associated with student enjoyment of distance education. This lack of association between Enjoyment and Active Learning does not tell us, though, that active learning diminishes the quality of distance education. It merely identifies a weak relationship between students being active in their learning and overall enjoyment of distance education. It could be the case that active learning is not associated well with enjoyment in the face-to-face learning environment either. Because Active Learning is a new scale that was developed exclusively for the DELES, and has not been used in other learning environment instruments, further research with this scale is desirable. 4.4.4 Consistency with previous research on associations between attitudes and learning environments Looking back to Walberg’s (1981) discussion of educational productivity (see Chapter 2, Subsection 2.2.5), it is evident that learning environments have influences on and associations with student outcomes, both cognitive and affective (Fraser, 2002a). Learning environment research has commonly investigated and demonstrated positive associations between students’ perceptions of their learning environments and students’ attitudes across subjects, across levels of study, and across international borders (Fraser, 1986, 2002a) (see Chapter 2, Subsection 2.2.7.3). While the individual results of this productive body of research are too broad to describe here, recent selected literature has reported positive 96 attitude/learning environment associations. For example, Scott and Fisher (2001) reported statistically significant (p<0.001) associations between a scale of Enjoyment of Science Lessons and the eight learning environment scales of a Malaysian version of the Questionnaire on Teacher Interaction for primary school students (QTIE) (the QTIE is the Malay translation of the English-language Questionnaire on Teacher Interaction for Primary schools (QTIP)). They found that students enjoyed their science lessons more when teachers demonstrated leadership and were strict, helpful/friendly, and understanding. They discovered a negative association indicating that students enjoyed their science lessons less when teachers were uncertain, dissatisfied and admonishing, and allowed student freedom. They noted that, when all the other learning environment scales were held constant, teachers’ helpful and friendly behaviour had the most impact (β=0.30) on the students’ enjoyment of their science lessons. Likewise, Zandvliet (2002) reported statistically significant (p<0.01) positive associations between students’ attitudes using selected items from the Test of Science Related Attitudes (TOSRA) and five scales of the popular What is Happening in this Class (WIHIC) questionnaire (Aldridge, Fraser, & Huang, 1999). When all the learning environment scales were mutually controlled, Task Orientation, the extent to which it is important to stay on the subject matter, had the strongest association with Student Satisfaction (β=0.46). This suggests that, in the technology-rich classroom, the focus of Zandvliet’s study, students are more satisfied, and perhaps more productive, when they are allowed to complete activities and are not pulled from their tasks. As with these previous studies and others, my investigation of associations between students’ attitudes and psychosocial scales resulted in statistically significant positive associations. This adds more support to Walberg’s (1981) notion that learning environments have influences on and associations with student outcomes. However, as with any study, my research is not without its limitations (see next section). 97 4.4.5 Limitations of analysis of Enjoyment-learning environment associations The present study involving DELES data provides the first study of post-secondary distance education that investigates associations between the distance education learning environment and student satisfaction. However, this is not to suggest my use of the DELES scales to investigate Enjoyment-environment associations is not without limitations. The investigation of associations between student enjoyment of distance education and the six scales of the DELES might have been more informative if class mean data were also captured so that so that satisfaction-environment associations could be explored at the class mean level. Likewise, the outcome of student enjoyment could be complemented by measures of additional cognitive and affective student outcomes. Further, the sample used for this study is not necessarily representative of any particular population. The sample would need to be better defined and associations further investigated in order to draw more confident and generalizable conclusions for use in distance education learning environments. Lastly, as in previous research, the correlational analyses used in my study do not establish a causal link between learning environment and student enjoyment. 4.5 CONCLUSION This chapter has presented the last of the three stages in the development and validation of the Distance Education Learning Environment Survey—data analysis for a sample of 680 students. Principal component factor analysis was conducted to support the six-scale a priori structure of this psychosocial learning environment instrument. Fourteen (14) faulty items of an original 48 items were identified and removed. The faulty items that were removed either loaded below a 0.50 threshold in their own scale or above 0.50 on scales other than their own. The remaining 34 items in the six a priori scales accounted for a cumulative variance of 67.15%, with the scale of Student Interaction and Collaboration accounting for the highest variance of 14.31%. 98 Internal consistency reliability analysis was calculated using Chronbach’s alpha coefficient to identify to what extent the items within each scale measure the same construct as other items within their scale. Overall, the alpha coefficients of the six scales ranged from 0.75 to 0.94. The analyses of these data based on 680 field test responses supported the efficacy of a new learning environment survey instrument suitable for use in exploring post-secondary distance education environments. Also reported were simple and multiple correlation analyses of the associations between six distance education psychosocial learning environment scales and a dependent variable of student enjoyment of distance education. Similar to previous investigations of associations between student attitudes and learning environments, the simple correlation analysis (r) resulted in statistically significant (p<0.01) positive associations for all scales. In the case of the DELES, these correlations ranged from 0.12 to 0.31. When all other learning environment scales were mutually controlled, the standardized regression coefficients (β) identified five statistically significant independent associations ranging from 0.11 to 0.23, with the scale of Personal Relevance having the strongest association. The multiple correlation between student enjoyment and the set of DELES scales (R) was 0.46 and statistically significant. Questions about student satisfaction with distance education learning environments are important ones that have not been asked before. Too many ‘feel good’ distance education assumptions are made and must be substantiated. Practically speaking, in terms of the bottom line in many higher education policy rooms, student retention in distance education is a real concern. Researchers and practitioners must explore these associations further, not only for the sake of college and university fiscal standing, but also for the sake of developing distance education learning environments that support positive student attitudes. It follows that students who are satisfied with distance education are also likely to produce higher achievement outcomes, which is another crucial consideration in the ongoing development of high-quality distance education. Finally, the limitations acknowledged in this chapter include the fact that preliminary validation and reliability analyses for the DELES did not involve class mean data. 99 Perhaps future inclusion of a larger sample, that would permit validation and reliability analyses to be conducted between countries, would provide replication of the present findings, improve generalisability, and further support the DELES as a tool for assessing learning environments in distance education. Similarly, in the investigation of associations between student enjoyment and the six DELES scales, the use of class mean data could offer further insight. The investigation of associations between student outcomes and the learning environment could also be complemented by the inclusion of a range of cognitive and affective scales. Additionally, the broad sample could be narrowed and better defined for more salient and practical results in specific distance education settings. Chapter 5 includes an expanded discussion of my study’s limitations, as well as a discussion of future directions that one could take in learning environments research using the DELES. 100 Chapter 5 CONCLUSION, IMPLICATIONS, AND RECOMMENDATIONS 5.1 INTRODUCTION Using a learning environment framework in research in distance education is relatively new within the fields of both learning environments and distance education. With rapid changes in technology and telecommunications, more post- secondary education institutions are conducting some form of education by distance. This study has developed a new learning environment instrument that allows researchers and practitioners to gain insights into the psychosocial environments in distance education classes—an uncharted territory. This chapter begins with a summary of my thesis in Section 5.2, with each chapter summarised. Section 5.3 addresses the major findings of my study, followed by Section 5.4 that describes three major contributions of my study. The constraints and limitations of my study are outlined in Section 5.5, whereas Section 5.6 sketches out recommendations for future research related to my study. 5.2 OVERVIEW OF THIS THESIS The first chapter of this thesis provides a general distance education and learning environments background, in addition to the aims and significance of this study. The look at distance education (Section 1.1) focuses on post-secondary distance education and continuing trends toward significant increases in this means of reaching student populations. I attempt to make the point that much distance education today is still based on a face-to-face, lecture-oriented model and that there is the need for re-thinking how we teach and how students learn in this mode on its own terms. I also point out that there is a prolific commoditisation of post-secondary distance education and strong considerations must be made in terms of quality so that universities and colleges do not become merely players in a business-driven education market. I also presented a brief introduction to psychosocial learning environments (Section 1.3) and an outline of the aims and significance of this study 101 (Sections 1.4 & 1.5). My study’s aims are 1) to develop and validate a new post- secondary distance education learning environment instrument and 2) to examine attitude-environment associations related specifically to distance education. Finally, I point out that this study is significant in three ways. First, in terms of distance education, this study expands on current and past distance education research. Second, in terms of contributing to knowledge regarding the maintenance of high- quality higher education it grounds distance education in thoughtful teaching- learning theory with respect to the social and psychological determinants and effects of the distance education learning environment. Third, it contributes to a bricolage of learning environments research, whereby distance education is studied on its own terms as a genuine alternative form of education, rather than as an add-on to traditional studies. Chapter 2 is a review of literature in four parts. It covers the 1) evolution of psychosocial environment research, 2) distance education research in general, 3) distance education psychosocial learning environments, and 4) student satisfaction. Section 2.2 takes into account the evolution of learning environments research by including elemental theories behind the study of psychosocial environments and by reviewing work that eventually led to the study of environments in educational settings. This includes a presentation and description of the ever-increasing body of learning environment instruments, as well as an examination of several lines of learning environments research. Section 2.3 outlines distance education research in general terms, followed by Section 2.4, which brings the two disciplines of learning environments and distance education together, beginning with a description of fragmented research on distance education learning environments found to date. Finally, Section 2.5 explores aspects of student satisfaction, especially in terms its value in post-secondary education and in ways in which it can be measured. Chapter 3 describes the methods that I used to develop and validate the Distance Education Learning Environments Survey (DELES). It also presents the categories of analysis (Section 3.2) and the method that I used for collecting data (Section 3.3). The approach I utilised was one consisting of three stages. Stage 1, consisting of four steps, was the identification and development of salient scales. New scales were developed based on the literature on high-quality distance education, while other 102 scales were modified from previous learning environments instruments (Section 3.5). Stage 2 consisted of three steps for writing individual survey items, both new items and those adapted from other learning environment instruments (Section 3.6). Stage 3 was a two-step process that consisted of field testing the new instrument and analysing the data (Section 3.7). Chapter 4 is a detailed expansion of the data analysis step in Stage 3 of the development of the DELES, as well as a statistical examination of associations between the dependent variable of students’ enjoyment of distance education and the psychosocial scales of the DELES. Validity and reliability analyses were conducted by means of factor analysis (Subsection 4.2.1), which supported the factorial validity of the DELES, and item analysis (Subsection 4.2.2), which verified the reliability of the DELES. Section 4.4 details the analyses used to identify enjoyment-environment associations and goes on to discuss the strongest and weakest associations, speculating on potential reasons for each. Chapter 4 is concluded with a look at previous outcome-environment research and limitations found in this study. 5.3 MAJOR FINDINGS The aims of this study were twofold. First, I set out to develop and validate a new learning environment instrument exclusively for use in post-secondary distance education learning situations. Second, I investigated associations between enjoyment of distance education and psychosocial factors related to distance education learning environments in order to identify which factors influence affective aspects of distance education as perceived by students. Each of these two aspects of my study are presented below. 5.3.1 Development and validation of a new learning environment instrument for post-secondary distance eduction The field-test version of the Distance Education Learning Environments Survey (DELES), along with an attitude scale of Enjoyment, were administered to a sample of distance education students via a form on the World Wide Web. The survey was administered over a four-month period, resulting in 680 responses. The data were 103 analysed to determine the validity and reliability of the instrument in terms of factor structure and internal consistency reliability. The six-scale a priori structure of the DELES was supported through principal component factor analysis. Fourteen faulty items of the original 48-item field test version were identified and removed. The faulty items that were removed either loaded below a 0.50 threshold in their own scale or above 0.50 on scales other than their own. The remaining 34 items accounted for a cumulative six-scale variance of 67.15%, with the scale of Student Interaction and Collaboration accounting for the highest proportion of variance (14.31%). Internal consistency reliability analysis (Cronbach’s alpha coefficient), used to identify the extent to which items within each scale measure the same construct as other items within their scale, ranged from 0.75 to 0.94. This analysis resulted in a new 34-item, six-scale learning environment survey suitable for exploring post- secondary distance education learning environments. 5.3.2 Investigation of associations between enjoyment of distance education and six psychosocial learning environment scales Simple correlation and multiple regression analyses were used to determine any associations between students’ perceptions of their distance education learning environment and their enjoyment of distance education. The simple correlation analysis resulted in statistically significant positive associations between all six DELES scales and the scale of student enjoyment of distance education. When all other psychosocial scales were mutually controlled, the scale of Personal Relevance stood out as having the strongest association with Enjoyment, suggesting that the extent to which there are opportunities for students to make connections between their out-of-class experiences and their in-class experiences has a significant link with their enjoyment of distance education. The second strongest outcome-environment association was between the scale of Authentic Learning and Enjoyment. Authentic learning, the extent to which students have the opportunity to solve real-world problems in class, has an association with 104 their attitude toward distance education. Closely following the Authentic Learning- Enjoyment association is the Student Interaction and Collaboration-Enjoyment association. The extent to which students have opportunities to interact with one another, exchange information, and engage in collaboration is related to students’ enjoyment of learning by distance education. At the opposite end of the outcome-environment spectrum was the Active Learning- Enjoyment association. Their was negligible association between the scale of Active Learning and students’ enjoyment of distance education. In other words, in my study, the extent to which students have the opportunity to take an active role in their learning has little bearing on their enjoyment of distance education. Given this information, it is safe to speculate that, with this population— predominantly graduate-level distance education students (72.6% of the field-test respondents were graduate-level students)—psychosocial aspects of instructional design of distance education courses should focus on personally-relevant learning. This should be closely followed by instructional activities that are as authentic as possible. And, the instruction should be designed to allow students to have ample opportunities for collaboration and interaction amongst themselves. 5.4 DISTINCTIVE CONTRIBUTIONS OF MY STUDY My study has contributions to offer in at least three areas: 1) distance education research in general, 2) higher education studies, and 3) in learning environments research. First, in terms of distance education, this study supports the widely-held notion (Butler, 2001; Chickering & Ehrmann, 1996; Fahy, 2003; Frederick, Cannon, Umble, Steckler, & Shay, 2001; Golbeck & Sinagra, 2000; Hartley, 2000; Ho & Tabata, 2001; Levine, 2003; Morihara, 2001; Nader, 2001; Oren, Mioduser, & Machmias, 2002; Owen, 2000; Park-Kim, 2001; Zhu & McKnight, 2001) that student interaction and collaboration are noteworthy factors in high-quality distance learning. The scale of Student Interaction and Collaboration ranks in the top half of the positive outcomes-environment associations when all other scales are mutually controlled. While student interaction and collaboration are important, my study also suggests that interaction and collaboration amongst students is not an exclusive 105 element to creating high-quality distance education. Instructors and instructional designers seeking to develop high-quality distance education must look further than strategies for developing opportunities for student interaction and collaboration. My study suggests they must also lend strong credence to the extent to which they approach personal relevance for students and how authenticity is addressed in the instructional design of the distance education psychosocial learning environment. Second, in terms of contributions toward higher education studies, we can clearly see in my study that learning environment characteristics have positive associations with student satisfaction, which addresses recent calls for research into student satisfaction in distance education (Morgan & McKenzie, 2003). The DELES was developed without the pretence that distance education is add-on education, or somehow a secondary quality of education. It was designed with the notion in mind that distance education is a unique and alternative form of education. Given the information contained herein, we can see that if colleges and universities want post- secondary students to be satisfied with the way in which we aim to teach them and expect them to learn in this environment, we must then consider the role these psychosocial influences play on distance education learning environments. Likewise, not only has this study demonstrated that there is value in paying attention to the distance education learning environment, college and university educators now have an economical instrument they can use for measuring what is going in their distance education classes in terms of the six psychosocial scales contained within the DELES. In fact, given the nature of the ease in distribution of the DELES as a Web-based instrument, it has been gaining use, even during the writing of this thesis, to aid instructors in over 100 distance education classes to measure the psychosocial learning environments in their educational settings. Third, in terms of contributions toward the field of learning environments research, this new instrument is designed to consider altogether new learning environments. Although distance education is not chronologically new, there is certainly a recent proliferation and new widespread acceptance of it (Cookson, 2002), especially in higher education. Following this trend in the rapid expansion of distance education is this new instrument. Learning environment researchers and education practitioners 106 now have a new instrument with which they can measure that which goes on in our distance education classes. As a concrete demonstration of this contribution toward learning environment research, I should point out that I have already given permission in two cases for components of the DELES to be modified for use in other learning environments research—exactly the type of contribution research should make toward its own discipline. 5.5 CONSTRAINTS AND LIMITATIONS As with any research study there are potential flaws in design, data, and interpretation. In my study one consideration that must be taken into account is the fact that the one-sample design does not lend itself to cross-validation. And, as with other research on psychosocial environmental factors, it is constrained on the principle that the potential effect on any one scale depends upon the milieu in which the research is conducted (Moos, 2002), providing only limited quantitative generalisability by any one study. Further, given that my instrument was Web-based, the sample is not as well defined as it would be with conventional samples deliberately drawn from particular university courses. Regarding the validity and reliability of the scales, the newly developed scale of Active Learning, with its three items in the final version of the instrument, demonstrates validity and reliability based on factor and reliability analyses. However, given that this scale originally contained eight items in the field-test version of the instrument, it calls into question the true value of considering active learning in psychosocial learning environments studies in distance education. The validation of the Distance Education Learning Environment Survey (DELES) is restricted by the notion that respondents were students with an affinity toward distance education. Those students who might demonstrate less of an attraction to distance education may have opted not to participate, leaving responses to those students with favourable opinions. Likewise, in terms of investigating outcome- environment associations, the same consideration may be true that only those students who enjoy distance education to begin with may have been the predominant 107 respondents, resulting in more positive outcome-environment associations than if all distance education students in a given population responded. Further related to outcome-environment associations is the consideration that the scale assessing student attitude is narrow and restricted to items related only to student enjoyment of the distance education environment. Attitudes toward distance education can be more comprehensively defined than just by enjoyment. An expansion on considerations toward student satisfaction is warranted in future studies. 5.6 RECOMMENDATIONS FOR FUTURE RESEARCH Distance education learning environments research is still in its initial stages. This study is distinctive because it contributes to the overall body of knowledge related to distance education, to studies of higher education, and to learning environments research in a new and emerging area. However, further studies related to these three areas needs to be continued. A larger, more targeted sample and capturing class data would be desirable in order to expand upon the generalisability of what I have reported in this initial development and validation. A cross-national study would be advantageous in order to introduce larger variation in learning environment perceptions and student attitudes. Also, a set of studies differentiating between undergraduate students and graduate students may shed light on differences in the perceptions of students seeking their first degree versus students seeking second or third university degrees. There are likely to be differences between these two categories of students and what distance education learning environment factors are most suitable for each. Additional studies could expand on the popular line of learning environment research of investigating outcome-environment associations and consider students’ cognitive achievement in terms of its association with the psychosocial scales validated in my study. Likewise, investigations between affective outcomes other than student enjoyment and psychosocial environment factors could be investigated 108 to address previously made calls for further research in distance education by others (Cookson, 2002; Morgan & McKenzie, 2003). It is also interesting to note that the scale of Student Interaction and Collaboration represents an active mode of learning for students. Participants in my study gave responses that indicated this scale is associated with their enjoyment of distance education. However, when students responded specifically to items in the scale of Active Learning, the results indicated no association with their enjoyment of distance education—an unexplained dichotomy that could stand further investigation, likely by qualitative methods. Further, the validation and reliability analyses, following previous learning environment research methods, were only conducted on data from the actual form of the Distance Education Learning Environment Survey (DELES). An expanded study could be conducted using the same analyses of data from the preferred version and the instructor version in order to create a more holistically validated instrument. Finally, the structure of my six scale-DELES could be further supported and investigated by confirmatory factor analysis within a structural equation modelling framework, following a recent call for the need for such to determine the structural attributes of learning environment instruments (Dorman, in press). 5.7 CONCLUDING COMMENTS Research on psychosocial learning environments in distance education faces the dilemma that the environment itself has only ephemeral effects because it is quickly outmoded by the changes resulting from the introduction of new technology, thus the introduction of new ways of teaching and learning in this new learning environment. I support the notion that “This growth of online educational opportunities needs to be matched by a similar growth in educational research focused upon the learning environments created in the digital world” (Clayton, in press). My study provides support for investigating the fleeting aspects of this genre of higher education so that as we move into the next generation of digitally-based teaching and learning, however it may look, we have investigated the fundamental factors that impact learning, not factors based exclusively in student achievement or in instructor and 109 student satisfaction, but in what goes on socially and psychologically to shape the classroom environment that does not exist in the face-to-face world. 110 REFERENCES Admiraal, W. F., Lockhorst, D., Wubbels, T., Korthagen, F., & Veen, W. (1998). Computer-mediated communication environments in teacher education: Computer conferencing and the supervision of student teachers. Learning Environment Research, 1, 59-74. Aldridge, J. M., & Fraser, B. J. (1997). Examining science classroom environments in a cross- national study. Proceedings Western Australian Institute for Educational Research Forum 1997. Retrieved March 27, 2003 from http://education.curtin.edu.au/waier/forums/1997/aldridge.html Aldridge, J. M., Fraser, B. J., & Huang, T.-C.I. (1999). Investigating classroom environments in Taiwan and Australia with multiple research methods. Journal of Educational Research, 93, 48-57. Aldridge, J. M., Fraser, B. J., Fisher, D. L., Trinidad, S., & Wood, D. (2003, April). Monitoring the success of an outcomes-based, technology-rich learning environment. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. Aldridge, J. M., Fraser, B. J., Taylor, P. C., & Chen, C.-C. (2000). Constructivist learning environments in a cross-national study in Taiwan and Australia. International Journal of Science Education, 22, 37-55. Allen, D., & Fraser, B. J. (2002, April). Parent and student perceptions of the classroom learning environment and its influences on students outcomes. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Anderson, G. J., & Walberg, H. J. (1974). Learning environments. In H. J. Walberg (Ed.), Evaluating educational performance: A sourcebook of methods, instruments and examples (pp. 81-98). Berkeley, CA: McCutchan Publishing. Bain, J. D., McNaught, C., Mills, C., & Lueckenhausen, G. (1998). Describing computer-facilitated learning environments in higher education. Learning Environments Research, 1, 163-180. Ballantyne, C. (2000, November). Why survey online? A practical look at issues in the use of the Internet for surveys in higher education. Paper presented at the annual conference of the American Evaluation Association, Honolulu, HI. 111 Ballard, G. (2003). Identifying and addressing adult learning principles in Web-enhanced/Web-based courses. In R. Ham and J. Woosley (Eds.), Proceedings of the 10th annual International Distance Education Conference (pp. 4-1 – 4-10). College Station, TX: Center for Distance Learning Research. Barnette, J. J. (2000). Effects of stem and Likert response option reversal on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60, 361-370. Benson, J. (1987). Detecting item bias in affective scales. Educational and Psychological Measurement, 47, 55-67. Boling, N. C., & Robinson, D. H. (1999). Individual study, interactive multimedia, or cooperative learning: Which best supplements lecture-based distance education? Journal of Educational Psychology, 91, 169-174. Bowker, D., & Dillman, D. A. (2000, May). An experimental evaluation of left and right oriented screens for Web questionnaires. Paper presented at the annual meeting of the American Association for Public Opinion, Portland, Oregon. Britain, S., & Liber, O. (1999, October). A framework for pedagogical evaluation of virtual learning environments (Report 41). Bangor: University of Wales, Joint Information Systems Committee's Technology Applications Programme (JTAP). Brookfield, S. (2003). Overcoming alienation as the practice of adult education: The contribution of Erich Fromm to a critical theory of adult learning and education. Adult Education Quarterly, 52, 96-111. Butler, S. M. (2001, May). Creating a constructivist on-line learning environment. Paper presented at Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, United States. Retrieved June 12, 2001, from http://leahi.kcc.hawaii.edu/org/tcon01/papers/butler.html Canino, M. J. (2002, May-June). Bridging walls and crossing borders in the Caribbean. Academe, 88(3), 39-41. Cannon, M. M., Umble, K. E., Steckler, A., & Shay, S. (2001). We’re living what we’re learning: Student perspectives in distance learning degree and certificate programs in public health. Journal of Public Health Management and Practice 7 (1), 49+. Retrieved June 25, 2001, from the Curtin University of Technology Library Scholarly Education Database. 112 Cattell, R. B., & Cattell, H. E. P. (1995). Personality structure and the new fifth edition of the 16PF. Educational and Psychological Measurement, 55, 926-929. Chamberlain, V. M., & Cummings, M. N. (1984). Development of an instructor/course evaluation instrument. College Student Journal, 18, 246-250. Chang, V. (1999). Evaluating the effectiveness of online learning using a new web based learning instrument. Proceedings of the Western Australian Institute for Educational Research Forum 1999. Retrieved January 1, 2003, from http://education.curtin.edu.au/waier/forums/1999/chang.html Chang, V., & Fisher, D. L. (2001a). A new learning instrument to evaluate online learning in higher education. In M. Kulske & A. Herrmann (Eds.), New horizons in university teaching and learning (pp. 23-34). Perth: Curtin University of Technology. Chang, V., & Fisher, D. L. (2001b, December). The validation and application of a new learning environment instrument to evaluate online learning in higher education. Paper presented at the Annual Conference of Australian Association for Research in Education, Perth, Western Australia. Chesebro, J. L., & McCroskey, J. C. (2002). Communication for teachers. Boston: Allyn and Bacon. Chickering, A. W., & Ehrmann, S. C. (1996, October). Implementing the seven principles: Technology as the lever. American Association for Higher Education Bulletin. Retrieved June 12, 2001, from http://www.aahe.org/bulletin/Implementing%20the%20Seven%20Principles.htm Chionh, Y. H., & Fraser, B. J. (1998, April). Validation and use of the ‘What is Happening in this Class’ (WIHIC) questionnaire in Singapore. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Chua, S. L., Wong, A. F. L., & Chen, D.-T. (2001, December). Validation of the Chinese Language Classroom Environment Inventory (CLCEI) for us in Singapore secondary schools. Paper presented at the Annual Conference of the Australian Association for Research in Education, Fremantle, Australia. Cifuentes, L., & Murphy, K. L. (2000). Images of Texan and Mexican cultures shared in a telecommunications partnership. Distance Education, 21, 300-322. 113 Clayton, J. (in press). Assessing and researching the online learning environment. In M. S. Khine & D. Fisher (Eds.), Technology-rich learning environments: A future perspective. Singapore: World Scientific. Clinton, P. (2002, July/August). Viewpoint: Not so distant. University Business, 5, 13. Commonwealth Department of Education, Science & Training. (2002, May 2). Changing contexts. In Higher education at the crossroads: An overview paper (chap. 3). Retrieved May 10, 2002 from http://www.dest.gov.au/crossroads/pubs/3.htm Commonwealth Department of Education, Training & Youth Affairs. (2000). The way forward: Higher education action plan for the information economy. In Learning for the knowledge society: An education and training action plan for the information economy (DETYA Publication No 6465SCHP00A). Canberra: AusInfo. Retrieved December 16, 2002, from http://www.deet.gov.au/schools/publications/reports/learning/learning.htm Commonwealth of Australia. (2002). Australia’s overseas aid program 2002-2003. Retrieved October 17, 2002 from http://www.ausaid.gov.au/budget02/budget_2002_2003.html Connell, R. (2001, March). Educommerce: Online learning migrates to the e-commerce arena (Eduventures White Paper). Boston: Eduventures.com, Inc. Retrieved December 31, 2002, from http://www.eduventures.com/research/industry_research_resources/educommerce.cfm Cook, K. C. (2000). Online professional communication: Pedagogy, instructional design, and student preference in Internet-based distance education. Business Communication Quarterly, 63, 106-110. Cookson, P. S. (2002). Editorial: Online postgraduate education: Some reflections. International Review of Research in Open and Distance Learning, 3. Retrieved May 13, 2003 from http://www.irrodl.org/content/v3.2/editorial.html Cox, M., Preston, C., & Cox, K. (1999, September). What motivates teachers to use ICT? Paper presented at the British Educational Research Association Annual Conference, University of Sussex at Brighton, Great Britain. Retrieved June 12, 2001, from http://www.leeds.ac.uk/educol/documents/00001329.htm Cresswell, J., & Fisher, D. L. (1997, March). A comparison of actual and preferred principal interpersonal behavior. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. 114 Cuban, L. (2001). Oversold and underused: Computers in the classroom. London: Harvard University Press. Cummings, R., & Ballantyne, C. (1999, October). Student feedback on teaching: Online! On target? Paper presented at the Australasian Society Annual Conference, Perth, Western Australia. Curtin University of Technology. (2002). Curtin African virtual university project proposal. Retrieved October 17, 2002 from http://cea.curtin.edu.au/avup/about.html de Vaus, D. A. (2001). Research design in social research. London: Sage Publications. den Brok, P. J., Levy, J., Rodriguez, R., & Wubbels, T. (2002). Perceptions of Asian-American and Hispanic-American teachers and their students on teacher interpersonal communication style. Teaching and Teacher Education, 18, 447-467. Diaz, D. P. (2000). Carving a new path for distance education research. The Technology Source. Retrieved March 26, 2002, from http://ts.mivu.org/default.asp?show=article&id=648 Diaz, D. P., & Cartnal, R. B. (1999). Students' learning styles in two classes: Online distance learning and equivalent on-campus. College Teaching, 47, 130-135. Dillman, D. A., & Bowker, D. K. (2001). The Web questionnaire challenge to survey methodologists. In U. Reips & M. Bosnjak (Eds.), Dimensions of Internet science (pp. 159-178). Berlin: Pabst Science Publishers. Dorman, J. P. (in press). Cross-national validation of the What Is Happening In This Class? (WIHIC) questionnaire using confirmatory factor analysis. Learning Environments Research: An International Journal. Dorman, J. P., Adams, J. E., & Ferguson, J. M. (2001, April). Cross-national validation and use of classroom environments scales. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA. Dougiamas, M., & Taylor, P. C. (2002, July). Interpretive analysis of an Internet-based course constructed using a new courseware tool called Moodle. In A. Goody, J. Herrington, & M. Northcote (Eds.), Proceedings of the 2002 Annual International Conference of the Higher Education Research and Development Society of Australasia (HERDSA). Perth: HERDSA. Retrieved December 31, 2002, from http://www.ecu.edu.au/conferences/herdsa/papers/nonref/pdf/MartinDougiamas.pdf 115 Dryden, M., & Fraser, B. (1998, April). The impact of systemic reform efforts in promoting constructivist approaches in high school science. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Eastmond, D. (2000). Realizing the promise of distance education in low technology countries. Educational Technology, Research and Development, 48(2), 100+. Retrieved April 24, 2002, from ProQuest database. Eastmond, J. N., & Kim, Y. (2000). International review [Review of the book Instructional technology in Asia: Focus on Japan and Korea]. Educational Technology, Research and Development 48(1), 100+. Retrieved June 25, 2001, from the Curtin University of Technology Library Scholarly Education Database. Englemann, S., Granzin, A., & Severson, H. (1979). Diagnosing instruction. Journal of Special Education, 13, 355-365. Fahy, P. J. (2003). Indicators of support in online interaction. International Review of Research in Open and Distance Learning, 4. Retrieved May 13, 2003 from http://www.irrodl.org/content/v4.1/fahy.html Felix, U. (2001). A multivariate analysis of students’ experience of web based learning. Australian Journal of Educational Technology, 17, 21-36. Fish, M. C., & Dane, E. (2000). The classroom systems observation scale: Development of an instrument to asses classrooms using a systems perspective. Learning Environments Research 3, 67-92. Fisher, D. L., Aldridge, J. M., Fraser, B. J., & Wood, D. (2001, December). Development, validation and use of a questionnaire to assess students’ perceptions of outcomes-focused, technology- rich learning environments. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/fis01028.htm Fisher, D. L., & Fraser, B. J. (1983). A comparison of actual and preferred classroom environment as perceived by science teachers and students. Journal of Research in Science Teaching, 20, 55- 61. Fisher, D., & Fraser, B. J. (1990). School climate: Assessing and improving school environments (SET research information for teachers No. 2). Melbourne: Australian Council for Educational Research. 116 Fisher, D. L., Goh, S. C., Wong, A. F. L., & Rickards, T. W. J. (1996, November). Perceptions of interpersonal teacher behaviour in secondary science classrooms: A cross-national study. Paper presented at the Conference of the Educational Research Association, Singapore and the Australian Association of Research in Education, Singapore. Fisher, D. L., Henderson, D., & Fraser, B. J. (1995). Interpersonal behaviour in senior high school biology classes. Research in Science Education, 25, 125-133. Fisher, D., & Stolarchuk, E. (1998, August). The effect of using laptop computers on achievement, attitude to science and classroom environment in science. In Proceedings Western Australian Institute for Educational Research Forum 1998, Freemantle, Western Australia: University of Notre Dame. Retrieved March 2, 2003, from http://education.curtin.edu.au/waier/forums/1998/fisher.html Fisher, D., Rickards, T., & Fraser, B. J. (1996). Assessing teacher-student interpersonal relationships in science classes. Australian Science Teachers Journal, 42(3), 28-33. Fisher, D. L., & Waldrip, B. C. (2002). Measuring culturally sensitive factors of classroom learning environments with the CLEQ. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 27-48). River Edge, NJ: World Scientific. Fraser, B. J. (1977). Selection and validation of attitude scales for curriculum evaluation. Science Education, 61, 317-329. Fraser, B. J. (1981). Test of science-related attitudes. Melbourne, Victoria: Australian Council for Educational Research. Fraser, B. J. (1986). Classroom environment. London: Croom Helm, Ltd. Fraser, B. J. (1989). Assessing and improving classroom environment (What Research Says, No. 2). Perth: Curtin University of Technology. Fraser, B. J. (1990). Students’ perceptions of their classroom environments. In K. Tobin, J. B. Kahle, and B. J. Fraser (Eds.), Windows into science classrooms: Problems associated with higher- level cognitive learning (pp. 199-221). London: The Falmer Press. Fraser, B. J. (1997). Classroom environments. In H. J. Walberg & G. D. Haertel (Eds.), Psychology and educational practice (pp. 323-341). Berkeley, CA: McCutchan Publishing. 117 Fraser, B. J. (1998a). Classroom environment instruments: Development, validity and applications. Learning Environments Research, 1, 7-33. Fraser, B. J. (1998b). Science learning environments: Assessment, effects and determinants. In B. J. Fraser & K.G. Tobin (Eds.), International handbook of science education (pp. 527-564). London: Kluwer Academic Publishers. Fraser, B. J. (2002a). Learning environments research: Yesterday, today and tomorrow. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 1-25). River Edge, NJ: World Scientific. Fraser, B. J. (2002b). Preface. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. vii-viii). River Edge, NJ: World Scientific. Fraser, B. J., & Chionh, Y. H. (2000, April). Classroom environment, self-esteem, achievement, and attitudes in geography and mathematics in Singapore. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Fraser, B. J., & Fisher, D. L. (1986). Using short forms of classroom climate instruments to assess and improve classroom psychosocial environment. Journal of Research in Science Teaching, 5, 387–413. Fraser, B. J., Fisher, D. L., & McRobbie, C. J. (1996, April). Development, validation, and use of personal and class forms of a new classroom environment instrument. Paper presented at the annual meeting of the American Educational Research Association, New York. Fraser, B. J., Giddings, G. J., & McRobbie, C. J. (1992). Assessing the climate of science laboratory classes (What Research Says, No. 8). Perth: Curtin University of Technology. Fraser, B. J., Gidding, G. J., & McRobbie, C. J. (1995). Evolution and validation of a personal form of an instrument for assessing science laboratory classroom environments. Journal of Research in Science Teaching, 32, 399-422. Fraser, B., Sinclair, B., & Ledbetter, C. (2001, December). Assessing and changing classroom environments in urban middle schools in Texas. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved May 7, 2003 from http://www.aare.edu.au/01pap/fra01449.htm 118 Fraser, B. J., Treagust, D. F., & Dennis, N. C. (1986). Development of an instrument for assessing classroom psychosocial environment at universities and colleges. Studies in Higher Education, 11, 43-54. Frederick, J., Cannon , M. M., Umble, K. E., Steckler, A., & Shay, S. (2001). “We’re living what we’re learning”: Student perspectives in distance learning degree and certification programs in public health. Journal of Public Health Management and Practice, 7, 49-60. Fulkerth, R. (2002, November). Managing for course and program quality in the online environment. Paper presented at the 2002 Teaching Online in Higher Education Conference, Fort Wayne, IN. Retrieved November 13, 2002, from http://www.ipfw.edu/as/tohe/2002/Papers/fulkerth.htm Garson, D. (2001). Factor analysis. Retrieved July 29, 2002, from http://www2.chass.ncsu.edu/garson/pa765/factor.htm George, D., & Mallery, P. (2001). SPSS for Windows step by step: A simple guide and reference 10.0 update (3rd ed.). Toronto: Allyn and Bacon. Gibbs, G. R. (1999). Learning how to learn using a virtual learning environment for philosophy. Journal of Computer Assisted Learning, 15, 221-231. Giddings, G. J., & Fraser, B. J. (1990, April). Cross-national development, validation and use of an instrument for assessing the environment of science laboratory classes. Paper presented at the annual meeting of the American Educational Research Association, Boston. Goh, S. C. (2002). Studies on learning environments in Singapore classrooms. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 197-216). River Edge, NJ: World Scientific. Goh, S. C., & Fraser, B. J. (1996). Validation o fan elementary school version of the Questionnaire of Teacher Interaction. Psychological Reports, 79, 512-522. Goh, S. C., & Fraser, B. J. (1998). Teacher interpersonal behaviour, classroom environment and student outcomes in primary mathematics in Singapore. Learning Environments Research, 1, 199-229. Goh, S. C., & Fraser, B. J. (2000). Teacher interpersonal behavior and elementary students’ outcomes. Journal of Research in Childhood Education, 14, 216-231. 119 Goh, S. C., & Khine, M. S. (Eds.). (2002). Studies in educational learning environments: An international perspective. River Edge, NJ: World Scientific. Goh, S. C., & Tobin, K. (1999). Student and teacher perspectives in computer-mediated learning environments in teacher education. Learning Environments Research, 2, 169-190. Golbeck, S. L., & Sinagra, K. (2000). Effects of gender and collaboration on college students’ performance on a Piagetian spatial task. The Journal of Experimental Education, 69, 22+. Retrieved May 27, 2001, from XanEdu Research database. Gopinathan, S. (2002). Foreward. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (p. v). River Edge, NJ: World Scientific. Graham, M., & Scarborough, H. (2001). Enhancing the learning environment for distance education students. Distance Education, 22, 232-244. Grasinger, M. F. (1999). Successful distance learning: Teaching via synchronous video. College Teaching, 47(2), 70-73. Gunawardena, C. N., Nolla, A. C., Wilson, P. L., & Lopez-Islas, J. R. (2001). A cross-cultural study of group process and development in online conferences. Distance Education, 22, 85-121. Gururajan, V. (2002). Knowledge management a new game in higher education: A discussion paper on the new trends of technology influencing the higher education landscape. In Focusing on the Student. Proceedings of the 11th Annual Teaching Learning Forum, 5-6 February 2002. Perth: Edith Cowan University. Retrieved May 19, 2002, from http://cea.curtin.edu.au/tlf/tlf2002/abstracts/gururajan-abs.html Hanley, L. (2002, May-June). Globalization and the university. Academe, 88(3), 2. Harnar, M. A., Brown, S. W., & Mayall, H. J. (2000). Measuring the effects of distance education on the learning experience: Teaching accounting via PictureTel. International Journal of Instructional Media, 27, 37-49. Harnish, D., & Reeves, P. (2000). Issues in the evaluation of large-scale two-way interactive distance learning systems. International Journal of Educational Telecommunications, 6, 267-281. Harris, J. (1998). Virtual architecture: Designing and directing curriculum-based telecomputing. Eugene, OR: International Society of Technology in Education. 120 Harrison, A., Fisher, D., & Henderson, D. (1997). Student perceptions of practical tasks in senior biology, chemistry and physics classes. Proceedings Western Australian Institute for Educational Research Forum 1997. Retrieved May 5, 2003, from http://education.curtin.edu.au/waier/forums/1997/harrison.html Hartley, R. (2000). On-Line collaborative learning environments. Educational Technology & Society 3(3). Retrieved June 12, 2001, from http://ifets.ieee.org/periodical/vol_3_2000/hartley.html Hartshorne, H., & May, M. A. (1928). Studies in the nature of character: Studies in deceit. New York: Macmillan. Hase, H. D., & Goldberg, L. R. (1967). Comparative validity of different strategies of constructing personality inventory scales. Psychological Bulletin, 67, 231-248. Henderson, D., Fisher, D., & Fraser, B. J. (2000). Interpersonal behavior, laboratory learning environments, and student outcomes in senior biology classes. Journal of Research in Science Teaching, 37, 26-43. Hinde, J. (2000, November). Made in Australia. University Business, 3, 31-35 & 80. Ho, C. P., & Tabata, L. N. (2001). Strategies for designing online courses to engage student learning. Paper presented at Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, United States. Retrieved June 12, 2001, from http://leahi.kcc.hawaii.edu/org/tcon01/papers/ho.html Howland, J. L., & Moore, J. L. (2002). Student perceptions as distance learners in Internet-based courses. Distance Education, 23, 183-195. Huck, S. W. (2004). Reading statistics and research (4th ed.). New York: Allyn and Bacon. Insel, P. M., & Moos, R. H. (1974). The social environment. In P. M. Insel and R. H. Moos (Eds.), Health and the social environment (pp. 3-12). Lexington, MA: D.C. Heath and Company. Institute for Higher Education Policy (IHEP). (1999). What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education. Washington DC: Institute for Higher Education Policy. Retrieved December 31, 2002, from http://www.ihep.com/PUB.htm Jaeger, R. M. (1993). Statistics: A spectator sport (2nd ed.). London: Sage Publications. 121 Jamieson, P. (1999). Improving teaching by telecommunications media: Emphasising pedagogy rather than technology. In B. Collis and R. Oliver (Eds.), Proceedings of Ed-Media 1999: World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Seattle, Washington, June 19-24, 1999. (pp. 85-90). Charlottesville, VA: Association for the Advancement of Computing in Education. Jayroe, L. J. (1998). The evolution of distance education: Impact and implications for the 21st century educator. Retrieved December 31, 2002, from http://www.hapsweb.org/Library/Jayroe_HAPS98_presentation/sld015.htm Jegede, O. J. (1992). Constructivist epistemology and its implications for contemporary research in distance education. In T. Evans & P. Juler (Eds.), Research in distance education (Vol. 2) (pp. 21-29). Geelong, Victoria: Deakin University Press. Jegede, O. J., Fraser, B. J., & Fisher, D. L. (1995). The development and validation of a Distance and Open Learning Environment Scale. Educational Technology Research and Development, 43, 90-93. Jegede, O., Fraser, B. J., & Fisher, D. L. (1998, April). The Distance and Open Learning Environment Scale: Its development, validation and use. Paper presented at the 69th Annual Meeting of the National Association for Research in Science Teaching, San Diego, CA. Jegede, O. J., & Kirkwood, J. (1994). Students’ anxiety in learning through distance education. The American Journal of Distance Education, 15, 279-290. Jegede, O., Taplin, M., Fan, R. Y. K., Chan, M. S. C., & Yum, J. (1999). Differences between low and high achieving distance learners in locus of control and metacognition. Distance Education, 20, 255-273. Kercher, S. (2002). E-learning market trends. Faulkner information services. Retrieved October 5, 2002, from Faulkner Advisory for IT Studies database. Kerlinger, F. N. (1986). Foundations of behavioral research (3rd ed.). New York: Holt, Rinehart and Winston. Khine, M. S. (2002). Study of learning environment for improving science. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 131-151). River Edge, NJ: World Scientific. 122 Kim, H.-B., Fisher, D. L., & Fraser, B. J. (1999). Assessment and investigation of constructivist science learning environments in Korea. Research in Science and Technological Education, 17, 239-249. Kim, H. B., Fisher, D. L., & Fraser, B. J. (2000). Classroom environment and teacher interpersonal behavior in secondary school classes in Korea. Evaluation and Research in Education, 14, 3- 22. Kim, J., & Mueller, C. W. (1978). Introduction to factor analysis: What it is and how to do it. London: Sage Publications. Kiritz, S., & Moos, R. H. (1974). Physiological effects of the social environment. In P. M. Insel and R. H. Moos (Eds.), Health and the social environment (pp. 13-35). Lexington, MA: D.C. Heath and Company. Knight, R. G., Chisholm, B. J., Marsh, N. V., & Godfrey, H. P. (1988). Some normative, reliability, and factor analytic data for the revised UCLA Loneliness Scale. Journal of Clinical Psychology, 44, 203-206. Kollock, P., & Smith, P. A. (1999). Communities in cyberspace. In M. A. Smith & P. Kollock (Eds.), Communities in cyberspace (pp. 3-25). London: Routledge. Koul, R. B., & Fisher, D. (2002, December). Science classroom learning environments in India. Paper presented at the annual conference of the International Educational Research Conference of the Australian Association for Research in Education, Brisbane, Australia. Kreijns, K., Kirschner, P. A., & Jochems, W. (2002). The sociability of computer-supported collaborative learning environments. Educational Technology & Society, 5, 8-22. Kuh, G. D. (2001a). Assessing what really matters to student learning. Change, 33(3), 10-17. Kuh, G. D. (2001b). The National Survey of Student Engagement: Conceptual framework and overview of psychometric properties. (National Survey of Student Engagement: The College Student Report). Retrieved June 25, 2001, from http://www.indiana.edu/~nsse/html/framework.html Kyle, W. C. (1997). Editorial: Assessing students' understanding of science. Journal of Research in Science Teaching, 34, 851-852. 123 Lane, C. (n.d.). Research in distance education. In Distance Learning Resource Network’s technology resource guide. Retrieved December 31, 2002, from http://www.dlrn.org/library/dl/guide.html LaRose, R., & Whitten, P. (2000). Re-thinking instructional immediacy for Web courses: A social cognitive exploration. Communication Education, 49, 320-338. Lattu, M. (2000, September). Interaction in distance education. Paper presented at the European Conference on Educational Research, Edinburgh, Scotland. Leach, K., & Walker, S. L. (2000). Internet-based distance education: Barriers, models, and new research. In Proceedings of WebNet World Conference on the WWW and the Internet 2000, October 30-November 4, 2000. Charlottesville, VA: Association for the Advancement of Computing in Education. Leary, T. (1957). An interpersonal diagnosis of personality. New York: Ronald Press Company. Lee, S., & Fraser, B. (2001, December). Laboratory classroom environments in Korean high schools. Paper presented at the annual meeting of the Australian Association for Research in Education, Freemantle, Western Australia. Lee, S. U., & Kim, H. B. (2002). Curriculum development and learning environment research in Korea. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 169-196). River Edge, NJ: World Scientific. Lefoe, G., Gunn, C., & Hedberg, J. (2001, December). Recommendations for teaching in a distributed learning environment: The students’ perspective. In Proceedings of the 18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education (ASCILITE) (pp. 355-366). Melbourne: Biomedical Multimedia Unit, The University of Melbourne. Retrieved December 31, 2002, from http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/lefoeg.pdf Leh, A. (1999). Computer-mediated communication and foreign language learning via telecommunication technology. In B. Collis and R. Oliver (Eds.), Proceedings of Ed-Media 1999: World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Seattle, Washington, June 19-24, 1999 (pp. 68-73). Charlottesville, VA: Association for the Advancement of Computing in Education. 124 Levine, J. (2003). Quality distance education: Appropriate interdependent and collaborative environments. In R. Ham and J. Woosley (Eds.), Proceedings of the 10th annual International Distance Education Conference (pp. 18-1 – 18-5). College Station, TX: Center for Distance Learning Research. Lewin, K. (1936). Principles of topological psychology (F. Heider & G. M. Heider, Trans.). New York: McGraw-Hill Book Company, Inc. Lorenzo, G. (2002). EArmyU and the future of distance education. The Technology Source. Retrieved May 19, 2002 from http://ts.mivu.org/default.asp?show=article&id=998 Majeed, A., Fraser, B. J., & Aldridge, J. M. (2001, December). Junior secondary mathematics students’ learning environment and satisfaction in Brunei Darussalam. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/maj01681.htm Majeed, A., Fraser, B. J., & Aldridge, J. M. (2002). Learning environment and its association with student satisfaction among mathematics students in Brunei Darussalam. Learning Environments Research, 5, 203-226. Maor, D. (1999). A teacher professional development program on using constructivist multimedia learning environment. Learning Environment Research, 2, 307-330. Margianti, E. S. (2002). Learning environment research in Indonesia. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 153- 167). River Edge, NJ: World Scientific. Margianti, E. S., Fraser, B. J., & Aldridge, J. M. (2001, December). Investigating the learning environment and students’ outcomes at the university level in Indonesia. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/mar01668.htm Marginson, S. (2002, May-June). Education in the global market: Lessons from Australia. Academe, 88(3), 22-24. Markel, M. (1999). Distance education and the myth of the new pedagogy. The Journal of Business and Technical Communication, 13, 208-222. 125 McAllister, G. (2001). Attitudes and issues associated with distance learning. Unpublished manuscript, University of West Florida. Retrieved May 16, 2002, from http://scholar.coe.uwf.edu/lkcurda/summeritseminar/Projects/gina.htm McRobbie, C. J., Fisher, D. L., & Wong, A. F. L. (1998). Personal and class forms of classroom environment instruments. In B. J. Fraser and K. G. Tobin (Eds.), International handbook of science education (pp. 581-594). London: Kluwer Academic Publishers. McRobbie, C. J., & Thomas, G. P. (2001). Changing the learning environment to enhance explaining and understanding in a year 12 chemistry classroom. Learning Environments Research, 3, 209-227. Melland, H. I., & Volden, C. M. (2001). A nurturing learning environment – on- or off-line. Nursing Forum, 36, 23+. Retrieved October 16, 2002, from Health and Wellness Resource Center database. Merisotis, J. P., & Olsen, J. K. (2000). The ‘effectiveness’ debate: What we know about the quality of distance learning in the U.S. TechKnowLogia, 2, 42-44. Merrill, M. D. (2001). First principles of instruction. Utah State University, UT: Second Generation Instructional Design (ID2). Retrieved June 12, 2001, from http://www.id2.usu.edu/Papers/5FirstPrinciples.PDF Mioduser, D., Nachmias, R., Lahav, O., & Oren, A. (2000). Web-based learning environments: Current pedagogical and technological state. Journal of Research on Computing in Education, 33, 55-76. Molenda, M., & Harris, P. (2001). Issues and trends in instructional technology. In R.M. Branch and M. A. Fitzgerald (Eds.), Educational media and technology yearbook (Vol. 26, pp. 3-15). Englewood, CO: Libraries Unlimited, Inc. Moos, R. H. (1974). Systems for the assessment and classification of human environments: An overview. In R.H. Moos and P. M. Insel (Eds.), Issues in social ecology: Human milieus (pp. 5-29). Palo Alto, CA: National Press Books. Moos, R. H. (1976). The human context: Environmental determinants of behavior. New York: John Wiley & Sons. Moos, R. H. (1979). Evaluating educational environments. San Francisco: Jossey-Bass Publishers. 126 Moos, R. H. (1996). Understanding environments: The key to improving social processes and program outcomes. American Journal of Community Psychology, 24, 193+. Retrieved October 5, 2002, from ProQuest Social Science Plus database. Moos, R. H. (2002). The mystery of human context and coping: An unraveling of clues. American Journal of Community Psychology, 30(1), 67+. Retrieved October 5, 2002, from ProQuest- Social Science Plus database. Moos, R. H., & Trickett, E. J. (1974). Classroom environment scale manual. Palo Alto, CA: Consulting Psychologists Press. Morgan, C. K., & McKenzie, A. D. (2003). Is enough too much? The dilemma for online distance learner supporters. International Review of Research in Open and Distance Learning, 4. Retrieved May 13, 2003 from http://www.irrodl.org/content/v4.1/mckenzie_morgan.html Morihara, B. (2001, May). Practice and pedagogy in university web teaching. Paper presented at Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, United States. Retrieved June 12, 2001, from http://leahi.kcc.hawaii.edu/org/tcon01/papers/morihara.html Murphy, K. L., Chen, C., Mahoney, S. E., Gazi, Y., Yang, X., Fite, S., et al. (2003, January). Extending teaching presence: Students as teachers in online courses. Paper presented at the 10th Annual Distance Education Conference, Austin, TX. Murphy, K. L., & Cifuentes, L. (2001). Using Web tools, collaboration, and learning online. Distance Education, 22, 285-305. Murray, H. A. (1938). Explorations in personality: A clinical and experimental study of fifty men of college age. New York: Oxford University Press. Myint, S. K., & Goh, S. C. (2001, December). Investigation of tertiary classroom learning environment in Singapore. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/myi01168.htm Nader, N. A. (2001, May). What do you want to be when you grow up? Keynote address presented at Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, United States. Retrieved June 12, 2001, from http://leahi.kcc.hawaii.edu/org/tcon01/greetings/nader.html 127 Nair, C. S., & Fisher, D. L. (2001). Learning environments and student attitudes to science at the senior secondary and tertiary levels. Issues in Educational Research, 11(2), 12-13. National Survey of Student Engagement. (2000). The NSSE 2000 report: National Benchmarks of Effective Educational Practice. (National Survey of Student Engagement: The College Report). Retrieved June 25, 2001, from http://www.indiana.edu/~nsse/acrobat/excerpts.pdf Newby, M., & Fisher, D. L. (1997). An instrument for assessing the learning environment of a computer laboratory. Journal of Educational Computing Research, 16, 179-190. Newcomb, T. M. (1929). The consistency of certain extrovert-introvert behavior patterns in 51 problem boys. New York: Columbia University Teachers College Bureau of Publications. Newhouse, P. (1994). Creating computer supported learning environments: A three year study. In J. Steele and J. G. Hedberg (Eds.), Learning Environment Technology: Selected Papers from LETA. Canberra: AJET Publications. Retrieved November 3, 2002, from http://wwwtlc1.murdoch.edu.au/gen/aset/confs/edtech94/mp/newhouse.html Noble, D. (1998). Digital diploma mills: The automation of higher education. First Monday, 3(1). Retrieved May 19, 2002, from http://www.firstmonday.dk/issues/issue3_1/noble/index.html Ntuli, S. (2001). Evaluating and improving teaching practice and distance education teacher training programs using learning environments assessments. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/index.htm Ntuli, S., Aldridge, J. M., & Fraser, B. J. (2003, April). Using learning environment assessments to improve elementary school teaching during a distance inservice course in South Africa. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. Oelschlaeger, M. (1991). The idea of wilderness: From prehistory to the age of ecology. New Haven, CT: Yale University Press. Ogburn, W. (1964). On culture and social change. Chicago: University of Chicago Press. Olsen, J. K. (2000). Is virtual education for real?: Issues of quality and accreditation. TechKnowLogia, 2(1), 16-18. 128 Olsen, T. M., & Wisher, R. A. (2002). The effectiveness of Web-based instruction: An initial inquiry. International Review of Research in Open and Distance Learning, 3. Retrieved May 13, 2003 from http://www.irrodl.org/content/v3.2/olsen.html O’Reilly, M., & Newton, D. (2001, December). Interaction online: Above and beyond requirements of assessment. In Proceedings of the 18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education (ASCILITE) (pp. 461-470). Melbourne: Biomedical Multimedia Unit, The University of Melbourne. Retrieved December 31, 2002, from http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/oreillym.pdf Oren, A., Mioduser, D., & Nachmias, R. (2002, April). The development of social climate in virtual learning discussion groups. International Review of Research in Open and Distance Learning, 3. Retrieved July 12, 2002, from http://www.irrodl.org/content/v3.1/mioduser.html Organisation for Economic Co-operation & Development (OECD). (2000, September). Science, technology and innovation in the new economy. OECD Policy Brief. Retrieved December 31, 2002, from http://www.oecd.org/publications/Pol_brief/2000/2000_04.pdf Owen, M. (2000). Structure and discourse in telematic learning environment. Education Technology & Society 3(3). Retrieved June 12, 2001, from http://ifets.ieee.org/periodical/vol_3_2000/b04.html Owens, R. G. (2001). Organizational behavior in education: Instructional leadership and school reform (7th ed.). Boston: Allyn and Bacon. Pace, C. R. (1962). Implications of differences in campus atmosphere for evaluation and planning of college programs. In R. Sutherland, W. Holtzman, E. Koile, and B. Smith (Eds.), Personality factors on the college campus. Austin, TX: University of Texas Press. Pace, C. R. (1967). College and university environment scales: Technical manual (2nd ed.). Princeton, NJ: Educational Testing Service. Pace, C. R., & Stern, G. (1958). An approach to the measurement of psychological characteristics of college environments. Journal of Educational Psychology, 49, 269-277. Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace: Effective strategies for the online classroom. San Francisco: Jossey-Bass Publishers. 129 Park-Kim, K. (2001, May). Improving interactivity in online courses. Paper presented at Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, United States. Retrieved June 12, 2001, from http://leahi.kcc.hawaii.edu/org/tcon01/papers/park-kim.html Phillips, J. J. (1991). Handbook of training evaluation and measurement methods (2nd ed.). Houston, TX: Gulf Publishing Company. Pickett, L. H., & Fraser, B. J. (2002, April). The role of learning environment, achievement, and student and teacher attitudes in a science mentoring program for beginning elementary school teachers. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Ponzurick, T. G., France, K. R., & Logar, C. M. (2000). Delivering graduate marketing education: An analysis of face-to-face versus distance education. Journal of Marketing Education, 22, 180- 187. Quek, C. L., Wong, A. F. L., & Fraser, B. J. (2001, December). Determinants and effects of perceptions of chemistry classroom learning environments in secondary school gifted education classes in Singapore. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/que01434.htm Raaflaub, C. A., & Fraser, B. J. (2002, April). Investigating the learning environment in Canadian mathematics and science classrooms in which laptop computers are used. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Raaflaub, C. A., & Fraser, B. J. (2003, March). The learning environment associated with the use of laptop computers in Canadian science classes. Paper presented at the annual meeting of the National Association of Research in Science Teaching, Philadelphia, PA. Riah, H., & Fraser, B. (1998a, April). Chemistry learning environment and its association with students’ achievement in chemistry. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Riah, H., & Fraser, B. J. (1998b, April). The learning environment of high school chemistry classes. Paper presented at the annual meeting of the National Association for Research in Science Teaching, San Diego, CA. 130 Rose Communications, Inc. (Producer). (2001, May 16). The Charlie Rose show. New York: Public Broadcasting Service. Salmi, J. (2000, January/February). Higher education: Facing the challenges of the 21st century. TechKnowLogia, 2(1), 7-10. Retrieved December 31, 2002, from http://www.techknowlogia.org/TKL_active_pages2/CurrentArticles/main.asp?IssueNumber= 3&FileType=HTML&ArticleID=56 Schmidt, W. C. (1996). WWW Survey Assistant [Web-based software]. Halifax, Nova Scotia, Canada: S-Ware. Retrieved March 19, 2003 from, http://or.psychology.dal.ca/~wcs/hidden/home.html Schriesheim, C. A., Eisenbach, R. J., & Hill, K. D. (1991). The effect of negation and polar opposite item reversals on questionnaire reliability and validity: An experimental investigation. Educational and Psychological Measurement, 51, 67-78. Schriesheim, C. A., & Hill, K. D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41, 1101- 1114. Scott, R. H., & Fisher, D. L. (2001, December). The impact of teachers’ interpersonal behaviour on examination results. Paper presented at the annual conference of the Australian Association for Research in Education, Perth, Western Australia. Seagren, A. T., & Stick, S. L. (1999). Enhancing quality through distributed education. Phi Delta Kappan, 80, 793-794. Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance: Foundations of distance education (2nd ed.). Upper Saddle River, NJ: Merrill-Prentice Hall. Sinagra, K., & Golbeck, S. L. (2000). Effects of gender and collaboration on college students’ performance on a Piagetian spatial task. Journal of Experimental Education, 69, 22-36. Skilbeck, M. (2001). Panel discussion: The e-learning revolution—Implications for academic work and university culture. In Online learning in a borderless market. Proceedings of a conference held 15–16 February 2001 at Griffith University Gold Coast Campus (pp. 59-63). Canberra: Department of Education, Training, and Youth Affairs. Slavin, R. E. (2000). Educational psychology: Theory and practice (6th ed.). Boston: Allyn and Bacon. 131 Slay, J. (1998, August). Enhancing the learning environment. Paper presented at the British Education Research Association Annual Conference, The Queen’s University of Belfast, Northern Ireland. Snowman, J., & Biehler, R. (2003). Psychology applied to teaching (10th ed.). Boston: Allyn and Bacon. Soerjaningsih, W., Fraser, B. J., & Aldridge, J. M. (2001). Teacher-student interpersonal behaviour and student outcomes among university students in Indonesia. In P. L. Jeffrey (Compiler), AARE 2001 Conference Papers. Melbourne, Victoria, Australia: Australian Association for Research in Education. Retrieved March 2, 2003 from http://www.aare.edu.au/01pap/soe01034.htm Spector, M. J., Wasson, B., & Davidson, P. I. (1999). Designing collaborative distance learning environments for complex domains. In B. Collis and R. Oliver (Eds.), Proceedings of Ed- Media 1999: World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Seattle, Washington, June 19-24, 1999 (pp. 323-329). Charlottesville, VA: Association for the Advancement of Computing in Education. Spender, D. (2001). Panel discussion: Online education–are universities prepared? In Online learning in a borderless market. Proceedings of a conference held 15–16 February 2001 at Griffith University Gold Coast Campus (pp. 21-26). Canberra: Department of Education, Training, and Youth Affairs. Squire, K. D., & Johnson, C. B. (2000). Supporting distributed communities of practice with interactive television. Educational Technology Research and Development, 48(1), 23-42. Steele, J. M., House, E. R., & Kering, T. (1971). An instrument for assessing instructional climate through low-inference student judgment. American Educational Research Journal, 8, 447- 466. Steiner, V. (1995). What is distance education? Distance Learning Resource Network. Retrieved December 31, 2002, from http://www.dlrn.org/library/dl/whatis.html Stern, G. G. (1974). B = f(P, E). In R. H. Moos and P. M. Insel (Eds.), Issues in social ecology: Human milieus (pp. 559-568). Palo Alto, CA: National Press Books. Stern, G. G., Stein, M. J., & Bloom, B. S. (1956). Methods in personality assessment. Glencoe, IL: Free Press. 132 Stolarchuk, E., & Fisher, D. (1998, July). The effect of using laptop computers on achievement, attitude to science and teacher-student interpersonal behaviour in science. Paper presented at the annual conference of the Australasian Science Education Research Association, Darwin, Australia. Sutcliffe, R. G., Cogdell, B., Hansell, M. H., & McAteer, E. (1999). Active learning in a large first year biology class: A collaborative resource-based study project on AIDS in science and society. Innovations in Education and Training International, 36, 53-65. Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education, 22, 306-331. Tait, A. (2003). Editorial: Reflections on student support in open and distance learning. International Review of Research in Open and Distance Learning, 4. Retrieved May 13, 2003 from http://www.irrodl.org/content/v4.1/tait_editorial.html Taylor, J. (2001). Fifth generation distance education. E-Journal of Instructional Science and Technology, 4, 1-14. Retrieved December 29, 2002, from http://www.usq.edu.au/electpub/e- jist/docs/old/vol4no1/contents.htm Taylor, P. C., Fraser, B. J., & Fisher, D. L. (1997). Monitoring constructivist classroom learning environments. International Journal of Educational Research, 27, 293-302. Taylor, P. C., & Maor, D. (1998). Constructivist virtual learning environment survey. Retrieved December 31, 2002, from http://www.curtin.edu.au/learn/unit/05474/forms/CVLES_form.html Taylor, P. C., & Maor, D. (2000, February). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. In A. Herrmann and M. M. Kulski (Eds.), Flexible Futures in Tertiary Teaching, proceedings of the 9th Annual Teaching Learning Forum. Perth: Curtin University of Technology. Retrieved December 31, 2002, from http://cea.curtin.edu.au/tlf/tlf2000/taylor.html Teh, G. P. L. (1999). Assessing student perceptions of Internet-based online learning environments. International Journal of Instructional Media, 26(4), 397-402. Teh, G. P. L., & Fraser, B. J. (1994). An evaluation of computer assisted learning in geography in Singapore. Australian Journal of Educational Technology, 10(1), 55-68. 133 Tinker, R. (2000, September). Ice machines, steamboats and education: Structural change and educational technologies. Paper delivered at the Secretary’s Conference on Education Technology 2000, Alexandria, VA. Retrieved April 10, 2003 from http://www.ed.gov/Technology/techconf/2000/tinker_paper.pdf Tobin, K. (1998). Issues and trends in the teaching of science. In B. J. Fraser and K. G. Tobin (Eds.), International handbook of science education (pp. 129-151). London: Kluwer Academic Publishers. Tobin, K. (2000). Catalysing changes in research on learning environments: Regional editor’s introduction. Learning Environments Research, 2, 223-224. Tobin, K., & Fraser, B. J. (1998). Qualitative and quantitative landscapes of classroom learning environments. In B. J. Fraser and K.G. Tobin (Eds.), International handbook of science education (pp. 623-640). London: Kluwer Academic Publishers. Traub, J. (2000, November 19). This campus is being simulated. The New York Times Magazine, p. 90. Retrieved May 17, 2001, from LEXIS-NEXIS ACADEMIC database. Tu, C., & Corry, M. (2001). A paradigm shift for online community research. Distance Education, 22, 245-263. Tynjälä, P. (1999). Towards expert knowledge? A comparison between a constructivist and a traditional learning environment in the university. International Journal of Educational Research, 31, 357-442. University of the Incarnate Word. (2002). China Incarnate Word. Retrieved October 17, 2002, from http://www.uiw.edu/ciw/ University of the Incarnate Word. (n.d.). Universe online. Retrieved October 17, 2002, from http://antares5.net/uiw/about.html Wagner, A. (1998, October/November). Redefining tertiary education. The OECD Observer No. 214. Retrieved December 31, 2002, from http://www.oecd.org/publications/observer/214/article4- eng.htm Walberg, H. J. (1974). Evaluating educational performance. In H.J. Walberg (Ed.), Evaluating educational performance: A sourcebook of methods, instruments, and examples (pp. 1-9). Berkeley, CA: McCutchan Publishing. 134 Walberg, H. J. (1981). A psychological theory of educational productivity. In F. H. Farley & N. J. Gordon (Eds.), Psychology and education: The state of the union (pp. 81-108). Berkeley, CA: McCutchan Publishing Corporation. Walberg, H. J., & Anderson, G. J. (1968). Classroom climate and individual learning. Journal of Educational Psychology, 49, 414-419. Walberg, H. J., Singh, R., & Rasher, S. P. (1977). Predictive validity of students’ perceptions: A cross-cultural replication. American Educational Research Journal, 14, 45-49. Walker, S. L. (1998). The value of building skills with online technology: Online training costs and evaluation at the Texas Natural Resource Conservation Commission. In D. A. Schreiber & Z. L. Berge (Eds.), Distance training: How innovative organizations are using technology to maximize learning and meet business objectives (pp. 270-286). San Francisco: Jossey-Bass Publishers. Walker, S. L. (2001a, May). Online learning environments research. Paper presented at the Sixth Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Honolulu, HI. Walker, S. L. (2001b, November). What is going on in your classroom? Two formative methods for evaluating your online class. Paper presented at the 2001 Teaching Online in Higher Education Annual Conference. Purdue University, Fort Wayne, Indiana. Walker, S. L. (2002a). Evaluation, description and effects of distance education learning environments in higher education. In R. Ham & J. Woosley (Eds.), Proceedings of the Ninth Annual Distance Education Conference. College Station: Center for Distance Learning Research, Texas A&M University, TX. Walker, S. L. (2002b, May). Measuring the distance education psychosocial environment. Paper presented at the Seventh Annual Teaching in the Community Colleges Online Conference, Kapi'olani Community College & University of Hawaii, Hawaii, Honolulu, HI. Walker, S. L., & Resta, P. (2002, September). Collaborating for online instruction. Paper presented at the Technology Leadership Academy 2002 Fall Institute, Austin, TX. Wang, A. Y., & Newlin, M. H. (2002). Predictors of performance in the virtual classroom: Identifying and helping at-risk cyber-students. Technology Horizons in Education Journal, May, 21+. Retrieved October 16, 2002, from EDUmark database. 135 Wilson, J. (2002). The power of distance learning. Education, 122, 638. Wong, A. F. L., & Waldrip, B. G. (1996, November). Science classroom learning environments and student attitudes in Singapore, Australia, and the South Pacific. Paper presented at the annual conference of the Singapore Educational Research Association and Australian Association for Research in Education, Singapore. Wong, A. F. L., Young, D. J., & Fraser, B. J. (1997). A multilevel analysis of learning environments and student attitudes. Educational Psychology, 17, 449-468. Woolfolk, A. (2001). Educational psychology (8th ed.). Boston: Allyn and Bacon. Wubbels, T. (1993). Teacher-student relationships in science and mathematics classes (What Research Says to the Science and Mathematics Teacher, No. 11). Perth, Western Australia: National Key Centre for School Science and Mathematics, Curtin University of Technology. Wubbels, T., Créton, H. A., & Hooymayers, H. P. (1992). Review of research on teacher communication styles with use of the Leary model. Journal of Classroom Interaction, 27, 1- 12. Wubbels, T., & Levy, J. (1993). Do you know what you look like? Interpersonal relationships in education. London: Falmer Press. Yang, J., Huang, I. T., & Aldridge, J. M. (2002). Investigating factors that prevent science teachers from creating positive learning environments in Taiwan. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 217-233). River Edge, NJ: World Scientific. Yarrow, A., Millwater, J., & Fraser, B. J. (1997). Improving university and primary school classroom environments through preservice teachers’ action research. International Journal of Practical Experiences in Professional Education, 1, 68-93. Youngblood, P., Trede, F., & DeCorpo, S. (2001). Facilitating online learning: A descriptive study. Distance Education, 22, 264-284. Ysseldyke, J., & Elliott, J. (1999). Effective instructional practices: Implications for assessing educational environments. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (3rd ed.). New York: John Wiley and Sons, Inc. 136 Zandvliet, D. B. (1999). The physical and psychosocial environment associated with classrooms using new information technologies: A cross-national study. Unpublished doctoral dissertation, Curtin University of Technology, Perth, Western Australia, Australia. Retrieved March 19, 2003 from the Australian Digital Theses Program Distributed Database http://adt.curtin.edu.au/theses/available/adt-WCU20020502.121823/ Zandvliet, D. B. (2002). Learning environments in technology-rich classrooms. In S. C. Goh & M. S. Khine (Eds.), Studies in educational learning environments: An international perspective (pp. 49-72). River Edge, NJ: World Scientific. Zhu, E., & McKnight, R. (Eds.). (2001). Principles of online design. Fort Myers, FL: Gulf Coast University, Office of Instructional Technology. Retrieved May 25, 2001, from http://www.fgcu.edu/onlinedesign 137 Appendix A SCALE AND ITEM REVIEW PANELLISTS SCALE REVIEW PANELLISTS George Bagwell Professor, Anthropology and Psychology Colorado Mountain College, USA Edward Bressan Instructional Designer Intuition Publishing, London, England Angela S. Burger Professor of Political Science University of Wisconsin Colleges, USA Susan M. Butler, Ph.D. Science Education North Carolina State University, USA Daph Crane Instructional Designer/Program Evaluator Bow Valley College, Calgary Alberta, Canada Fran Hunter Distance Learning Coordinator, College of Education Texas A&M University, USA Rozhan M. Idrus Associate Professor Universiti Sains Malaysia, Malaysia Rick Merritt, Ph.D. Manager of Technical Training/Adjunct Instructor Correspondent Services Corporation/UBS PaineWebber/University of Denver, USA Bonnie Morihara, Ph.D. Program Associate for Distance Learning Oregon University System, USA Martin Omukuba Monitoring and Evaluation Officer CARE International, Kenya 138 Carolyn C. Robertson, Ph.D. Director of Distance Learning Tarrant County College, USA John M. Slatin, Ph.D. Director, Institute for Technology and Learning University of Texas at Austin, USA Frank M. Slapar Adjunct Professor/Professor Emeritus California National University for Advanced Studies/Pittsburg State University, USA Arun Kumar Tripathi Research Assistant Telecooperation Research Group, Technical University of Darmstadt, Germany ITEM REVIEW PANELLISTS Angela S. Burger Professor of Political Science University of Wisconsin Colleges, USA Susan M. Butler, Ph.D. Science Education North Carolina State University, USA Daph Crane Instructional Designer/Program Evaluator Bow Valley College, Calgary Alberta, Canada Fran Hunter Distance Learning Coordinator, College of Education Texas A&M University, USA Rozhan M. Idrus Associate Professor Universiti Sains Malaysia, Malaysia Rick Merritt, Ph.D. Manager of Technical Training/Adjunct Instructor Correspondent Services Corporation/UBS PaineWebber/University of Denver, USA Carolyn C. Robertson, Ph.D. Director of Distance Learning Tarrant County College, USA 139 Frank M. Slapar Adjunct Professor/Professor Emeritus California National University for Advanced Studies/Pittsburgh State University, USA 140 Appendix B FINAL REVISED VERSION OF THE DISTANCE EDUCATION LEARNING ENVIRONMENT SURVEY (DELES) 141 142 143 Appendix C SATISFACTION SCALE OF ENJOYMENT OF DISTANCE EDUCATION 144 14270_downloaded_stream_14270 14271_downloaded_stream_14271 14272_downloaded_stream_14272 14273_downloaded_stream_14273 14274_downloaded_stream_14274 14275_downloaded_stream_14275 14276_downloaded_stream_14276 14277_downloaded_stream_14277 14278_downloaded_stream_14278 14279_downloaded_stream_14279