200 BOOK REVIEW MEASUREMENT IN DISTANCE EDUCATION Written by Amy J. Catalano Dr. Mehmet KOKOC ORCID: 0000-0002-1347-8033 Fatih Faculty of Education Trabzon University Trabzon, TURKEY ISBN 9781315229447 (eBook) 9781138714465 (Paperback) 9781138714472 (Hardback) Publication Date 2018 Publication Formats Print and eBook Publisher Routledge Publishing/Taylor & Francis INTRODUCTION The growing interest in distance education technologies bring together a critical question on whether online and distance learning is as effective as face-to-face learning. This skepticism requires employing best approaches and robust instruments to evaluate online and distance education (Lowenthal & Davidson-Shivers, 2019). Researchers and instructional designers examine learning process in distance education for building better online learning experiences. To ensure the quality of distance education, institutions basically emphasis on well-designed course, effective technologies, online instructors with high competence, and effective learning outcomes. What is important at this point is to evaluate efficacy of distance education using robust and valid instruments, scales and measurements. In this respect, the book titled Measurements in distance education: A Compendium of instruments, scales, and measures for evaluating online learning by Catalano (2018) has set out to provide a large manual including more than fifty instruments, scales and questionnaire with useful description on their psychometric properties. The author of the book is Amy J. Catalano. Dr. Catalano is currently an Associate Professor of teaching, learning and technology at Hofstra University, United States of America. Her recent studies have encompassed psychometric evaluation of instruments. The book aims to be an organized guide to assist novice and experienced researchers in the field of distance education in using reliable and valid instruments, scales, and measures that are suitable to their research questions. Its intended audience are anyone interested in understanding assessment of distance education and learning. The main thesis of the book is that post- secondary institutions become engaged in distance education increasingly and assessing the quality of distance programs with appropriate instruments is very important for them. Thus, the book is a collection of many measures applicable distance learners to evaluate distance education and teaching, online learning experience of learners and online instructors. Turkish Online Journal of Distance Education-TOJDE October 2019 ISSN 1302-6488 Volume: 20 Number: 4 Book Review 2 201 REVIEW OF THE BOOK The overall structure of the book takes the form of five chapters, including, besides the introduction and index, chapter on “engagement and satisfaction” (Chapter 1), “student readiness to learn online and self- efficacy” (Chapter 2), “evaluation of the distance education teaching and learning environment” (Chapter 3), “student learning and behaviors” (Chapter 4), and “student achievement, retention, and attrition” (Chapter 5). Considering the structure of the book, Catalano’s The Measurements in distance education seems to be a well-organized and useful book. In the five chapters of the book, the author presents a range of prominent themes related to effectiveness and efficiency of online and distance learning. The book begins with an introduction section in which the author explains the goal and rationale of the book, criteria for inclusion and search strategy, most commonly used metrics of validity and reliability. The introduction section provides useful description of reliability and validity concepts that should be considered to review whether a scale is reliable and valid. The sections on instruments follow an organized format, starting with source, purpose and description of the scale; details on development and validation of the scale; scale items if accessed. Prefaces of the chapters apart from the fifth chapter have explanations of concepts related to the instruments. The first chapter includes 11 instruments assessing engagement and satisfaction. It defines engagement based on Dixon (2015, p.2) as “the extent to which students actively engage by thinking, talking, and interacting with the content of a course, the other students in the course, and the instructor”. While the author emphasizes importance of engagement in online courses success, there is only two engagement scale presented in the chapter. Interestingly, three social presence scales are included in the chapter due to being associated with satisfaction and engagement. This chapter could be more informative if the author argues for a clear distinction among social presence, engagement and satisfaction. The following scale is about online student connectedness which is related to student feelings leading to drop-out. Most instruments in the chapter are satisfaction scales about perceived satisfaction with course content, learning environment, learning experiences, interactions and support services. The second chapter includes 16 instruments assessing student readiness to learn online and self-efficacy. In this chapter, the author states that there are many instruments measuring online readiness, which are developed based on different perspectives and for various populations. It is pointed out that student readiness encompasses self-efficacy in context of online learning. Thus, the concept of student readiness is paired with the concept of self-efficacy by the author. Most instruments in the chapter are online learning readiness scales measuring student competencies for successful online learning. It is remarkable that readiness scales have been developed considering various learning context and technologies such as internet, e-learning, computer-supported collaborative learning, self-regulated online learning and mobile learning. In addition, there are two scales about teacher readiness and online learning self-efficacy/satisfaction in the chapter. The author emphasizes possible risks of using readiness scales for not at-risk online learners as discouraging learners’ participation in distance education. The third chapter includes 17 instruments assessing evaluation of the distance education teaching and learning environment. The instruments represented in the chapter focus on measuring students’ perception of critical factors which are playing important role in effective online learning and teaching. Some of these scales can be used to measure the evaluation of distance education environment with pedagogical and technological elements as a holistic perspective, while others examine evaluation of distance education environment by considering specific educational concept and learning contexts based on different theoretical frameworks. The author put emphasis on online constructivism learning environments as a followed path when classifying scales in evaluation of the distance education. Thus, the scales titled COLLES, CMLES and OCLES are included in the third chapter as measures on evaluation of constructivist online learning environments. Similarly, COI survey based on social constructivist model is described with detail information. It is remarkable that there is only a scale measuring competencies and tasks for teaching online in the third chapter. 202 The fourth chapter covers 11 instruments assessing student learning and behaviors. Most of the instruments included in this chapter are intended to be used to assess students’ attitudes toward online learning. Two scales of technology acceptance focus on students’ intention to accept and use online learning technologies. Using these scales can provide meaningful results to researchers and learning designers to assist students in learning and support sustainable learning. In the fourth chapter, there are specific scales measuring enjoyment, misbehavior, learning styles and critical thinking skills of students in context of online learning. In addition, a scale that can be used to examine how students use digital technologies and social media in online learning is described here with full text of the items. The last chapter includes five instruments assessing student achievement, retention and attrition. Interestingly, it doesn’t begin with an introduction of the chapter and description of the relevant concepts. Most instruments in the chapter are about retention and attrition of students in higher education. These instruments aim to identify students at risk of drop out. Apart from these, an online learning achievement questionnaire is provided in the chapter. This questionnaire encompasses factors that predicts students’ achievement such as confidence in online skills, desire for interaction with others, self-management of learning and beliefs about distance education. The most appealing feature of the book is that it provides detailed information needed by researchers related to development and validation of the instruments including their psychometric properties. Considering description of the instruments in the book, the reader can choice appropriate data collection tool to use in their research. When the scales provided in the book are taken into consideration based on their purpose and published years, it is noticed that there is an association between measurements in distance education, online learning context and technologies. The scales developed based on constructivism, social learning, community of inquiry, computer-supported collaborative learning, massive open online courses, content and learning management systems confirm this inference. A possible explanation for this might be that widespread use of new technologies and triggering role of the pedagogical models have an effect on emergence of new measurements in distance education. CONCLUSION The book could be beneficial for researchers to assess and evaluate impact of open and distance learning courses on individual differences of learners and essential elements to success in distance learning. Also, it provides valuable collection of valid instruments and scales that can be used to answer critical questions on efficiency of learners’ learning experiences, teaching experiences and online learning design. It is hoped that the instruments and scales in the book will be of interest to researchers, online learning designers, instructional designers and practitioners in the field of distance education. Nonetheless, the reader should bear in mind that the book is based valid measures identified following the specific search strategy by the author. In conclusion, my recommendation to researchers and especially PhD candidates in the field of distance education is that you should benefit from the book as a starting point to select appropriate data collection tools after deciding on your research problems. In addition, the book could be a source of information to researchers, who intent to develop new instruments, to help them know what is developed before they begin instrument development. 203 BIODATA and CONTACT ADDRESSES of AUTHOR Dr. Mehmet KOKOC, is currently a researcher in the Department of Computer Education and Instructional Technology at Trabzon University, Turkey. He received his Ph.D. in Computer Education and Instructional Technology from Hacettepe University. He teaches undergraduate-level instructional technology courses and graduate-level educational measurement and statistics courses. Dr. Kokoc’s research interests primarily focus on learning analytics, cognitive profiling, video use in e-learning, social media in education, open and distance learning. He is also broadly interested in excessive use of digital technologies such as social media and professional development of teachers. Mehmet KOKOC Department of Computer Education and Instructional Technology, Fatih Faculty of Education, Trabzon University Address: Trabzon University Fatih Campus, F Block 3rd Floor, 61330, Trabzon, Turkey Phone: +90 462 4551582 E-mail: kokoc@trabzon.edu.tr REFERENCES Catalano, A. J. (2018). Measurements in distance education - A compendium of instruments, scales, and measures for evaluating online learning. London: Routledge. Dixson, M. (2015). Measuring student engagement in the online course: The online student engagement scale (OSE). Online Learning, 19(4). doi: http://dx.doi.org/10.24059/olj.v19i4.561 Lowenthal, P. R., & Davidson-Shivers, G. V. (2019). Strategies used to evaluate online education. In M. G. Moore & W. C. Diehl (Eds.), Handbook of distance education (pp. 415 –427; 4th ed.). Routledge.