Student opinions about the period of measurement and evaluation in distance education: the difficulties Available online at www.sciencedirect.com 1877-0428 © 2010 Published by Elsevier Ltd. doi:10.1016/j.sbspro.2010.12.371 Procedia Social and Behavioral Sciences 9 (2010) 1597–1601 WCLTA 2010 Student opinions about the period of measurement and evaluation in distance education: the difficulties Hasan Karala, Ayça Çebib*, Memnüne Pek enc a,c Department of Computer Education and Instructional Technology, Karadeniz Technical University, Trabzon 61040, Turkey bDistance Education Center, Karadeniz Technical University, Trabzon 61040, Turkey Abstract The purpose of this study is to identify the views of the students' who have been trained by the distance education about the measurement-evaluation period. 91 students who are from different cities and registered in the distance education program of the graduate degree without thesis in Karadeniz Technique University, Social Sciences Institute, Education Management, Inspection, Planning and Economy have constituted the sample of this study. The data in the study have been gathered through the interview form in which the open-ended questions take part. Students experience anxiety and concern due to the probable system problems and the outage of electricity and internet which may occur in the period. Also it has been expressed that some students can't motivate enough for the online exams because of the atmosphere differences. © 2010 Published by Elsevier Ltd. Keywords: Distance education, Measurement and evaluation, Students' opinions; 1. Introduction Compared to traditional teaching methods, distance education, with the developments in information and communication technologies, has enabled the realization of different distance education applications. The assessment and evaluation caused by developments in distance education on the current trends in education take place between these applications. Assessment process is important for students and distance education as well as all the institutions to determine how much students have learned as a result of their education. This process provides important feedback for both sides (Altan & Sefero lu, 2009; Kara, 2009). From this point of view, it is expressed by researchers that students’ opinions about the process of assessment and evaluation have a great importance (Altan & Sefero lu, 2009). To increase the effectiveness of distance education assessment activities, alternative to the traditional measurement tools, different vehicles can be used (Dali, 2008). For this purpose, using new assessment methods such as constructive learning, performance assessment and evaluation of real-life; according to this approach, assessment is one of the basic elements of the learning process and regardless of being in contrast with the process, evaluation begins with the process (Simonson et al., 2003; Semerci & Bekta , 2005). The flexibility in distance education provided by developments in Internet technology has made the assessment and evaluation flexible also; online applications have begun to carry out. There are some benefits and limitations about online measurement used in assessment activities Providing students with flexibility in terms of time, the collection of data more quickly, * Hasan Karal. Tel.: +90 462 377 71 25; fax: +90 462 248 73 44 E-mail address: aycacebi@ktu.edu.tr Open access under CC BY-NC-ND license. http://creativecommons.org/licenses/by-nc-nd/3.0/ 1598 Hasan Karal et al. / Procedia Social and Behavioral Sciences 9 (2010) 1597–1601 evaluation of the results more quickly and reducing costs are among these benefits (Dommeyer et al., 2004; Anderson et al., 2005). Limitations encountered are examined, it requires examination of computer and internet access, lack of security caused by examination itself, the probability of the students’ cheating or controlling whether the student enter into the exam his own or not, student isolation, evaluation, deficiency of instructor control over the evaluation conditions, lack of communication, and lack of skill and information about online education are stated as the issues in such environments (Kerka & Wonacot, 2000; Graham et al., 2000; Shuey, 2002; Benson, 2003; Tany ld z & Semerci, 2005, McCombs, & Vakil, 2005). 2. Aim As with all systems, in distance education evaluation must be considered to determine effectiveness of teaching- learning process. Jung (2005) stated that measurement and evaluation are seen as the last step distance education institutions and generally hasn’t been given the necessary importance. Yet, determining students’ positive and negative opinions about the evaluation process helps reviewing assessment methods in distance education and students’ contribution to the creation of more comfortable environments (Altan & Sefero lu, 2009) is stated by the researcher. In this framework, a deep qualitative study was needed about how the students who are registered in distance education perceive analyzing and evaluation process. The purpose of this study is to determine the opinions of the students of distance education concerning assessment and evaluation process. 3. Method In this study case study, a qualitative research pattern, were used. Case study searches for factors related to a situation with a holistic approach and focuses on how they affect the status (Y ld r m & im ek, 2006). In other words, case studies are the works in that life experiences are reflected. 91 students who are from different cities and registered in the distance education program of the graduate degree without thesis in Karadeniz Technique University, Social Sciences Institute, Education Management, Inspection, Planning and Economy have constituted the sample of this study. As the participants are experts from different levels of The Turkish Ministry of National Education, this study enabled them to consider assessment and evaluation process not only through the eyes of a student but also through the eyes of a teacher. The study data were collected through open-ended interview form including questions. In the preparation of the interview form, the theoretical information obtained from the literature searching, data collection tools used in researches on this subject and expert opinions are based on. The data gathered from the interview were assessed using content analysis method. In order to increase the reliability and validity of the research findings the views of participants were frequently quoted. In addition, the data obtained from subject matter were discussed with experts in the field of education at every stage and these people’s comments related to the results were taken. 4. Finding In this section, the students’ demographic characteristics and experiences they has lived in the process of assessment and evaluation in distance education will be given. Table 1 shows the demographic characteristics of students participating in the study. Table 1. Demographic Characteristics of Participants FEATURES N % Gender Male 84 92.3 Female 7 7.7 Ministry of Education service class Inspector 19 20.9 Branch Manager 12 13.2 Principal 22 24.2 School Assistant 29 31.8 Teacher 9 9.9 Computer Literacy Level Basic level (On-off the computer, browsing etc..) 7 7.7 Middle-level (basic skills programs, as well as office use) 64 70.3 Hasan Karal et al. / Procedia Social and Behavioral Sciences 9 (2010) 1597–1601 1599 Advanced level (use of all kinds of program in addition to middle level) 20 22.0 Online Exam Experience Yes, I have online exam experience before. 19 20.9 No, I haven’t got online exam experience before. 72 79.1 If data in Table 1 is analyzed, it is seen that 84 students participated in the study (92.3%) are male, 7 (7.7%) are the female. Sample group is designated from inspectors (20.9%), the branch managers (13.2%), the school principals (24.2%), deputy director of the school (31.8%) and teachers (9.9%) who are served at least 5 years in Ministry of Education. When the participants assessed according to their computer literacy, it is seen that 7.7% of the basic level, 70.3% moderate reputation, while 22% of advanced. When the students’ online exam experiences in distance education are examined, it is stated that 79.1% of the students hasn’t lived online exam experience, besides, 20.9% have lived such an experience before. When the students' opinions about the process of assessment and evaluation in distance education are examined, it is found that students have experienced some difficulties in this process. Table 2 summarizes the difficulties that students have experienced the process. Table 2. Students’ views on difficulties experienced in the process of assessment and evaluation in distance education Problems f % Student (Individual) based problems Lack of interest and motivation 16 17.6 Lack of computer literacy 12 13.2 Online test anxiety 8 8.8 Technology based problems Internet problems 22 24.2 Technical (computer) problems 15 16.5 Power cut 9 9.9 Teacher based problems Lack of measuring instruments diversity 5 5.5 Alternative assessment means are not preferred 2 2.2 Distance (Media) based problems Lack of feedback 28 30.7 Communication Problems 18 19.8 Teachers and students are not know each other enough 13 14.3 The interaction problems in courses 7 7.7 The absence of guidance in the exams 5 5.5 The process cannot be evaluated 4 4.4 Validity problems Lack of coherence between the topics and questions in the tests 7 7.7 The degree of difficulty of the questions 2 2.2 Cheating on online exams 15 16.5 Reliability Problems Online exam time is short 15 16.5 Exam questions are not clear 3 3.3 The question is not sufficient in number 2 2.2 Table 2, when the data in table 2 is analyzed, it is seen that students have a number of difficulties in the process of assessment and evaluation. At the end of the analysis, it has stated that there are some problems based on students (individuals), teacher, technology and distance, and as a result it has concluded that there are validity and reliability problems. As a result of the investigations it has stated that students related problems are lack of interest and motivation, computer literacy and online test anxiety. Students views on this subject has stated as follows: “Environments in tests applied in distance education can affect concentration.” “Due to the domestic environment I cannot concentrate on courses and online tests so it causes tests anxiety on me.” “Distance education applications require efficiency in information technologies. Comparing an inadequate student with others may cause some disruptions. Students reached a particular qualification in a given program in distance education must be sufficient in the fields of information technology, also. Otherwise, the measurement and evaluation is not healthy. A person who cannot use keyboard and communication skills efficiently wastes his time. This will adversely affect academic performance, also.” 1600 Hasan Karal et al. / Procedia Social and Behavioral Sciences 9 (2010) 1597–1601 In distance education applications, internet problems such as disconnection from the Internet (24.2%), lack of computer hardware or software problems (16.5%), and power cuts (% 9.9) are the technology based problems. The process of assessment and evaluation in distance education technology issues that negatively affect the resources are defined as follows: technological problems affect the evaluation process in distance education and it is expressed below: “Disconnection from internet and power cuts in my current location affects me negatively so I cannot listen to courses and participate in tests effectively.” “I think that technical problems in the time of tests affect the assessment and evaluation process negatively.” “A sudden technologic problem may affect the participant’s success.” As students and teachers are away from each other, students and teachers cannot recognize each other enough (%14.3). In addition, it causes communication problems (%19.8), lack of interaction (%7.7) and deficiency in feedback (%30) which is one of the most important elements of evaluation process. It is stated that lack of guidance also affect students’ success (%5.5). In addition, it is reported that in distance education, evaluation process is carried out rather than product evaluation and it is hard to carry out process evaluation activities (4.4%). Students' opinions about the subject are as follows: “As it is distant, personal qualities cannot perceived as it is desired so evaluation process affected negatively.” “There is no face-to-face interaction so there is no interaction with teachers. We cannot talk or share lessons with other colleagues. As the lessons are in virtual environment it is difficult to understand or integrate issues. In addition, in a virtual environment, an adequate motivation cannot be provided in the exam.” “Differences in Teachers' expectations and students’ expectations affects academic success in negative.” “It is a disadvantage that there is no one to ask for help in the online exams. So it affects evaluation.” “There is no enough feedback in distance education. Because there are a large group of students. We ask questions, but teachers may pass without seeing some of these questions. Following the evaluation of test results or details about our faults, we cannot take feedbacks about our errors. We can only see the errors and the correct answer but we cannot talk about our errors.” Distance education teaching staffs do not prefer alternative evaluation techniques (%2.2) and reliability and validity of the expression was also experiencing some problems. Cheating on online exams (16.5%), described the issues are not consistent with the exam questions (7.7%), and questions the degree of difficulty (2.2%) is a good problem to set the terms of the validity of such examinations, plus online exams not given enough time (16.5%), exam questions is not clear (3.3%) and lack of a sufficient number of exam questions (2.2%) also affects the reliability. Some students' opinions about this situation are as follows: “Exams are generally in the form of test. Sometimes classic. But other question techniques should be given addition to these.” “Assessment techniques used by teachers are the same. Yet, we talk about the model of constructivist learning and alternative assessment and evaluation tools, but we do not apply them. In fact, if portfolios in which homework, projects or studies are carried out in electronic environment, it may be more useful for the evaluation process.” “Think that people that attend the same program answer the questions together. What can we say about the validity and reliability of this exam?” “As I cannot use communication skills and keyboard efficiently, I spent some extra time to write the answers. So, especially in classical exams, I cannot finish my exam in time.” 5. Discussion, Conclusion and Recommendations Based on the findings gained from the experts, students' interests and motivations, their computer literacy levels and technological infrastructures (Internet, computer hardware) have an important role in distant education. In distant education, teachers and learners do not know each other. So their expectations are different, also. Baturay and Bay (2009)’ result supports this situation. It is clear from the students’ opinions that evaluation of the process according to tests or written exams cannot be carried out in a healthy way. Altan and Sefero lu (2009) emphasize that feedback is important for students for an effective distant education. In the study, it is emphasize that feedback is important but because of the distance there is less and inadequate Hasan Karal et al. / Procedia Social and Behavioral Sciences 9 (2010) 1597–1601 1601 feedback. This situation affects students’ success negatively. In distance learning, while evaluation process, student may cheat. King and his friends (2009) findings show that students cheat more easily in online education. When the questions are not clear, students cannot find one to ask for help. This is another deficiency of the system. In online exams compared to classical exam patterns, there are some factors that affect success. These are students’ computer literacy, as a result of this having time problem, each person has been exposed to different environment and different stimuli (lack of unity in environment) as a result of lack of motivation in exam. Xuet (2007) has emphasized in his study that online exams are not affected by computer knowledge and attitude but these exams should be prepared in a simple way that those who don’t know much about computer can overcome. Students have exam anxiety even if their computer literacy level middle or higher. Because they haven’t experienced an online exam before. Students preferred online exams because they had online exam experience (Donovan et al., 2007). According to the students’ opinions; there isn’t variety in evaluation and assessment tools in distant education, there is no coherence between the questions in exams and the topics and the questions are not clear. To evaluate process through distance education forum, online chats and etc.. should be included to the process and should be active during the process. Questions should not only measure students’ level of knowledge but also they should be simple and understandable to prevent inappropriate behaviors. Exam environments should be prepared according to online exam. In distance education, selecting the most appropriate methods of assessment and evaluation, the process can be made more efficient. References Altan, T., & Sefero lu, S. S. (2009). Uzaktan e itimde de erlendirme süreci: Ö renci görü lerinin sistemin geli imine katk lar . 3.International Computer & Instructional Technologies Symposium (ICITS 2009). 7-9 October 2009, Karadeniz Technical University Fatih Faculty of Education, Trabzon. Anderson, H. M., Cain , J., & Bird, E. (2005). Online Student Course Evaluations: Review of Literature and a Pilot Study. American Journal of Pharmaceutical Education 2005, 69(1), 34-43. Baturay, M., & Bay, Ö.F. (2009). Uzaktan ö retimi tercih eden ö rencilerin demografik özellikleri (Demographic Characteristics of the Students That Chose Distance Education). Journal of Ziya Gokalp Faculty of Education, 7(13), 17-26. Benson, A. (2003). Assessing participant learning in online environments. New Directions for Adult and Continuing Education, 100, 69-78. Dali, H. (2008). Desing and implement of e-learning performance evaluation system. International Conference on Computer Science and Software Engineering. DOI 10.1109/CSSE.2008.157 Dommeyer, C.J., Baum, P., Hanna, R. W., & Chapman, K. S. (2004). Gathering faculty teaching evaluations by inclass and online surveys. Assessment & Evaluation in Higher Education, 29(4), 611-623. Donovan, J., Mader, C., & Shinsky, J. (2007). Traditional Course Evaluation Formats: Student Perceptions. Journal of Interactive Online Learning, 6(3), 158-180. Graham, C., Cagiltay, K., Craner, J., Lim, B., & Duffy, T. M. (2000). Teaching in Web Based Distance Learning Environment: an Evaluation Summary Based on Four Courses. Center for Research on Learning and Technology Technical Report No.13. Indiana University, Bloomington. Jung, I. (2005). Innovation and good practices of open and distance learning in Asia and the Pasific. Apeid, Unesco Bangkok Occasional Paper Series. Paper no: 3. Kara, E. (2009). Uzaktan yüksek ö retimde ölçme de erlendirme ve Anadolu Üniversitesi aç kö retim sistemi uygulamas (Measurement and evaluation in distance higher education and an application in Anadolu University open education system). Social Sciences Institute of Anadolu University, Eski ehir. Kerka, S., & Wonacot, M. (2000). Practitioner file: Assessing learners online. Columbus, OH: Clearinghouse for Adult, Career, and Vocational Education. Accessed May 16, 2009 from:http://ericacve.org/fulltext.asp King, C., Guyette, R., & Piotrowski, C. (2009). Online exams and cheating: An empirical analysis of business students’ views. The Journal of Educators Online, 6(1), 1-11. McCombs, B., & Vakili, D. (2005). A learner-centered framework for e-learning. Teachers College Record. 107(8), 1582-1600. Semerci, Ç., & Bekta , C. (2005). nternet Temelli Ölçmelerin Geçerli ini Sa lamada Yeni Yakla mlar (The New Approaches for Prov d ng Val d ty In Internet-Based Measurements), The Turkish Online Journal Of Educational Technology, 4(1), 130-134. Shuey, S. (2002). Assessing Online Learning in Higher Education. Journal of Instruction Delivery Systems, 16(2),13-18. Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance: Foundations of distance education (2nd ed.). Upper Saddle River, NJ: Merrill Prentice Hall. Tany ld z , M., & Semerci, Ç. (2005). Çevrim çi E itim Uygulamalar na li kin Ö retim Eleman ve Ö renci Görü lerinin Belirlenmesi (The opinions of students and teaching staff about online education implementations). Accessed May 22, 2009 from: http://www.tebd.gazi.edu.tr/arsiv/2005_cilt3/sayi_2/197-216.pdf Xu, Y., Iran-Nejad , A., & Thoma, S. J. ( 2007). Administering Defining Issues Test Online: Do Response Modes Matter?. Journal of Interactive Online Learning, 6(1), 10-27.