Views on Modular Assessment and Evaluation Process in Distance Education Procedia - Social and Behavioral Sciences 46 ( 2012 ) 2073 – 2077 1877-0428 © 2012 Published by Elsevier Ltd. Selection and/or peer review under responsibility of Prof. Dr. Hüseyin Uzunboylu doi: 10.1016/j.sbspro.2012.05.430 WCES 2012 Views on modular assessment and evaluation process in distance education Hasan Karala, Ayca Cebib* a Department of Computer Education and Instructional Technology, Karadeniz Technical University, Trabzon, 61040, Turkey bDistance Education Center, Karadeniz Technical University, Trabzon, 61040, Turkey Abstract Thanks to developing information and communication technologies, distance education enables to perform distance education applications different from traditional teaching methods. Assessment and evaluation in distance education is among current interest areas with the developing distinct distance education applications. The current study aims to determine views on assessment and evaluation process in distance education by identifying which modules (exam, forum, chat, assignment, wiki etc.) should be used for online assessment process and which criteria should be taken into consideration while evaluating a student performance. In this context, the study was carried out with 91 registered participants at Karadeniz Technical University. The study was a descriptive study, therefore; the data was gathered via a questionnaire developed by the researchers. The study concludes that assessment and evaluation process involves not only online exams but also modules such as forum, assignment, wiki, dictionaries which shows students’ process performance. In modular assessment and evaluation process, students’ answers should be considered qualitatively and quantitatively by teachers. © 2012 Published by Elsevier Ltd. Keywords: Moduler Assessment and Evaluation, Distance Education ; 1. Introduction With the rapid developments in communication and information technologies and the spread of the internet with each passing day has brought great opportunities in the field of education as well as in many fields. The distance education where students and teachers are in different places in terms of time and space or in both conditions has become more attractive (Özkul, 2003). With widespread distance learning activities, assessment and evaluation process in distance education is started to be discussed. Although activities in distance education are attempted to be continued as in the traditional education, it is stated that there has been some problems especially in assessment and evaluation process and also students cannot be evaluated well enough in the process (Shuey, 2002). Compared to the traditional classroom teaching, different educational problems are encountered in distance education. This is why distance educators asses students not only with exams and assignments but they also find useful to collect data with different tools (Altan & Seferoğlu, 2009). In distance learning applications, traditional methods of assessment and evaluation, multiple choice, fill in, short answer or long explanations, are generally used (Simonson et al., 2003). However Kalelioğlu and Gulbahar (2010) * Ayça Çebi Tel.: +90-462-377-7125 E-mail address: aycacebi@ktu.edu.tr Available online at www.sciencedirect.com © 2012 Published by Elsevier Ltd. Selection and/or peer review under responsibility of Prof. Dr. Hüseyin Uzunboylu Open access under CC BY-NC-ND license. Open access under CC BY-NC-ND license. http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/ 2074 Hasan Karal and Ayca Cebi / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 2073 – 2077 have emphasized the need to use alternative tools as well as traditional measurement tools to increase the effectiveness of activities and to execute process evaluation in terms of effectiveness of education and student achievement as the communication between teachers and students is limited in distance education process. Learning management systems (LMS) which are growing in parallel with the developments in distance education applications renew itself with new add-ons and new modules with each passing day. Learning management systems enable traditional learning activities and they also have modules which enable us to evaluate students in the process (Watson & Watson, 2007). Effective use of these modules (wiki, blog, workshop) and analysis of students' log records have great importance to contribute process evaluation (Ingram, 1999). In addition, many criteria such as participation in discussions on time such as online forums, the nature of participation, acting in accordance with the forum rules, length of the messages and supporting with references are reported as the criteria to be taken into account in process evaluation (Dabbagh, 2000; Fleming, 2008). With this frame, the purpose of the study is stating the comments on assessment and evaluation process in distance education to introduce the thoughts about which modules in on-line process should be participated in the 2. Method The study was conducted with 91 participants registered to distance education graduate program in Karadeniz Technical University. As the participants were the experts in different levels of The Turkish Ministry of National Education, they were able to assess the study in the eyes of teacher as well as in the eyes of students. Descriptive design was used for the research. These are the studies in which participants' views or interests and abilities of a subject are determined (Fraenkel & Wallen, 2006). Data of the study were obtained through questionnaires developed by researchers. The survey consists of likert and open-ended questions. Questionnaire used in this study was examined by experts and it was applied to working groups after necessary adjustments were made. Demographic characteristics of the survey working group are consist of 5-point likert-type questions related to which modules should be taken into account in online assessment and evaluation and the criteria in these modules. To determine the gap, the formula 5-1/5 was used and the value 80 was found. Accordingly, if the c -4.20, they are totally agree with the thoughts in the study, if it is 4.19-3.40, they are highly agree with them, if it is 3.39-2.60, they are partially agree, if it is 2.59-1.80, they are less agree, if it is 1.79-1.00, they are never agree with the thoughts in the survey. The data was analyzed with the descriptive analysis method. 3. Finding In this section, the demographic characteristics of participants and the views related to alternative evaluation criteria in assessment and evaluation process in distance education will be given respectively. Table 1 shows the demographic characteristics of students participating in the study. Table 1. Demographic characteristic of participants FEATURES N % Gender Male 84 92.3 Femal 7 7.7 Professional Experience 6-10 years 41 45.1 11-15 years 37 40.7 16 years and more 13 14.2 Distance Education Experiences One Semester 58 63.7 Two Semesters 33 36.3 2075 Hasan Karal and Ayca Cebi / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 2073 – 2077 When the data examined in Table 1, 84 of the participants (92.3%) are male, 7 of them (7.7%) are female participants. While the participants are students in graduate program with no thesis in distance education, they are also performing as teachers. In terms of professional experience, 41 participants (45.1%) have 6-10 years experience, 37 (40.7%) have 11-15 years of professional experience and 13 (14.2%) have and 16 years and more experience. When the participants are examined in terms of their distance education experiences, it is seen that the majority of participants (63.7%) have taken courses for one semester, while the others (36.3%) have taken courses for two semesters. Synchronous and asynchronous modules are used in distance education allow researchers to evaluate the participants outside the traditional assessments as online exams. Table 2. Participants' views on evaluation modules that can be used in distance education Items X Sd criteria that must be in the assessment and evaluation process. 3.35 1.46 The evaluation must be done only according to the results of the exams. 2.58 1.37 The number of discussions in forums (frequency) should be considered as an evaluation criterion. 2.32 1.30 Student participation in synchronous courses should be an evaluation criterion. 3.00 1.43 Notes from quizzes must be taken into account in assessment and evaluation. 2.95 1.29 Participation to the questionnaires is a criterion that should be taken into account in evaluation process. 3.10 1.51 Material sharing in system should be considered as an assessment and evaluation criterion. 2.60 1.44 Evaluation of the quality of homework and projects is a criterion to be considered in the assessment and evaluation process. 3.76 1.13 Loading homework and projects to the system on time should be evaluated as an evaluation criterion. 2.98 1.35 Notes taken as a result of evaluating the quality of the responses written in forums should be taken into account in assessment and evaluation process of distance learning. 3.38 1.14 Participating in discussions in the forums on time should be considered as an assessment and evaluation criteria. 1.95 1.39 Acting appropriate in forums should be considered as an evaluation criterion in distance education. 2.55 1.20 process as alternative evaluation options. 2.36 1.14 When the data in Table 2 is examined as a whole, it can be said that the results obtained from alternative evaluation modules that synchronous and asynchronous systems used in distance learning are submitted and the log records can be used to as a criterion to evaluate distance education. However, the participants are not totally agreed on seeing quantitative values such as frequency of participation, or the submission timely as evaluation criteria. In evaluation process (X = 3.35). It is thought that students should be evaluated only according to their exam results(X = 2.58). Also, it can be said that they agree less with the thought that the frequency of participation in online discussions (X = 2.32), participation in discussions on time (X = 1.95) and behaving properly in forums (X = 2.55) should be criteria in distance education. In addition, it is stated that they agree less with the thought that wiki, blog, workshop and etc that LMS provides should be alternative evaluation criteria in participating into modules(X=2.36). But they are 2076 Hasan Karal and Ayca Cebi / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 2073 – 2077 partially agree with the thought that the notes taken after the evaluation of the discussion in forums can be taken account as criteria in assessment process of distance learning (X=3.38). They are partially agree with the thought that participation in synchronous courses (X = 3.00), grades obtained from quiz (X = 2.95), participation in the questionnaires through the process (X = 3.10), material sharing in the system (X = 2.60), and loading homework and projects on the system on time (X = 2.98) should be considered as evaluation criteria. Also they think that the grades for the evaluated homeworks and projects should be taken into account as evaluation criteria (X=3.76). taught in distance education, which criteria do you take into account to do an evaluation?" are analyzed, the data in Table 3 were obtained. Tablo 3. Participant comments on the evaluation criteria in the process of distance education Evaluation criteria in distance learning process N=91 %=100 Exam results 72 79.1 Spent time in synchronous course 35 38.5 Active participation to the course 30 33.0 Homework or projects 28 30.8 Quality of the response in the forums 19 20.9 Spending time in asynchronous system 15 16.5 Numbers of the answers in the forums 5 5.5 Information sharing in environments as wiki, blog etc 3 3.3 When the data in Table 3 is analyzed, it can be seen that exams should be taken account in the evaluation process of distance education as well as it is in traditional education. Preferences are in high level in using synchronous attendance time (log records), level of active participation in courses as well as in traditional education, quality of homework and projects in assessment and evaluation process of distance education. While some of the participants see synchronous participation as a criterion, the others think that the time spent for engaging with course materials in an asynchronous environment is a criterion too. In addition, it is seen that the participants prefer the evaluation of the quality of the responses on the forums to the number of the responses. Moreover another small group think that 4. Conclusion It is stated that traditional evaluation methods can be used in distance education too, assessment and evaluation assignment, blog and forum can be thought as evaluation tools. Quantitative values (frequency of use, response time, etc. these modules. Qualitative assessment is perceived as more important than qualitative assessment. This result is similar to the results that Hara et al (2000) and Yorke et al (2006) obtained before. It is thought that security weaknesses in online exams can be solved by giving more importance to the evaluation process. Howland and Moore (2002) states that students dislike forum discussions. On t opinions on considering forum responses as an criterion in assessment and evaluation process contrast with Howland and Moore's (2002) study. This difference is thought to be due the sample group being different. In distance education, while students are evaluated, in addition to the exams, the log records, analysis of student behavior, participation in the discussions on forums, material and information sharing, sending homework and projects on time and properly, active participation to the synchronous courses should be considered as other criteria. 2077 Hasan Karal and Ayca Cebi / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 2073 – 2077 References Altan, T., & 3.International Computer & Instructional Technologies Symposium (ICITS 2009). 7-9 October 2009, Karadeniz Technical University Fatih Faculty of Education, Trabzon. Cotton, D. & Yorke, J. (2006) Analysing online discussions: What are students learning?in Who's learning? Whose Technology? (Editors: Markauskaite L, Goodyear P & Reimann P), Sydney University Press. Pages 163-171. Dabbagh, N. (2000). Online Discussion Protocols and Rubrics. (visited Oct. 3, 2011) http://mason.gmu.edu/~ndabbagh/wblg/online-protocol.html Fleming, D. L. (2008). Using Best Practices in Online Discussion and Assessment to Enhance Collaborative Learning. In College Teaching Methods & Styles Journal, 4(10), 21-39. Fraenkel, J.R., & Wallen, N.E. (2006). How to design and evaluate research in education. New York: McGraw-Hill. Hara, N., Bonk, C.J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28, 115-152. Howland, Jane L. & Joi L. Moore (2002). Student Perception as Distance Learners in Internet-Based Courses. Distance Education, 23 (2), 183- 195. Ingram, A. L. (1999). Using Web Server Logs in Evaluating Instuctional Web Sites. Journal of EducationTechnology Systems, Volume: 28. , 1(3). URL:http://www.etad.net/dergi/index.php?journal=etad&page=article&op=view&path%5B%5D=17 . E- - KKM, 30 Nisan 2003, Ankara. Shuey, S. (2002). Assessing Online Learning in Higher Education. Journal of Instruction Delivery Systems, 16(2),13-18. Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance: Foundations of distance education (2nd ed.). Upper Saddle River, NJ: Merrill Prentice Hall. Watson W. R. & Watson S. L. (2007). An Argument for Clarity: what are learning management systems, what are they not, and what should they become?. TechTrends, 51(2), 28-34.