Opinion of Teaching Staff in Distance Education Systems, Regarding the Assessment and Evaluation Process Procedia - Social and Behavioral Sciences 46 ( 2012 ) 4661 – 4665 1877-0428 © 2012 Published by Elsevier Ltd. Selection and/or peer review under responsibility of Prof. Dr. Hüseyin Uzunboylu doi: 10.1016/j.sbspro.2012.06.314 WCES 2012 Opinion of teaching staff in distance education systems, regarding the assessment and evaluation process Metin Iskenderoglu a, Tuba Aydogdu Iskenderoglu b*, Mehmet Palanci c a Karadeniz Technical University, Fatih Faculty of Education, Trabzon, 61335, Turkey b Karadeniz Technical University, Fatih Faculty of Education, Trabzon, 61335, Turkey cKaradeniz Technical University, Fatih Faculty of Education, Trabzon, 61335, Turkey Abstract The purpose of this study is to understand the opinion of teaching staff in distance education systems, on the assessment and evaluation process, and to compare their opinion with the one observed within the context of formal education. In this respect, interviews were carried out with 4 teaching staff working in both distance education and formal education frameworks. The participants revealed that exams and assignments were the most frequent means used in assessment and evaluation process in distance education, whereas the most important problems faced in this process is the inability to make observations, and the inevitable comparison with formal education. © 2012 Published by Elsevier Ltd. Keywords: Distance education, assessment, evaluation, 1. Introduction Distance education is defined as an institutional education activity wherein students, teachers, and teaching materials at separate locations are brought together with the help of communication technologies. Another definition would be to take classroom activities in conventional learning-education, to outside the class. Distance education activities have a history of more than 200 years. For instance, in 1728, Boston Gazette published advertisements for shorthand lessons by mail. In Turkey, however, the history of distance education activities began with “Teacher Training Report” presented by Dewey in 1924. As a concept, it began to materialize by 1927 (Alkan, 1997). Currently, distance education system is used in associate degree, undergraduate, and graduate programs at more than 10 universities. The fast-paced development of information technologies led to the implementation of distance education practices distinct from conventional teaching methods. Following such developments closely is the problem of how to evaluate the education and learning outputs (Karal et al. 2010). Crucial to, as in the case of all systems, the institutions that provide distance education as well as to students, is to determine the extent of what the students actually learn in the education process. This process is invaluable for the feedback it provides to both parties (Altan and Seferoğlu, 2009). Such feedback serves not only to the purpose of certifying the achievement of the students through grading, certificates, documents etc., but also enrich the education activities, and review and improve the education process and the assessment tools (Simonson et al., 2003) * Tuba AYDOGDU ISKENDEROGLU Tel.: +90-505 661 16 45 E-mail address: tiskenderoglu@ktu.edu.tr Available online at www.sciencedirect.com © 2012 Published by Elsevier Ltd. Selection and/or peer review under responsibility of Prof. Dr. Hüseyin Uzunboylu Open access under CC BY-NC-ND license. Open access under CC BY-NC-ND license. http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/ 4662 Metin Iskenderoglu et al. / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 4661 – 4665 The flexibility provided by developments in internet technologies was translated in flexibility in assessment and evaluation, and it became possible to launch online education projects. Online assessment activities employed in evaluation processes have both advantages and limitations. Providing to students flexibility in terms of timing, faster gathering of data, reduced costs, and reduced burden on teaching staff are among its most prominent advantages (Dommeyer, Baum, Hanna, and Chapman, 2004; Anderson et al., 2005). On the limitations side of the picture are; the need for computers and internet access to administer exams, lack of security inherent in examination design, the possibility that the students may cheat, the inability to check if the student himself/herself took the exam personally, conditions, obstacles of communication, and lack of information and skills regarding online education (Kerka and Wonacot, 2000; Shuey, 2002; Benson, 2003; certain prerequisites should be met to remove these limitations. The prerequisites they specified include reliability, which is a must for validity of internet-based assessment processes, prevention of cheating of students, making sure that the person who is supposed to take the exam is the one actually takes it, establishment of scope validity i.e. the balance between items-topics , the ability to update exam questions through continuous item analysis, provision of a comfortable environment for the student, and equipment of client computers with uninterrupted power supplies. Furthermore, the importance of active communication with a view to rendering online applications more effective is underlined. (Graham et al., 2000). Assessment and evaluation is considered to be crucial to ascertain the achievement rate regarding the objectives of education and how much the individuals took from the offering, and to provide feedback to the theory (Kara, 2009). The most important factor in achieving this aim lies with the teaching staff responsible with the course teaching staff engaged in distance education, regarding assessment and evaluation process, as well as the solutions they developed for the cases they come across. Accordingly, the aim of the study is to understand the opinion of teaching staff in distance education systems, on the assessment and evaluation process, and to compare their opinion with the one observed within the context of formal education. What is the opinion of teaching staff in distance education systems, on the assessment and evaluation process, and what are the differences/similarities compared with the opinions within the context of formal education? 2. Method Qualitative research is the approach adopted in this study. The method employed in the study is the action research, known as the model wherein the teacher acts as the researcher. Cohen and Monion (1989) define action research as the methods developed to provide practical solutions to problems faced at a particular moment of the education-learning process. The researcher took an active part and lectured in a distance education system. The sample of the study comprises the teaching staffs who teach both in formal education programs and the programs executed in Karadeniz Technical University Distance Education Application and Research Center. Two among the participants (A, B) are lecturers teaching for associate degree programs, while the other two participants (C, D) are faculty members, teaching for undergraduate programs. The participants were requested to describe the assessment and evaluation methods they use within the framework of distance education system, and how they execute the process, and finally, to compare this with its equivalent in formal education. In the study, data was collected through interviews where open-ended questions were asked. The analysis was carried out by classifying responses according to themes and codes. The fundamental purpose of content analysis is 3. Findings 3.1 Which method(s) do you employ in order to measure the achievement of the student in the distance education system? What are the criteria you consider in choosing the method(s)? 4663 Metin Iskenderoglu et al. / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 4661 – 4665 The responses to questions are provided in Table 1 and Table 2. Table 1: Assessment and evaluation methods employed by staff teaching in distance education system Examination Homework Lesson Evaluation Project Confidential Evaluation A X X X X X B X X X X X C X D X X Table 2: Criteria affecting the choice of assessment and evaluation methods employed by staff teaching in distance education system Verbal Lesson Rules of School Applied Course A X X X B X X X C X D X X A glance on Table 1 and Table 2 reveals that lecturers A and B use all assessment and evaluation methods. The participants explain why they use all methods by stating that the courses they teach are verbal-intensive courses focused on application. They also note that they are bound when calculating the grade of the students, with the percentiles specified in KTU Registrar Regulation. Teaching staff C, on the other hand, stated that (s)he applied only exams to measure the success of students, due to course-specific reasons. In the interview teaching staff C communicated his view on the subject as follows: ment, we apply tests as the system would allow it. The final exam on the other hand, comprises a classical exam focused on application. This is not the ideal I have in mind. I want to achieve the objective through assignments, and in particular, assignments replacing the second midterm for grading the achievement and level of the student. However, since I am unable to adapt to the system quite well, collecting the assignments looks like more work than it is worth. The students cannot voice inquires about assignments. Yet in formal education, feedback is a faster process. hours, that would cost me valuable lecturing time, which is not a very att Teaching staff D, on the other hand, uses exams and assignments to evaluate achievement. However, the (s)he stated his/her doubts about the grade reflecting the achievement of the student well. 3.2. What are the assessment and evaluation related problems you face when teaching over distance education system? The responses provided to the question are shown in Table 3. Table 3: The assessment and evaluation related problems faced by teaching staff in the distance education system. Encountered Problems A B C D Poor communication with students X X X Reaching to students X Understand that student understand or not understand the topic X X X X X X X X Student interaction X X Time limitation X X aprelininary information X X Difficulties of following process X Suspense and secrecy X X X X X Attendance-absence X Experience about system X X Range of examinations X Online examination reliance X Comparision of formal education and distance education students X X X Table 3 shows the problems faced by the teaching staff. The problems vary by the contents and characteristics (verbal, practical etc.) of courses. The common problems faced by the teaching staff mostly converge on the lines of 4664 Metin Iskenderoglu et al. / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 4661 – 4665 failure to see whether the students understood the topic taught, inability to make observations, uncertainty and confidentiality 40 students are shown online, I get no answers to my on the 15 which actually answers. On the other hand, in an actual classroom, I get a general idea if they understand or not. Can the student perform the application? I request answers through my questions, which subsequently make a 3.3. What are your solutions for better assessment and evaluation in the distance education system? Responses to the question are presented in Table 4. Table 4: The solutions developed by teaching staff for more efficient assessment and evaluation in the distance education system Developed methods for encountered problems A B C D Peer education X Active using of forums X Mail X Sustain asynchronous part X X X X Creating platform X X X X Blog, facebook X Web site X X Increasing motivation X Creating question bank X Consultancy service X Student-centered education X X X Following development of students X Homework-project X X X In Table 4 various methods used by teaching staff for better assessment and evaluation are summarized. The common method is to ensure participation in the class through feeding a discussion among students, and to or have missed. 3.4 Comparison of distance education with formal education The findings reveal that the teaching staff in distance education programs face various problems during their teaching experience, and develop different solutions to overcome these problems. When assessing a course offered, the teaching staff base their analysis on the assessment developed in formal training. This point is clear in the interviews. Teaching staff B summarizes it as follows: In effect, we use formal education as an index of 100%. We then strive to measure our success in distance education by comparing it against the formal education. We compare distance education applications against formal education. We teaching through the distance educ Teaching staff C also harbors similar views, while teaching staff D argues as follows: assessment tools for distance education, for assessment will never be the same as that in formal education. We know that for sure. We cannot decide if a student is to pass or not, through a single exam. We have to make that clear. Passing or failing through some last 4. Discussions, Conclusions and Suggestions The conclusions of this study put exams and assignments as the most widely used assessment-evaluation methods employed in the distance education system. Yet there are some problems faced in the evaluation process. According to teaching staff, the communication problems in distance education lead to failure of students and 4665 Metin Iskenderoglu et al. / Procedia - Social and Behavioral Sciences 46 ( 2012 ) 4661 – 4665 teachers in getting to know each other, and also to different perceptions of mutual expectations (about assignments, projects etc.) with respect to evaluation and assessment. This point supports the findings of Baturay and Bay (2009). The distant locations of students during the assessment activities applied in distance education, and lack of al. (2009) supports the finding that students find it easier to cheat in online education. Another problem with respect to assessment and evaluation process is the lack of any authority to provide clarification in case some matters regarding exam questions cannot be understood clearly and fully. The participants in the study recommend strengthening of the asynchronous part of the system, the provision of a discussion environment and feeding a discussion environment in order to preclude such problems. The environments where the students will take the exam should be conducive to online examinations. The selection of the most applicable one among assessment and evaluation methods within the framework of distance education, and rendering the process more effective may help in determining the learning rates of the students (Karal et al., 2010). It is also necessary to train teaching staff to take part in distance education, in order to ensure that they use the system more effectively. The system should be enriched with course related documents, lecture notes, and content. The education materials in the system should be understandable by the students, and should be compliant with standards. The students must be provided advisory services. The advisory services would also help accumulated detailed information about students, which would in turn help in better functioning of the system, and foreseeing as well as preventing any deficiencies in advance. When evaluating the achievement outputs of students, the student-oriented grading system should be employed to support the students in improving themselves on dimensions they are not up to par in. Furthermore, the courses should be grouped in accordance with their contents. This may help preventing the imposition of uniform teaching and evaluation for all courses. References Anderson, H. M., Cain , J., & Bird, E. (2005). Online Student Course Evaluations: Review of Literature and a Pilot Study. American Journal of Pharmaceutical Education, 69(1), 34-43. That Chose Distance Education). Journal of Ziya Gokalp Faculty of Education, 7(13), 17-26. Benson, A. (2003). Assessing participant learning in online environments. New Directions for Adult and Continuing Education, 100, 69 -78. Dommeyer, C.J., Baum, P., Hanna, R. W., & Chapman, K. S. (2004). Gathering faculty teaching evaluations by inclass and online surveys. Assessment & Evaluation in Higher Education, 29(4), 611-623. Graham, C., Cagiltay, K., Craner, J., Lim, B., & Duffy, T. M. (2000). Teaching in Web Based Distance Learning Environment: an Evaluation Summary Based on Four Courses. Center for Research on Learning and Technology Technical Report No.13. Indiana University, Bloomington. (2010). . WCLTA 2010, 1597-1601 Kerka, S., & Wonacot, M. (2000). Practitioner file: Assessing learners online. Columbus, OH: Clearinghouse for Adult, Career, and Vocational Education. Accessed May 16, 2009 from:http://ericacve.org/fulltext.asp McCombs, B., & Vakili, D. (2005). A learner-centered framework for e-learning. Teachers College Record. 107(8), 1582-1600. Shuey, S. (2002). Assessing Online Learning in Higher Education. Journal of Instruction Delivery Systems, 16(2),13-18. Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2003). Teaching and learning at a distance: Foundations of distance education (2nd ed.). Upper Saddle River, NJ: Merrill Prentice Hall. The opinions of students and teaching staff about online education implementations. Accessed May 22, 2009 from: http://www.tebd.gazi.edu.tr/arsiv/2005_cilt3/sayi_2/197-216.pdf Yavuz, C. & www.ilet.gazi.edu.tr/dergi/30 225-241 in social sciences). Ankara: