Microsoft Word - Kirsi_et_al_CR.doc Interactive Visualization Tools to Improve Learning and Teaching in Online Learning Environments Kirsi Kuosa1, Damiano Distante2, Anne Tervakari1, Luigi Cerulo3, Alejandro Fernandez4, Juho Koro1, Meri Kailanto1 1Tampere University of Technology, Tampere (Finland) 2Unitelma Sapienza University, Rome (Italy) 3University of Sannio, Benevento (Italy) 4LIFIA, CIC/FI, National University of La Plata, La Plata (Argentina) {kirsi.kuosa, anne.tervakari, juho.koro, meri.kailanto}@tut.fi damiano.distante@unitelma.it lcerulo@unisannio.it alejandro.fernandez@lifia.info.unlp.edu.ar ABSTRACT This paper presents two interactive visualization tools for learning management systems (LMS) in order to improve learning and teaching in online courses. The first tool was developed at the Intelligent Information Systems Laboratory (IISLab) at the Tampere University of Technology (TUT). The tool is used to analyse students’ activity from automatically recorded user log data and to build interactive visualizations. They provide valuable insights into the learning process and participation of students in a course offered to teachers and students. The second tool was developed at the Unitelma Sapienza University. It extends navigation and search functionalities in the discussion forum of an LMS with a topic-driven paradigm. The tool analyses forum con- tent and automatically identifies discussion topics. It then enhances the original forum with a topic-driven navigation structure and an interactive search graph. Both tools have been devel- oped as plug-ins for the Moodle LMS, but their analysis processes and techniques can be adopted into any LMS. Keywords: e-learning, learning analytics, information retrieval, data mining, information visuali- zation, visualization tools. INTRODUCTION As online learning spreads and becomes pervasive, the need for tools that monitor students’ par- ticipation in online courses and measure knowledge and skills development has greatly in- creased. Most learning management systems (LMS) and Web-based solutions that are adopted to offer online courses are able to record rich amounts of information about students’ activity, such as communication, collaboration, and participation in online courses. Many LMSs also offer re- ports and even simple graphs about each student's (elementary) actions. However, these reports are poor indicators of students’ activity and participation if they are considered singly (Soller, 2 2001). Exploiting the log data of LMSs in order to understand the activity in an e-learning envi- ronment, how to improve student learning processes, and how to support the process of teaching requires tools and strategies that help to select the right information in a timely fashion and pre- sent it at the right location in a correct, clear and effective way (Tufte, 2001; Tervakari et al., 2014). To achieve this goal, at IISLab1 was developed a visual analytics instrument called the TUT LA tool. The implementation of the tool was for TUT Circle2, an e-learning environment that is built using the Drupal3 content management system (Silius et al., 2011; Tervakari et al., 2013). In this paper, we present the basic principles of an extended version of this tool, which was re- implemented as a plug-in for the widely adopted Moodle4 LMS. Discussion forums are one of the main means of asynchronous communication used in e- learning environments. They are available in all LMSs. Forums are used in online courses to fa- cilitate the learning and teaching processes, such as supporting student-to-teacher interaction, student collaboration, group work and cooperative learning. Discussions held in a forum within a certain period are a potentially useful source of information for any student that subsequently accesses the forum online. However, the success of a forum as an information source and means of communication is determined by the richness of its content. Moreover, its content (discussions and messages) can be navigated and searched. While the first property depends on user participa- tion and use of the medium, the second depends on the navigation structure and the search fea- tures provided by the forum. To support the discovery and retrieval of relevant information in discussion forums in the e- learning environment of distance-learning through the Unitelma Sapienza University, we defined a data analysis process that exploits information retrieval techniques. It employs topic models (Blei, 2011) and formal concept analysis (Ganter and Wille, 1999) to identify the topics dis- cussed in a given forum and to provide topic-driven searches and navigation (Cerulo and Dis- tante, 2013)(Distante et al., 2014). In this paper, we briefly describe these processes and describe the implementations of the plug-ins for the Moodle LMS. They provide interactive visualizations that help both students and teachers find the information they need and monitor students’ learning and actions in the learning environment. RELATED WORK Recently, on-line education systems have become widespread tools adopted by both historical and newly founded educational institutions. E-learning and e-teaching are new contexts for edu- cation through which large amounts of information are generated and available to users world- wide. A large portion of the available information takes the form of free text without the struc- ture required for automated knowledge retrieval. Learning analytics provides valuable support for teachers and students to understand learning habits and to obtain rich information about learning and teaching processes. Learning analytics is 1 http://www.tut.fi/en/about-tut/departments/mathematics/research/iislab/index.htm (last accessed on 1 October 2014). 2 TUT Circle is a social media-enhanced learning environment developed by the IISLab of the Tampere University of Technolo- gy. TUT Circle promotes students’ networking, collaboration and communications in small groups by allowing them to send private messages, chat, create, contribute to and comment on content, exchange opinions, share resources and create communities for different needs. Students can also control the visibility of their information and allow access to the information either for the members of their small group or for all students. www.tut.fi/piiri 3 www.drupal.org 4 www.moodle.org 3 commonly used to measure, collect, analyse and report user data about learners in order to un- derstand and optimize learning, learning environments and teaching (Siemens and Long, 2011). For example, Carr (2012) reported that a great number of open online courses monitor every us- er’s action, such as pausing a video, increasing feedback speed, responding to quiz questions, revising assignments and participating in forums. The data are then used to analyse student be- haviour and test how people learn. Hence, the teacher can tailor the learning environment to fit each student’s learning style and needs. The lack of quality in collected tracking data represents a major challenge to ensure its effec- tive utilization. Moreover, large amounts of data can be difficult to analyze, interpret and under- stand because processing data requires complex cognitive processes. However, if the data are packed and presented in a correct, clear and effective manner, users can process an incredible amount. When the data presented are complex to perceive and process, adequate visual presenta- tions are mandatory (Berg, 2012). Appropriate visualizations can help to process information, such as by increasing memory and the number of available processing resources, reducing the number of searches for information, enhancing the recognition of patterns, enabling perceptual inference operations and using perceptual attention mechanisms for monitoring (Ware, 2012). A good visualization helps the users understand the data, confirms what the users already know, and provides insights into the data. However, to be understood, the context of the data used should be familiar and interesting to the users. Historically, data mining has been used to extract knowledge from free text (Baeza-Yates and Ribeiro-Neto, 1999). Knowledge extractions from e-learning systems, particularly from user generated data, was introduced in (Castro, Vellido, Nebot and Mugica, 2007b; Hanna, 2004). Tang and McCalla (2005) investigated patterns of system usage in teachers and students’ learn- ing behaviour. Data clustering was suggested to promote group-based collaborative learning and to diagnose students incrementally (Castro et al., 2007a). Information retrieval techniques consider that texts are represented by a vector of terms that are usually weighted by their occurrence in documents. The availability of different media (e.g., image, video, audio etc.) has recently demanded more sophisticated information retrieval tech- niques that are capable of treating new kinds of information (Goodrum, 2000; Foote, 1999; Beel et al., 2009). E-learning forums are usually represented as text. Hence, traditional information retrieval techniques are usually adopted to search for relevant information. Association-rules mining has been widely adopted in e-learning, specifically in recommendation systems (Za ́ıane, 2002; Yang et al., 2010), learning material organization (Tsai et al., 2001), student learning as- sessments (Romero et al., 2005), course adaptation to the student’s behaviour (Hogo, 2010) and evaluation of educational Websites (dos Santos Machado and Becker, 2003). The development of cooperative learning and knowledge sharing inside student groups predominates in recent re- search trends (Jakobsone et al., 2012). In particular, Web technologies take advantage of the op- portunities raised by mixing the social and the semantic Web (Ghenname et al., 2012). Dicheva and Dichev (2006) investigated concept maps and their use of navigation in educational contexts. They proposes a framework and a set of tools for the development of ontology-aware reposito- ries of learning materials. The idea is similar to our topic-driven navigation structure. In our ap- proach, topics are extracted from free text semi-automatically by leveraging information retrieval techniques which are then validated by the user, whereas concepts have to be manually defined by the authors of the learning materials (Dicheva and Dichev, 2006). THE PROPOSED INTERACTIVE VISUALIZATION TOOLS 18 words are by default linked with the AND operator, and the system retrieves a list of all posts containing the keywords used in the search. The second approach was used the TDForum Moodle plug-in. While performing the tasks, we collected the number of items inspected and the time needed to achieve the information search goal. In the Moodle full-text search, the number of inspected items was computed by counting the number of posts examined before finding the expected posts. In the topic-driven navigation approach, the number of inspected items was the sum of two quantities: the number of links fol- lowed to reach the closest topic in the Discussion Topics View of the TDForum plug-in and the number of posts examined before finding the correct posts. The findings showed that in general the number of items inspected using the full-text search was on average higher than the number of items inspected using TDForum. Moreover, the time necessary to obtain the correct answer was on average less in the TDForum. The reason is that in the full-text search more time was spent in choosing the correct search keywords. These results confirmed that the TDForum tool facilitates and improves information searches in the discussion forums on a Moodle platform. CONCLUSIONS AND FURTHER RESEARCH The interactive learning data visualizations provided by the TUT LA tool can offer concrete and practical support for students in their learning activities in online courses. They can help students to monitor and evaluate their performance processes and learning outcomes in order to make strategic adjustments to improve, which could bring real benefits to students. Especially on mas- sive open online courses (MOOCs) that require students to be responsible for their learning, self- regulation and other learning skills. Teachers of online courses can also benefit from visualizations in many ways. For example, the visual analytic tools, such as the TUT LA tool, could help teachers to make decisions regard- ing pedagogical strategies, instructional guidance, actions and interventions that may be used to support student participation and activity. In order to support teachers in obtaining increased practical value by using the visual analytics tool, the controls used to specify data and manipulate views need further development. Visual analytics could also be used to retrieve valuable infor- mation about how students use the course platform, how they study subtopics and how they use learning materials and other resources, such as how much time students spend in watching a vid- eo or reading a chapter of the learning material, in which order students study the learning mate- rials offered, and which parts of the assignments students have completed. This kind of infor- mation can help teachers to evaluate students’ learning performance and monitor their progress during courses. In MOOCs with thousands of students, the TUT LA tool could offer valuable practical support for teachers, such as identifying students at risk of failing or dropping out. The information can also be used to identify topics that students skip and learning materials that they do not use, which may help the teacher to improve the instructional design of the course Effective search and navigation capabilities associated with discussion forums in online cours- es are decisive in making forums a useful information source for students, as well as an addition- al means of communication. Moreover, according to earlier research, students prefer tools and information that can provide them with practical support in analysing and examining textual in- formation, such as messages on discussion forums, in order to discover new information (Silius et al., 2013; Tervakari et al. 2014). 19 In this paper, we presented two tools for use with Moodle LMS: a tool for learning analytics and a tool for enhancing content searches and navigation in a forum. Both tools were designed to support students and teachers in e-learning environments. IISLab at TUT in Finland, Unitelma Sapienza University in Italy, and the University of La Plata in Argentina have started a collaborative research that aims to extend and evaluate the use- fulness of their visual analytics tools and forum enhancement approach by means of a set of con- trolled experiments replicated in different classrooms. We expect the measured advantages to be twofold: i) information is retrieved in less time, and ii) contents stored in on-line forums are managed and tracked in a more effective manner. REFERENCES Baeza-Yates, R. A. and Ribeiro-Neto, B. (1999). Modern Information Retrieval. Addison- Wesley Longman Publishing Co., Inc., Boston, MA, USA. Beel, J., Gipp, B., Stiller, J. (2009). Information Retrieval On Mind Maps - What Could It Be Good For?. 5th International Conference on Collaborative Computing: Networking, Applica- tions and Worksharing (CollaborateCom'09). Washington, DC: IEEE. Berg, M. (2012). Human abilities to perceive understand and manage multidimensional infor- mation with visualizations. Doctoral dissertations 30/2012. Helsinki: Unigrafia Oy. Blei, D. M. (2011). Introduction to probabilistic topic models. Communications of the ACM, 55(4), pp.77-84, ACM Press. Bostock, M., Ogievetsky, V., and Heer, J.. (2011). D³ Data-Driven Documents, IEEE Transac- tions on Visualization and computer graphics. Carr, N. (2012, September 27). The crisis in higher education. MIT Technology Review. http://www.technologyreview.com/featuredstory/429376/the-crisis-in-higher-education/. Last accessed on August 15th, 2014. Castro, F., Nebot, A., and Mugica, F. (2007a). Extraction of logical rules to describe students’ learning behavior. 6th International Conference Web-Based Education - Volume 2, WBED’07, pp. 164–169, Anaheim, CA, USA. ACTA Press. Castro, F., Vellido, A., Nebot, A., and Mugica, F. (2007b). Applying data mining techniques to e-learning problems. In Jain, L., Tedman, R., and Tedman, D., editors, Evolution of Teaching and Learning Paradigms in Intelligent Environment, volume 62 of Studies in Computational In- telligence, pp. 183–221. Springer Berlin Heidelberg. Cerulo, L. and Distante, D., (2013) Topic-driven semi-automatic reorganization of online discus- sion forums: A case study in an e-learning context. In Proc. of the 2013 Global Engineering Ed- ucation Conference (EDUCON), pp. 303-310, IEEE. Dicheva, D. and Dichev, C. (2006). Tm4l: Creating and browsing educational topic maps. British Journal of Educational Technology, 37(3), pp. 391–404. Distante, D., Cerulo, L., Visaggio, C.A., Leone, M. (2014), Enhancing Online Discussion Fo- rums with a Topic-Driven Navigational Paradigm: A Plugin for the Moodle Learning Manage- ment System". In Proc. of the 6th International Conference on Knowledge Discovery and Infor- mation Retrieval (KDIR) 2014, pp. 97-106. Scitipress Co. 20 dos Santos Machado, L. and Becker, K. (2003). Distance education: A web usage mining case study for the evaluation of learning sites. In 2003 IEEE International Conference on Advanced Learning Technologies (ICALT 2003), 9-11 July 2003, Athens, Greece, pp. 360–361. IEEE Computer Society. Foote, J. (1999). An overview of audio information retrieval. Multimedia Systems, 7(1), pp. 2- 10, Springer. Ganter, B., and Wille, R. (1999). Formal concept analysis: mathematical foundations. ISBN 978-3-642-59830-2, Springer. Ghenname, M., Ajhoun, R., Gravier, C., and Subercaze, J. (2012). Combining the semantic and the social web for intelligent learning systems. In Proc. of the 2012 Global Engineering Educa- tion Conference (EDUCON), 2012, pp. 1–6, IEEE. Goodrum, A. A. (2000). Image Information Retrieval: An Overview of Current Research. In- forming Science 3(2). Hanna, M. (2004). Data Mining in the e-Learning Domain. Campus-Wide Information Systems, 21(1), pp. 29–34. ISSN: 1065-0741, Emerald Group Publishing Ltd. Hogo, M. A. (2010). Evaluation of e-learning systems based on fuzzy clustering models and sta- tistical tools. Expert Systems with Applications, 37(10), pp. 6891–6903, Elsevier. Jakobsone, A., Kulmane, V., and Cakula, S. (2012). Structurization of information for group work in an online environment. In Proc. of the 2012 Global Engineering Education Conference (EDUCON), pp. 1 –7, IEEE. Keim, D.A. (2002). Information Visualization and Visual Data Mining, IEEE Transactions on Visualization and computer graphics. 8(1), Jan/Mar 2002, IEEE Computer Society. Kosara, R. and Mackinlay, J. (2013). Storytelling: The Next Step for Visualization. IEEE Com- puter (Special Issue on Cutting-Edge Research in Visualization), 46(5), pp. 44–50. Nielsen, J. (1993). Usability Engineering. Academic Press. Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction design: Beyond human-computer interac- tion. New York: John Wiley & Sons. Rodriguez, C. O. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for massive open online courses. European Journal of Open, Distance and E- Learning. Retrieved from http://www.eric.ed.gov/PDFS/EJ982976.pdf Romero, C., Ventura, S., and Bra, P. D. (2005). Knowledge discovery with genetic programming for providing feedback to courseware authors. User Modeling and User-Adapted Interaction, 14(5), pp. 425–464. Siemens, G. and Long, P. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, vol. 46, no. 5. Retrieved from http://www.educause.edu/ero/article/penetrating-fog-analytics-learning-and-education Silius, K., Tervakari, A-M., and Kailanto, M. (2013). Visualizations of User Data in a Social Media Enhanced Web-based Environment in Higher Education. International Journal of Emerg- ing Technologies in Learning 8(2), pp. 13-19. 21 Silius, K., Tervakari, A-M., Huhtamäki, J., Tebest, T., Marttila, J., Kailanto, M. & Miilumäki, T. (2011). Programming of Hypermedia – Course Implementation in Social Media. 2nd Interna- tional Congress on Computer Applications and Computational Science. Advances in Intelligent and Soft Computing, 2012, Volume 144/2012, 369-376. Springer Berlin / Heidelberg Soller, A. (2001). Supporting Social Interaction in an Intelligent Collaborative Learning System. International Journal of Artificial Intelligence in Education, 12(1), pp. 40 - 62. Sowa, J. F. (1987). Semantic Networks. In Stuart C Shapiro. Encyclopedia of Artificial Intelli- gence. Retrieved 2008-04-29. Tang, T. and McCalla, G. (2005). Smart Recommendation for an Evolving e-Learning System: Architecture and Experiment. International Journal on e-Learning, 4(1), pp. 105–129, AACE, Chesapeake, VA. Tervakari, A-M., Marttila, J.,Kailanto, M., Huhtamäki, J., Koro, J., and Silius, K. (2013). Devel- oping learning analytics for TUT Circle. In: Ley, Tobias; Ruohonen, Mikko; Laanpere, Mart; Tatnall, Arthur (ed.). Open and Social Technologies for Networked Learning. IFIP WG 3.4 In- ternational Conference, OST 2012, Tallinn, Estonia, July 30-August 3, 2012, Revised Selected Papers. IFIP Advances in Information and Communication Technology vol. 395, pp. 101-110. Springer Berlin Heidelber. Tervakari, A.-M., Kuosa, K., Koro, J., Paukkeri, J., and Kailanto, M. (2014). Teacher’s learning analytics tools in social media enhanced learning environment. In Proc. of the International Con- ference of Interactive Collaborative Learning, ICL 2014, 3-6 December 2014, Dubai, United Arab Emirates. In press. Tervakari, A.-M., Silius, K., Koro, J., Paukkeri, J., and Pirttilä, O. (2014). Usefulness of infor- mation visualizations based on educational data. In Proc. of the 2014 IEEE Global Engineering Education Conference, EDUCON, 3-5 April 2014, Istanbul, Turkey. Tsai, C.-J., Tseng, S.-S., and Lin, C.-Y. (2001). A two-phase fuzzy mining and learning algo- rithm for adaptive learning environment. In Proceedings of the International Conference on Computational Science- Part II, ICCS ’01, pp. 429–438, London, UK, UK. Springer-Verlag. Tufte, E. 2001. The Visual Display of Quantitative Information. Second edition. Graphics Pr. ISBN 978-0961392147. Ware, C. (2012). Information visualization: perception for design. Third edition. Boston, Morgan Kaufmann. Yee, K. P., Fisher, D., Dhamija, R., and Hearst, M., (2001). Animated exploration of dynamic graphs with radial layout. In Proc. of the 2001 IEEE Symposium on Information Visualization (INFOVIS), pp. 43-50, IEEE. Yang, Q., Sun, J., Wang, J., and Jin, Z. (2010). Semantic web-based personalized recommenda- tion system of courses knowledge research. In Proc. of the 2010 International Conference on Intelligent Computing and Cognitive Informatics, ICICCI ’10, pp. 214–217, Washington, DC, USA. IEEE Computer Society. Za ́ıane, O. R. (2002). Building a recommender agent for e-learning systems. In Proc of the In- ternational Conference on Computers in Education, ICCE ’02, pp. 55–, Washington, DC, USA. IEEE Computer Society.