key: cord-0629120-xe1jdi0m authors: Aryal, Himali title: A literature survey on student feedback assessment tools and their usage in sentiment analysis date: 2021-09-09 journal: nan DOI: nan sha: 3f3f40c3455d490cf1430a1085f24233f3c35304 doc_id: 629120 cord_uid: xe1jdi0m Online learning is becoming increasingly popular, whether for convenience, to accommodate work hours, or simply to have the freedom to study from anywhere. Especially, during the Covid-19 pandemic, it has become the only viable option for learning. The effectiveness of teaching various hard-core programming courses with a mix of theoretical content is determined by the student interaction and responses. In contrast to a digital lecture through Zoom or Teams, a lecturer may rapidly acquire such responses from students' facial expressions, behavior, and attitude in a physical session, even if the listener is largely idle and non-interactive. However, student assessment in virtual learning is a challenging task. Despite the challenges, different technologies are progressively being integrated into teaching environments to boost student engagement and motivation. In this paper, we evaluate the effectiveness of various in-class feedback assessment methods such as Kahoot!, Mentimeter, Padlet, and polling to assist a lecturer in obtaining real-time feedback from students throughout a session and adapting the teaching style accordingly. Furthermore, some of the topics covered by student suggestions include tutor suggestions, enhancing teaching style, course content, and other subjects. Any input gives the instructor valuable insight into how to improve the student's learning experience, however, manually going through all of the qualitative comments and extracting the ideas is tedious. Thus, in this paper, we propose a sentiment analysis model for extracting the explicit suggestions from the students' qualitative feedback comments. Online education has gained in popularity over the years due to various aspects such as accessibility in form of massive open online courses [1] , affordability and flexibility as result of distance and blended education [2] [3] [4] , frameworks and tools [5] [6] [7] [8] , interactivity [9] , learning pedagogy and life-long learning [10] . It has proven to be a most effective solutions for learning, especially during the COVID-19 pandemic. According to World Health Organization (WHO) data, COVID-19 has been recorded in over 216 countries, with millions of confirmed cases [11] . Many countries have implemented precautionary measures, such as school and university lockdowns [12] . As a result, educational institutions began to provide the majority of their services online, including lectures and various examinations, to over 60% of students throughout the world via various platforms [9, 13] . Online learning environments represent a distinct way of interaction. It entails the development of educational materials, delivery of instruction, and program management using the internet and other essential technologies [14] . During online learning, it is critical to maintain student interaction to curb dropouts [15] [16] [17] , and react to their responses while also obtaining feedback to/from them. Feedback and assessment are used to track student progress, manage learning pace, and assess instructional methods [18] . There are two kinds of student evaluations: summative and formative assessment. Summative assessment is a method of attempting to summarize student learning at a specific point in time, such as the end of a course. The arXiv:2109.07904v1 [cs.CY] 9 Sep 2021 majority of standardized examinations are in the form of a summative assessment. They aren't designed to give teachers and students quick, contextualized feedback during the learning process. Formative assessment, on the other hand, occurs when teachers give students feedback in ways that help them learn more effectively, or when students can engage in a comparable, self-reflective process [19] . If the primary purpose of assessment is to support highquality learning, then formative assessment ought to be understood as the most important assessment practice [20] . In a typical classroom, different characteristics such as students' faces and gestures assist the teacher in evaluating the behaviour [21] and sentiment of the students even though they are idle. However, formative assessment in virtual learning is a challenging task. Despite the challenges, different technologies are progressively being integrated into teaching environments to boost student engagement and motivation [22, 23] . Different student response systems have been utilized to improve student participation and real-time responses, including Mentimeter, Plickers, GoSoapBox, and Poll Everywhere [24] . In addition, in recent years, the usage of game-based applications such as Kahoot! in the classroom has grown in popularity [25] . Although the concept is not new, its application in the classroom in the context of play or competition has a direct impact on various aspects such as classroom participation and encouragement to the development of the activity, perception of the learning, commitment to the subject, and interest in deepening theoretical concepts [26] . Furthermore, student responses are also important for improving pedagogic and assessing students' learning behavior, and to provide personalized services and content via recommender systems [27] . It helps to improve the interaction between teachers and students, and frequent input from students aids in the development of a better learning environment [18] . However, in most cases, feedback is provided after the course or a lecture in e-learning, which might be too late to reconsider. Also, feedback can come from a different source, and it has to be analyzed to understand the opinion of students. Thus, in this literature based research paper we would focus on the effectiveness of different feedback assessment tools and how the responses from these tools can be utilized in the sentiment analysis to identify the student's attitude towards the course or teaching approach. • How many papers were published between 2015 to 2021 that have used digital tools to collect feedback and/or to perform quizzes? • What publications have these papers published in? • Which tool has been explored more? • When did the majority of the research take place? • What are the most widely used evaluation metrics for feedback technologies that have been studied? • How effective it is to adapt lectures and teaching style on the go during a lecture-based on students' feedback? • How sentiment analysis can be utilized in the responses collected from feedback assessment tools to identify the student's attitude towards the course or teaching approach? There are currently a plethora of game-based learning tools available that use variations on game aspects to inspire, provide feedback, and structure participation pathways [28] . In this study, only the state-of-the-art of tools like Kahoot!, Mentimeter, and Padlet were explored. The bulk of research studies [26, [28] [29] [30] [31] [32] [33] [34] [35] [36] have looked at various aspects of these technologies, such as their effects on engagement, learning, classroom dynamics, concentration, motivation, and enjoyment, as well as how well they have been used in education. [37] Concluded that game-based learning has a positive impact on student performance. The students enjoy the lectures that include a quiz or survey in the middle. The studies [26, 28, 30, 34, 35, [37] [38] [39] also demonstrated that there are numerous chances of losing concentration in e-learning, and these technologies aided to nurture concentration and increase engagement. Additional focus of researches [37, 40, 41] were on how students and teachers perceived the use of these tools in education. The students' opinions of using these technologies in the classroom are also positive. According to the research [40] , students noticed that these technologies helped them pay attention in class, and that Kahoot! or Mentimeter quizzes were particularly engaging. It has the ability to boost student attendance and has been regarded as interactive and quick. They also stated that the anonymity aspect allowed all students, including those who lacked confidence, to participate. Furthermore, some studies [29, 34] compared the performance of students who utilized game-based learning to those who used followed traditional approaches. Their performance has greatly improved as a result of the findings. Students were divided into groups and taught using traditional methods such as PowerPoint presentations and encouraging students to ask questions and interact verbally, while others were taught using game-based learning, in which students played through a series of questions related to the topic while the teacher provided explanations in between. The evaluation [32] found that students who had the opportunity to learn in a game-based learning environment learned more than those who were taught using traditional methods. Another point of emphasis was the course teachers' real-time comments to the students. The [31, 35, 36, [38] [39] [40] studies looked at how the technologies studied could be used to interact with students and deliver immediate feedback based on their performance on a quiz or another activity. Only a few research, however, delved deeper into the analysis of feedback using machine learning techniques in order to use it to improve teaching style or other aspects of education. In the research [42] text pre-processing techniques were utilized to analyze student feedback obtained through various ways such as online polls and OMR sheets. According to the author, this experimental study of sentiment trees and feedback rating results in accurate polarities such as positive, negative, or neutral. Furthermore, the author stated that this technique would aid in determining better feedback findings, which faculty might use to improve their teaching process. The table 1 outlines the state-of-the-art of a few feedback assessment tools as well as summarizes the context in which these tools are assessed and aspects of which they have been studied. Sentiment analysis on student feedback has been an active research topic in the past years. The majority of the data utilized for the analysis comes from online courses, with only a few from the student feedback system. In this section we will highlight some of the research papers that uses student feedback in sentiment analysis. A survey table is also included at the end of the section, demonstrating the effectiveness of various feedback assessment tools. The research work led by Z. Kastrati et al. [45] highlighted a systematic mapping of sentiment analysis on student feedback using NLP and Deep Learning. The research was conducted on 92 articles out of 612 that were found in the subject matter in learning platform environments. The PRISMA framework was utilized to guide the search procedure, which included only articles published between 2015 and 2020. The research identified that sentiment analysis on student's feedback using NLP and Deep Learning is a rapidly growing field, however, in order to fully mature the research and development in the field, structured datasets, standardized solutions, and work concentrated on emotional expression are essential. Another study [46] presents a comparison of performances of 4 machine learning and natural language processing techniques: Naive Bayes, Maximum Entropy, Long Short-Term Memory, Bi-Directional Long Short-Term Memory for SA. According to the research, the Bi-Directional Long Short-Term Memory algorithm outperformed in terms of the F1-score measurement with 92.0% on the sentiment classification task and 89.6% on the topic classification task. In addition, a sentiment analysis application was developed that analyses student feedback and provides overview reports for the administrator to identify and recognize student interests in the institution. The research study conducted in [47] investigated on aspect-level sentiment analysis to automatically identifies sentiment or opinion polarity of the student reviews. The weakly supervised tool was applied to effectively identify the aspect categories discussed in the unlabeled students' reviews. In addition, the results show that the framework performs admirably in terms of both aspect category identification and sentiment categorization. Furthermore, it is intended that the proposed framework would replace labor-intensive sentiment analysis techniques that heavily rely on manually labeled data. A similar research work is also conducted in [48] where various traditional machine learning algorithms and deep neural networks are trained on real-world dataset composed of more than 21 thousands students' reviews manually classified into 5 different aspect categories and 3 polarity classes. Further improvements were made by applying deep learning models in [49] . These models, however, relies only on the basic NLP techniques to process data for finding sentiments. To better capture domain specific sentiments for educational feedback, there is a need to incorporate both semantic and contextual information form users input. Such a model could utilize domain-specific ontology [50] [51] [52] [53] , and publicly available lexical resources such as SenticNet, SentiWordNet, etc., [54] . In the research [42] text pre-processing techniques were utilized to analyze student feedback obtained through various ways such as online polls and OMR sheets. According to the author, this experimental study of sentiment trees and feedback rating results in accurate polarities such as positive, negative, or neutral. Furthermore, the author stated that this technique would aid in determining better feedback findings, which faculty might use to improve their teaching process. Padlet Evaluated Padlet as a repository which stores resources related to classroom activities and students' work-in-progress Improved accessibility to teacher's input and peers' work-in-progress [32] K. P. Nuci et al. In-lecture quizzes in online classes Significant increase in students' engagement, interaction, and interest in lectures [26] T. Parra-Santos et al. Kahoot! Quizzes, and Questionnaire Students' attention, motivation and enjoyment are increased [28] A. I. Wang We used systematic mapping as a research methodology to review the literature for this research. To conduct this literature review, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [55] . The PRISMA method of systematic review enables a comprehensive review of all relevant articles and aids in the discovery of a clear response to the research question. The approach uses a variety of inclusion and exclusion criteria to help with decision-making. For this study, we followed a desk work methodology where scientific papers that contained information feedback assessment tools were searched and gathered. The keywords search approach was used to locate these resources. Keywords like 'Kahoot!', 'Mentimeter', 'Padlet', 'Feedback Assessment Tools, 'Game-Based approach' were used to search these resources. Following the initial research, we limited our search only to a few feedback assessment tools ('Kahoot!, 'Mentimeter, 'Padlet') and focused our survey on which context of these has been studied and which aspect i.e. information about the themes/topics for which students have commented on these tools are in the learning environment and whether they are beneficial to employ in the future. The table 2 includes the query string used for searching the articles. Search String (Query) Kahoot!, "Kahoot!" AND Feedback, "Kahoot!" AND "Feedback" AND "Sentiment" Mentimeter, "Mentimeter" AND "Feedback", "Mentimeter" AND "Feedback" AND "Sentiment" Padlet, "Padlet" AND "Feedback", "Padlet" AND "Feedback" AND "Sentiment" The study incorporates the articles published between 2015 and 2021. Since the study was conducted in 2021, it encompassed papers published up until July of that year. We used the following online research databases and engines to conduct our search: • ACM Digital Library The search began with a simple query, which we narrowed down sequentially. In the first stage, using the queries string 'Kahoot!', 'Mentimeter', and 'Padlet', a total of 1638 articles published in the English language were gathered. The figure 2 depicts the statistics of all the articles acquired initially in the various digital libraries. The papers were screened using a number of inclusive criteria. Articles published only in English, papers released between 2015 and 2021, and prioritized peer-reviewed articles were among the inclusion criteria utilized for the filtering. The table 3 displays annual data for articles published on various feedback tools on different database sources. The search is narrowed down to specific sorts of articles, such as journal articles and conference papers. The research was also restricted to the subject of computer science. A total of 154 articles were identified to meet the eligibility requirements using the following inclusion criteria. • Type of publication • Language (English) The table 4 shows the final 154 articles obtained with the above mentioned eligibility criteria. On the query string "sentiment AND feedback AND (mentimeter OR Kahoot! OR Padlet)" a few articles were retrieved; however, upon closer inspection, it was discovered that sentiment analysis was not done on the feedback received from the tools (Kahoot!, Mentimeter, and Padlet). The answers to the research questions are included in this section. How many papers were published between 2015 to 2021 that have used digital tools to collect feedback and/or to perform quizzes? Since the development of Kahoot!, it has become a popular research topic in the field of education, with a large number of articles published since then. In the Mentimeter and Padlet, it's the same. Without taking into account varied inclusion and exclusion criteria, we found a total of 1638 articles in these three tools. Whereas, considering different inclusion/exclusion criteria, 154 articles were found in journals and conferences in the field of Computer Science between 2015 to 2021. These 154 articles were published either on Kahoot! or Mentimeter or Padlet in the context of feedback assessment or feedback analysis. When did the majority of the research take place? The total number of publications published in the year 2021 was the highest, meeting the inclusion criteria and eligibility criteria as shown in the table 5. After 2018, the number of articles published each year is higher than before. This increased number of articles in the recent years could be due to the COVID-19 outbreak. Because of the Covid-19 pandemic, educational institutions have been obliged to shift to an online teaching method all around the world [12] . As a result, an increased study on the topic of e-learning and various online feedback assessment systems was observed. Which tool has been explored more? We looked into Kahoot!, Mentimeter, and Padlet, which are three different feedback assessment tools. According to the data, Kahoot! is the most extensively used tool. We discovered 103 articles solely on the Kahoot! tools with considering the inclusion/eligibility criteria. Whereas 29 studies focused on Padlet feedback analysis and 20 publications on mentimeter were identified. According to our observations, Kahoot! is the most popular tool among the technologies considered for feedback assessment in the learning environment. What are the most widely used evaluation metrics for feedback technologies that have been studied? The 154 articles for the metadata analysis were chosen once the final inclusion and exclusion criteria were determined. Upon closer inspection, we discovered that studies had been undertaken in the following areas: • Student perception • Collaborative knowledge building • Improve student learning motivation and performance How effective it is to adapt lectures and teaching style on the go during a lecture-based on students' feedback? The implementation of a feedback system in the lecture is seen positively by both students and teachers [24, 40, 41, 43] . The perspectives and experiences of others who have used these tools in the classroom offer excellent insight all the while diversifying the learning process [43] . According to the results of the survey, several in-class feedback assessment tools such as Kahoot!, Mentimeter, Padlet are quite effective. However, we were unable to locate any research that focused on sentiment analysis on data collected from feedback tools. We can't deny that sentiment analysis research on student review isn't new; nonetheless, the most of studies have focused on data acquired from Coursera reviews [47] , social media pages [56] , or other online courses platforms [57] . The research on the sentiment analysis based on the feedback assessment tools especially Kahoot!, Mentimeter and Padlet is mostly unexplored. One approach to determine the polarity of student comments received via feedback assessment tools (Kahoot!, Mentimeter, and Padlet) is using sentiment analysis [49] . The methodology of Student Feedback Sentiment Analysis is depicted in the flowchart as shown in the figure 3. The depicted model is inspired from [58, 59] . We will focus on the following aspects when conducting sentiment analysis: Figure 3 : Proposed Sentiment Analysis Model 1. Data Collection: Data collecting is an important phase in the Sentiment Analysis. The more data there is, the better the result. In a course during the Fall semester of 2021, testing will be conducted for a short-list of tools such as Kahoot!, Mentimeter, and Padlet. 2. Data pre-processing: Data consist of the mixture of text, numbers, and special characters. Data preprocessing is required before further data processing, including converting text to lowercase, reducing stop words, removing unnecessary punctuation, locating exclamation marks or question marks, and removing inconsistent letter casing. 3. Feature Extraction and Conceptualization of Sentiment: We can better interpret the text by employing feature extraction, which will improve the accuracy of training the data using a new method. Term Presence and Frequency, Part of Speech Tagging, and Negation are some of the features that can be used. Also incorporating the semantic context using publicly available lexical databases (i.e WordNet, SentiWordNet, SenticNet, etc.) [54] or semantically rich representations using ontologies [60, 61] and their thesaurus [62, 63] to identify opinion and attitude of users from text would be an import aspect to further investigate. Training Model with different classifiers: Following feature extraction, we can use a number of existing algorithms to model our data, including Naive Bayes, Support Vector Machines, CNN, and others. The algorithm depends on the features of the data and there is no said algorithm that is perfect for any data. To identify the optimal model for the analysis, we'll run the data through a variety of features and algorithms. Due to the increasing demand of online education and distant learning as a result of the COVID19 epidemic, students' feedback analysis is the most vital task for professors as well as educational institutes. Hence implementing various feedback assessment tool is essentials. Although a lot of study has been done on the effectiveness of feedback assessment tools, there is still work to be done on sentiment analysis on data received from feedback assessment tools (Kahoot!, Mentimeter, and Padlet). This article focused on the current state of research on feedback assessment tools, as well as the contexts in which these tools have been investigated and the status of sentiment analysis. The findings imply that feedback evaluation tools are now vital in the field of education, and that sentiment analysis on data acquired from these feedback assessment tools is rarely investigated. Towards understanding the MOOC trend: pedagogical challenges and business opportunities Interactive media learning object in distance and blended education Implications of online and blended learning for the conceptual development and practice of distance education The effect of a flipped classroom in a spoc: Students' perceptions and attitudes Multimedia learning objects framework for e-learning An e-learning theoretical framework Automatic annotation of lecture videos for multimedia driven pedagogical platforms A conceptual elearning framework HIP-a technology-rich and interactive multimedia pedagogical platform An overview of online education: Attractiveness, benefits, challenges, concerns and recommendations 2019-12) Who coronavirus disease (covid-19) dashboard. Accessed on 2021-08-09 Advantages, limitations and recommendations for online learning during COVID-19 pandemic era A framework for institutional adoption and implementation of blended learning in higher education Virtual learning environments MOOC dropout prediction using machine learning techniques: Review and research challenges Early dropout prediction using data mining: a case study with high school students Predicting student dropout in a MOOC: An evaluation of a deep neural network model Formative assessment and self-regulated learning: A model and seven principles of good feedback practice Assessment -summative and formative -some theoretical reflections The best value in formative assessment User behaviour analysis on LMS and MOOC Evaluating learners' emotional states by monitoring brain waves for comparing game-based learning approach to pen-and-paper An analysis of learner experience with MOOCs in mobile and desktop learning environment The effect of mentimeter and kahoot applications on university students' e-learning Digital game-based learning and serious games in education Gamification for formative assessment in the framework of engineering learning A systematic mapping review on MOOC recommender systems The effect of points and audio on concentration, engagement, enjoyment, learning, motivation, and classroom dynamics using kahoot Evaluation of interactive and gamified approaches for teaching ict theory: A study of powerpoint, sembly, and kahoot Effective usage of gamification techniques to boost student engagement Using padlet for project-based learning in documentary filmmaking Game-based digital quiz as a tool for improving students' engagement and learning in online lectures Using the game-based student response tool kahoot! in an online class: Perspectives of online learners Enhancing learning and engagement through gamification of student response systems Using kahoot! in the classroom to create engagement and active learning: A gamebased technology solution for elearning novices Students' perception of kahoot!'s influence on teaching and learning The effect of using kahoot! for learning-a literature review Effectiveness of kahoot as a revision tool in studying waves Mentimeter app in computer science courses: Integration model and students' reception Using mentimeter to enhance learning and teaching in a large class Perceptions of students for gamification approach: Kahoot as a case study Analysis of student feedback by ranking the polarities The views of the teacher candidates on the use of kahoot as a gaming tool Cheat sheets and padlet: A metacognitive learning tool Sentiment analysis of students' feedback with NLP and deep learning: A systematic mapping study Deep learning versus traditional classifiers on vietnamese students' feedback corpus Weakly supervised framework for aspect-based sentiment analysis on students' reviews of MOOCs Aspect-based opinion mining of students' reviews on online courses The potential of machine learning algorithms for sentiment classification of students' feedback on MOOC SEMCON: semantic and contextual objective metric SEMCON: a semantic and contextual objective metric for enriching domain ontology concepts Domain-specific sentiment analysis using contextual feature generation An improved concept vector space model for ontology based classification SENTIWORDNET: A publicly available lexical resource for opinion mining European Language Resources Association (ELRA) Preferred reporting items for systematic reviews and meta-analyses: the prisma statement Sentiment analysis of students feedback: A study towards optimal tools WET: Word embedding-topic distribution vectors for MOOC video lectures dataset Cross-cultural polarity and emotion detection using sentiment analysis and deep learning on COVID-19 related tweets Cross-cultural polarity and emotion detection using sentiment analysis and deep learning-a case study on COVID-19 The impact of deep learning on document classification using semantically rich representations Ontology-based sentiment analysis process for social media content Performance analysis of machine learning classifiers on improved concept vector space models Multilevel sentiment analysis using domain thesaurus