key: cord-0996903-6qx9j4bj
authors: Celik, Ismail; Gedrimiene, Egle; Silvola, Anni; Muukkonen, Hanni
title: Response of learning analytics to the online education challenges during pandemic: Opportunities and key examples in higher education
date: 2022-03-12
journal: Policy Futures in Education
DOI: 10.1177/14782103221078401
sha: 6c3ef52e1fc30a388b07f533c3355d2ce46282e1
doc_id: 996903
cord_uid: 6qx9j4bj

Emerging technological advancements can play an essential role in overcoming challenges caused by the COVID-19 pandemic. As a promising educational technology field, Learning Analytics (LA) tools or systems can offer solutions to COVID-19 pandemic-related needs, obstacles, and expectations in higher education. In the current study, we systematically reviewed 20 papers to better understand the responses of LA tools to the online learning challenges that higher education students, instructors, and institutions faced during the pandemic. In addition, we attempted to provide key cases in which LA has been effectively deployed for various purposes during the pandemic in the higher education context. We found out several prominent challenges for stakeholders. Accordingly, learners needed of timely support and interaction, and experienced difficulty of time management. Instructors lacked pedagogical knowledge for online teaching. In particular, individual and collaborative assessment have been a challenge for them. Institutions have not been ready for a digital transformation and online teaching. In response to these challenges, LA tools have been deployed for the following opportunities: monitoring, planning online learning process, fostering learners’ engagement and motivation, facilitating assessment process; increasing interaction, improving retention, being easy to use. Understanding these promises can also give insight into future higher education policies.

The ongoing COVID-19 pandemic has dramatically disrupted higher education institutions all over the world. Therefore, as a response to the rapid spread of the pandemic, higher education institutions have suspended face-to-face education temporarily (Bao, 2020; Usher et al., 2021) . For instance, at the end of April 2020, higher education institutions from approximately 180 countries closed their campuses (Marinoni et al., 2020) . After the closure of the campus, the institutions had swiftly changed their instruction to online education. This is most probably because online education has been regarded as the safest and most responsible approach during the pandemic for keeping social isolation (Openo, 2020) .

However, the rapid transition to online education has led to many challenges for all stakeholders including the institutions, teachers, and learners (Chetty et al., 2020; Hodges et al., 2020) . For example, the institutions had to update and broaden their servers to meet the mass deployment of online learning environments (Balderas and Caballero-Hernández, 2020) . In addition to these technical issues, the institutions had to struggle in providing the instructors with pedagogical support for online teaching. In fact, many instructors conceived online teaching and learning as "painful, worrisome, and anxiety-inducing" during the pandemic (Kimmons et al., 2020, p.2) . For instance, the instructors failed to effectively monitor students' behavioral, emotional, and cognitive aspects of learning because of the lack of an effectively designed system (Van der Spoel et al., 2020) . Also, instructors were unable to assess and evaluate the learners due to students' cheating behavior, which is hard to detect in online learning environments (Balderas and Caballero-Hernández, 2020) . From the student perspective, the concern regarding cheating behavior was leading instructors to take immensely demanding exams (Lancaster and Cotarlan, 2021) . It has been evidenced that higher education students also have dissatisfied with and challenged the online learning process (Ludlow, 2020). One possible reason for the dissatisfaction is regarded as non-stop online lectures throughout all day, causing Zoom fatigue in the students (Bonk, 2020) . Therefore, pure online lectures that are far from meeting the psychological well-being, needs, and expectations of students may escalate the drop-out rate during pandemic (Baber, 2020; Diver and Martinez, 2015; Teuber et al., 2021) . Furthermore, the pandemic process increased the inequality among the students. In this regard, Chetty et al. (2020) found out students with high socio-economic status outperform those of low socio-economic status during the pandemic. This is because students from low-income areas may lack the necessary technical equipment such as the Internet or personal computer; these students may receive less support from their parents .

Emerging technological advancements can play an essential role in overcoming challenges caused by the pandemic (Richter, 2020) . Indeed, this period might be a unique opportunity to better understand the implication of technological solutions in learning and teaching (Dwivedi et al., 2020; Sein, 2020) . As an emerging technology field, Learning Analytics (LA) tools or systems can offer solutions to pandemic-related needs, obstacles, and expectations in higher education (O'Leary, 2020) .

For example, dashboards as one of the most common LA tools can provide teachers with important information about their students, including time management, material use, and social interactions (Safsouf et al., 2021) . Hence, LA dashboards can serve as a monitoring tool for students by creating and reporting several visualizations on student progress and learning outcomes during the pandemic (Conde et al., 2020) . Instructors can straightforwardly recognize specific learning/ support needs of students through LA dashboards; ultimately, this provides instructors with possibilities for timely and individualized interventions (He et al., 2021; Molenaar and Knoop van Campen, 2018) . Moreover, with the help of machine learning algorithms, the dashboards can create predictive models based on student real-time data (Dabbebi et al., 2017; Diana et al., 2017) . These models can be specified in the process of the pandemic, considering students' needs and guidance (Safsouf et al., 2021) . In addition to dashboards, LA tools might be utilized by instructors, for measuring and monitoring students' individual and group performance and for assessing progress (Conde et al., 2020) . Also, some LA tools have been developed for the detection of students' cheating behavior during online exams (Balderas and Caballero-Hernández, 2020) .

In order to better support learners with current and emerging technologies, there is a need to understand how LA has been utilized for mitigating pandemic-related challenges. In the current study, we systematically reviewed a selection of the papers to better understand the responses of LA tools to the online learning challenges that higher education students, instructors, and institutions faced during the pandemic. In addition, we attempted to provide key cases in which LA has been effectively deployed for various purposes during the pandemic in the higher education context. Having a positive understanding of the advantages regarding LA tools accelerates the adoption of LA (De Laet et al., 2020; Park and Jo, 2019) . Then, it is likely that higher education stakeholders, who perceive LA as useful, can benefit from tools in future natural crises or emergency situations.

The extensive use of big data technologies enables us to collect a massive amount of data on the learning process (Gašević et al., 2016) . This data has been regarded as a valuable asset and promising source of information to support and enhance effective learning behavior and played a role in the emergence of the field of LA (Siemens and Gasevic, 2012; Larrabee Sønderlund et al., 2019) . LA is defined as the measurement, collection, analysis, and reporting of data about learners and their contexts to better understand and optimize the learning process (Siemens and Gasevic, 2012) . Meanwhile, LA tools are designed to provide customized information for supporting the learning process by analysis of stored data when learners interact with the learning environment (Dyckhoff et al., 2012) . Considering these definitions, online learning environments might be one of the most suitable platforms for the successful deployment of LA (Heath, 2021) . In fact, students in online learning leave a large amount of data while interacting with course materials and assignments (Alexandron et al., 2019; Schwendimann et al., 2017) . Therefore, these environments may contribute to data-driven decision-making in higher education, also in non-emergency days (Tsai et al., 2019; .

LA tools have provided possibilities and approaches to support learners in online and hybrid settings in higher education (e.g., Larsen et al., 2019) . We argue that these tools could be used effectively to support higher education institutions, instructors and learners in post-pandemic higher education. To better understand the ways of how and the purposes to which these tools could be used, an overview of LA use during pandemic is needed. Understanding LA promises can also give insight for future higher education policies (Ifenthaler and Yau, 2020) . In this context, the synthesis of the current study is important for higher education students, instructors, and policy-makers.

We addressed the following research questions (RQs) in the current study:

· RQ1-What are the online education challenges that institutions, instructors, and students face during the COVID-19 pandemic? · RQ2-What are the advantages of LA with the purpose of optimizing online education during the COVID-19 pandemic? · RQ3-What are the key cases from the pandemic process in which LA tools have been effectively deployed in response to online education challenges during the COVID-19 pandemic?

The manuscript search processes We adopted the guidelines of systematic literature review procedure suggested by Kitchenham and Charters (2007) . These guidelines start with a manuscript search process followed by employment of selection of criteria. Next, we performed analysis and synthesis of findings of the selected manuscripts. In the current study, four databases have been used for the manuscript search processes: ACM Digital-Library, IEEE XPLORE, Thomson Reuters' Web of Science, Elsevier's SCOPUS. These databases consist of several high-ranking journals (Shih, et al., 2008) including high-quality studies (Crescenzi-Lanna, 2020). The search terms used comprised the following words and combinations: "Learning analytics," "educational data mining," "dashboard," "visualization," "remote teaching," "remote learning," "online learning," "distance education," "higher education," "pandemic," and "covid" (Table1). After accessing the articles, we employed a set of inclusion and exclusion criteria. Following the relevant criteria, we selected empirical (1) articles and conference papers that have been conducted in higher education context (2) during the COVID-19 pandemic (3). Lastly, one more inclusion criterion is the actual use of LA analytical tools as a response to online learning challenges. As the time period, the conference papers and articles (in English) published started from the first of January 2020 to first of September 2021. The editorials, reviews and the studies conducted in the K-12 context have been excluded. As a result of the application of the criteria, 20 articles and conferences remained for the further analysis and synthesis process. Table 1 indicates the search string for each database and the number of articles selected/accessed.

In order to get an overview of the studies, we coded the selected articles in terms of data modality, analytical tool, country, and domain. Table 2 shows the overview of the studies on the response of LA to the online learning challenges during the COVID-19 pandemic. We performed the qualitative content analysis for these 20 studies. Considering that the preliminary or template coding scheme could unnecessarily direct researchers to the out of research aims (Ş imşek and Yıldırım, 2011), we Table 1 . The search string used in each database and its results.

Search string (Selected/Accessed) Web of Science TS = Topic. covers title, abstract and keywords (((TS = ("learning analytics" or "analytic* " or "educational data mining")) AND TS=("pandemic" OR "COVID")) AND TS=("remote teaching" or "remote learning" or "online learning" or "distance education")) AND TS=("higher education" or "univers*") ((TS=("automated feedback" or "dashboard" or "visualization")) AND TS=("higher education")) AND TS=("pandemic" or "covid") (5/23) Scopus TITLE-ABS-KEY = Title, abstract and keywords TITLE-ABS-KEY (((("learning analytics" or "analytic* " or "educational data mining") AND ("higher education" or "univers*") AND ("remote teaching" or "remote learning" or "online learning" or "distance education") AND ("pandemic" OR "covid")))) (4/38) ACM Digital-Library "learning analytics" AND pandemic (6/52) IEEE Xplore "learning analytics" AND pandemic (5/45) employed the open coding approach (Williamson, 2015) in the process of qualitative content analysis. For this, we firstly familiarized with the whole by picking a few of papers randomly and considered their basic meaning. Secondly, we defined preliminary thought in the margin, made a list of all thoughts, and grouped similar thoughts. In this process, we also made columns to differentiate key, unique, and leftover thoughts. Thirdly, after coding the text, we decided the most illustrative phrasing for our thoughts and turned them into categories. Fourthly, a certain abbreviation was decided for each category, to alphabetize these codes. Lastly, we incorporated the last code and employed initial analysis and recode, if necessary. The first author initially completed coding the studies individually and then shared the codes with the second and third authors. Disagreements were negotiated by checking the code list and the relevant studies. Some categories were merged and renamed. Finally, we recorded the 20 studies according to the latest code list (Table 2) . According to Table 2 , log data was found to be the main source of data for the LA tool. Also, Moodle and dashboards have been the most deployed tools for analytical purposes. LA tools have been commonly utilized in engineering and computer science departments.

Online distance education challenges (RQ1)

We found a number of reported challenges that stakeholders faced during online distance education in the COVID-19 pandemic. We addressed these challenges in terms of higher education stakeholders, namely, learners, instructors and institutions (see Table 3 ). According to the results of our review, the most commonly reported problem for students was lack of timely support. For instance, Maher et al. (2020) revealed that students needed support for their learning but could not receive it sufficiently during online education. The support is needed because students were less familiar with online learning before pandemic (Landrum et al., 2021) . Students had also struggled to understand course goals and instructor's expectations; thus, they had a sense of ambiguity about the assessment process of their online learning (Platt et al., 2014) . Similarly, lack of interaction appeared to be another challenge from the learner side. In fact, having less interaction chance is an important factor negatively affecting learners' online learning satisfaction (Cole et al., 2014) .

We also found that having difficulty with time management was a prominent challenge for students (e.g., Tabuenca et al., 2021) . This result is similarly evidenced with another study conducted during pandemic (Barrot et al., 2021) . During COVID-19 pandemic, empirical studies have shown that poor quality Internet access and less digital literacy skills were associated with lower use of the Internet for educational purposes (Van Deursen, 2020) . Similarly, in the current review, we reported students' digital inequalities to properly access learning content and lack of digital skills to utilize online environments as two important challenges (e.g., Günther, 2021; Rahmah et al., 2020) .

From the instructor`s side, our review revealed that assessment was a challenging issue for online learning (Conde et al., 2020; Naidoo and Naidoo, 2021) . Therefore, reliability of online exam results has been a conflicting issue for instructors. We also observed that instructors rarely had trust in the reliability level of the online assessment process (Balderas and Caballero-Hernández, 2020) . For instructors, this may be due to the decreased chance/opportunity for real time monitoring of learners. Similar to students, higher education instructors were not ready for online learning; therefore, we found that instructors were far from delivering best quality online lecture pedagogically and technically (e.g., Rahmah et al., 2020) .

It has been also found that the online learning process due to the pandemic situations lead to stressful situations for in the higher education instructors (Boyer-Davis, 2020). One of the reasons for this stress was depending on online education technologies without necessary digital skills. In the current review, we also observed that instructors had stress from heavy dependence on online learning technologies (Maher et al., 2020) . Table 3 . The challenges that stakeholders faced during online distance education.

Challenges ( The advantages of LA for online distance education challenges (RQ2)

As a result of the qualitative analysis, nine main categories were formed for the opportunities: providing timely support (1), automated adaptive feedback (2), planning online learning process (3), fostering learners' engagement and motivation (4), monitoring (5), facilitating assessment process (6), increasing interaction (7), improving retention (8), easy to be deployed (9). We then grouped each main category into separate sub-categories, based on where the reported opportunity naturally fit (see Table 4 ). It is important to note that some papers showed more than one opportunity. For such papers, more than one code was assigned for the opportunities. We also used "+" to indicate which stakeholders benefited from the relevant opportunity. According to results from the reviewed papers, two of the most remarkable opportunities of LA tools were that the tools supported learners and instructors in the planning the online learning process and monitoring. In the process of planning, learners and instructors can determine learning goals (Tabuenca et al., 2021) and set up completion times and conditions (Liang et al., 2020) by utilizing LA tools. This helps the learners with self-regulating their learning and the instructors with pedagogical planning (e.g., . Also, LA tools helped the institutions' and instructors' resource allocation during planning online learning (e.g., Anteby et al., 2021) .

Instructors benefited from monitoring opportunities when they monitor their students during a lecture or the whole online course (Farah et al., 2020) or when they receive information of students' early detected outcome (Dias et al., 2020) . In addition, students can monitor their own online learning progress using LA tools (e.g., Lapitan et al., 2021). As a result of our review, providing timely support was found to be a crucial opportunity for LA tools (e.g., Liu et al., 2020) . Given that one of the challenging issues for online learning is lack of support, LA tools can be a solution for this challenge. Similarly, the opportunity of automated adaptive feedback from LA tools can save Table 4 . Opportunities of LA tools for online distance education challenges.

Relevant stakeholders

Monitoring ( student-student (f = 1) + student-instructor (f = 1) + + Improving retention (f = 2) + + + Being easy to use (f = 2) + + + teachers' time and may help learners when they have a difficulty (Günther et al., 2021) . This adaptive feedback promotes personalized learning. According to our findings, LA tools might play an important role in fostering learners' engagement and motivation in online learning environments. In their study, Safsouf et al., (2021) identified that students shared their positive experience when they used the LA dashboard. In particular, students might have positive feelings when they monitor their learning progress through various useful visualizations. Also, students are more satisfied with having a chance of setting up their learning goal through the reminder systems (Safsouf et al., 2021) . When students face tough conditions such as pandemic, it is difficult for them to be highly motivated for learning and ultimately engage in the learning process (Mishra et al., 2020) . For this, successful deployment of LA tools offer opportunities for supporting their engagement .

LA facilitated online learning assessment was identified as an important support for instructors during pandemic. Specifically, the formative (Lapitan et al., 2021) and/or summative (Mata et al., 2021) assessment can be performed with the help of LA tools. For instance, Naidoo and Naidoo (2021) utilized visualizations of students' learning progress, which is beneficial for formative assessment. Further, it is possible to use LA individual (Balderas and Caballero-Hernández, 2020) and group assessment (Conde et al., 2020) in the process of online learning. Another LA opportunity is found to be the increment of student-student (Maher et al., 2020) and students-instructor interaction (Campanyà et al., 2021) . One possible explanation for increment in students-instructor interaction is the advantages of LA tools to teachers for understanding students' learning progress. This contributes to timely support from teachers, eventually resulting in the students-instructor interaction (Hernández-Lara et al., 2019) .

Some studies reported that LA tools have benefits for improving students' retention as a response to increased dropping-out risk (Perez-Sanagustin et al., 2021) . Instructors generally have time limitation for detecting students at-risk and offering adaptive feedback for students (Günther, 2021) . Students at-risk who cannot receive adaptive support are more likely to drop out the course (Martin et al., 2020) . In such times, LA tools with the advantages of timely support and intervention can increase student retention (Perez-Sanagustin et al., 2021) . It has been also reported that LA tools were perceived as easy to use (Maher et al., 2020; Liu et al., 2020) . Hence, this could be regarded as an opportunity for deployment.

Key case examples regarding deployment of LA tools during the pandemic (RQ3) Anteby et al. (2021) , aimed at exploring the cost and effectiveness of implementation of a set of podcasts for medical students in online distance education during the COVID-19 pandemic. The podcast series were released on a website and well-known podcast platforms. A total of 10 podcast episodes were available for free download within 9 months starting from March 2020. Podcast analytics acquired from transistor.fm that is a hosting platform for measuring audience and interest. The measuring is based on a set of metrics such as the number of downloads per episode by the day. Utilizing podcast analytics, the researchers conducted a cost-analysis process of podcast-supported online teaching methods. As a result of this process, it has been estimated that the total cost to create podcast series is comparatively less than other educational modalities. Hence, podcast analytics as a LA tool might offer premises for decision-making on the efficacy and effectiveness of a teaching and content delivery method.

In a study by Balderas and Caballero-Hernández (2020), a LA tool has been developed to detect students' cheating behavior when higher education students take online exams. For this, the students' data on the Moodle platform has been obtained. Then, this data has been transformed into a spreadsheet format and analyzed using process mining (Disco) and cheating detection software (By-Cheat). After the analysis, a variety of reports and visualizations have been formed representing students' information (grade, starting exam time, IP address from the student`s location, resources accessed during the exam, etc.). The LA tool has been deployed in the Degree in Computer Science and Engineering of the University of Cadiz (Spain). Results indicated that students have taken online exams in groups, so they have been seeking the answers to the questions among a few. It has been detected that some students who prefer taking the exam later, obtained higher grades within less time. This study highlighted that LA tools might be utilized for contributing to a valid and reliable assessment for distance education during or even after the pandemic. Safsouf et al. (2021) , utilized a LA dashboard entitled TaBAT that enabled instructors and students to monitor students' learning process with numerous visualizations. Specifically, the dashboard provided instructors with descriptive and/or predictive graphs regarding students. These graphs and visualizations helped to understand students' engagement, academic performance and probability of drop-out. This process led to timely interventions that might promote learners' selfreflection, autonomy, and academic performance. In the development process of TaBAT, researchers used a set of indicators (e.g., course, participation, success) for the three types of notifications, namely, students in difficulty, students in risk of dropping-out, and recommendations. According to the authors, automated feedback and timely support offered by LA tools such as TaBAT dashboard might play an important role in reducing drop-out rate at online distance education.

In another study, a LA tool has been created and employed with the purposes of evaluation of students' interactions (Conde et al., 2020) . The tool has been tested in a computer animation course during the pandemic. The students worked in groups of four members for a collaborative task. It was possible to monitor and assess student participation and interaction by using the number of short and long messages. Further, the number of initiated conversations was considered as an indicator of student leadership. Therefore, the Telegram Instant Messaging Tool facilitated teachers to better understand the individual acquisition of teamwork competence. The results pointed out that the usage of the tools was associated with better academic performance. Higher education students were willing to use Telegram Instant Messaging Tool, since they were familiar with instant messaging software. According to Conde et al. (2020) , the individual assessment from a collaborative work during online education requires online tools. This study indicated that LA tools revealed the frequency and contributions from students' messages which can be an important asset for collaborative work evaluation. Maher et al. (2020) integrated gamification principles into the LA dashboard. In particular, they suggested the Personalized Adaptive Gamied E-learning (PAGE) model as a response to online learning challenges such as learner's lack of motivation and lack of enjoyment. The suggested model is designed as independent of any domain so that it fits any sort of course for learners with different backgrounds. These flexible options enable teachers to make a dynamic course plan or/and practice. In addition, the PAGE model supports learners in overcoming the miscommunication challenges caused by distance learning. The results presented that LA dashboard supported with gamification principles had the potential for personalizing the higher education courses to each individual learner. During pandemic times, it is necessary to make the learning process as enjoyable as possible especially when higher education students feel isolated. Hence, the study of Maher et al. (2020) offers empirical evidence that using gamificated-LA dashboard might support instructors for making the courses more interesting.

Previous review studies indicated opportunities of LA in higher education (e.g., Viberg et al., 2018; Tsai and Gasevic, 2017) . However, the educational needs and expectations of stakeholders are most likely to vary in crisis situations such as pandemic. Therefore, the deployment of LA in higher education can differentiate compared to regular life conditions. In this regard, it is crucial to understand the role of LA in mitigating the learning and teaching challenges related to pandemic. The current study is the first attempt to provide an overview on the usage of the LA tools for overcoming online education challenges during COVID-19 pandemic. The implications of the current study might be important for post-pandemic policies of the higher education institutions. In fact, successful deployment of LA can be inspiring for another educational context such as K-12 level. For instance, when pre-service teachers increase their awareness of LA benefits in the higher education, they can utilize these tools in their professional life, specifically in K-12 context.

Based on our findings students have faced the most difficulties during online learning in COVID-19 pandemic. Further, the instructors have been identified as the stakeholder group potentially most benefiting from LA services. Thus, this leads to believe that LA has the potential to provide solutions to the challenges that institutions and especially instructors face during crises now and in the future. However, the nature of students' difficulties remains a challenge to address. From the student perspective, lack of timely support for learning activities, lack of human interaction and time management difficulties were the most prominent problems in the online education. As responses to these challenges, LA tools served as supportive tools for monitoring and planning online learning process and increasing their motivation for engagement. In light of this result, we suggest that the institutions and policy-makers should encourage the instructors and students to use LA tools effectively to overcome interaction, planning and time management challenges in particularly crisis time. According the results from our synthesis, instructors had difficulties in terms of individual and group assessment reliability of those assessments. In addition, they had less chance for real time monitoring of learners. For the difficulties that the instructors had, LA offers opportunities such as detecting students at-risk as a function of monitoring and facilitating assessment process. Moreover, the current study pointed out instructors might use some LA tools for more valid and reliable online exams. As seen from the current and previous studies, higher education institutions are not ready for digital transformation with insufficient infrastructure. This might lead to insufficient online learning management. Particularly, the current study showed that universities failed to provide the instructors with pedagogical support for online teaching during pandemic. We argue that preparing the instructors for giving online lecture should be agenda in the post-pandemic higher education policies. In this preparation, the opportunities of LA tools for online learning should be introduced to instructors. This can lessen instructors' stress stem from intense use of technologies. However, online education challenges identified at the institutional level also indicated that similar initiatives are needed to support learners. For example, improvement of the data infrastructure and improving the readiness for digital transformation could involve actions such as providing students with the platforms that enable students' study planning, monitoring and time management, or instructors' online monitoring and reliable assessment of student work during online courses. These implications might be considerable for post-pandemic higher education.

The results of this study are in line with the previous findings of LA opportunities and challenges in higher education context. As identified in this study, previous studies have identified LA tools opportunities to support learners' self-regulation of learning, including study planning and monitoring (e.g., Matcha et al., 2019) , as well utilization of LA tools as an instructors support for pedagogical design, monitoring and assessment of online learning processes (e.g., Molenaar and Knoop van Campen, 2018) . Additionally, institutional leadership and investment on the development of data infrastructure and development of LA tools has been identified as one of the key issues in implementing high quality LA supported practices and processes in educational institutions (Tsai et al., 2019) .

Despite addressing a topical and highly relevant area of research, this study has several limitations. First, extant literature focuses on the topic of LA and challenges experienced by the stakeholders of higher education are analyzed in this context. Literature reviews with different foci might uncover a variety of other online learning challenges, which would be important to take into account for a holistic perspective needed to organize distance education in the future. Some challenges such as difficulties with time management experiences by students have so far been only addressed by LA to a very limited extent. We see further possibilities for exploration in this area, as problems with time management will likely remain a challenge after a pandemic as well, for example, in organized hybrid education. Other identified challenges, such as a lack of social interactions, need to be investigated as a complex problem of students' well-being and the LA role in solving this issue has to be evaluated critically. We, however, see possibilities in this largely unaddressed area with involvement of other institutional resources, such as academic advisors, whose role in LA utilization has so far been underexplored. For future research, the impact of implementing LA tools in post-pandemic education as supporting teacher and student engagement, fluency of institutional practices and student learning should be explored.

According to the results of this study, LA as an emerging technology provides promising solutions to tackle the online education challenges faced by higher education institutions during COVID-19 pandemic. However, it is still unclear, what kind of educational practices higher education institutions will build for post-pandemic time. LA tools should be further investigated in the context of online and hybrid education practices. Key finding from our study is that while students were identified to experience the greatest number of challenges during distance education, instructors were potentially greatest beneficiaries from the LA tools. Key implication from this finding is the need for the institutions to initiate, supervise, and continually examine that investments into LA tools for teachers translates into support for students. Hence, the results of this study show that educational institutions have a responsible and critical role in understanding the challenges and needs of their stakeholders and in exploration of LA tools to address unique needs of these stakeholders in post-pandemic educational practices.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ismail Celik  https://orcid.org/0000-0002-5027-8284 Anni Silvola  https://orcid.org/0000-0002-2191-2194 technology supported transitions in education. Her research interests are career guidance, selfefficacy and learning analytics.

Anni Silvola is a PhD student at the Faculty of Education, at the University of Oulu, Finland. Her doctoral thesis focuses on researching how to support study engagement with learning analytics during university studies. Her research interests are study engagement and competence development during higher education studies, collaborative learning and learning analytics. Currently, she is working on a national learning analytics research and development project, funded by the Ministry of Culture and Education

Hanni Muukkonen is a professor in Educational Psychology at the Faculty of Education, University of Oulu (Finland). She holds a PhD in psychology. Her research has addressed collaborative learning and knowledge creation in higher education, design for learning and technologymediated collaboration. Technology design and development of related knowledge practices has continued through multiple large R&D projects in educational technology and learning analytics.

Are MOOC learning analytics results trustworthy? With fake learners, they might not be!

Development and utilization of a medical student surgery podcast during COVID-19

Determinants of students' perceived learning outcome and satisfaction in online learning during the pandemic of COVID19

Analysis of learning records to detect student cheating on online exams: case study during COVID-19 Pandemic

COVID -19 and online teaching in higher education: a case study of Peking university

Students' online learning challenges During the pandemic and how they cope with them: the case of the Philippines. Education and Information Technologies online ahead of print

Pandemic ponderings, 30 years to today: Synchronous signals, saviors, or survivors?

Technostress in higher education: An examination of faculty perceptions before and during the COVID-19 pandemic

Mixed Analysis of the Flipped Classroom in the Concrete and Steel Structures Subject in the Context of COVID-19 Crisis Outbreak

Promoting student engagement in online collaborative writing through a student-facing social learning analytics tool

The Economic Impacts of COVID-19: Evidence from a New Public Database Built from Private Sector Data

A learning analytics tool for the analysis of students' Telegram messages in the context of teamwork virtual activities

Multimodal learning analytics research with young children: a systematic review

Towards adaptive dashboards for learning analytic: An approach for conceptual design and implementation

Adoption and impact of a learning analytics dashboard supporting the advisor-Student dialogue in a higher education institute in Latin America

An instructor dashboard for real-time analytics in interactive programming assignments

DeepLMS: a deep learning predictive model for supporting online learning in the Covid-19 era

MOOCs as a massive research laboratory: opportunities and challenges

Design and implementation of a learning analytics toolkit for teachers

Impact of COVID-19 pandemic on information management research and practice: Transforming education, work and life

Bringing computational thinking to non-STEM undergraduates through an integrated notebook application

How do we start? State and directions of learning analytics adoption

The impact of social norms on students' online learning behavior: insights from two randomized controlled trials

Information technology solutions, challenges, and suggestions for tackling the COVID-19 pandemic

Buried treasure or Ill-gotten spoils: the ethics of data mining and learning analytics in online instruction

Applying learning analytics to students' interaction in business simulation games the usefulness of learning analytics to know what students really learn

Remote teaching and online learning

Utilising learning analytics to support study success in higher education: a systematic review

What (some) faculty are saying about the shift to remote teaching and learning

Guidelines for Performing Systematic Literature Reviews in Software Engineering

An effective blended online teaching and learning strategy during the COVID-19 pandemic. Education for Chemical Engineers

Contract cheating by STEM students through a file-sharing website: a Covid-19 pandemic perspective

A class of one: students' satisfaction with online learning

The efficacy of learning analytics interventions in higher education: a systematic review

Opportunities for improving the learning/teaching experience in a virtual online environment

MapOnLearn: the use of maps in online learning systems for education sustainability

Learners on focus: visualizing analytics through an integrated model for learning analytics in adaptive gamified e-learning

The impact of Covid-19 on higher education around the world

A synthesis of systematic review research on emerging learning environments and technologies

How to Teach Online? Recommendations for the assessment of online exams with University students in the USA in times of pandemic

Detection of learning strategies: A comparison of process, sequence and network analytic approaches

Online teaching-learning in higher education during lockdown period of COVID-19 pandemic

How teachers make dashboard information actionable

Linking Formative and Summative Performance in an online L2 module: Insights from Learning Analytics

Evolving information systems and technology research issues for COVID-19 and other pandemics

Education's Response to the COVID-19 Pandemic Reveals Online Education's Three Enduring Challenges

Factors that affect the success of learning analytics dashboards

Can feedback based on predictive data improve learners' passing rates in MOOCs? A preliminary analysis

A methodology to study the university's online teaching activity from virtual platform indicators: the effect of the covid-19 pandemic at universitat politècnica de catalunya

Virtually the same?: student perceptions of the equivalence of online classes to face-to-face classes

Developing distance learning monitoring dashboard with google sheet: an approach for flexible and low-price solution in pandemic era

Locked-down digital work

TaBAT: design and experimentation of a learning analysis dashboard for teachers and learners

Perceiving learning at a glance: a systematic literature review of learning dashboard research

The serendipitous impact of COVID-19 pandemic: A rare opportunity for research and practice

Research and trends in the field of e-learning from 2001 to 2005: a content analysis of cognitive studies in selected journals

Learning analytics special issue

Mind the gap: smoothing the transition to higher education fostering time management skills

Satisfying students' psychological needs during the covid-19 outbreak in German higher education institutions

Learning analytics in higher education-challenges and policies: a review of eight learning analytics policies

Complexity leadership in learning analytics: drivers, challenges and opportunities

From data to actions: Instructors' decision-making based on learners' data in online emergency remote teaching

Teachers' online teaching expectations and experiences during the Covid19-pandemic in the Netherlands

Digital inequality during a pandemic: quantitative study of differences in COVID-19-related internet uses and outcomes among the general population

t Reinventing the Wheel Just Operating the Tools": The Evolution of the Writing Processes of Online First-Year Composition Students (Unpublished Doctoral Dissertation

The current landscape of learning analytics in higher education

Measuring the impact of covid-19 induced campus closure on student self-regulated learning in physics online learning modules

Covid-19-VR strikes back: innovative medical VR training

Ismail Celik is a postdoctoral researcher in the Learning and Learning Processes Research Unit at the Faculty of Education, University of Oulu (Finland). He holds a PhD in educational technology. His research areas cover social media use, technology integration models in education, epistemological beliefs, artificial intelligence, and learning analytics.Egle Gedrimiene is a PhD student in Faculty of Education, at the University of Oulu, Finland. She has a background in psychology and is currently involved in research project investigating