key: cord-0431857-1jwlk089
authors: Sakr, Nourhan; Salama, Aya; Tameesh, Nadeen; Osman, Gihan
title: EduPal leaves no professor behind: Supporting faculty via a peer-powered recommender system
date: 2021-04-20
journal: nan
DOI: nan
sha: 4adedaea98cb6e5f776b10473dd47004dddb2520
doc_id: 431857
cord_uid: 1jwlk089

The swift transitions in higher education after the COVID-19 outbreak identified a gap in the pedagogical support available to faculty. We propose a smart, knowledge-based chatbot that addresses issues of knowledge distillation and provides faculty with personalized recommendations. Our collaborative system crowdsources useful pedagogical practices and continuously filters recommendations based on theory and user feedback, thus enhancing the experiences of subsequent peers. We build a prototype for our local STEM faculty as a proof concept and receive favorable feedback that encourages us to extend our development and outreach, especially to underresourced faculty.

The COVID-19 lock-down forced many higher education institutions globally to continue instruction via online modalities at an unprecedented pace and scale [5, 10] . With many faculty scantily trained in teaching strategies or with little support on best online practices [4, 10, 22] , instruction was maintained at the cost of education quality, equity and sound pedagogy [3, 5, 18] . Online education requires deliberate design and development [5, 10, 16, 22] , yet, the pandemic forced the adoption of emergency remote learning, regardless of any obstacles.

In non-emergency times, faculty in resourced institutions are often supported by instructional designers who provide personalized guidance on making sound design and technology decisions for the faculty's particular context [1] . However, given the sheer number of "overnight" transitions, individualized help became rather challenging [10] . The pandemic revealed the lacking capacities for support and infrastructure in institutions [3, 11, 14, 21] , thereby questioning readiness for the digital era. Looking further into under-resourced institutions, general capacity building and high-quality instructional guidance are considered a luxury.

In light of this extreme global test, we identify a gap in the pedagogical support available to educators. Social media and online webinars attempted closing this gap by providing platforms for sharing experiences and sound tips online. However, we see three issues with such channels: They are (1) less personalized,

(2) suffer from information overflow and (3) are not guaranteed to continue after the pandemic. These issues make it hard for some faculty to find relevant resources or apply what they find to their personalized contexts. Based on our survey (n = 103), 86% believed that being able to readily access relevant experiences shared by peers would be beneficial to them, even in the long run.

Within this framework, we propose, EduPal, a virtual educational consultant. Our user-centric design provides a crowd-sourcing platform augmented with collaborative filtration to automate experience sharing and knowledge distillation. We wrap these within a recommendation system that provides personalized, context-aware guidance on best-fit pedagogical practices, as supported by research theory and faculty practice. As a proof of concept, EduPal was customized for our local STEM community at the American University in Cairo (AUC), classifies as an M1 university 1 . In this paper, we present our pilot's data collection methodology, system design and show positive user feedback declaring the system as promising for extension and generalization.

Our data-driven design builds on a taxonomy that is the product of an elaborate data collection process, outlined briefly in this section. Despite our focus on supporting STEM faculty at AUC (for the pilot study), we consider various populations to build our data. We apply maximum variation sampling in recruiting participants and conclude any stage when the research team agrees on information saturation. Our findings heavily rely on qualitative analysis. Our secondary research builds on education and psychology literature, as well as social media narratives from all stakeholders, i.e. faculty, students and instructional designers.

Community Feedback. The first stage distills knowledge from 50 hours of semistructured interviewing of faculty at AUC, spanning all schools. Faculty reflected on their teaching challenges, need for pedagogical support, and practices most effective for their specific class types. Results were augmented with secondary research to determine sets of: 1)features that identify profiles of instructors/classes and 2)pedagogical practices and technology tools that best fit each profile.

Filtration and Validation for STEM. The second stage refines the features and recommendations to those most applicable to STEM courses via a semi-open survey aimed at STEM faculty at AUC (n = 100).We, then, ran two seminars for STEM faculty at AUC (n = 50) and two seminars for instructional designers (n = 14). Our recommendations were presented for discussions regarding their viability and feasibility. This process seeds our recommendations bank.

Learning from Experts. Finally, we conducted semi-structured interviews with global educational consultants (n = 9, 60 mins each). The interviews simulated a pedagogical consultation followed by a discussion on the usual process. The 

EduPal is an instance of knowledge-based recommender systems [2, 6, 7, 12, 15, 19, 20] . Via a chatbot interface, the user (assumed to be an instructor) interacts with the system as if talking to an instructional designer. The conversational session collects session features (f ) about the user and their course. The design encompasses a two-staged pipeline that allows users to benefit from the perspectives of practicing peers and pedagogical experts, as well as rate and suggest recommendations, thus automating experience sharing. The proposed recommendations are meant to enhance the three class room interaction modes defined in [17] . Communication with the server is secure and confidential. An "anonymous" mode is added to maintain privacy of users who do not want to share their information. Figure 1 depicts the system components and flow.

Collaborative filtering based on educators feedback. A user-based collaborative filtering approach [20] is used to compile an initial set of recommendations. When a new session starts, f is collected and cosine similarity [8] is used to identify the most similar users and extract the set of recommendations S, top rated by those users. This approach is suitable only for recommendations that have received ratings, while unrated recommendations are sent to the expert system, which is authorized to update the knowledge bank R.

Expert system. This module mimics the decision making processes of an instructional designer and addresses the cold start problem [13] . It can be considered as a symbolic AI system where rules are constructed based on feedback from subject matter experts [9] . It ensures that the recommendations are not only based on popularity but are also pedagogically sound with support from research and practice. For each recommendation r ∈ R, experts identify the factors defined in f that should match r with a user, based on the learning sciences. Those rules are then translated into system logic. where, each element s ∈ S)is either accepted as part of the final set, S , or rejected. This logic is also used to add recommendations, deemed as best fit by experts, to S .

Feedback. Each recommendation in S is presented in a conversational format and rated by the user. The ratings are fed-back to the system to inform future selections 2 . Finally, the user may share other effective pedagogical practices, that are considered for extending R and fully streamlining experience sharing.

AUC faculty 3 (n = 10) evaluated the prototype via 30-minute usability tests. We first learned about (1) their means and frequency of seeking help with pedagogical matters and (2) their experience with chatbots. They, then used EduPal and provided ratings on the overall quality of the experience and the received recommendations. On a Likert scale out of 5, the mean responses to those questions were 3.8 & 3.7, respectively. Testers were also asked to share what they liked and disliked about EduPal as well as its advantages and disadvantages over their default methods of seeking help. The majority found EduPal beneficial and user-friendly. They highlighted fast feedback and constant availability as immediate advantages over other aid methods. The chatbot interface made their experience feel interactive, engaging and personalized. Our recommendations were deemed of good quality but users suggested providing more specific examples for application. Lastly, EduPal's anonymous mode provided a "safe zone" for instructors who usually avoid sharing experiences or asking for help.

Based on the favorable feedback received, we conclude that EduPal shows a successful pilot system and is worth generalizing to address global knowledge distillation, experience sharing automation, recommendation personalization and support scalability, in addition to promoting fairness in access to resources, given that EduPal is available 24/7 for free. We also recognize the potential of Edu-Pal becoming a screening tool for educational consultations, thus augmenting the impact of instructional designers and making appointments run faster in times of high demand, e.g. during the pandemic. between emergency remote teaching and online learning. Educause review 27, 

Instructional design in higher education: Defining an evolving field

Knowledge-based recommender systems. Encyclopedia of library and information systems

Covid-19: 20 countries' higher education intra-period digital pedagogy responses

A critical reconceptualization of faculty readiness for online teaching

What we learnt from "going online" during university shutdowns in south africa

An architecture for making recommendations to courseware authors using association rule mining and collaborative filtering

Educational recommender systems: A pedagogical-focused perspective

2 -getting to know your data

Artificial intelligence: The very idea

The difference 2020

Recommender systems in elearning environments: a survey of the state-of-the-art and possible extensions

Addressing cold-start problem in recommendation systems

Online classes try to fill education gap during epidemic

Improving recommender systems with adaptive conversational strategies

Learning online: What research tells us about whether, when and how

Three types of interaction

In search of the 'new normal': Reflections on teaching and learning during covid-19 in a south african university

Management and selection of visual metaphors for courseware development in web based learning

Introduction to recommender systems handbook

How a top chinese university is responding to coronavirus

Instructional designers' roles in emergency remote teaching during covid-19