key: cord-0079077-6xdbi0qp
authors: Shaw, Rabi; Mohanty, Chinmay; Patra, Bidyut Kr.; Pradhan, Animesh
title: 1D Multi-Point Local Ternary Pattern: A Novel Feature Extraction Method for Analyzing Cognitive Engagement of students in Flipped Learning Pedagogy
date: 2022-05-26
journal: Cognit Comput
DOI: 10.1007/s12559-022-10023-5
sha: 23a66355e789d87518aa44f8ce5e5a186004d686
doc_id: 79077
cord_uid: 6xdbi0qp

Flipped learning is a blended learning method based on academic engagement of students online (outside class) and offline (inside class). In this learning pedagogy, students receive lesson any time from lecture videos pre-loaded on digital platform at their convenience places and it is followed by in-classroom activities such as doubt clearing, problem solving, etc. However, students are constantly exposed to high levels of distraction in this age of the Internet. Therefore, it is hard for an instructor to know whether a student has paid attention while watching pre-loaded lecture video. In order to analyze attention level of individual students, captured brain signal or electroencephalogram (EEG) of students can be utilized. In this study, we utilize a popular feature extraction technique called Local Binary Pattern (LBP) and improvise it to develop an enhanced feature selection method. The adapted feature selection method termed as 1D Multi-Point Local Ternary Pattern (1D MP-LTP) is used to extract unique features from collected electroencephalogram (EEG) signals. Standard classification techniques are exploited to classify the attention level of students. Experiments are conducted with the data captured at Intelligent Data Analysis Lab, NIT Rourkela, to show effectiveness of the proposed feature extraction technique. The proposed 1D Multi-Point Local Ternary Pattern (1D MP-LTP)-based classification techniques outperform traditional and state-of-the-art classification techniques using LBP. This research can be helpful for instructors to identify students who need special care for improving their learning ability. Researchers in educational technology can extend this work by adopting this methodology in other online teaching pedagogy such as Massive Open Online Courses (MOOC).

Knowledge is the foundation upon which civilizations are built. Learning is the single most primary attribute required for the evolution of human beings. Over the years, the purpose of learning has remained the same, but methods have evolved. In ancient India, the Gurukul system was prevalent, with students staying in ashrams of their gurus and learning under the shade of banyan trees. Modernization and urbanization led to concrete buildings to be used as schools and institutions to impart education. Teachers deliver lessons inside a classroom through verbal instruction using boards or multimedia projectors while students take notes, listen and understand the lessons. Doubt clearing and assessment happen inside the four walls of the classroom. This is the traditional classroom pedagogy, which is still ubiquitous and relevant today. This provides an opportunity of physical interaction between the teachers and the students. The teacher can address the doubts and difficulties of each student in-person. However, this face-to-face mode of learning has a few drawbacks. A greater emphasis is given in delivering the lectures and completing the syllabus than practice or problem solving, due to time constraints. Thus, students lack the capacity to solve the practical problems using the theoretical knowledge. Due to the larger batch size of around 150 − 200 students, it becomes nearly impossible to identify the students who are inattentive. However, this is not the only method of learning today. The twenty-first century world is synonymous with digital revolution and Artificial Intelligence (AI). AI has spread its roots into almost every domain today. In the past few years, there has been a massive infusion of technology in the education domain. Innovative learning techniques like Massive Open Online Courses (MOOC), e-learning, m-learning (mobile learning) have enabled seamless access to learning for students anywhere anytime. Modern education tools like projectors, smartboards, audio-video devices, digital textbooks, even seating arrangements [1] , all powered by AI have provided useful alternatives for learning and can even improve the traditional approach. The shortcomings of the traditional classroom can be addressed by online learning and m-learning, where live online lectures or pre-recorded videos are available for the students at their fingertips. The instructors can also reap benefits of online assessment methods [2] [3] [4] . However, online learning comes with its own share of shortcomings. There is no scope for direct physical interaction between students and teachers. Problems such as lack of focus on critical thinking and problem solving still prevail in this mode. The instructor cannot identify the inattentive students, which hinders the potential success of online learning.

The blending of traditional and online learning methods can be a solution to this issue. One such popular learning mode is Flipped Learning [5] . In this mode, students complete their lessons in their homes or hostels. They can access reading material and pre-recorded lecture videos outside class as per their convenience. Inside the classrooms, the students engage in discussions and get their doubts and queries resolved by the instructor. This gives the instructor sufficient time to focus on imparting critical thinking skills and real-life problem solving inside the classroom. Unfortunately, the flipped classroom is vulnerable to shortcomings as well. It is hard for an instructor to know whether a student paid attention while watching pre-loaded lecture video. Monitoring the attention engagement levels of learners becomes important in this context. Sensors can be used to monitor their activities as they watch video lectures. The signals recorded by these sensors can be then analyzed and used to classify their attention levels [6, 7] . More signals can be captured using multiple sensors on different body parts [8] . But this multi-model approach needs expensive equipments. In addition, the students may not be able to give natural attention and get irritated when more than one sensor are put on their bodies. Multiple sensors may produce more noise compared to using single sensor. Coordinating multiple sensors is another major concern and system becomes complex. Therefore, researchers focus on using only single sensor in various domains [1, [9] [10] [11] .

In this paper, we propose a technique to record EEG signals of students while they watch video lectures, extract features from the recordings, and use them to classify students' attention levels. We call our proposed method 1D Multi-Point Local Ternary Pattern (1D MP-LTP). This method involves the use of Discrete Wavelet Packet Transform (WPT) to extract the useful features and reduce the dimension of data. Our proposed 1D MP-LTP method is then applied over it to obtain the desired features. Ternary Patterns are designed on varying block sizes for performing comparative analysis. The obtained Ternary Patterns are also converted into Binary Patterns in order to analyze and understand the need and effectiveness of Ternary patterns. We obtain several feature sets from the different conversion operations on different block sizes. To validate these features, standard classifier models and evaluation metrics are used. The classifiers using our proposed 1D MP-LTP method outperform traditional and LBP-based classifiers. EEG signal is a key attribute for analyzing the cognitive state of the student and has been used in different sub-domains of educational technology [1, 12, 13] . However, its utility is yet to be explored in the field of flipped learning. The novelties and contributions are summarized as follows.

-We propose the 1D MP-LTP method for novel feature extraction from brain wave signals in order to improve the learning ability of the student in the flipped learning pedagogy. -Data acquired at our lab for research on flipped learning has been utilized. Experimental results of our two proposed methods are evaluated with standard performance metrics such as accuracy, recall, precision, and F1 for classification purposes. Results showed the effectiveness of our proposed approach in the field of flipped learning pedagogy.

This work is an extended part of our accepted paper in proceedings of the 21 st IEEE International Conference on Advanced Learning Technologies (ICALT 2021) [14] . This paper is organized as follows. We provide a brief literature related to this article in "Related Research". A background of the proposed method is laid in "Background". In "Dataset Description" and "Proposed Method", dataset description and the proposed method have been described, respectively. Experimental results are demonstrated in "Experimental Results". Finally, "Conclusions and Future Scope" concludes this work and discusses possible directions and scope of this work in the future.

In this section, we describe related research work about academic engagement of students in Flipped Learning (FL) and feature extraction using Local Binary Patterns (LBP) [15] in various domains such as image processing [16, 17] , signal analysis [18, 19] , healthcare [20] , emotion recognition [21] , bio-metric identification [22] , rotary machines [23] , etc.

Extensive research has been accomplished to enhance the quality of education in traditional as well as online and blended learning. Cognitive Analysis in Education (CAE) involves understanding the underlying mechanism of thought process while learning. Rogaten et al. [24] review a rich but diverse variety of adopted methodologies to measure behavioral and cognitive learning gains.

To understand how efficiently a person is understanding or learning the lectures, cognitive engagement needs to be assessed. It becomes a challenging task to differentiate between cognitively engaged students from the rest. This is because, the unengaged student usually imitates the engaged ones for the sake of evading from the hurdles of learning. Giacomone et al. [25] take up a research gap in mathematics education research to provide an exhaustive characterization of mathematical activity carried out by students. The author uses the answers of 30 prospective primary education teachers to a specific mathematics problem on fractions using area and tree diagrams. Roohr et al. [26] examine learning gain of college students based on their performances in mathematics, writing, reading, and critical thinking as assessed by the ETS Proficiency Profile (EPP). They concluded that the student gained less knowledge in the first two years (first and second year), compared with last two years (third and fourth year). While comparing with the conventional curriculum-based learning, R. Amanda et al. [27] examine the effectiveness of Team-Based Learning (TBL) on persistent retention of knowledge. An elaborate analysis is carried out in [28] , which tries to show the effect of lecture video along with in-video assignments on students. In a recent study, [29] by Pappas et al., extensive research has been shown to analyze student engagement in adaptive learning through feedback, monitoring, and personalizing of contents. Questionnaire data, electrodermal signals, EEG signals, Clickstreams, and eye-tracking were used to measure arousal, cognitive load, response time, attention, and emotions for analysis. Sharma et al. [30] also utilize wristband data, eyetracking, facial video, and EEG data as features to predict performance levels of students in a "grey-box" approach on building pipelines for educational data. Mangaroska et al. [31] explore the utility of multiple e-learning systems to enhance learning methods using cross-platform analytics. All these works are directly related to the development of Flipped Learning models as a pertinent source of learning.

Subramaniam and Muniandy [32] show the comparison of performances in flipped learning (43 students) and traditional settings (41 students) based on emotional, behavioral, agentic and cognitive engagement levels. Sojayapan and Khlaisang [33] divided students into groups for flipped learning, traditional learning and online classroom to investigate their team learning ability. From t test analysis of posttest and pre-test results, flipped classroom was concluded to be more effective. To enhance learning and motivation in MOOCs, eye movements, gaze movements, stimuli and reading patterns are analyzed in [34] .

Giannakos et al. attempted to capture psychological data from devices that could be worn on wrists [35] . The authors used feature extraction techniques such as FFT, statistical measures. They were able to prove successfully that psychological senses could be a plausible replacement to measure attention and focus levels of students. Similar to attention, relaxation state of a student can also be analyzed [36] . The authors compared the mobile polling and Just-in-Time Teaching (JiTT) strategy. In [37] , the authors compared traditional click-stream models to current models based on psychological responses. Along with EEG data, eye-tracker, wristbands and webcam were used to collect psychological data. The findings showed EEG data and eye-tracking mechanisms to be especially dependable for attention measurement. A SVM-GA methodology has been adopted by Chen et al. [9] to analyze the attentiveness of the students using EEG data. The experiment employs Discrete Wavelet Transform and is used for feature extraction. A novel approach involving Genetic Algorithm (GA) is used to select seven optimum features and further pass through the SVM (Support-Vector Machine) model to classify the students with high or low attention rates. The model also identifies the portions of the video lectures on which students pay less attention. Similarly, sustained cognitive load, emotion, attention, and learning performance for different kinds of video lectures have been evaluated by authors in [38] [39] [40] [41] . Chen and Wu [39] used three different kinds of videos described as voice-over presentation type, lecture capture format, and picture-in-picture method. Along with evaluating attention, they also evaluated stress levels.

In the fields of image processing [42] [43] [44] , signal analysis [21, 22] , healthcare [20] , emotion recognition [21] , biometric identification [22] , etc., local information is also necessary while extracting notable features. This led to the foundation of a concept called Local Binary Patterns. The idea is to utilize the intensities of neighboring points (pixels in the case of image) and develop a representative value for that point. Thus the entire feature vector is transformed into a new one, which has the texture significance directly associated with it. This updated feature vector is then used for classification purposes.

Local Binary Pattern has found immense utility in the field of medical diagnosis. The LBP has been utilized to detect epileptic seizures in the brain from EEG signals [20, 45] . Epilepsy diagnosis is also the target in [18] , where Discrete Wavelet Transform has been utilized to decompose EEG signals into 5 levels using 1-Dimensional LBP. Quad Binary Patterns are also used to diagnose epileptic [46] . One-dimensional LBP has been used to detect Alzheimer's Disease [47] . A very recent work in the field of medicine where a Multi-Kernel Local Binary Pattern has been used to classify COVID-19 pneumonia from medical images [48] . LBP is extended to classify emotions from EEG signals. The extended LBP named as Fractal Patterns uses edges of the image as central point unlike traditional in LBP [21] . LBP is also extended to one-dimensional Local Difference Patterns for classification ECG signal [22] . LBP has been extended to 1-D Local Ternary Pattern (LTP) on rotary machines [23] . While two-dimensional LBP has been used in the domain of image processing, the same can be extended to signals with one-dimensional LBP or LTP [23, 47] .

As we discussed in "Student Academic Engagement in the Flipped Learning", EEG signals, electrodermal signals, eye gaze, gaze gesture or hand movement are analyzed to know the degree of engagement of the students with learning contents such as lecture video [2-4, 13, 49-55] . It is found that EEG signals are the most deciding factor for cognitive ability. However, EEG signals need to be processed to extract important features for further analysis [56, 57] .

On the other hand, LBP ("Feature Extraction by Local Binary Patterns") has been found as an effective feature extraction technique in many domains such as image processing, signal analysis, healthcare, emotion recognition, bio-metric identification, rotary machines, etc. This motivates us to explore LBP as feature extraction technique to analyze EEG signals obtained from one-channel headset which is easy to wear and cost-effective. In the current scenario (COVD 19) online learning is prevalent; however, teachers may not be able to know the engagement level of student. This will help teachers analyze the engagement level of student.

In our proposed method, we exploit two important feature extraction techniques, namely Local Binary Pattern (LBP), and Discrete Wavelet Packet Transform (DPT), which are discussed next.

In the fields of image processing or signal analysis, local information can be useful while extracting notable features. Ojala et al. [15] introduced a concept for the comparative study of texture measures in 1996, known as local binary pattern (LBP). LBP is mainly focused on the texture features of the image. Usually, a block of size 3 × 3 is chosen. And the central pixel is compared with all of its neighboring pixels. Each of the neighboring pixels are assigned values based on their comparison with the central pixel. If a neighbor pixel value is less than the value of central pixel, it is assigned 0, else 1. After assigning the binary values, the binary string is generated by going over all the neighbors (clockwise or anticlockwise). The binary string is then converted to its decimal value, which is the updated value of the pixel point. A histogram is then generated based on the frequency of each number occurring. This histogram is used as the feature vector. An example of the process is shown in Fig. 1 . This process is continued for all pixels. Several LBP [47, 58, 59] operators can be defined according to their neighbors as shown in Fig. 2 .

Wavelet Transforms are multi-resolution-based signal analysis techniques having their credibility in frequency as well as time domain analysis. Although Fourier transform has been historically used for frequency band analysis, Wavelet Transforms are The classical Discrete Wavelet transform (DWT) decomposes a signal into multiple orthogonal wavelets. The mother signal subjected to Discrete Wavelet transform provides two sets of coefficients symbolizing time-frequency domain representations. The low-ranged frequency coefficients termed as approximation coefficients are obtained using low-pass filters. Similarly, the high-ranged frequency coefficients are obtained using high-pass filters. The conventional DWT allows further decomposition of approximation coefficients and facilitates multi-resolution analysis. Tackling this very limitation, Discrete Packet Transform (DPT) [56, 57] allows the further decomposition of both approximation and detailed coefficients. The multi-resolution-based decomposition yields a tree-like decomposition facilitating decomposition at each level. The number of levels of decomposition and the choice of mother wavelet is specific to the use case being dealt with. Discrete Packet Transform, being more flexible and having better control, is more suitable over the conventional DWT.

Thirty-one male and thirteen female participants were selected for our experiment. All participants were between 17 and 20 years old and belong to departments of our institute from B. Tech course (first and second year). The selection procedure of participants for the experiment was compliant with the guidelines of the Institutional Review Boards (IRB) of NIT Rourkela. For our experiment, we used NeuroSky's MindWave Mobile2 (Fig. 3 ) headset 1 which was connected to the computer via adapter or Bluetooth. This is done to capture the EEG signals which are the responses by brain's stimulus. Sampling rate of headset is 512 Hz. The stimuli are continuous and the produced time signals are converted into the frequency domain subsequently.

We need to assign labels to the recorded data for classification. For this, two tests are conducted, before (pre-test) and after (posttest) a student watches a lecture video. We computed Pearson correlation between two tests marks for a student and EEG signals with positive correlation are considered in the experiment. The pre-test marks achieved by a student are considered, x which determines the prior knowledge possessed by the pupil on the video's topic. The post-test marks are considered to be y, which indicates how much he/she learnt from the video lesson. Let the mean value of the post-test marks of all students is a and the difference between mean values of post-test and pre-test marks is b. We assume that student is speculated to have sufficient knowledge about the topic if x is greater than or equal to 80% of total marks. His (her) EEG analysis will not be effective in the primary purpose of the experiment. In a case where a student obtains a marks in the post-test, this reveals that she performed worse than the average mass even after watching the video lectures. In addition, if the improvement over pre-test marks of a student, referred as (ImpX) ( ImpX = y − x ), is less than b, this indicates that the student did not even improve in his (her) own performance, which clearly shows lack of focus during the lecture video. This explains that the concerned student did not comprehend the lecture and it is labeled as Inattentive. All other cases are labeled as Attentive. Mathematically, the labeling approach is determined by using Eq. 1.

In this section, we utilize the concept of LBP in the proposed 1D Multi-point Local Ternary Pattern (1D MP-LTP) method for generating novel features from EEG signals.

During pre-processing, each EEG signal corresponding to a lecture video is subjected to Discrete Wavelet Packet Transform (WPT). The WPT is used in decomposition of the signal into set of wavelets, which helps to get relevant features in time-frequency domain [60] . Each level of DPT reduces the dimension of data by almost half. In our work, after experimentation with various levels of decomposition, four levels of decomposition are carried out, the approximation coefficients of the fourth decomposition are used (Fig. 4) . Having applied WPT, we use 1D MP-LTP method which is described below.

Let L =< l 1 , l 2 , … , l n > be the single dimensional data obtained after applying WPT on EEG signal corresponding to a video watched by a student. We group k consecutive values in L, say l 1 , l 2 ....l k to form one block. After grouping, we get |L| k non-overlapping blocks in total. In a single block, we assign a ternary bit to each value. This assignment of ternary operator is based on a threshold value . The value of threshold decides the boundaries for ternary values. In our study, the value of is taken as half of the mean amplitude ( ) value of each record L. We calculate the LTP value of each block. We convert each value of a block L j into ternary bits as follows. The center element is referred to as E c and neighbors (other elements) are referred to as E i , where i indicates the index of the element. Ternary values are then assigned to create the Ternary string. If (E c − E i ) is greater than or equal to a threshold then we assign a −1 to i th element. If (E i − E c ) is greater than or equal to a threshold , we assign it as +1 . For all other cases, we assign 0. We get a ternary string X using Eq. 2.

(2)

where X i is the ternary bit i th position. This process is carried out for each block.

Obtained ternary strings from each block can also be converted into binary string. Therefore, we can obtain two primary variants of proposed method based on the string conversion used. These variants are termed as 1D MP-LTP TT where we directly read string in the ternary format and 1D MP-LTP TB , if we convert the ternary strings to binary.

We can further obtain two variants of 1D MP-LTP TT based on the way we read the ternary string of each block (clockwise, anticlockwise). These variants are 1D MP-LTP TT (A) , and 1D MP-LTP TT (C) . The entire procedure is mentioned in Algorithm 1. After encoding a string block using Local Ternary Operator, we replace −1 in the strings with either 0 or 1. We can read the strings of ternary bits from left (anti-clockwise (A)) or right (clockwise (C)) end. Each ternary string is converted in four different ways, which are described as follows.

-1D MP-LTP −1,1 TB (A) : Each block is read in anti-clockwise, and value -1 is replacement with 1. It generates lower binary codes. Lower binary code value must be greater than or equal to upper binary code. We finally encode string to it corresponding decimal representations using the formula given in Eq. 3. We repeat this process for the next block and so on. At the end, we combine all the decimal-coded string blocks to obtain the features for each 1D data.

where m is the base (in our case, 2 for binary and 3 for ternary) and k represents the string's block size. In our experiment, we consider the values of k as 5, 7, and 9, respectively. The exact value of k is decided by trial and error method. We use odd numbers for the value of k, so that we get an equal number of cells on the both sides of the central (median)

Performance of the proposed method was not satisfactory beyond the value of k equal to 9. So, we restricted to use these values (5, 7 and 9) only. The entire 1D MP-LTP process is shown in Fig. 5 . In this way, we have two feature sets generated from 1D MP-LTP TT method and four different feature sets generated from 1D MP-LTP TB method. The entire procedure is mentioned in Algorithm 2.

As the preliminary idea arrives from LBP, we also create features using LBP. Based on the LBP, we obtain two different types methods, i.e., (1D MP-LBP(C) and 1D MP-LBP(A)) after using 1D Multi-point Local Binary Pattern (1D MP-LBP).

To show the effectiveness of our proposed feature extraction technique, we need to validate using standard classifier models. For classification, we split the datasets L1  L1  L1  L1  L2  L3  L4  L2  L3  L4  L2  L3  L4  L2  L3  L4   -288  320  3 into 70% for training, 20% for testing and 10% for validation purposes. In order to perform classification, we use Artificial Neural Network (ANN) with Backpropagation as our first classifier. In the ANN, two hidden layers are considered for mapping the input to output. The number of nodes in output layer is the number of class labels. In our case, it is 2, i.e., two class labels (Inattentive and Attentive). The sigmoid activation function was used between input-hidden layers. And a softmax activation function was used between the hidden layer and output layer. The loss function used in the error adjustments is the cross-entropy loss function, also known as log loss function. The network was trained for 1000 epochs with learning rate 0.0001. We also use standard classifiers like Decision Trees (DT), Random Forests (RF), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and state-of-the-art technique LBP [47] , and LTP [23] .

In KNN, the value of K is varied from 1 − 50 and validation-error for each is recorded. The K value corresponding to the minimum error is considered for the metrics evaluation. The Decision Tree classifier uses Gini index for splitting. The number of trees or estimators in Random Forest classifier is taken to be 100, with Gini index as the split criterion. An RBF-kernel-based Support Vector Machine(SVM) Classifier is also used for comparative analysis.

Two variants of proposed methods, namely 1D MP-LTP TT and 1D MP-LTP TB are applied on the dataset collected at our laboratory (IDA LAB, NIT Rourkela). We use the 1D MP-LBP feature selection methods for performance analysis purpose.

We report the performances of classification using proposed technique 1D MP-LTP TT in Table 1 . The results are also summarized with varying block sizes of 5, 7, and 9 in both clockwise and anti-clockwise arrangements. For the 1D MP-LTP TT (C) technique with a non-overlapping segment of size 5, in terms of F1 and accuracy, DT reports a value of 59.26% and 50.00%, respectively. The random forest outperforms DT. The ANN and SVM report F1 of 81.08% and accuracy of 68.18% each, respectively. However, KNN is better than ANN and SVM in terms of accuracy (70.45%). For block size 7, KNN records the highest F1 (81.16%) and the highest accuracy (70.45%), and DT records the least F1 (65.57%) and least accuracy (52.27%). The KNN outperforms ANN and SVM. Similar observation is seen for block size of 9, KNN is better than ANN, SVM, DT, and RF. The KNN achieves the value of F1 and accuracy 78.90% and 65.90%, respectively. The block length 7 (seven) is found to be most suitable for this dataset. Performance of individual classifier is reported to be best with block length 7.

For the 1D MP-LTP TT (A) technique with a non-overlapping segment of size 5, the DT is the worst performing classifier (F1 = 74.62%, and accuracy = 61.36%). The RF performs better than DT. The ANN and SVM achieve the highest value of F1 (82.66%). However, KNN outperforms SVM as well as ANN in terms of accuracy (72.72). Similar observation is seen for block size 7, where KNN, ANN and SVM achieve similar results for accuracy. However, ANN and SVM are better than KNN in term F1 (87.18). Similarly, for block size of 9, KNN records the highest F1 (86.84%) and the highest accuracy (77.27%). So, we conclude that 1D MP-LTP TT (A) technique performs better for SVM and ANN classifier with block size 7.

The performances of classifiers using 1D MP-LTP TB are displayed in Table 2 . We compared our results with the 1D-TP method [23] . Performance of ANN, RF and SVM using 1D MP-LTP TB as feature extraction technique is reported to be the same. The KNN is found to be the best with F1 (80.55%) and accuracy (68.18%). Performance of classifiers using 1D-TP(A) is found to be better than using 1D-TP(C). Results obtained from four variants of proposed 1D MP-LTP TB are also displayed. The proposed D MP-LTP −1,1 TB (A) is found to be outperforming all other variants as well as 1D MP-LTP TB . Table 3 . The ANN classifier using 1D LBP outperforms other classifiers using 1D LBP. The ANN approach outperforms all the models with F1 (78.86%) and accuracy (65.90%). For the 1D MP-LBP(C) method, DT is the worst performing classifier (F1 = 67.74% and accuracy = 54.54%). The ANN and SVM perform better than RF with F1 (81.08%) and accuracy (68.18%) for both. The KNN approach outperforms all other models with F1 (82.19%) and accuracy (70.45%). Likewise, for 1D MP-LBP(A) method, KNN classifier has the highest accuracy (61.36%) and the highest F1 (72.13%) among the models. The proposed 1D MP-LBP(C) is better than 1D LBP.

Average accuracy of popular 1D LBP and our proposed method is displayed in Fig. 6 . From this plot, it can be clearly seen that our proposed method is better than 1D LBP.

The single-dimensional EEG signals are divided into a number of non-overlapping blocks in the proposed feature extraction technique. Subsequently, each block is converted into ternary (binary) string. Each string is read from clockwise or anti-clockwise. We read the string from both sides to see the effect of position of feature value with in a block. We observed that ternary bits +1 located in lower index contribute very less value if string is read from left while it can contribute significantly if it is read from right. We noticed that the performance of anti-clockwise method is better than clockwise method. It can be noted that the classifiers perform better for 1D MP-LTP TT while retaining the -1 values in ternary formats.

Accuracy of the classifier using proposed feature extraction technique ranges between 70 and 80%. To further improve the accuracy, one can use other biological signals such as facial expressions, eye gaze, gaze gesture or hand movement which can be collected using web camera or frontal camera. It can be noted that this sensor is not placed on the body part. Placing multiple sensor or multi-channel headset may be a bar for natural attention. 

Flipped Learning is found to be an effective learning pedagogy and has been adopted in many higher learning educational institutes across the world. However, major issue with this learning pedagogy is that it cannot identify whether the students pay sufficient attention while watching pre-recorded lecture video.

In this study, we proposed a feature extraction approach to obtain the significant feature for analyzing captured brain signal of students while watching lecture videos. The adapted feature extraction approach is used to classify students involved in a flipped classroom based on the attention level. The results showed that the classifiers combining with proposed 1D MP-LTP method outperform standard classifier as well as LBP-based classifiers. This research can be helpful for instructors to identify students who need special care for improving their learning ability.

This work can be extended in two ways, i) recommend the inattentive lecture video to the students for rewatching, ii) to analyze the multi-source data along with singlechannel EEG data in the flipped learning pedagogy. 

Ethics Approval Experiments were conducted with IRB approval from NIT Rourkela (Our institute).

Consent of each participant is obtained for the experiment.

Consent for Publication Consent for publications was obtained for publishing their data and photographs.

The authors are declaring that there are no conflicts of interests.

Designing a technology-augmented school desk for the future classroom

Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. Higher Education Research & Development

Exploring undergraduates' perspectives and flipped learning readiness in their flipped classrooms

Student perceptions toward flipped learning: New methods to increase interaction and active learning in economics

A flipped classroom approach to teaching empirical software engineering

Estimation of User's attention in mobile Moocs from eye gaze and gaze gesture tracking

Towards attentive, bi-directional MOOC learning on mobile devices

Towards emotionally aware AI smart classroom: current issues and directions for engineering and education

Assessing the attention levels of students by using a novel attention aware system based on brainwave signals

Validation of stimulus for EEG signal based cognitive load analysis

Classification of imagined speech using Siamese neural network

Predicting when teachers look at their students in 1-on-1 tutoring sessions

Effect of the flipped classroom on the mathematics performance of middle school students

Attention analysis in flipped classroom using 1d multi-point local ternary patterns

A comparative study of texture measures with classification based on featured distributions. Pattern Recogn

Face detection based on multi-block lbp representation

An LBP encoding scheme jointly using quaternionic representation and angular information

A hybrid Local Binary Pattern and wavelets based approach for EEG classification for diagnosing epilepsy

A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals

1D-local binary pattern based feature extraction for classification of epileptic EEG signals

A new fractal pattern feature generation function based emotion recognition method using EEG

ECG based biometric identification using one-dimensional local difference pattern

A novel feature extraction method for bearing fault classification with one dimensional ternary patterns

Reviewing affective, behavioural and cognitive learning gains in higher education

Cognitive analysis on prospective mathematics teachers' reasoning using area and tree diagrams

Investigating student learning gains in college: a longitudinal study. Stud High Educ

Effects of team-based learning on short-term and long-term retention of factual knowledge

Investigating students' use and adoption of with-video assignments: lessons learnt for video-based open educational resources

Utilizing multimodal data through fsqca to explain engagement in adaptive learning

Building pipelines for educational data using AI and multimodal analytics: A "grey-box" approach

Architecting analytics across multiple e-learning systems to enhance learning design

The effect of flipped classroom on students' engagement

The effect of a flipped classroom with online group investigation on students' team learning ability

Eye-tracking and artificial intelligence to enhance motivation and learning

Fitbit for learning: Towards capturing the learning experience using wearable sensing

Mental effort detection using EEG data in E-learning contexts

Multimodal data as a means to understand the learning experience

Construction of multi-mode affective learning system: taking affective design as an example

Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance

Using humorous images to lighten the learning experience through questioning in class

Using positive visual stimuli to lighten the online learning experience through in class questioning

Enhanced local texture feature sets for face recognition under difficult lighting conditions

Local tetra patterns: a new feature descriptor for content-based image retrieval

Multi-directional multi-level dual-cross patterns for robust face recognition

Epileptic seizure detection using constrained singular spectrum analysis and 1D-local binary patterns

A new approach for automatic detection of focal EEG signals using wavelet packet decomposition and quad binary pattern method

One dimensional local binary patterns of electroencephalogram signals for detecting Alzheimer's disease

A novel COVID-19 and pneumonia classification method based on F-transform

Video-based learning ecosystem to support active learning: application to an introductory computer science course

Predicting student retention in massive open online courses using hidden Markov models

Applying educational data mining to explore students' learning patterns in the flipped learning approach for coding education

Exploring students' learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education

Seamless flipped learning: a mobile technology-enhanced flipped classroom with effective learning strategies

Classifying students based on cognitive state in flipped learning pedagogy

ARTFul: adaptive review technology for flipped learning

Continuous and discrete wavelet transforms

The discrete wavelet transform: wedding the a Trous and Mallat algorithms

Channel binary pattern based global-local spatial information fusion for motor imagery tasks

Multiresolution gray-scale and rotation invariant texture classification with local binary patterns

EEG data classification for mental state analysis using wavelet packet transform and Gaussian process classifier