key: cord-1033307-yei1nz5r authors: Paxinou, Evgenia; Kalles, Dimitrios; Panagiotakopoulos, Christos T.; Verykios, Vassilios S. title: Analyzing Sequence Data with Markov Chain Models in Scientific Experiments date: 2021-07-21 journal: SN Comput Sci DOI: 10.1007/s42979-021-00768-5 sha: 04031398fb0a0778db3696d11cc6fe180bf8e14a doc_id: 1033307 cord_uid: yei1nz5r Virtual reality-based instruction is becoming an important resource to improve learning outcomes and communicate hands-on skills in science laboratory courses. Our study attempts first to investigate whether a Markov chain model can predict the students’ performance in conducting an experiment and whether simulations improve learner achievement in handling lab equipment and conducting science experiments in physical labs. In the present study, three cohorts of graduate students are trained on a microscopy experiment using different teaching methodologies. The effectiveness of the teaching strategies is evaluated by observing the sequences of students’ actions, while engaging in the microscopy experiment in real-lab situations. The students’ ability in performing the science experiment is estimated by sequential analysis using a Markov chain model. According to the Markov chain analysis, the students who are trained via a virtual reality software exhibit a higher probability to perform the steps of the experiment without difficulty and without assistance than their fellow students who attend more traditional training scenarios. Our study indicates that a Markov chain model is a powerful tool that can lead to a dynamic evaluation of the students’ performance in science experiments by tracing the students’ knowledge states and by predicting their innate abilities. In science courses, instructors do their best to communicate knowledge in terms of content and skills. A student who pursues to successfully complete a science course must not only have understood and assimilated the basic principles of specific science subjects, but he/she should have also acquired specific practical laboratory skills [21] . Besides, it is commonly acceptable that laboratory hands-on skills have always been a key pillar of science education. Μany researchers claim that the best way to obtain them is through practicing and not through passively watching face-to-face demonstrations in science labs or simply watching instructional videos [4] . Although practicing in a physical lab is an ideal way of being trained in experimental techniques, constantly equipping and maintaining these labs is becoming more and more expensive and prohibitive for educational institutions [34] . A robust and affordable solution to overcome: (a) the expensive update, (b) the safety issues that result from the misuse of sensitive and complex lab instruments from novice or unprepared students, and (c) the current needs for distance experimental training that arose from the new COVID-19 situations, is to immerse in a virtual lab and interact with the virtual lab equipment. Virtual Reality (VR) is a cutting edge emerging technology that the last decades demonstrates a great potential to change and modernize the way learners are trained in handson skills in many fields such as medical, engineering, natural sciences, etc. [6, 29, 44, 49, 57, 60] . Makransky and Lilleholt [35] mention that many business analyses and reports (e.g., Belini et al. [3] , Greenlight & RoadToVR [17] ) predicted that VR would be the biggest future computing platform of all time as it could revolutionize the entertainment, gaming and education industries. Review papers mention the research methodologies used in the area of adaptive systems like 3D virtual learning environments [51] . Many supporters of VR technology claim that this alternative educational approach facilitates learning due to the ability of the human brain to perceive better and assimilate easier a three-dimensional (3D) computer-graphics representation than a simple text [11] . Many studies show that simulations can be a very promising and affordable tool for learning and instruction [30, 56, 64] , especially for users who are not aware of information technologies [14] . Virtual laboratories have overall positive effects on students' cognitive load, skills development and motivation [35] . Several VR educational applications have been designed for STEM (Science, Technology, Engineering, Mathematics) domains [9, 45, 59] , as interaction with such environments has shown, among other things, gains in deep and certain conceptual understanding, experimental experience and problem-solving ability [23] . The exploratory character of the virtual worlds that offers free navigation using the first-person user viewpoint may be the reason why VR applications are educationally superior also to multimedia [66] . There are several research studies that explore whether in science courses, attending traditional educational scenarios in physical labs is more beneficial than following more novice teaching strategies that include interaction with virtual environments [5, 36, 37, 40, 43, 53] . All researchers agree on the fact that physical labs play a unique role as it is a sine-qua-non in science [20, 22, 69] , but more and more the VR technology reveals its substantial contribution to the successful achievement of the learning outcomes in laboratory courses. Paxinou et al. [42] provided evidence in favor of the use of a VR educational software, as a supplementary tool to the traditional laboratory learning methods in Biology Makransky et al. [36, 37] claimed that simulations must be used as a tool for preparation for the lab experiments. Authors in Xu et al. [67] showed that in developmental biology the combination of a virtual oriented and a traditional methodology in teaching promotes effective student learning. In a science laboratory course, a way to evaluate the effectiveness of a certain teaching procedure is to explore whether the learning outcomes have been fulfilled. Have the students understood the introduced concepts? Have the students managed to obtain the required laboratory skills? The level of the students' understanding of the new topics can be easily assessed, for example, through scoring specially designed written tests, but assessing the gained practical skills is a quite multidimensional task [46] . Data, derived from the participation of students in educational research, are a powerful tool to researchers as they can utilize them to identify hidden patterns by using analytics techniques [15, 16, 27, 33, 41, 61, 63, 65] . This study is initially based on the assumption that the effective completion of all the steps comprised an experiment, is a robust indicator that the learner has a high perception of the lab environment and has also acquired all the necessary hands-on skills. Therefore, we trained three groups of students on the Microscopy Experiment, by applying three different teaching scenarios, to investigate the predominance of the best scenario. After the different educational interventions, the students' ability to handle and operate properly a microscope, was evaluated through a specially designed worksheet. According to that worksheet, the microscopy experiment was divided into 13 steps. As is the case in every science experiment, the 13 steps had to be performed strictly in the given order and without skipping any of them. As a result, the data derived from the observed students' actions when performing those 13 steps, were sequential. In this study, we decided to use a popular method for analyzing our sequential data, a Markov-based technique, and more specifically, a Markov chain model. Markov model-based techniques are useful methods to analyze data, where order matters [32, 48] . Markov chain models and Hidden Markov models (HMMs) are both statistical models that belong in this category. Considerable research has been conducted on Markov chain models in many different settings, such as to predict enrolments for an education system or to model teachers' behavior in the decision-making process or finally to analyze genetic algorithms [26, 39, 47, 55] . On the other hand, HMMs have extensively been used to model the behavior of individual students regarding their engagement and their motivation towards the learning procedure [2, 10, 18, 50, 54, 62] . Arieli-Attali et al. [1] used a HMM to learn about test takers' choice-making behavior in a self-adapted test, Shih et al. [52] proposed a HMM that could discover student learning tactics, Tadayon and Pottie [59] and He and Gao [19] used a HMM to analyze and make predictions of the students' performance in educational games, and Jeong et al. [25] used HMMs to examine the effect of metacognitive prompting on students' learning in the context of our computer-based learning-by-teaching environment. Although the HMM is based on augmenting the Markov chain model, in this study we used the latter, as we aimed to model the sequences of observable events, like the observed student's actions when conducting an experiment in a science lab, and not any unobservable influences. To the best of our knowledge, no other studies have relied on a Markov chain model to provide evidence in favor of an educational intervention by modeling the students' performance in a lab environment. Neither such a model has been used to further investigate whether the integration of Information and Communication Technologies into teaching, helps students acquiring those experimental skills that are necessary for performing successfully an experiment. Based on the fact that a science experiment, by nature, is a prototypical procedure to obtain sequential data, our main research question is whether a Markov chain model can offer additional information about the comparison of different teaching methods that are applied in laboratory science courses. At this point, it is worth mentioning Deep Learning Techniques are also very popular techniques that could have been considered for the processing of the data in this study. Deep Learning comprises a state-of-the-art learning paradigm that sheds new light on neural network approaches. Long-Short Term Memory (LSTM) components, Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) bring in the Machine Learning Community a great deal of innovation that provides accurate complex predictions on a diverse set of problems like speech processing, image processing, text processing and the like [24, 28, 68] . But, despite their groundbreaking impact on recent application problems like self-driving cars, in this study, we decided to choose simpler white box algorithms that fitted better for our problem where the understanding of the proposed solution was necessary. The main aim of our study was twofold. First, we investigated whether a Markov chain model is a useful tool that can evaluate the students' performance in science experiments by tracing their knowledge states and by predicting their innate abilities. Second, we interpreted the produced data to compare the effectiveness of three different teaching strategies applied in a laboratory science course. Such a comparison could lead to serious decisions regarding the improvement of teaching and learning practices. Furthermore, in our study, the Markov Chain methodology used for the sequence data analysis is presented as a step-by-step procedure in an attempt to make it totally understandable. Our results provided evidence in favor of using such a statistical model for modeling and predicting the students' actions when conducting hands-on exercises. In addition, the Markov chain methodology helped us come to the conclusion that the students who were trained on the microscopy experiment by interacting with a VR educational application, exhibited a greater ability in performing the specific experiment in the physical lab. Sixty-two, 4th-year, undergraduate students at the Department of Primary Education of University of Patras, in Greece, participated in this study. The students were enrolled in the Computers and Education course, where the learning outcomes are (a) to practice in computer use, (b) to be informed about the latest developments in educational software for primary and secondary education and (c) to be educated on technology-assisted teaching and learning. This research focuses on the acquisition of hands-on skills after the training through a specific methodology. In particular, the participants are trained to operate a photonic microscope, the most basic and essential instrument in a biology lab. They are educated through three different teaching methods, one of which is the use of educational software. Our sample is a novice audience, as it brings a zero to minimum prior knowledge on the subject of microscopy. We chose the specific participants, as the students who attended the aforementioned graduate program, will face the challenge to train their young pupils, in simple experiments such as that of microscopy, when becoming teachers in primary schools. To educate the 62 students on the use of the photonic microscope we separated them into three different groups; the T-Group who attended a traditional face-to-face demonstration of the microscopy experiment, the V-Group who watched an instructional video on the microscopy experiment, and the VR-Group who downloaded the Onlabs software (https:// sites. google. com/ site/ onlab seap/) and interacted with the simulated microscope in a VR environment to perform the microscopy experiment. The specific experiment included the use of all of the 4 objective lenses of an optical (light) microscope to focus on human, animal and plant cells. Table 1 presents briefly the three phases of the educational experiment. The current study is focusing on the 2nd Phase and especially on the 3rd Phase of the scenario. As presented in Table 1 , in the 2nd Phase, the VR-Group was trained on the microscopy topic via the Onlabs, and more precisely by using the Instruction Mode of this VR application. The Onlabs is offered in three modes: The Instruction Mode, the Evaluation Mode and the Experimentation Mode. When using the Instruction Mode, specific instructions keep appearing on the screen guiding the students to operate properly the microscope. In case they cannot respond to the instruction, they have the option to click on the globe-button on the left up corner of the screen and a written hint is appearing to their help ( Fig. 1 ). If and only if, the students complete an instruction, they can move on to the next one. The students keep following the instructions and taking into consideration the hints until they manage to focus successfully on a specimen by using all the objective lenses of the microscope. Only once, the VR-Group interacted with Onlabs, the V-Group watched the instructional video and the T-Group attended the face-toface tutorial. According to the 3rd Phase of the project, after the training in the 2nd Phase, the three groups entered the biology lab. There, all students used their own optical microscope to set the instrument and focus on different cells. In parallel with conducting the specific microscopy experiment, the students had to fill in a worksheet (hard copy) following the method presented by Paxinou et al. [45] . According to this worksheet, the experiment was a procedure divided into 13 steps. For each step, an instruction was given. The students had to follow the given instruction to perform each step, in a specific order. Each time a student performed a step, he/she had to tick on one of the three given outcomes (states), A, B or C, (Table 2) , before moving on to the next step. Table 3 demonstrates the worksheet where instructions for each one of the 13 steps are given. In this point it is important to highlight that in laboratory experiments the order of the given instructions must be followed in a strict way. As a result, the students had to perform each step in the order presented in Table 3 , otherwise, the experiment would not have been successfully completed. At the end of the assessment, each student's data record included an ordered sequence of states, representing her or his performance on the microscopy experiment. The corresponding state sequence is, for exam-ple⟨A, A, C, B, A, B, C, A, ...⟩ . Subsequently, the student performance is modeled using a Markov sequence model called the Markov chain model [38] , which is presented in detail in the following section. In this study, and in the context of an educational setting, we focus on the completion of an experiment executed by three groups of students (observed subjects) after been educated on microscopy, each by a different educational methodology. The experiment is presented as a sequence of steps, each with a number of possible outcomes. As an outcome, we consider the observable student's action, among the possible ones, while performing a certain step in the experiment. If the probability of observing any of the possible outcomes in a single trial/step, depends on a predetermined timeinvariant probability distribution, then we can claim that the Table 1 A brief presentation of the designed educational experiment 1st Phase (Introduction to microscopy)-1 h 1. All students attended a general-oriented tutorial on the light microscopy 2. All students filled in a quiz on microscopy 3. The students were divided in three cognitive balanced groups based on their quiz scores 2nd Phase (Trained on the microscopy experiment) The T-Group attended a live demonstration of the light microscopy experiment, performed by the lab tutor The V-Group watched an instructional video on the light microscopy experiment The VR-Group was trained on the light microscopy experiment by using the Instruction Mode of the Virtual Reality application Onlabs 3rd Phase (In the physical biology lab)-1/2 h 1. All students used a light microscopy experiment to focus on a specimen in the Biology lab 2. In parallel, all students filled in a worksheet designed to assess the acquired skills on operating an optical microscope experiment follows the so-called multinomial distribution model [8, 12] . Furthermore, we assume that the set of the possible outcomes is the same for all steps in the experiment. The multinomial model is not a precise and accurate representation of a science experiment as the different steps of such an experiment might have varied difficulties, and because of that, varied probabilities of the observing outcomes. The coincidence of having different steps with the same difficulty may only happen in case different steps exhibit the same complexity, which is rarely the case in real-lab situations. Another assumption while using a multinomial model, which does not hold true for a science experiment, is the fact that the probability of observing a certain outcome at a particular step along the way, has a data distribution which is independent of the outcomes of the experiment in the nearby positions. Usually what happens in a series of actions is that the probability of observing a certain outcome at a particular position in the sequence is affected by the outcomes found at adjacent positions in the sequence. I finally completed the step but on difficulty State C I couldn't complete the step by myself so I asked for help (from the supervisor or a fellow student) Table 3 The assessment worksheet Step Instruction State Step Instruction State Given the previous discussion, it appears that a Markov sequence model seems to be a more accurate representation of a science experiment. A Markov sequence model assumes that an outcome at a particular position in the sequence depends on the outcome in the previous position. That is, if an outcome Α is observed at the current position, then the probability of observing any one of the outcomes at the next position depends on a predetermined probability distribution. Markov models embody randomly changing systems [13] and it is a popular method to model sequential data. There are four common Markov models used in different situations: the Markov chain, the Hidden Markov Model, the Markov Decision Process and the Partially Observed Markov Decision Process. In our study, we use the Markov chain model which represents a class of stochastic processes of great interest for the wide spectrum of practical applications [7] . In a Markov chain model, there is a certain number of outcomes (states) which are observed while the subject (e.g. the student) is conducting a sequential process (e.g. a science experiment divided into numerous steps). The state space is represented as S and is simply a set containing the possible states of the process, that is, S = s 1 , s 2 , … , s r [31] . Each state corresponds to a possible outcome that can be observed while performing a step in the aforementioned science experiment. The process starts in one of these states and moves on successively from one state to another, creating a sequence of states x where x = x N , x N−1 , … , x 1 and N denotes the number of the steps and the length of the sequential process. The changes in the observed states across the sequence are called transitions. The probabilities associated with various state changes are called transition probabilities. For example, if the x sequence at the (k − 1)th step is at the state s i (current state) and when moving to the next kth step, is at state s j (next state), the transition probability is represented as p ij where: As it is already mentioned, this probability does not depend upon any states other than the current state. The process can also remain in the state it is in, and this occurs with probability p ii . An initial probability distribution, defined on S , specifies the starting state. Usually, this is done by specifying a particular state as the starting state. The overall process is then characterized by a state space, a probability transition matrix describing the probabilities of particular transitions and a probability distribution over the state space for the initial state. By applying many times the Bayes' Rule, P(X, Y) = P(X|Y)P(Y), for any probabilistic model of sequences, we can write the probability of the sequence as: The key property of a Markov chain is that the probability of each symbol x k depends only on the value of the preceding symbol x k−1 , not on the entire previous sequence, e.g. [10] . Therefore, Eq. 2 becomes The main use of Eq. 3 is to find the values for a likelihood ratio test. To do that we use real data from our experiment. Our goal is to be able to discriminate between two Markov chain models and decide upon which one is the most probable to have generated a certain sequence. Please note that in our study we induce three such Markov chain models, each one corresponding to one of the three participating groups (T, V and VR-Group). Given that in our study we observe the students' performance from three different groups, it is safe to run three such discrimination tests ( , each one corresponding to one pair from our three Markov chain models. For demonstration purposes, we present how to use the likelihood ratio test to discriminate between the Markov chain model induced by the VR-Group and the T-Group data. The transition probabilities for each one of these two models were set using the following equations: where count VR g ij and count T g ij is the number of times, state j follows state i in all of the sequences of students in VR-Group and T-Group (the so-called training sets), respectively. These are actually the maximum likelihood (ML) estimators for the transitions probabilities. To use these two models for discrimination we calculate the log-odds ratio (Score): (3) where x k−1 x k are the log-likelihood ratios of corresponding transition probabilities. Please observe that a positive value for the above Score function indicates that the sequence x has been more probably generated from the Markov chain model of the VR-Group. On the other hand, a negative value indicates that the sequence x has been more probably generated from the Markov chain model of the T-Group. In our study, the state space S is a set of three states ( Table 2) . As a result, the state space S = s 1 , s 2 , ..., s r is presented as S = {A, B, C} , where r = |S| = 3 . Each student has to perform individually the microscopy experiment which is divided into 13 steps. As a result, the length of the sequential process N appearing in Eq. 2, is now equal to 13. While following the given instructions and conducting the sequential steps, strictly in the given order, each student moves from state s i to state s j . At this point, we have to highlight that not knowing how to carry out an instruction for a specific step, leads to failure, as there is no option to move on to the next step without having completed the current one. Fortunately, lab supervisors, in physical lab environments, or avatars and help/hint buttons, in intelligent tutoring systems like VR environments, offer learners second chances and the capacity to move on. Based on that, in our study, state A corresponds to the "I completed the step easily" action, state B corresponds to the "I finally completed the step but on difficulty" action, and state C corresponds to the "I couldn't complete the step by myself so I asked for help (from the supervisor or a fellow student)" action ( Table 2) . After the completion of the experiment, sequential data that represent the specific actions each student executes while performing the steps, are produced. For example, in the VR-Group, the student with ID 16, exhibits the following pattern of actions in the 13 steps of the experiment: A, A, A, B, B, B, A, B, C, A, A, A, A. By observing this individual sequence, we count 12 transitions in total, when moving from one state to another. More specifically, we count five transitions from state A to itself, 2 transitions from state A to state B, zero transitions from state A to state C, one transition from state B to state A, 2 transitions from state B to itself, etc. Based on this counting, p AA = 5/7, p AB = 2/7, p AC = 0, p BA = 1/4, p BB = 2/4, etc. These transition probabilities can be organized in a transition matrix, such as the one presented in Table 4 . A transition matrix is a common way to store the transition probabilities for a Markov chain model. The rows of the transition matrix represent the states observed at the current step in the sequence, while the columns represent the states that will be encountered at the next step in this sequence. It is easy to notice that in the transition matrix, the rows of the transition probabilities sum to 1. The transition probabilities of students during the execution of the experiment comprise the basic parameters for a stochastic/probabilistic model such as the Markov chain model. These parameters are typically estimated from large sets of cases often called a training set. For instance, the probability p BC = 1/4 was estimated as the observed frequency of transitions from state B to state C, in the training set of the students' performance data. In this way, we obtain the transition probabilities by counting the nine probabilities which represent the entire space of possible transitions among states in our experiment, as long as the training sequences are not systematically biased towards a peculiar state transition pattern. We expect that these observed frequencies constitute reasonable estimates of the underlining transition probabilities of our Markov chain model. It can be shown that using the frequencies with which a student transitions from one state to another, as the aforementioned transition probabilities, maximizes the total probability of all the sequences given the specific Markov chain model (the likelihood). The specific way for estimating these models is known as Maximum Likelihood Estimation (MLE). Table 5 presents the average transition probabilities matrices for T, V and VR-Group. According to these matrices, a T-Group student who is currently in state C and, therefore, ask for help to move on, has a 63.0% chance of moving into the higher ability state A at the next step. The diagonal entries represent the probabilities of remaining in the same state. For example, a VR-Group student has a 45.2% chance of remaining in state B at the next step. A VR-Group student who is currently in state C has a 0.0% chance of remaining in this state. All three transition probability matrices suggest that there is a degree of inertia regarding state A. The probabilities for persisting in state A are higher than those of shifting to state B or C (77.6%, 78.7% and 86.6% for T, V and VR-Group, respectively). On the other hand, regarding states B and C, the probabilities for shifting to state A are higher than any other probability, (except from the V-Group when the current state is C). The powers of the transition matrices give us interesting information about the experiment as it evolves. Let P T g be the transition matrix of the Markov chain for the T-Group: The ijth entry p (n) ij of the matrix P n T g gives the probability that the Markov chain, starting in state s i , will be in state s j after n steps. For example, let us compute p (2) AC which represents the probability that a Markov chain starting in state A will be in state C after 2 steps. We can easily observe that to move from state A to state C in 2 steps, we can either (a) remain to state A in the first step and move to state C in the second step, or (b) move from state A to state B in the first step and move from state B to state C in the second step or (c) move from state A to state C in the first step and remain to state C in the second step. This can be represented by the following equation: Equation 7 indicates that p (2) AC is given by the dot product between the first row and the third column of the transition matrix P T g . By substituting the corresponding values for the row and column vectors of the aforementioned matrix P T g we get The result above (0.097) is actually the value of the cell (1,3) in the matrix P T g computed below: Powers of transition matrices indicate the long-term behavior of a Markov chain model. In Table 6 , successive powers of an adequate size of the transition probability matrices for T, V and VR-Group are presented. We note that after a certain number of steps for each group, the state predictions are independent of the current state. For instance, looking at the fourth power of the transition matrix for T-Group, P 4 T g , the transition probabilities for the three states A, B and C are 0.756, 0.146 and 0.097 respectively, no matter where the chain started. This is an example of a type of Markov chain called a regular Markov chain. For this type of chain, it is true that the long-range predictions of states are independent of the starting state. Subsequently, we illustrate the long-term behavior of a Markov chain, when it starts in a state chosen by a probability distribution on the set of states. In our study, a probability distribution is a row vector that comprises three components whose entries are non-negative and sum to one. If v is such a probability distribution which represents an initial state of a Markov chain, then we view the ith component of v as representing the probability P x 1 = s i in Eq. 3, that the chain starts in state s i . To denote this, we choose v to be the probability vector with its ith entry equal to 1 and all other entries equal to 0. For example, the probability vector v = (0, 1, 0) is the initial probability distribution when our Markov chain starts in state B. We note that if we want to estimate the distribution of the states after a number of steps, we simply need to multiply the initial probability distribution with the corresponding power of the transition matrix of the Markov chain model. So, if our initial probability distribution vector is v = 1 3 , 1 3 , 1 3 , then the probability distribution vector after three steps is going to be v (3) = v ⋅ P 3 . If in the formula before, we consider the transition matrix P , as the transition matrix for the VR-Group, the probability distribution vector after three steps is going to be v (Table 7) . In other words, although n increases theoretically without a bound, after a certain number of steps its value attains a limit (for example, p n AA becomes 0.756 for the T-Group, 0.719 for the V-Group and 0.820 for the VR-Group). Since state A is related to the response I completed the step easily, it is obviously a state that corresponds to higher student's ability than state B and C, in terms of how to treat and operate successfully a microscope. As a result, the observation where VR p n AA > T p n AA > V p n AA , is an indicator that the interaction with the VR lab environment helped the students in VR-Group become more capable of handling the microscope than their fellow students in other groups. These findings agree with the students' scores in a written assessment test based on the subject of microscopy. In detail, in a zero to ten scale, the students in the T-Group received a score of 6.52 ± 1.31, in the T-Group 6.64 ± 1.19 and in the VR-Group 7.39 ± 1.18 (T-Group: t = − 2.538; df = 29; p < 0.05, V-Group: t = − 4.353; df = 28; p < 0.001, VR-Group: t = − 8.823; df = 23; p < 0.001). Based on the results presented in Table 7 , it is also noteworthy that watching passively an instruction video is a less effective teaching method than attending a face-to-face live demonstration of the experiment, as it is less probable that the V-Group students will be in state A than the students in the T-Group. While in the previous discussion we argued upon the collective performance of the three groups of students, we now proceed to characterize the performance of each individual student, based on the observed sequence of actions in the experiment that he/she conducted. To accomplish this, we make use of Eq. 6 that assigns a score to a student, based on the sequence x that he/she exhibits. The score in Eq. 6 can be used to discriminate between a pair of Markov chain models by summing over the logarithm of ratios of corresponding transition probabilities, in these two models. Let us consider the transition matrices for T and VR-Group presented in Table 5 . To apply Eq. 6, we need to take the ratio of these two matrices, element by element, as shown in the following equation: followed by the logarithm base 2 of the ratio of the two matrices above, which is displayed below: If we now consider any possible sequence observed by a student and by applying Eq. 6 along with the information in matrix LR, we can easily tell whether the sequence has been generated by the Markov chain model of the VR-Group or of the T-Group. Let us consider the sequence that the student with ID 11 exhibits : A, A, A, A, A, A, B, B, B, A, A, A, A . It is obvious that this sequence contains eight AA transitions, one AB transition, two BB transitions and one BA transition. For each one of these transitions, Eq. 6 tells us to consider the corresponding value of the cell from the LR matrix. In this way, based on Eq. 6 for the sequence of the student with ID 11, x = A, A, A, A, A, A, B, B, B , A, A, A, A , we get The positive value (3.045) of the score for the given sequence x, suggests that the sequence conforms to the Markov chain model of the VR-Group. In the following table (Table 8) , we present the data derived from the observed sequences of the 19 students in the VR-Group. For students with IDs 7, 9, 13 and 15, the Score(x) is negative (all the elements that correspond to these students are marked in bold in Table 8 ). This finding indicates that although these three students originally belonged to the VR-Group, the group where the students have the highest probability to perform the experiment without difficulty and without any help, the log-likelihood discrimination test assigned them to the T-Group. The analogue table for the T-Group is presented below ( Table 9 ). For students with IDs 8, 12-18 and 23, the Score(x) is positive. As a result, although these students originally belonged to the T-Group, the log-likelihood discrimination test assigned them to the VR-Group. Figure 2 presents the histogram of the scores for all the sequences of the T and VR-Groups. The T-Group sequences are shown with light blue, the VR-Group sequences with salmon, whereas the purple color results from the overlap of the light blue and the salmon color. This overlap of the values is around zero. Given our ability to discriminate sequences coming from different groups of students, we can easily segment a given (possibly) unknown sequence, to the most probable group path. For example, if we were segmenting the sequence of the student from the T-Group with ID 15 (Table 10) , we would see that in the 1st and from 7th to 12th step his/her actions are closer to the VR-Group while during the 2nd-6th step are closer to the T-Group. AAC AAA AAA CAA A 0.308657 VR Table 9 The T-Group student ID followed by his/her sequence along with the Score(x) computed by Eq. 6 and the assigned group In this paper we investigated the capability of a Markov chain model to evaluate the students' performance in science experiments by tracing the students' knowledge states and by predicting their innate abilities. Consequently, by analyzing the data derived from the Markov model-based techniques, we tried to compare the effectiveness of three different teaching methodologies in communicating experimental skills to students. To accomplish that, three groups of students were trained on microscopy through three different teaching scenarios: the T-group attended a classical face-to-face lab tutorial, the V-Group watched, in class, an instructional video, and the VR-Group entered the Computers lab, downloads a VR educational software and interacts with a simulated microscope. After the educational interventions, the students assessed their gained hands-on skills by performing a microscopy experiment of 13 steps, in real lab situations. There, we recorded through a worksheet, the students' actions and subsequently we analyzed their states. Each time a student performed one of the 13 steps, he/she had to tick on one of the three given states A, B or C, before moving on to the next step. State A corresponded to "I completed the step easily", State B to "I finally completed the step but on difficulty" and State C to "I couldn't complete the step by myself so I asked for help". Using training data from three groups of students, we came up with three different Markov chain models each one represented by a different transition matrix. We made use of the stationary distribution property of the transition matrices to assess the performance of each one of the three groups. We exploited the underlying Markov property of a sequence to build a scoring function that discriminates between a pair of input models. It was also shown that the Markov chain models are a promising solution in educational lab environments to model students' actions and make the prediction throughout the experiment. The results indicate that examining sequential data from the experiment could lead to dynamic evaluation of student experimental skills before he/ she even finishes the specific experiment. The three Markov chain models presented in this project could assign every student to the T, V or VR-Group, (or in other words, to the low, medium, or high ability group), giving the instructor the potentiality to make a beneficial intervention and provide the students' efficient feedback. Our results indicated that a VR-oriented learning procedure is more beneficial and more effective in helping the students to acquire the necessary experimental skills for a specific lab experiment. Students from the VR-Group were better prepared and exhibited a higher probability to conduct the steps of the experiment easily and without any help than the students in V and T-Group. Although nowadays video is the most important digital media on the Internet, the role of the watchers is still the same; they are a passive audience. Therefore, according to the Markov chain analysis, the students in the V-Groups had a lower probability to conduct the steps easily and by themselves than the students in T-Group who also attended passively a demonstration of the experiment but this demonstration was live and they had the opportunity to stop the tutor and pose questions. The average transition probability of the V-Group students to move on to the higher ability State A at their next step, when they were in State C, was 42.1% whereas this probability was 63.0% and 90.0% for students in T and VR-Group, respectively. Furthermore, the probabilities for persisting in State A were higher for VR-Group (86.6%) compared to the percentages of 77.6% and 78.7%, for T and V-Groups, respectively. Our findings reinforce the point of view that educational institutions should take advantage of technological innovations such as VR, and enrich the conventionally applied learning methods and curriculums in an attempt to successfully engage their students and satisfy the initially defined learning outcomes. A VR application such Onlabs could be offered to students that attend laboratory courses to engage them in a creative way into the educational procedure and prepare them for the real lab environment. The higher performance of the VR-Group was probably a consequence of the active participation, the high interactivity and the individualization, features that are highlighted when interacting with a VR environment. Furthermore, models like the one suggested in this study, could be a useful tool for educators to predict their students' performances in science experimentation through tracking their knowledge state. Such Table 10 The probable group path for the student with ID 15 from the T-Group Step Score( Experimental data indicated the good fitness of our models to the training data. To validate the accuracy of our findings, further experimentation with larger training data sets is necessary. A focus of the future research will be on designing an empirical study where a HMM can be applied to model students' emotional states like frustration, anxiety or interest, when performing a science experiment in a physical lab. Acknowledgements Not applicable. Our project was supported by: Dr. EP. She is currently a Postdoctoral Researcher in the School of Science and Technology at the Hellenic Open University, with a PhD in the field of Natural Sciences and Education. She designed and implemented the educational scenario and the assessment material. She organized and supervised the whole project and she was the instructor during the implementation of the teaching methodologies. As she has a degree in Chemistry and a MSc in Pharmacy, she has great experience in science labs and science experiments. Finally, she was the major contributor in writing the manuscript and she participated in the interpretation of the data from an educational aspect. Dr. DK. He is a professor in the School of Science and Technology at the Hellenic Open University in the field of the artificial intelligence. He is supervising the Onlabs team, an interdisciplinary team that develops the VR-based software Onlabs which is used in this research (https:// sites. google. com/ site/ onlab seap/). As a result, he is the major contributor in creating the new software used in this work. Dr. CTP. He is a Professor at the Department of Primary Education, at the University of Patras in the field of the Educational Technology, and Development and Implementation of the Educational Material. He participated in designing the teaching methodologies and he closely supervised the evolving experimental procedure as his class (students enrolled in Computers and Education course) participated in this project. He contributed to the interpretation of data. Dr. VV. He is a professor in the School of Science and Technology at the Hellenic Open University, in the field of Data Management, and the director of the Big Data Analytics and Anonymization Lab. He is an experienced professor with a demonstrated history of working in the higher education industry, skilled in Big Data, Learning Analytics and Lecturing. He has a main contribution to the concept of this work as he has a deep knowledge on statistical models, such as the Markov Chain model used in this study, and he organized and implemented the statistical analysis in our study. He was the major contributor in interpreting the generated data. All authors read and approved the final manuscript. Funding Not applicable. The datasets generated during study are available from the corresponding author on reasonable request. Code Availability Not applicable. understanding test takers' choices in a self-adapted test: a hidden markov modeling of process data Modeling learning patterns of students with a tutoring system using Hidden Markov Models Virtual & augmented reality: Understanding the race for the next computing platform Control learning: present and future Improving biotech education through gamified laboratory simulations Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: a review of the empirical research The theory of stochastic processes Introduction to computational genomics Peppy: a virtual reality environment for exploring the principles of polypeptide structure Biological sequence analysis: Probabilistic models of proteins and nucleic acids The encyclopedia of virtual environments-education Statistical Distributions Markov chains: from theory to implementation and experimentation A practical guide to developing virtual and augmented reality exercises for teaching structural biology An effective LA approach to predict student achievement Polarity, emotions and online activity of students and tutors as features in predicting grades virtual reality industry report Stock market forecasting using hidden Markov model: a new approach Hidden Markov Bayesian game with application to Chinese education game The role of laboratory in science teaching: Neglected aspect of research The laboratory in science education: foundations for the twenty-first century The role of faculty mentors in the research training of counseling psychology doctoral students A review of the use of virtual reality head-mounted displays in education and training Deep joint spatiotemporal network (DJSTN) for efficient facial expression recognition. Sensors Using hidden Markov models to characterize student behaviors in learning-by-teaching environments The application of a Markov Chain in educational planning A learning analytics methodology for detecting sentiment in student fora: a case study in distance education Efficient facial expression recognition algorithm based on hierarchical deep neural network structure Extended Reality Interactive Laboratories Literature review in games and learning. Bristol: Futurelab Applied statistics for bioinformatics using R Video-based face recognition using adaptive Hidden Markov Models A learning analytics methodology for student profiling Hands-on, simulated, and remote laboratories: a comparative literature review A structural equation modeling investigation of the emotional value of immersive virtual reality in education Simulation based virtual learning environment in medical genetics counseling: an example of bridging the gap between theory and practice in medical education Virtual simulations as preparation for lab exercises: assessing learning of key laboratory skills in microbiology and improvement of essential non-cognitive skills Investigation of a remarkable example of dependent trials An analysis on genetic algorithms using Markov process with rewards Blending Physical and Virtual manipulatives: An effort to improve students' conceptual understanding through science laboratory experimentation Mechanism to capture learner's interaction in VR-based learning environment: design and application Methods of assessing the students' performance upon utilization of a virtual reality educational tool for laboratory biology courses Achieving educational goals in microscopy education by adopting Virtual Reality labs on top of face-to-face tutorials 3D virtual reality laboratory as a supplementary educational preparation tool for a biology course Implementation and evaluation of a three-dimensional virtual reality biology lab versus conventional didactic practices in lab experimenting with the photonic microscope Assessing the impact of virtualizing physical labs A Markov chain model in teachers' decision making A tutorial on HMM and selected applications in speech recognition The learning effects of computer simulation in science education Hidden Markov model-based speech emotion recognition Adaptive 3D Virtual Learning Environments-A Review of the Literature Unsupervised discovery of student strategies Computer simulations to support science instruction and learning: a critical review of the literature Probabilities and prediction: modeling the development of scientific problem solving skills A Markov chain analysis on simple genetic algorithms Virtual laboratories in biology, biotechnology and chemistry education: a literature review The educational application of virtual laboratories in archaeometry Predicting student performance in an educational game using a hidden Markov model Embodied experiment of levitation in microgravity in a simulated virtual reality environment for science learning Exploiting extended reality technologies for educational microscopy On the equivalence between bimodal and unimodal students' collaboration networks in Distance Learning Hidden Markov model decomposition of speech and noise The envisioning report for empowering universities. Maastricht: EADTU; 2021 Computer gaming and interactive simulations for learning: a meta-analysis An analysis of the use and effect of questions in interactive learning-videos A conceptual basis for educational applications of virtual reality Exploration of an interactive "Virtual and Actual Combined" teaching mode in medical developmental biology StoolNet for color classification of stool medical images Physical versus virtual manipulative experimentation in physics learning Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations The authors declare that they have no conflict of interest and no competing interests.