key: cord-0202024-js0nb90l authors: Muzaffar, Abdul Wahab; Tahir, Muhammad; Anwar, Muhammad Waseem; Chaudry, Qaiser; Mir, Shamaila Rasheed title: A Systematic Review on Online Exams Solutions in E-learning: Techniques, Tools, and Global Adoption date: 2020-10-13 journal: nan DOI: nan sha: fca16d1ba5c09023255e3de725a4a8b52a7e11a1 doc_id: 202024 cord_uid: js0nb90l E-learning in higher education is exponentially increased during the past decade due to its inevitable benefits in critical situations like natural disasters, and pandemic. The reliable, fair, and seamless execution of online exams in E-learning is highly significant. Particularly, online exams are conducted on E-learning platforms without the physical presence of students and instructors at the same place. This poses several issues like integrity and security during online exams. To address such issues, researchers frequently proposed different techniques and tools. However, a study summarizing and analyzing latest developments, particularly in the area of online examination, is hard to find in the literature. In this article, an SLR for online examination is performed to select and analyze 53 studies published during the last five years. Subsequently, five leading online exams features targeted in the selected studies are identified and underlying development approaches for the implementation of online exams solutions are explored. Furthermore, 16 important techniques and 11 datasets are presented. In addition, 21 online exams tools proposed in the selected studies are identified. Additionally, 25 leading existing tools used in the selected studies are also presented. Finally, the participation of countries in online exam research is investigated. Key factors for the global adoption of online exams are identified and investigated. This facilitates the selection of right online exam system for a particular country on the basis of existing E-learning infrastructure and overall cost. To conclude, the findings of this article provide a solid platform for the researchers and practitioners of the domain to select appropriate features along with underlying development approaches, tools and techniques for the implementation of a particular online exams solution as per given requirements. E-learning has shown promising results during critical circumstances like natural disasters, wars, and pandemics like COVID 2019. For that reason, numerous methodologies and learning management systems have been introduced during the last three decades in order to deliver and promote E-learning successfully [1] . The usage of E-learning is continuously growing as yet, which creates opportunities as well as challenges from online lecture delivery, content management, and handling the online exams effectively. Particularly, different technological advancements with reliable and high-speed internet infrastructure allows the exploitation of advanced image processing and machine learning techniques for the realistic accomplishment of educational activities through E-learning [2] . This urges colleges and universities for the adoption of E-learning as a reliable educational platform. Online examination is an integral part of E-learning solutions for the genuine and fair assessment of students' performance [3] . The design and execution of online exams are the most challenging aspects in E-learning. Particularly, online exams are usually conducted on E-learning platforms without the physical presence of students and instructors at the same place. This creates several loopholes in terms of integrity and security, of online exams. For example, the verification of an examinee is extremely problematic in online environment particularly in the absence of continuous monitoring. Moreover, online exam settings are highly supportive for cheating as thousands of online information resources are accessible to students without any check and balance. Furthermore, it is very difficult to ensure the high speed and continuous availability of internet connection for all students during exams. The development of effective question banks, impartial setting of exam papers and marking of descriptive questions are few more challenges in online exams. All aforementioned issues eventually compromise the integrity, security, and objectivity of online exams. To confront the concerns accompanied by online exams, researchers frequently propose different solutions to ensure integrity and security. There exist several studies [4] where biometric techniques have been utilized for the trustworthy verification of examinee. Moreover, researchers also applied machine learning / artificial intelligence approaches to develop applications for the prevention of cheating during online exams [5] . Furthermore, different approaches have also been proposed [6] to automatically generate question banks for the effective assessment of student's performance in online exams. In addition to this, several exam management systems with broader features have been developed (e.g. Secure Exam Environment [7] etc.) for the efficient online exam execution. The proposed techniques and tools certainly improved the integrity, security and fairness of online exams. In literature, there exist several studies [1, 8] where intensive reviews and surveys are performed for the investigation of E-learning as a whole. On the other hand, there are few attempts to analyze particular aspects of online examination like user authentication [4] , relationship to student learning [9] etc. However, a study systematically analyzing and summarizing across-the-board online exams developments is hard to find in the literature to the best of our knowledge. As online exam is a critical part of E-learning, it is a need of the day to investigate and summarize the latest online exams progress within a single study. To achieve this, a Systematic Literature Review (SLR) [10] is performed in this article to find the answers of the following questions: RQ9. What are the major challenges in current online exams research and how to improve upon these challenges? To answer aforementioned questions, this article performs SLR to select and analyze 53 studies , which are published during Jan 2016 to July 2020. The outline of SLR is shown in Figure 1 . Particularly, a review protocol is developed (Section 2.2) with inclusion and exclusion criteria (Section 2. This research is conducted by utilizing the guidelines of Systematic Literature Review (SLR) [10] where development of review protocol is an integral and methodical design of review protocol (Section 2.2). The selected studies are classified into three major categories (Section 3.1) in order to simplify the data extraction and synthesis process. The definitions of these categories are as follows: 1. Biometric Category: The verification of examinee and prevention of cheating are two major challenges in online examination. In this regard, researchers frequently utilized biometric features like fingerprints, face, and head movements etc. in order to provide reliable solutions. For example, authors in [11] utilized head movements to analyze the abnormal behavior of examinee. All such studies where biometric features are utilized for particular purposes are placed under Biometric Category. There are studies where a complete software application for online exams are proposed for different purposes. For example, G. Frankl et al. [7] propose a complete software application named "Secure Exam Environment" for the execution of online exams. Similarly, in another study [12] , ViLLE tool is developed to accomplish the automated assessment in online exams. All such studies where a complete tool is developed to achieve particular online exams objectives are placed under Software Applications Category. Category. On the other hand, there are few studies (e.g. [14] ) where conceptual framework is proposed. Furthermore, few studies proposed certain techniques (e.g. improved online exams user interface [15] etc.) which do not belong to either Biometric or Software Applications categories. Such multidisciplinary studies are also placed in the General category. The development of review protocol involves six steps as per the standard SLR guidelines [10] . The first two steps (i.e. background and research questions) are already performed in the introduction (Section 1) of the article. The details of the remaining four steps (i.e. Inclusion and exclusion criteria, Search process, Quality assessment and data extraction / synthesis) are given in subsequent sections. The inclusion and exclusion criteria are the most important part of SLR. Particularly, the studies are selected or rejected on the basis of this criteria. We develop 6 parameters for the inclusion and exclusion of studies as follows: 1. Subject: The selected study must belong to online exams substantially. Studies dealing with E-learning as a whole but discussing online exams marginally should be discarded. Description: In this SLR, online exams is a major subject. Therefore, the study should only be included where the improvement in online exams is the major concern. In fact, there exist studies (e.g. [16] ) where solution for different aspects of E-learning is proposed and online exams are discussed / researched marginally. Such studies are excluded as online exams is a major area of research for this SLR. The study should only be selected if some genuine framework, technique, or software / prototype is proposed for the improvement of online exams. Description: This SLR only considers studies that are dealing with application research. Particularly, only those studies are selected where some genuine technique, framework or software / prototype is proposed to improve certain aspect(s) of online exams. In this context, review studies (e.g. [9] ) are not considered. Furthermore, empirical studies dealing with some particular hypothesis without any genuine proposal (e.g. [17] ) are also discarded. [2] proposed machine learning approach for the analysis of online exams. However, all the details are summarized in one page and validation is discussed in only few lines. In another study [5] , the cheating prevention algorithm for online exams is proposed with sufficient details, however, the validation information is totally missing. Consequently, all such studies with insufficient / missing validation details are discarded during the SLR. 6 . Repetition: Multiple studies having similar research contents are analyzed first and only one with most reliable contents is selected. Description: In the literature, there exist studies that present similar research contents. Particularly, the researchers usually propose the initial technique in some relevant conference. Subsequently, the full approach including complete implementation details are published in journal. For example, Ullah el al. [18] initially proposed dynamic profile questions approach for the authentication of online exams. Subsequently, authors published the extended version in [19] with complete details. In this SLR, we discard studies having almost similar research contents and we only select one of them with most reliable contents e.g. in aforementioned case, we select study [19] . We performed this SLR on the basis of aforementioned inclusion and exclusion parameters. Particularly, the study is only selected if it completely follows all inclusion and exclusion parameters. The study is discarded even if a single inclusion and exclusion parameter is violated. We performed the search process through six databases (Section 2.2.1) in order to select the relevant studies as per inclusion and exclusion criteria. The summary of search terms used in the search process is given in Table 1 Similarly, we also used advance search options like "Where Title or Abstract Contains" etc. to speed up the search process. After applying several filters and advance search options, we were able to get optimum and most relevant results that could be completely analyzed. For example, we only got 131 search results regarding "Online Exams" search term from IEEE database after applying different filters e.g. publication year between 2016 to 2020 . Similarly, we got 46 results for "e-learning Exams" search term as given in Sr. # 2 of Table 1. In the same way, we applied different filters in each scientific repository and got the filtered results as given in third column of Table 1 . Initially, we used simple and most relevant search terms like "Online Exams", "e-learning Exams", "Digital Exams" and "Electronic Exams" as given in Sr. # 1 to 4 of Table 1 . Once we analyzed the search results of these simple terms, we found certain keywords that are frequently associated with online exams subject and could be utilized to find the relevant studies effectively. For example, we found that the terms like "Proctoring" and "Biometric" is frequently utilized while performing the authentication of examinees in online exams. Similarly, the terms like "Assessment" and "Question Bank" are frequently utilized in the context of online exams. Therefore, we developed more advanced and intelli- gent search terms like "Exam Proctoring", "e-learning Assessment", "Exams Biometric" and "Online Question Bank", as given in Sr. 5 to 8 of Table 1 , in order to get relevant studies. These advanced search terms enabled us to find and select the studies, which cannot be picked through simple search terms. The detailed investigation of search results was performed for the selection of studies as per inclusion and exclusion criteria. Particularly, the parameter # 3 (Publication Year) and parameter # 4 (Publisher) of inclusion and exclusion criteria were already ensured during search process. However, to guarantee the compliance of other parameters, the search results were systematically analyzed through different steps as shown in Figure 2 . • Overall, we analyzed 1588 search results. Initially, we checked the titles of studies to confirm the relevance as per inclusion and exclusion criteria. The title of few studies clearly indicated their irrelevance with the given subject i.e. online exams. Therefore, we excluded 509 such studies by only analyzing the titles. • In the next step, the abstracts of remaining 1079 studies were investigated. It is observed that the abstracts of few studies clearly violating parameter # 1 (Subject) and parameter # 2 (Application Research) of inclusion and As a result, we found 332 studies violating parameter # 5, therefore, we excluded them as well. • At this stage, we had 126 remaining studies. We performed detailed analysis of these studies by reading each section to ensure their compliance with all six parameters of inclusion and exclusion criteria. We found that few studies were still violating parameter # 5. Furthermore, we also found repetition of few studies that was the violation of parameter # 6 (Repetition). Therefore, we excluded 81 such studies and selected 45 remaining studies, which were fully compliant with all inclusion and exclusion parameters. • In the final stage, we executed snowballing process on 45 selected studies. Particularly, the citations of selected studies were analyzed to find and pick additional relevant studies. As a result, we found 22 studies that seemed relevant in the given research context. After detailed analysis, we selected 8 studies, which were fully compliant with inclusion and exclusion criteria. Finally, 53 studies (45 + 8) were selected for further analysis in order to get the realistic and trustworthy answers to our research questions. We systematically developed the inclusion and exclusion criteria (Section [20] , [21] , [11] , [22] , [23] , [24] , [12] , [15] 8 2 2017 [25] , [26] , [27] , [28] , [29] , [30] , [31] , [32] , [33] , [34] , [35] [36], [37] , [38] , [39] , [40] , [41] , [42] , [43] , 15 [14] , [44] , [45] , [46] , [47] , [19] , [48] 4 2019 [49] , [13] , [50] , [51] , [52] , [53] , [54] , [55] , [56] , [57] , [58] , [59] , [ The six most renowned and trustworthy scientific databases are considered for the selection of studies as per parameter 4 of inclusion and exclusion criteria (Section 2.2.1). This significantly improves the quality of this SLR. The distribution of selected studies with respect to scientific databases is given in Table 3 . It can be seen from Table 3 The type (i.e. Journal or Conference) of selected study is another important factor for ensuring the quality of SLR. Although, we tried to select journal studies as much as possible, we were able to find 15 journal studies (out of 53), which were fully compliant with the inclusion and exclusion criteria. The distribution of selected studies on the basis of type is shown in Figure 3 where 28% of selected studies are from reputed journals while remaining 72% studies are from [36] , [20] , [13] , [21] , [25] , [50] , [11] , [37] , [22] , [38] , 22 [61] , [51] , [62] , [52] , [26] , [27] , [23] , [24] , [53] , [54] , [39] 2 ELSIVER [65] , [15] , [66] ,[59] 4 3 SPRINGER [28] , [14] , [44] , [29] , [30] , [58] , [45] , [31] , 15 [63] , [46] , [47] , [64] , [32] , [19] , [33] 4 ACM [40] , [41] , [55] , [12] , [42] , [43] , [56] , [ conferences. It is important to note that two book chapters (i.e. [28, 29] ) are selected from springer, however, we placed these book chapters in conference studies to keep the discussion simple. The data extraction and synthesis process are executed after the selection of studies as per inclusion and exclusion criteria (Section 2.2.1). This leads to find the realistic answers to research questions as defined in Section 1. The data extraction template is defined, as given in Table 4 , to extract and analyze the elements of concerns from selected studies. Particularly, the primary elements are first extracted from selected studies as given from Sr. # 1 to 4 of Table 4 . Subsequently, the data extraction with synthesis is performed to extract relevant elements, which are essential to answer RQs. For example, categorization of studies is performed to give the answer of RQ1. Similarly, the data extraction with synthesis is carried out for other important elements as given from Sr. # 5 to 12 of Table 4 . To this point, we have developed a review protocol analytically and selected the 53 studies accordingly. Furthermore, data extraction is performed to extract the relevant information from selected studies. This leads to compile the results precisely as given in subsequent section. This section presents precise results to provide authentic answers to research questions. Particularly, the primary categorization of studies is performed in Section 3.1. Moreover, the leading online exams attributes and underlying development approaches are identified in Sections 3.2 and 3.3, respectively. Furthermore, the proposed techniques / algorithms are presented in Section 3.4. In addition to this, the leading tools used and proposed by researchers are given in Section 3.5. Finally, the important datasets and summary of countries participation in online exams research are provided in Section 3.6 and Section 3.7, respectively. The three main categories are defined in Section 2.1 to perform the primary classification of selected studies. These categories are: 1) Biometric 2) Software Applications and 3) General. The categorization results are given in Table 5 . It can be seen from Table 5 software tool using biometric features (e.g. [36] ) are placed in General category. There are several associated attributes while executing the online exams. simultaneously in the study [13] . The aforementioned online exams feature classes, targeted in the selected studies, are given in the Table 6 . It can be seen from the Table 6 that Question Behavior (19 Studies) are the most frequently targeted features in the selected studies. Particularly, the researchers commonly tried to improve the question bank generation and automatic assessment of answers in online exams. For example, Zhengyang Wu et al. [66] proposed an AI approach for the effective generation of question bank in order to improve the overall assessment in online exams. In another study [39] , authors proposed a novel approach for the detection of plagiarism in online exams to automatically evaluate the answers swiftly. Similarly, Verification and Abnormal Behavior feature class is also an attractive area for researchers where several techniques have been proposed to ensure the integrity of online exams. For example, Diedenhofen et al. [33] developed PageFocus JavaScript to assess the abnormal events in examinee's system for cheating prevention. Similarly, there are studies (e.g. [49, 36] ) where the verification / authentication of examinee is ensured. We identified 9 studies to be categorized in the Other class where different feature classes are targeted simultaneously. For example, Ghizlane et al. [13] proposed an approach for continuous monitoring of online exam where examinee authentication is performed through face recognition technique. A security model is also proposed to ensure communication between sever and clients remains secure. In this way, the authors are simultaneously targeting Verification and Abnormal Behavior as well as Security classes. Similar is the case with study [29] . On this basis of [63] , the authors developed a web-based online exam system using PHP programming language. It can be seen from Table 7 In the selected studies, several techniques / algorithms have been proposed to achieve a particular objective for the improvement of online exams. The summary of leading techniques / algorithms proposed in the selected studies is given in Automata and π-calculus, to assess the violations in online exams. It is important to note that we only present leading techniques / algorithms in Table 8 and trivial proposals are not included for simplicity. For example, Abisado et al. [40] proposed simple divide and conquer algorithm for the detection of abnormal behavior during online exams. In another study [35] , standard logistic regression model without any significant variation is used to predict cheating in online exams. Therefore, we do not include such trivial techniques in Table 8 . It is important to mention that proper information regarding proposed technique / algorithm is not available in some of the studies. For example, S. Aisyah et al. [36] developed online exams authentication system with two components i.e. authentication and supervision. However, authors did not provide any substantial information about underlying techniques / algorithms employed for the system development. Therefore, such studies are not included in Table 8 . This section presents the tools that have been proposed as well as utilized Table 9 . The tool name is given in second column of Table 9 . Results of the proposed tools, as given in Table 9 , are really interesting. Overall, we identified 21 tools for online examination where 3 tools belong to Verification and Abnormal Behavior feature, 1 tool belongs to Security feature and 11 tools belong to Question Bank Generation and Evaluation feature. Moreover, 3 tools target both Verification and Abnormal Behavior as well as Security features whereas 3 tools support all three online exams features. Therefore, it can be concluded that Question Bank Generation and Evaluation is the frequently targeted feature in the proposed tools. It is important to note that the researchers claim the development of tool in few studies (e.g. [27] etc.). However, proper details about the proposed tools are missing in such studies. Therefore, we do not include such studies and their proposed tools in Table 9 due to lack of sufficient relevant information. For example, S. Prathish et al. [22] claim the development of intelligent system to monitor online exams where multi-modal biometrics are utilized. Authors explained the proposed approach properly, however, the details about the developed tool (e.g. interface, language / platform used for implementation etc.) are not provided. In addition to this, there are few studies particularly dealing with novel techniques / approaches without the development of tool. For example, Kassem et al. [31] proposed an interesting approach where formal methods are utilized to ensure the integrity of online exams. The development of tool with proper interface is usually not required for such proposals. Therefore, such types of studies are also not included in Table 9 . The results pertaining to the availability of proposed tools are really surprising. We found only one tool (i.e. ViLLE [12] ) where a web link is available to download few components (without actual source code). Other than ViLLE, rest of the tools proposed in studies under consideration do not provide any availability information (e.g. download link, source code etc.). Therefore, the proposed tools are of least significance for researchers and practitioners since further customization / extension or even evaluation is not possible. The tools availability results are really surprising and require further investigation. Therefore, we performed search (Google) for each proposed tool to find any additional information or web link. However, we were unable to find some proper download or source code link for any of the proposed tool. In fact, we only found very basic information about few proposed tools. For example, we found a login link 2 for Online Item Exam System [21] , where neither the language was known to us nor username and password was available due to which further evaluation of the system was also not possible. Similarly, we found a web link 3 where basic information regarding Online Exam Proctoring (OEP) system [25] was provided and relevant dataset was also available. However, it was not possible to download OEP or its code for evaluation purposes. So far, we presented the tools that have been proposed and developed as part of a research. However, it is equally important to highlight the existing tools that have been used in the selected studies for the implementation of proposed techniques and tools. This facilitates researchers and practitioner of the domain to select right tool as per requirements. Therefore, we present 25 important exiting tools that have been used in the selected studies as given in Table 10 . The tool name is given in the second column and the purpose of tool is given in the third column of is a formal verification tool, which was used in [31] for the formal analysis of violations in online exams. To summarize, all aforementioned existing tools are utilized in the existing studies to achieve particular objective. It is important to mention that few studies did not provide any information about the languages and tools, which were used for implementation. For example, Mahatme et al. [23] proposed fuzzy logic-based approach for the intelligent classification of question bank, however, the information about the implementation language / tool was not given. Therefore, information about the utilized tools for such studies is not available in Table 10 . Similarly, in few studies (e.g. [38, 33] etc.), very basic approaches like HTML, CSS, and JavaScript etc. were utilized for implementation and we therefore did not include this information in Table 10 for simplicity. Tunisia. Such studies only perform the integration with Moodle, therefore, we did not include Moodle and corresponding studies in Table 10 . Datasets are really important for reliable validation of a proposed technique / tool. Therefore, trustworthy datasets are essential while authenticating the outcomes of a proposal. We identified 11 datasets used / proposed in the selected studies for validation as given in Table 11 . The dataset name is given in second column and characteristics of dataset are given in third column of Table 11 . Characteristics of dataset include format (Video, audio, text etc.), number of records and purpose (i.e. targeted online exam feature through dataset). The availability of dataset (i.e. Public, Private and Not-Applicable -N-A) is given in fourth column of Table 11 . Finally, the reference of relevant study where the given dataset is actually utilized / proposed, is given in last column. We found six publicly available datasets as given in Sr. # 1 to 6 of Table 11 . Out of these six public datasets, only Online Exam Proctoring (OEP) dataset was newly constructed in [25] [16] whereas the rest were benchmark datasets that were reused by [38, 55, 28, 64, 66] . The reference of each publicly available dataset is provided against the name (second column) for further investigation. On the other hand, we identified five datasets (Sr. # 7 to 11 of developed a dataset for validation by utilizing different existing online exam items like teacher assistance, government, and company exams. However, the details of developed dataset were not properly explained and availability information (e.g. download link etc.) was totally missing. Likewise, authors in [11] developed a dataset comprising of 6 videos with 25311 frames, however, the availability details were totally missing. In another study [41] , authors claimed the development of a dataset, but relevant details including total number of records were missing. It can be analyzed from students with four exam occurrences. In another study [58] , survey comprising 20 teachers has been conducted to validate the usability of the proposed system. To analyze the participation of countries in online exams research, the selected studies were thoroughly investigated to identify the contributing institutes and corresponding countries. We identified 25 countries that contributed to online exams research as given in Table (Table 12 ) on the basis of developed and developing countries is shown in Figure 5 . It can be analyzed from Figure 5 that the 70% (37 studies) of the contributions in online exams are from developing countries during past five years. On the other hand, 30% (16 studies i.e. [25, 11, 51, 55, 12, 42, 28, 44, 58, 31, 19, 33, 65, 59, 34, 35] ) contributions came from developed countries. Therefore, it seems that the adoption of online exams is more frequent in developing 24 https://www.imf.org/external/index.htm and can be applied in real environment. To summarize, the proposals like [25] are actually implemented in developed countries. On the other hand, such pro-posals (e.g. [25] cannot be applied in developing countries due to financial issues and unstable infrastructure. For example, the solution in [16] Certain characteristics of the aforementioned factors are highly important while adopting particular online exams features in real environment. These factors are directly linked with the system's overall cost, which is a major concerning element for most of the developing countries. In this context, it is required to investigate the requirements and effects of leading online exams features (Section 3.2) on each factor. This facilitates the selection of a right online exam system for a particular country on the basis of existing E-learning infrastructure and overall cost. Therefore, we performed comparative analysis of key online exam features with respect to adoption factors as given in Table 13 . The important online exam features are given in first column of Table 13 . Moreover, four important factors (i.e. Network Infrastructure, Hardware Requirements, Implementation Complexity and Training Requirements) along with aforementioned characteristics are given in second, third, fourth and fifth columns of Table 13 , respectively. Finally, respective overall cost is given in last column. To evaluate the requirement / effect of each feature with respect to a particular factor, three symbols / abbreviations are utilized. The tick symbol ( ) represents that a specified characteristic of a given factor is sufficient for the implementation of a particular online exam feature. On the other hand, the cross symbol (×) represents that a given online exam feature cannot be implemented through the specified characteristic of a factor. Finally, the essential characteristic of a particular factor, which is at least required for the implementation of a given feature is represented through Mandatory -(M) abbreviation as shown in Table 13 . It is important to note that four key online exams attributes are already defined in Section 3.2. Here, each attribute is logically divided into two groups in order to perform realistic comparative analysis as shown in Table 13 . For Based and Application Based groups where Biometric approaches (e.g. [25] ) utilize examinee images, videos etc. to evaluate examinee verification and / or abnormal behavior. On the other hand, the application-based approaches (e.g. [33] ) utilize examinee system's events (e.g. Browser Window Status etc.) to detect cheating abnormalities. In addition to this, Security attribute is classified as basic and advanced where advanced security provides complete and secure communication mechanism between all servers and clients of online exam system (e.g. [64] ). In the same way, Question Bank Generation and Evaluation attribute is divided into two groups (i.e. ML / AI based and Traditional) where the ML / AI based approaches (e.g. [41] ) apply advanced techniques for the generation and assessment of question bank. In contrast, Traditional approaches (e.g. [51] ) utilize different languages (e.g. Java, PHP etc.) without employing any latest ML / AI techniques. Finally, Usability attribute can be divided as Good or Fair where Good usability (e.g. [15] ) significantly improves the interaction of examinee with a given online exam system. It can be analyzed from Table 13 Therefore, training requirements for biometric solutions are usually high. On the basis of overall comparative analysis, it can be concluded that higher costs are required for the implementation of biometric solutions as given in Table 13 . On the other hand, Verification and Abnormal Behavior feature can be achieved through Application based approaches with low network infrastructure. Particularly, application-based solutions do not require the transfer of huge real time data between servers and clients. A typical example of such solution is PageFocus [33] where the behavior of examinee's browser window is analyzed to detect cheating abnormalities. Similarly, application-based solutions do not require large hardware requirements and system can be operational with minimum hardware. Furthermore, traditional languages like PHP, JavaScript etc. can be utilized for system development without employing any special ML / AI approaches. Therefore, application-based solutions can be implemented with low implementation complexity as given in Table 13 . In addition to this, application-based solutions usually do not entail intensive training requirements and system can be operational with basic training. Finally, the overall cost of application-based systems is much lower than the biometric based solution. The basic security feature in online exams can be achieved with low network infrastructure and small hardware requirements. It can be attained with low implementation complexity without employing any special trainings. On the other hand, advanced security aspects of online exams usually require good network infrastructure as continuous internet availability is important. Moreover, different types of firewalls, servers may be required for highly secured systems, therefore, advanced security may have medium hardware requirements. Furthermore, medium level implementation complexity is required for advanced security and basic training of network / system engineer may also be needed for the execution of system. On the basis of security feature analysis, it can be concluded that the implementation of basic and advanced security in online exams may require lower and higher overall costs, respectively. requirements. Moreover, the implementation complexity of such approaches is also low. These approaches can operate without performing any major training. Finally, the overall cost of such approaches is relatively low as compared to ML / AI based approaches. Usability feature in online exam systems typically is not directly linked with network infrastructure and hardware requirements. However, to ensure the availability of online exam system, usability feature may require low network infrastructure. On the other hand, any particular hardware requirements are not usually involved, and good usability can be achieved even with very limited hardware. Particularly, Good usability can be achieved with low implementation complexity while fair usability is naturally evolved during system development without employing any special implementation strategy. In addition to this, Good usability leads to simple and self-explanatory execution of online exam system where examinee and invigilator training is not usually required. On the other hand, basic training of a system is commonly required in case of fair usability. Overall, the lower costs are involved while achieving good usability in online exams systems. The analysis of major online exam features with respect to key factors, as given in Table 13 , is a significant step towards the adoption of online exams globally. Given the facts in Table 13 , the institutes and countries may initiate Table 13 . It is important to note that analysis performed in Table 13 is based on general observations, which are derived from the investigation of selected studies. For example, the observation like "biometric based studies usually require good network infrastructure" is based on the facts, which are given in several selected studies e.g. [25, 11] etc. Similar is the case with other observations. Therefore, the analysis performed in Table 13 is authentic. In fact, it is a significant step towards the global adoption of online exams. To this point, the selected studies are thoroughly investigated and required results are precisely presented in Section 3. Table 3 . The distribution of studies with respect to publication year is provided in Table 2 . The distribution of studies with respect to publication type (i.e. conference or Journal) is given in Figure 3 . The classification of selected studies with respect to major categorizes (i.e. Biometric, Software Applications and General) is performed in Section 3.1 (Table 5) . Table 6 . The summary of targeted features, based on the results of Section 3.2, is shown in Figure 6 . It is concluded from Figure 6 Table 7 . The summary of results, based on Table 7 , is given in Figure 7 . Answer: We overall identified 11 datasets as given in Section 3.6 (Table 11 ). Six datasets are publicly available whereas the accessibility information of remaining five datasets is unknown. Further details are presented in Section 3.6. RQ7. What are the main countries contributed / participated in the online exam research? Answer: We overall identified 25 countries along with respective institutes that have participated for online exams research in the selected studies, as given in Table 12 . It has been analyzed that most of the studies (62%) belong to Asian countries followed by European countries (17%). The complete details are available in Section 3.7. Though, we have carefully followed standard SLR guidelines [10] is given as an input and the system will predict the feasibility of online exams adoption with optimum features as per given input. We intend to develop such system in our next article. None declared. The classification of selected studies as per defined categories (Section 2.1). The results are summarized in Section 3.1 (Table 5) 6 Targeted Online Exams Attributes Leading online exam attributes targeted in selected studies. The results are summarized in Section 3.2 ( Table 6) . Underlying development approach used in each selected study for the implementation of a particular solution. The results are summarized in Section 3.3 (Table 7). Techniques / Algorithms Techniques / algorithms proposed in each study to achieve required objective. The results are summarized in Section 3.4 (Table 8 ). 10 Tools Leading tools proposed and utilized in selected studies. The results are summarized in Section 3.5 (Table 9 ) and (Table 10 ). The important datasets used / proposed in selected studies. The results are summarized in Section 3.6 ( Table 11 ). Country of Research Countries / institutions participated in selected study. The results are summarized in Section 3.7 (Table 12) . Biometric [49] , [11] , [38] , [61] , [24] , [40] , [47] 7 2 Software Applications [21] , [51] , [26] , [54] , [39] , [12] , [56] , [44] , [63] , [64] ,[60] 11 3 General [36] , [20] , [13] , [25] , [50] , [37] , [22] , [62] , [52] , [27] , [23] , 35 [53] , [41] , [55] , [42] , [43] , [57] , [28] , [14] , [29] , [30] , [58] , [45] , [31] , [46] , [32] , [19] , [33] , [65] , [15] , [66] , [59] , [34] , [48] , [35] Verification and [49] , [36] , [25] , [50] , [11] , [22] , [38] , [61] , [24] , [40] , 19 Abnormal Behavior [42] , [28] , [14] , [45] , [31] , [47] , [19] , [33] , [35] 2 Security [26] , [64] , [37] 3 3 Question Bank Generation [21] , [51] , [62] , [23] , [53] , [54] , [39] , [41] , [55] , [12] , 20 and Evaluation [43] , [56] , [30] , [58] , [63] , [46] , [32] , [65] , [66] , [59] 4 Usability [57] , [15 Other [20] , [13] , [52] , [27] , [44] , [29] , [34] , [60] ,[48] 9 Machine Learning [49] , [11] , [38] , [61] , [40] , [41] , [55] , [42] , [46] , [47] , [35 Artificial Intelligence [62] , [23] , [24] , [53] , [39] , [43] , [32] , [66] , [ Traditional Development [36] , [21] , [51] , [52] , [26] , [27] , [54] , [12] , [56] , [44] , [45] , 15 [63] , [64] , [33] , [59] 5 Additional [20] , [13] , [25] , [50] , [37] , [22] , [57] , [28] , [14] , [29] , [30] , 17 [58] , [19] , [65] , [15] , [34] , [60] Python [49] , [50] , [61] , [53] , [55] , [42] 2 PHP [21] , [51] , [29] , [30] , [58] 3 Java [27] , [60] 4 Matlab Implementation Languages [25] , [ 10 Open CV 8 Image Processing and [49] , [11] , [61] , [28] 11 Emgu CV 9 Machine Learning Libraries [50] 12 NLTK 10 Natural Language Processing [53] 13 OpenNLP 11 Libraries [32] 14 MySQL [21] , [51] , [29] , [58] 15 FireBase [36] , [51] , [55] 16 SQLlite Databases / Storage [50] , [53] 17 SQL Server [27] , [64] 18 JSON [36] 19 Hidden Camera Activity 12 Image Capturing [36] 20 Face++ 13 Facial Recognition Platform [27] 21 WeScan 14 Documents Scanning [55] 22 Bayesian Network tools in Java (BNJ) 15 Probability Models Toolkit [30] 23 ProVerif 16 Formal Verification Tool [31] 24 Disco 17 process mining toolkit [48] 25 Weka 18 Machine Learning Tool [35] An author co-citation analysis: Examining the intellectual structure of e-learning from Modeling filipino academic affect during online examination using machine learning Adapting engineering examinations from paper to online Review of user authentication methods in online examination Automated detection for student cheating during written exams: An updated algorithm supported by biometric of intent Applying k-means clustering on questionnaires item bank to improve students' academic performance The "secure exam environment" for online testing at the alpen-adria-universität klagenfurt/austria Review and discussion: E-learning for academia and industry Envisioning the use of online tests in assessing twenty-first century learning: a literature review Procedures for performing systematic reviews Video summarization for remote invigilation of online exams Automatically assessed electronic exams in programming courses A new model of automatic and continuous online exam monitoring Using aadhaar for continuous test-taker presence verification in online exams Proposed features of an online examination interface design and its optimal values Key components of an e-learning module in statistics for computer science students The impact of proctored online exams on the educational experience A multi-factor authentication method for security of online examinations A dynamic profile questions approach to mitigate impersonation in online examinations Virtualization methods for securing online exam Item-based approach for online exam performance and its application An intelligent system for online exam monitoring Data mining with fuzzy method towards intelligent questions categorization in e-learning Gesture based misbehavior detection in online examination Automated online exam proctoring Secure online examination by using graphical own image password scheme Research and development of intelligent online examination monitoring system Ensuring online exam integrity through continuous biometric authentication Security of online examinations Evaluation of online assignments and quizzes using bayesian networks Formal analysis and offline monitoring of electronic exams A novel automated assessment technique in e-learning using short answer type questions Pagefocus: Using paradata to detect and prevent cheating on online achievement tests A conceptual framework for detecting cheating in online and take-home exams Detecting probable cheating during online assessments based on time delay and head pose Development of continuous authentication system on android-based online exam application Introducing tamex model for availability of e-exam in wireless environment Research on abnormal behavior detection of online examination based on image information Prototype of online examination on molearn applications using text similarity to detect plagiarism Towards academic affect modeling through experimental hybrid gesture recognition algorithm An application of online exam in discrete mathematics course An application to discover cheating in digital exams Analysis and design of item bank system based on improved genetic algorithm Pathways to successful online testing: eexams with the "secure exam environment"(see An online lab examination management system (olems) to avoid malpractice Automatic assessment of descriptive answers in online examination system using semantic relational features An eexam platform approach to enhance university academic student's learning performance Process mining based approach to performance evaluation in computer-aided examinations A design of continuous user verification for online exam proctoring on m-learning Face recognition method for online exams Exam wizard: A novel eassessment system Online examination system with cheating prevention using question bank randomization and tab locking An examination system automation using natural language processing Online assessment system based on ims-qti specification Automatic exam grading by a mobile camera: snap a picture to grade your tests Automatically generating exams via programmable plug-ins, and generic xml exam support An interactive mobile technology to improve the usability of exam application for disabled student Intelligent on-line exam management and evaluation system Cheat-resistant multiple-choice examinations using personalization Integrated and secure web-based examination management system Convolutional neural network based virtual exam controller A virtual dialogue assistant for conducting remote exams The design and application of an web-based online examination system Fog-assisted secure data exchange for examination and testing in e-learning system Addressing cheating when using test bank questions in online classes Exam paper generation based on performance prediction of student group A unified model-based framework for the simplified execution of static and dynamic assertion-based verification Gamalel-Din, E-exam cheating detection system Convolutional Neural Networks (CNN) Natural Language Processing Hardware / Software Virtualization Technique Fuzzy Clustering Technique K Means Clustering and Rule Mining Algorithms Bayesian Network based Technique [30] Quantified Event Automata and π-calculus Technique Dynamic Profile Questions Technique Syntactical Relational Feature Extraction Technique Country and Institution participated in the selected studies No Bijing Institute of Technology Institute of Business and Informatics Stikom Surabaya Medi-Caps University (MU) Kavikulguru Institute of Technology & Science Institute of the Philippines Cubao Saudi Arabia Prince Sattam Bin Abdulaziz University The authors extend their appreciation to the Deanship of Scientific Research at Saudi Electronic University for funding this work.