Review Article

Gamification in psychological assessment in South Africa: A narrative review

Yaseerah Akoodie

Received: 21 Jan. 2020; Accepted: 30 June 2020; Published: 10 Sept. 2020

Copyright: © 2020. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Gamification is defined as the implementation of game design elements in real-world contexts for non-gaming purposes. Gamification is increasingly being used in psychological assessment as it is thought to increase the attractiveness, motivation and performance of test takers. The ease of use of gaming principles is also a strong enabling factor for gamification in assessment. However, not much is known about the field in the South African context. Hence, this article uses the narrative review method to present the latest research on gamification in assessment. More specifically, the article discusses the benefits, costs, validity and scoring methods used with gamification in assessment. Research was conducted through electronic databases as well as the World Wide Web using Google Content analysis. Based on the review it was evident that individuals performed similarly in traditional and gamified assessments. Based on the results, the use of gamification was shown to decrease anxiety and stress and to increase motivation, loyalty and efficiency, especially in corporate environments. Despite the benefits, critics point out that gamification may be viewed as less important because of the inclusion of game elements as candidates may pay less attention to the assessment than required. Further, gamification has the ability to manipulate individuals as well as to bias certain groups of individuals that may be more accustomed to the use of technology than others. This raises ethical concerns, which are discussed in the article. The results also demonstrate a gap in research and practice in South African contexts with few gamified assessments available in the market.

Keywords: artificial intelligence; game-based assessment; gamification; gamified assessments; psychological assessment.

Introduction

Gaming is increasingly accepted as a form of entertainment with individuals of all ages playing games regularly. Technological developments have led to a new use of gaming called gamification. Gamification involves the addition of game elements to different contexts from engineering through to education and more recently to psychological assessment. Gamified assessment can be employed in different sectors such as clinical, industrial or educational systems (Karagiorgas & Niemann, 2017). This study adopts the form of a narrative review and presents the latest research on gamification in assessment, its benefits and costs, validity and scoring methods using examples as applicable from the literature. Ethics and the appropriateness of gamified assessment will also be discussed specifically as it pertains to the South African context. However, it is necessary that the concepts in the field are first discussed. Hence, the concepts used in gamification in assessment are clarified. Following this the design and scoring of gamified assessments are presented as well as current uses of artificial intelligence (AI) in gamified assessments.

Core concepts used in gamification in assessment

Gamification refers to the inclusion of game design elements into a nongaming activity in different contexts, for example, the workplace or educational settings (Georgiou, Gouras, & Nikolaou, 2019). Games are known to possess positive effects such as collaborative learning, increased levels of participation, continual interest and enjoyment (Kocadere & Caglar, 2015). Gamification is used to increase the attractiveness and ease of use and when applied it increases the engagement and motivation of individuals’ (Mekler, Brühlmann, Tuch, & Opwis, 2017). It enhances performance on tasks and according to empirical evidence is believed to do so by providing external motivators in the form of game elements such as points, leader boards, badges, levels, challenges and more. However, the effects on intrinsic motivation are unclear as the implementation of isolated game elements does not seem to make any observable changes in intrinsic motivation and applicants’ well-being (Mekler et al., 2017).

Within assessment, the concepts of gamified assessment and game-based assessments need to be distinguished. These different approaches vary on where they are implemented and for what purpose they are employed. The psychometric properties used in a traditional assessment remain unchanged in a gamified assessment, but the application of game elements is used to give the assessment the effects of a game, therefore making it more playful and enjoyable (Georgiou et al., 2019). Fatehi, Holmgard, Snodgrass and Harteveld (2018) gamified the Thematic Apperception Test (TAT) and participants were required to complete the traditional version and the gamified version. Participants had reported that the gamified TAT had been increasingly motivating and engaging and had correlated with the validity and reliability scores of the traditional TAT assisting in evidencing its appropriateness. Therefore, because of the nature of gamified assessments, test publishers and companies employing this construct tend to create robust psychometrics that increases the validity of assessments (Landers, Armstrong, & Collmus, 2017). This makes gamified assessment an attractive new development. Haydt (2008) mentioned as cited in Menezes and Bortolli (2016) that gamified assessments are used in schools to test whether students are reaching the objectives they are expected to reach, to assess student learning and to test the cognitive ability of individuals before and after instructions are administered. Because of these purposes, gamified assessments can take the form of a diagnostic, summative or formative assessment.

Game-based assessments change the core of a traditional assessment model by harnessing the full scope of game thinking to capitalise on the inherent psychometric properties of games and better applicant reactions (Georgiou et al., 2019). Game-based assessments aim to rebuild an assessment as a game (Landers et al., 2017). These types of assessments assist in recording a player’s choices’ and the data about how a player arrived at a particular choice. This allows for game-based assessments to analyse information that traditional assessments cannot capture, specifically internal thought processes over lengthy periods.

Designing game-based assessments

In the design of game-based assessment three aspects need to be considered for the creation of an efficient system, namely dynamics, mechanics and components. Firstly, particular dynamics need to be established. The major dynamics as proposed by Werbach and Hunter (2012) included adding constraints to challenge the test taker, emotions to attract and maintain interest in the gamified assessment, narratives for a storytelling effect, progression and the chance to build relationships or status. The behaviours and interactions that the player has towards these dynamics help assessors to analyse an individual’s cognition as well as their engagement in an organisation. For example, if a player knows they are being watched and scored they tend to be more competitive and this can be monitored through the dynamic progression (Wiklund & Wakerius, 2016). Therefore, these dynamics are added to game-based assessment so that they can encourage participation.

A gamified environment further consists of mechanics used to create player engagement. For example, a reward which is a mechanic process may be found by the player and may stimulate happiness, a sense of achievement or curiosity (Wiklund and Wakerius, 2016). Other game mechanics include challenges, chance, competitions, cooperation, feedback, rewards, transactions and resource acquisition. These properties contribute to the aspect of motivation needed for an individual to engage with an assessment (Werbach & Hunter, 2012).

In application, challenges require the player to extend effort in order to solve. An example of a challenge may be to include time restrictions that create a sense of pressure on the player. This could be used to assess an employee’s ability to work in such conditions. The idea of feedback may be to increase the chance that certain behaviours may be repeated allowing for a clearer observation to be made (Wiklund & Wakerius, 2016). Direct feedback is further required to observe the progress an individual is making towards a specific goal as different dynamic processes can impact the positive or negative feelings related to an assessment. The feedback evaluation can activate the mechanics of reward in order to formulate the scoring measures of the assessment and increase engagement and happiness (Kocadere & Caglar, 2015).

When designing a game-based assessment researchers also consider components – the minute parts that directly affect the design of gamification. Examples of components as proposed in Wiklund and Wakerius (2016) consisted of avatars, levels, leader boards, points, teams, virtual goods, content unlocking and badges. Each component impacts the gamification process differently, for instance the levels demonstrate a player’s position in the game and can act as a method of feedback, whereas content unlocking that requires individuals to meet certain criteria to move forward, serves the mechanics of challenge, feedback and reward. Alternatively, badges can be used for setting goals, providing explanations for learned activities, identifying particular players, providing feedback and encouraging competition (Wiklund & Wakerius, 2016).

Ultimately, the dynamics, mechanics and components of a game-based assessment are considered in a pedantic nature as they create the environment of the assessment and the environment of an assessment helps to improve its purpose, as well as the gamified experience (Werbach & Hunter, 2012). An improved gamified experience results in lower anxiety, increased motivation and the feeling of flow resulting in informed decision-making. Gamified assessments are distinguished from traditional assessments because of the emphasis placed on the complete experience of the assessment as well as the promotion of user engagement rather than just focusing on the final scores (Lopes, Pereira, Magalhães, Oliveira, & Rosário, 2019).

Scoring in game-based assessments

A potent mechanic used for scoring individuals is the concept of a point system. Point systems are put in place to inform players about the scores they acquire and further provide insight into the progress an individual is making (Werbach & Hunter, 2012). Points can also serve as an information provider for the game designers as these points can be stored, tracked and be of help for the developers to understand occurrences between the game and participant (Wiklund & Wakerius, 2016). Points can be used to encourage competition by demonstrating scores between players or to feel progression by demonstrating scores only to the specific player (Odyssey, 2019). Some talent management companies may provide feedback through a point system. These point systems may include an overall score as well as learning potential scores. Odyssey, a South African company, also indicates whether or not a candidate is recommended for employment in order to account for the large populations (Odyssey, 2019; Werbach & Hunter, 2012).

Alternatively, individuals could be scored through badges. Badges could serve as a goal-setting device in order to encourage players to progress towards the goal. Badges can also guide and educate the player whilst also acting as a status symbol as they communicate a player’s accomplishments. They can also serve as an identification marker for a specific group of individuals (Werbach & Hunter, 2012).

Furthermore, levels can be used to score an individual’s progress, as levels display a player’s position at any point throughout the game. In a study conducted by Kocadere and Caglar (2015), the concept of levels was used to provide feedback and the designed levels were based on Bloom’s revised taxonomy, which consists of six steps; remembering, understanding, applying, creating, analysing and evaluating. The gamified assessment that was created had covered these topics through the development of levels based on each construct. In this manner the standard of cognitive ability of test-takers based on how well they had progressed was able to be assessed.

Lastly, leader boards are also used for scoring as they are in place to make simple comparisons. They provide the player with a description of their performance in relation to others. This can be a motivator as individuals can see how a few more points may lead them to an upward movement in the leader board. Leader boards can also be discouraging to individuals if they are not performing as well as others taking the gamified assessment (Werbach & Hunter, 2012).

Alternatively, individuals can be scored depending on the speed of their responses, the number of correct responses chosen or through the item response theory formula. The item response theory is a scoring method where psychometric scores are assigned to individuals based upon their ability to interact with difficult situations, the probability of guessing, discriminability and thinking capability. Data collected from these four constructs are integrated into a mathematical formula in order to calculate the psychometric score of a test taker (Coetzee, 2018).

Using artificial intelligence in game-based assessment

Recent developments such as AI are making appearances in almost every sector of the economy. With specific regard to the field of psychometrics, AI has been implemented in many organisations in processes such as the screening of candidates and in employee selection tests (Geetha & Reddy, 2018). For example, realistic chatbot-type conversations with candidates occur in situational judgements tests. These conversations give insight into talent decision-making as they are capable of tracking the cognitive progress and adapting to the individual in order to further analyse the ability they present (AI in Assessment, 2019; Geetha & Reddy, 2018). Chatbot-type conversations can also assist by screening individuals for specific job requirements on preferred channels,that is, Whatsapp, WeChat, Facebook messenger, iMessage. Chatbot conversations can then organise candidates based on salary expectations, willingness to relocate, interests and more. Candidates who answer favourably towards the company’s requirements could be referred to the suitable recruiters for further assessment via gamified assessments to further test their suitability (AI in Assessment, 2019). These advances also assist in hiring with quality and talent. It also provides an easy solution to the mapping of talents so that candidates can be placed in optimal positions (Geetha & Reddy, 2018).

Furthermore, AI initiates automated scoring and computer-generated interpretive reports hereby eradicating the need for developers to perform these tasks. Artificial intelligence produces benefits in assessment. Firstly, precision is assured through the use of AI as these systems can analyse large amounts of data faster than any human could. This can improve the time and cost of selection decisions. Next, efficiency can be ascertained when a system is automated as this eliminates the room for human error (Verma & Bandi, 2019). Artificial intelligence allows recruiters to conduct consistent and objective assessments of relevant data at an earlier stage than expected (AI in Assessment, 2019).

Specifically, regarding video interviews AI has simplified the process as AI systems can transcribe and analyse data quickly, as well as help to analyse the visual elements through emotion tracking and facial recognition. Following these techniques, AI provides a new insight and efficiency into scoring gamified assessments (AI in Assessment, 2019).

Beyond the screening and selection uses, AI has also proven to have positive benefits in the educational sectors in which students can be assessed in a new superior system that can track the progress of individuals and form an evaluation of the knowledge students have in a specific area of study (Luckin, 2017). This can aid teachers in understanding their students better and make them more aware of the performance of individuals thus enabling them to prepare and focus on necessary areas at a later stage in a semester or educational year. It also assists students by encouraging them to reflect on their learning and current grades (Luckin, 2017).

In contrast to the compelling developments in AI, a limitation surrounding bias presents itself. It has been discovered that algorithms are a reflection of the bias in the world and the impact of an AI system functioning through an algorithm is massive. Algorithms are trained on data documented in the world, thus the data contracted should consider different cultures, environments, socio-economic profiles, preferences, lifestyles and genetic endowments and should reflect this rich diversity (Panch, Mattie, & Atun, 2019). However, data is not uniformly available for all groups, hence an imbalance is created, for example, a certain group is not sampled as much as others are or some groups are overlooked completely. This creates insufficient data and an inaccurate prediction for under-represented categories (Panch et al., 2019). Recent research conducted by Amazon’s machine learning specialists acts as evidence to these discoveries. Amazon frequently makes use of automation and with regard to their AI based hiring tool researchers have found that the system was not gender-neutral (Dastin, 2018).

In addition, researchers have found problems with facial recognition intelligence. In the majority of the contemporary facial recognition algorithms evaluated in a different study it was found that demographic differentials exist (Grother et al., 2019). False positive and false negative results are common for many algorithms where false positives are more commonly found and false negatives are algorithm-specific. Ultimately, it was concluded that more accurate algorithms produce fewer errors and it had been suggested that having smaller demographic differentials in a study is favourable (Grother et al., 2019). These efforts display that AI can be reengineered to produce fairer results, however it also accentuates the dependency of AI on human training and presents how challenging and complex the problem of bias can be specifically because of the belief that by adding to the data sets the balance of the system will be affected (Knight, 2019). Therefore, it is critical to evolve and instruct AI systems with data that is unbiased and algorithms that can easily be explained.

To date there is not much research available on the uses and limitations of gamification as well as the ethics associated with this type of assessment. This study intends to present an overview of the uses and limitations of gamification in assessment and the ethics associated with gamification with a specific focus on the South African context.

Methods

This study used a narrative review method to explore gamification in assessment. A narrative review approach was chosen as this study provides a broad perspective on the topic area and explores the general debates in the area (Green, Johnson, & Adams, 2006). Hence, narrative reviews provide an examination of literature in the topic area in order to summarise information on the topic area and identify gaps for future research (Grant & Booth, 2009). Literature searches were conducted on Google Scholar; SAGE research methods; National Centre for Biotechnology Information (NCBI); Semantic Scholar and Research Gate using keywords as follows ‘Gamification’; ‘Gamified Psychological Assessments’; ‘Scoring Methods in Assessment’ and ‘Psychometrics and gamification’. Grey literature was also located by searching the World Wide Web using Google Content analysis.

Review findings

Benefits and limitations of gamification in assessment

The first benefit of gamification in assessment was that individuals have been found to perform similarly in traditional assessments and in gamified assessments demonstrating that gamified assessments have an equal footing in psychometrics deeming it an acceptable technique (AI in Assessment, 2019). Further, gamified assessments improve the users’ experience and in organisations, it improves the employers’ brand perception too. Because of this belief, gamified assessment can be considered an appropriate tool to use in businesses. They help reduce stress and any stereotype threats. They can test a participant’s decision-making skills, reactions, preferences and biases aiding in making informed employee selection decisions and finding appropriate roles for candidates. In addition, a gamified assessment environment might distract candidates from the idea that they are under assessment unlike traditional assessments. This can reduce test anxiety, which is beneficial as low levels of anxiety increase performance. It can promote behaviours that are more likely to appear unconsciously instead of socially acceptable or desired outcomes (Fetzer, McNamara, & Geimer, 2017). Gamification has also been found to boost motivation and loyalty as it allows a sense of motivation and confidence in one’s abilities, as well as ensure individuals know just how valued they are by a company. It allows individuals to take pride in what they are good at and improve on what they may not be even if this is not through extended hours of play and is discovered by approaching individuals who rank higher than you for advice. In this manner gamified assessments provide a more accurate interpretation of the candidates under examination, increasing the reliability of these types of assessments and the quality of staff members employed (Fetzer et al., 2017; Guy, 2019; Maltitz, 2014).

Further, game engagement and the use of contexts can aid in diagnosing how an individual handles a particular problem and this may lead to more robust inferences about performance (Fetzer et al., 2017). Because of these advantages the use of game elements improves the criterion validity of assessments too (Guy, 2019). Gamification utilises three dimensional graphics, sounds and avatars that give the assessment a realistic feel. This can enhance the transferability of the assessment, which increases the face and ecological validity related to it (Lumsden et al., 2015). According to a test partnership company called MindmetriQ all the assessments that they distribute to companies such as HTC, Steinhoff UK Retail LTD, Unilever, Barclays and many more, withhold high validity and reliability. The tests are measured against alternate measures and have been found to have correlations that are statistically significant. This suggests evidence of validity in the test partnership. Further, the company tests for reliability using Rasch item reliability and Pearsons’ reliability producing high levels of reliability that ensures accuracy and precision, therefore, permitting the use of these assessments for potent selections and assessment (Guy, 2019).

Fairness and objectivity are also increased, as well as efficiency as administration time is reduced significantly (Guy, 2019). Gamified assessments reduce the costs of administration as professionals are not required to spend the time and money it takes to incorporate a telephonic interview, and by making assessments available on online networks recruiters can attract a certain crowd that may not be interested in traditional assessments and may align with the skill set the organisation is interested in (Guy, 2019; Krasulak, 2015). This too impacts efficiency as it creates a skill-specific talent pool. Alongside this, the ability of scoring helps to ease the costs of recruitment as selections can be made by a simple pass or fail rule, for example, candidates are required to obtain a minimum score in order to progress in selections. This reduces the time administrators put in regarding traditional assessments as they are not expected to work case-by-case. Time to hire can also be reduced by combining tests allowing different constructs to be measured simultaneously, such as performance, personality and cognitive abilities (Nikolaou, Georgiou, & Kotsasarlidou, 2019). Many companies are affected by open vacancies as the longer the vacancy is left open the greater the cost is for the organisation because of loss in productivity. Gamified assessment systems aid in increasing productivity in selections as a single assessor can oversee and invite thousands of candidates whilst ensuring that they complete the assessment within a desired time frame instead of contacting candidates individually to arrange a mutually agreed upon interview time and wait for each candidates response (Guy, 2019).

In the health sector, gamified assessments are used to promote positive health behaviour and to eliminate the stigma surrounding certain health issues (Hamari & Koivisto, 2015). Mobile devices are an effective medium for individuals to monitor their health, for example, a Cognitive Behavioural Therapy (CBT) based application (app) called MyCompass is a self-guided psychological treatment that is designed to monitor and reduce mild to moderate anxiety, stress and depression as well as suggest techniques to improve social and work functioning (Giota & Kleftaras, 2014). This is conducted through a set of monitored interactive activities and assistance in restructuring the way an individual thinks or behaves under these conditions. Resultantly, this gamified app can assist clients and potential clients in increasing their health-seeking behaviours positively (Giota & Kleftaras, 2014). Furthermore, rehabilitation centers assist individuals who have suffered from brain injuries, diseases or disorders by aiding patients in relearning how to complete daily tasks individually (Vourvopoulos, Ponnam, Faria, & Badia, 2014). Instead of utilising the traditional usage of questionnaires and scales a gamified assessment known as RehabCity has been developed in the United States so that individuals can assess the cognitive deficits associated with the injury they face (Vourvopoulos et al., 2014). Players are placed in real-life environments in a virtual world in order to familiarise themselves with the daily situations they may encounter, for example, visiting the grocery store or abiding by traffic laws. The RehabCity gamified assessment correlates strongly with the Mini Mental State Examination test used in clinical assessments for cognitive functioning and therefore provides potent assistance in the health sector.

With gamified assessment candidates may also complete the test at any location without any transport cost or potentially wasting time of the interviewer (Guy, 2019). However, some concerns surrounding this benefit exist. Firstly, standardising the environmental conditions of the test-takers would present as a difficult task and could leave room for inaccurate representations of the participants (Foxcroft & Roodt, 2018). Next, all candidates or potential candidates may not have access to a stable Internet connection (Du Plessis, 2014). This could impact the gamified assessments accuracy or it may require candidates to travel to areas that can provide a stable Internet connection, which could be costly. Alternatively candidates may not have access to WiFi causing an increase in data expenses. Nonetheless, cancellations or candidates that don’t show up are no longer of concern when using online psychometric testing and with regard to providing feedback, online gamified assessment may have a customisable automatic email that can be sent to candidates, containing a feedback report for both successful and unsuccessful participants. This eradicates the need for administrators to contact each candidate individually and managing a candidate’s negative emotions when faced with rejection (Guy, 2019).

Critics argue further that gamification is a form of manipulation. Awarding someone a badge may only motivate them for a short period of time and could demonstrate a patronising nature. However, mandating skills as a prerequisite gives people a goal to work towards and assigns real value to the task and respective badge earned (Maltitz, 2014). In addition, a consideration surrounds the scepticism related to gamified assessment as certain groups who may be tested may be less likely to play assessment games, for example, millennials are more drawn to play assessment games yet older candidates may be less familiar with the activity (Guy, 2019). This could affect senior positions. In order to counteract this difficulty organisations have considered the idea of using a combination of assessments (Evalex, 2014).

Another risk that exists regarding the appropriate nature of gamified assessment is that some individuals may take the assessment less seriously because it is a game and may pay less attention than they would in a more formal traditional method of testing. The results may not represent true abilities. By setting assessments that are goal-specific and motivational companies can eradicate this hurdle (Evalex, 2014).

With reference to cultural differences an alternate limitation arises. According to Khaled (2014), because of the multicultural nature of the world it is increasingly important that cultural differences that may present in the utility of gamified assessments be taken into account. This may be a difficult task to complete specifically because some cultural convictions contradict and can negate each other, for example, in Danish and other Scandinavian cultures it is frowned upon to try to stand out and if you do it is perceived as though you believe that you are superior to others. Amongst this belief, there are a set of rules that encourage social equality, uniformity and social stability. In such a culture where competition is looked down upon gamified assessments may not flourish. On the other hand, in countries such as the United States it might be considered to be admirable for someone to pursue themselves and their personal objectives against all odds (Khaled, 2014). Cultural importance is in accord with risk taking, competitiveness, achievements, self-assertion and success. These cultures hold opposing beliefs and utilising one gamified assessment for both, may result in dishonest representations of the individuals’ abilities. Although gamification consultancy companies recognise the need to address cross-cultural differences, a lack of how to rectify these difficulties exists. Research notes that reform is necessary in the design phase, however more detail is required and for the present, designers working on international or national designs focus their attention on company culture or national culture rather than individual culture (Guhl, 2017).

Using gamification for assessment in South Africa

Currently gamification for assessment purposes is primarily used in organisational settings in South Africa. Deloitte, a multinational professional accounting services network, utilises a gamified assessment called Firefly Freedom. This gamified assessment maps an individual’s personality in order to gain insight into the mannerisms that candidates make use of in different situations (Hanna & Dettmer, 2004; Whitelock, 2019). L’Oreal, a multinational, beauty focused organisation uses a gamified assessment – ‘Reveal’ – in which candidates move through the Reveal platform as avatars. Candidates face various challenges from different departments in order to grant applicants the chance to explore the available positions in the company as well as to demonstrate the skills they possess (L’Oreal, 2019). Another example of gamified assessments in South Africa is Pricewater House Cooper’s (PWC) use of a game-based psychometric assessment called Ascender. Ascender is a web-based assessment set that takes the form of a novel-styled intergalactic journey in which candidates are required to apply their judgement and make decisions in different situations (PwC, 2019). Ascender evaluates applicant’s personal values by linking each decision to a specific personal value and with the use of a scoring algorithm, PWC recruiters are able to determine the degree of alignment of each candidate to a set of values and therefore align employee’s personal and corporate values to ensure efficient inter-team dynamics (PwC, 2019).

Evalex, a talent management company in South Africa, promotes the use of Odyssey – a gamified assessment. Odyssey was created to cater for the specific requirements found in developing economies where a minimum level of education and literacy cannot be assumed. Odyssey aims to identify talent and potential regardless of any previous formal training individuals may have. Further, recruiters often overlook entry level candidates with huge potential simply because the traditional assessments that are administered require a much more advanced level of education than the job requires. Odyssey uses a gamified assessment approach in order to measure the real skills that employees need in order to operate at the lower levels of work. These include problem-solving skills, instruction assimilation, trainability, English literacy, numerical literacy and productivity (Evalex, 2014).

Du Plessis (2014) discussed the challenges associated with using gamification in schools given the social and economic inequalities that exist in South Africa. Even though gamification offers an alternative for large-scale assessments in a climate of limited resources, difficulties arise in the appropriateness of this implementation. Structural problems exist in terms of access to technology, electricity and data. Many areas in South Africa do not have the necessary high-speed delivery systems that are required to make online technologies work optimally and many individuals do not have the budget to own any form of technology and have endless access to electricity and data. WiFi is also not freely accessible everywhere in South Africa making it increasingly difficult to afford enough data to run gamified assessments remotely (Du Plessis, 2014; Xala, 2018). Thus, transportation costs to and from organisations or WiFi hotspots has to be considered too. These factors affect the access to gamified assessments and therefore, affect the opportunities for individuals in South Africa. However, the South African government aims to provide broadband connectivity for all citizens through public WiFi by 2030. Public WiFi programmes are important as they can assist in addressing the issues of inequity for communities that are unable to afford high costs of data (Xala, 2018).

Beyond the difficulties of access to resources it was discovered that teachers in educational systems struggle to utilise game-based assessments or learning techniques because of a lack of knowledge or familiarity with technological devices and systems and they tend to avoid using such advancements (Connolly & Boyle, 2016). In South Africa this may pose as a difficulty too because of the vast inequalities of the country as not all individuals have access to technology and therefore may lack familiarity with certain devices and the manner in which they function. As a result, it is critical that training programs are made available in order to integrate technology into educational systems and other sectors (Botha, Herselman, & Ford, 2014). Further, gamification in assessment still deals with the challenges in traditional assessment ranging from language proficiency, quality of schooling, test-wiseness and multilingualism amongst others (Laher & Cockcroft, 2013).

Ethical considerations of gamified assessment

With regard to ethical considerations, it is important to follow the principles of fair-testing as every applicant should have equal chances of success. The assessment should appeal to all candidates regardless of their cultural background, age or physical ability. One way to ascertain that the gamified assessment is applicable to all is to omit the demographic variables generally requested from the participant (Psymetric Company, 2019). Hiring managers tend to favour candidates who are similar to themselves either based on a similar experience, university qualification, demographic or personality trait and they tend to disfavour candidates who are similar to an employee whom they had a bad experience with even though the new candidate could potentially be the correct choice. Similarly, younger candidates tend to suffer when hiring managers utilise traditional methods that require the aforementioned demographic details because of the lack of depth displayed on their resumes and older candidates tend to be impacted by irrelevant past experiences. In this manner, demographic information or identifiable information can cause companies to neglect the skills that candidates can really offer (Keijzer, 2018). Therefore, by removing this retrieval of information, individuals can be assessed or selected based on their ability presented in a game rather than their race, gender or other demographic details deeming gamified assessments increasingly fair.

In relation to fair-testing and applicability a potent concept to consider is cultural differences. It has been discovered that there is no solitary arrangement of qualities, beliefs or values that individuals from every single world culture accept to be similarly significant (Khaled, 2014). Because of this claim, it is safe to deduce that specific representations found in gamified assessments would not be interpreted in the exact same way cross-culturally, causing in-game behaviour to be impacted. These behaviours may then be misinterpreted by the assessor evaluating the scoring system in place or they may be evaluated in a manner that is positive to some scoring systems and negative to others. For example, participant A responds in a manner that is culturally acceptable to his standard; however this action may not be viewed as positive to the systems scoring. Alternatively, a candidate from the United States may approach the assessment with success, ambition and competitiveness in mind because of the cultural importance standards demonstrated in the United States and they may constantly try to better their score, whereas individuals in New Zealand who view markers of achievement as needless and almost offensive may react with less zest to score high. In this manner participants may be overlooked because of cultural beliefs rather than skill and this may infringe on discrimination (Khaled, 2014).

In order to further follow through on applicability requirements, developers’ should consider the country and the laws of the country where the gamified assessment will be implemented in too, as the laws of different countries may differ resulting in differing belief systems of the inhabitants (Wiklund & Wakerius, 2016). Beyond the beliefs of the people of a country, the laws and policies must be considered too as there may be severe consequences for non-compliance of labour laws and data privacy laws. Therefore, in order for gamified assessments to be appropriate they need to be adaptable to the contexts in which they will be used. This requires a flexible and configurable design that provides capability for users to turn features on and off based on their geographical preferences (Kumar & Herger, 2013).

In addition, data privacy is a necessary concern as particular European laws disallow collection, processing and the use of individual data without consent from the participants, therefore, candidates should be required to perform an action such as signing a document, clicking a button or checking a box to agree that their data are being collected and used (Kumar & Herger, 2013; Wiklund & Wakerius, 2016). In some countries the law gives power to workers councils’ to approve assessments utilised in a company. Such councils are generally concerned with the purpose of the data collection and the amount of data collected, as well as the justification behind it. Further, they consider where data is stored, if the purpose of the assessment can be reached with less data, the anonymity of the data and if the data serves as a basis for performance review decision such as salary increases, bonus calculation, promotions or expulsions. In addition, the negative impacts of these considerations on employee’s must be addressed too so that employee well-being and fairness is consistent. These factors need to be contemplated and approved by a council before usage of the gamified assessment is allowed. In order to avoid the prohibition of the entire game design the ability to turn off some features could help assure the game is flexible enough for worldwide use (Kumar & Herger, 2013).

As yet there are no guidelines, legislative or otherwise in South Africa with specific regard to gamification. However, game-based assessments and gamified assessments are still considered tests and in the event they are utilised in South Africa they are required to comply with the Employment Equity Act (EEA) (no. 55 of 1998) and the HPCSA guidelines (Tomu, 2013). This is specifically important when testing for psychological constructs. The EEA and the HPCSA work together in order to regulate the ethical conduct of psychologists (Tomu, 2013). In accordance with the EEA, when using gamification in assessments individuals need to ensure that the test is valid, reliable, applied fairly and is not biased against any employee or group. The HPCSA complements the professional laws and codes in order to regulate test use and it assumes that the tests being used are compliant with it.

Furthermore, the Protection of Personal Information Act (POPI) (no. 4 of 2013) should be considered. This act protects the privacy rights of all individuals, hence it impacts all parties that collect, process, store and disseminate personal information, therefore, directly impacting gamified assessments or game-based assessments (De Bruyn, 2014). The POPI enforces that a responsible party utilising an individual’s private data should obtain consent for collecting and storing the data, the purpose of the data collection must be made known to the individual, access should be provided or removed if requested, the individual providing the data should be aware of who will have access to their information and how and where the data will be stored and lastly, the measures that are put in place in order to safeguard your information (De Bruyn, 2014). The act allows for an exclusion with regard to processing information in one’s personal capacity, information that has been de-identified, that is, using an anonymisation technique or information that has been collected on behalf of a public body that promotes national security and public safety (De Bruyn, 2014). Ultimately, it would be necessary for a default setting to be used in which individuals are made aware that data are being collected, assessed or used (Wiklund & Wakerius, 2016).

The methods that the assessments are administered should also perform equally well on all devices as technology advancements may distort what is being assessed. It cannot be assumed that all measures of an assessment will be equivalent across different modes of delivery as it is more common that non-cognitive measures in a test can be transferred equivalently but cognitive measures do not always transfer correctly, for example, when moving from PC to mobile devices (Ryan & Derous, 2019). Whilst the reasons behind why different devices respond differently to the transfer of data has to be studied further as this would call for reliability trial sessions spread across all different types of devices in order to assure assessments fit their purpose (Aon, 2018). Despite reliability trial sessions, technology itself poses a set of ethical dilemmas as it cannot be classified as a fully reliable source because of battery failures and Internet connectivity issues, which have the potential to interfere with assessments or data recordings. Furthermore, privacy is not ascertained as the chances of theft and hacking exist in which important, private information could be exposed and confidentiality agreements can be broken (Giota & Kleftaras, 2014).

Another ethical concern surrounds the risk of individuals cheating the system. This means that the gamified experience needs to be built with the belief that players will try to cheat the system and as a result add cheat protection to the assessment as any form of cheating could result in inconsistent or dishonest results (Wiklund & Wakerius, 2016). In order to reduce cheating, gamified assessments should decrease the perceived values of rewards, for example, intrinsic rewards should be employed and they should not be transferable in the real world or rewards should have a large perceived value for the target audience, but not for the rest of the population. Notably, transferability of virtual rewards in the real world could further infringe on country laws and should be avoided. Total transparency between developers’ and participants is required in order to maintain such a system too (Kumar & Herger, 2013).

Other factors that could result in manipulation could form from the developers’ side. Companies that recruit through gamified assessment do not clearly disclose to test-takers the contents and aims of a gamification system. This edges into the exploitation of individuals’ vulnerabilities and requires further research (Kim, 2016).

With regard to manipulation, companies that employ gamification to increase employee skills and competitiveness tend to generate increased productivity in the workplace; however, the employees receive no physical or monetary reward for these improvements rather they are awarded virtual rewards, that is, badges or higher placement on a leader-board (Kim & Werbach, 2016). In this manner, the employee may receive less satisfying rewards whereas the employer will receive monetary rewards and recognition. For example, target utilises a gamification technique to assess the speed of cashiers through a game called ‘checkout’ in which cashiers are rated with a green light when they work fast enough or a red light when they are too slow. Cashiers are then awarded badges for reaching higher levels of speed and can be promoted to new levels as their efficiency increases. Checkout has proven to increase the speed of checkout lines and cashiers report an increased satisfaction in their job experiences; however, no monetary or real reward exists for the employees – only for employers (Kim, 2016). This relates to the issue of deceit, which is considered unethical and should be addressed (Kim & Werbach, 2016). A gamified assessment can be considered meaningful and effective if it prioritises the needs of the participant rather than those of the assessor or organisation (Bhavani et al., 2019).

Furthermore, Kumar and Herger (2013) had stated that gamification designers mimic the role of a social architect to a certain extent and with this title comes a responsibility. Assessors should be aware of the manipulation of the influence they may have on participants as gamification connects the virtual world to the real world and decisions that candidates make in a gamified system tends to affect their reality. Misuse of this power infringes on manipulation as individuals’ decisions may be affected by the assessment in order to benefit the assessor rather than the participant. For example, reminding players to save water and electricity, organise their workspace, create lists to organise their minds or to recycle materials would be a positive influence whereas describing players’ rights in a confusing or incomprehensible manner to them or constantly demonstrating an unhealthy work environment would demonstrate a negative influence (Kumar & Herger, 2013).

Alternatively, significant psychological harms can be found in gamified assessments and some scoring methods. Video-screening leader board systems can generate anxiety, shame and embarrassment among employees. According to empirical research carried out in Disneyland hotels, it was discovered that individuals found that seeing their performances listed and ranked against other co-workers caused them to panic and express anxiety about losing their jobs (Kim, 2016). In addition, scoring at the bottom of a leaderboard has proven to be a disliked position and in an assessment context this causes individuals to disengage from the assessment (Ferrell et al., 2015). In turn, this decreases their test-taking motivation and directly impacts their likelihood to complete the assessment. If these psychological impacts were to occur in a high-stakes assessment context, the organisation utilising the assessment might end up with a smaller talent pool. Therefore, gamification developers should aim to avoid such harms and grant attention to the method so that scoring is correctly expressed.

Lastly, it has been said that whilst gamified assessments have eradicated the ethical issue of adaptability regarding language because of their visual characteristics, it has been argued that these types of assessments do not take into consideration individuals that may be colour blind or may have sight impairments. These candidates should not be disadvantaged as such action would be considered as discriminatory. Furthermore, with regard to discrimination, it is possible that the instructions of some gamified assessments may not be understandable cross-culturally and in countries such as South Africa with a multiplicity of cultures and languages, this may impose difficulty on assessors and developers when adapting a gamified assessment (Aon, 2018).

Conclusion

Ultimately, the development of gamification had led to the creation of gamified assessments. This technological advancement has climbed to the top of the talent acquisition agenda globally. The uses of gamification have expanded significantly with games being used to acquire job for candidates, assess skills before and after instructions and assess abilities for specific roles across fields. Gamification in assessment is also being used internationally in educational and clinical settings. This review has highlighted the benefits and limits as well as the ethics of gamification in assessment with a specific focus on the South African context. From the review, it is evident that gamification has a lot to offer towards providing greater access to assessment for the South African population provided that ethical concerns are addressed.

Acknowledgements

We wish to thank and express our appreciation towards Professor Sumaya Laher, who had assisted in the production of this review by overseeing all proceedings and guiding the author through the process.

The author has declared that no competing interest exists.

I declare that I am the sole author of this article.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data sharing is not applicable to this article as no new data were created or analysed in this study.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position or any affiliated agency of the authors.

References