Applying Gamification to the Library Orientation: A Study of Interactive User Experience and Engagement Preferences ARTICLES Applying Gamification to the Library Orientation A Study of Interactive User Experience and Engagement Preferences Karen Nourse Reed and A. Miller INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2020 https://doi.org/10.6017/ital.v39i3.12209 Karen Nourse Reed (karen.reed@mtsu.edu) is Associate Professor, Middle Tennessee State University. A. Miller (a.miller@mtsu.edu) is Associate Professor, Middle Tennessee State University. © 2020. ABSTRACT By providing an overview of library services as well as the building layout, the library orientation can help newcomers make optimal use of the library. The benefits of this outreach can be curtailed, however, by the significant staffing required to offer in-person tours. One academic library overcame this issue by turning to user experience research and gamification to provide an individualized online library orientation for four specific user groups: undergraduate students, graduate students, faculty, and community members. The library surveyed 167 users to investigate preferences regarding orientation format, as well as likelihood of future library use as a result of the gamified orientation format. Results demonstrated a preference for the gamified experience among undergraduate students as compared to other surveyed groups. INTRODUCTION Background Newcomers to the academic campus can be a bit overwhelmed by their unfamiliar environment: there are faces to learn, services and processes to navigate, and an unexplored landscape of academic buildings to traverse. Whether one is an incoming student or recently hired employee of the university, all need to become quickly oriented to their surroundings to ensure productivity. In the midst of this transition, the academic library may or may not be on the list of immediate inquiries; however, the library is an important place to start. Newcomers would be wise to familiarize themselves with the building and its services so that they can make optimal use of its offerings. Two studies found that students who used the library received better grades and had higher retention rates. 1 Another study regarding university employees revealed that untenured faculty made less use of the library than tenured faculty, a problem attributed to lack of familiarity with the library.2 Researchers have also found that faculty will often express interest in different library services without realizing that these services are in fact available.3 It is safe to say that libraries cannot always rely on newcomers to discover the physical and electronic services on their own; they need to be shown these items in order to mitigate the risk of unawareness. In consideration of these issues, the Walker Library at Middle Tennessee State University (MTSU) recognized that more could be done to welcome its new arrivals to campus. The public university enrolls approximately 21,000 students, the majority of whom are undergraduates. However, with a Carnegie classification of doctoral/professional and over one hundred graduate degree programs, there was a strong need for specialized research among the university’s graduate students and faculty. Other groups needed to use the library too: non-faculty employees on campus as well as community users who frequently used Walker Library for its specialized and general collections. The authors realized that when new members of these different groups mailto:karen.reed@mtsu.edu mailto:a.miller@mtsu.edu INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 2 arrived on campus, few opportunities were available for acclimation to the library’s services or building layout. Limited orientation experiences were conducted within library instruction classes, but these sessions primarily taught research skills and targeted freshman general- education classes as well as select upper-division and graduate classes. In short, it appeared that students, employees, and visitors to the university would largely have to discover the library’s services on their own through a search on the library website or an exploration of the physical library. It was very likely that, in doing so, the newcomers might miss out on valuable services and information. As MTSU librarians, the authors felt strongly that library orientations were important to everyone at the university so that they might make optimal use of the library’s offerings. The authors based this opinion on their knowledge of relevant scholarly literature as well as their own anecdotal experiences with students and faculty.4 The authors defined the library orientation differently from library instruction: in their view, an orientation should acquaint users with the services and physical spaces of the library, as compared to instruction that would teach users how to use the library’s electronic resources such as databases. The desired new approach would structure orientations in response to the different needs of the library’s users. For example, the authors found that undergraduates typically had distinct library interests compared to faculty. It was recognized that library orientations were time-consuming for everyone: library patrons at MTSU often did not want to take the time for a physical tour, nor did the library have the staffing to accommodate large-scale requests. The authors turned to the gamification trend, and specifically interactive storytelling, as a solution. Interactive storytelling has previous applications in librarianship as a means of creating an immersive and self-guided user experience.5 However, no previous research appears to have been conducted to understand the different online, gamified orientation needs of various library groups. To overcome this gap, the authors developed an online, interactive, game-like experience via storytelling software to orient four different groups of users to the library’s services. These groups were undergraduate students, graduate students, faculty members (which included both faculty and staff at the university), and community members (i.e., visitors to the university or alumni); see figure 1 for an illustration of each groups’ game avatars. These groups were invited to participate in the gamified experience called LibGO (short for library game orientation). After playing LibGO, participants gave feedback through an online survey. This paper will give a brief explanation of the creation of the game, as well as describe the results of research conducted to understand the impact of the gamified experience across the four user groups. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 3 Figure 1. LibGO players were allowed to self-select their user group upon entering the game. Each of the four user groups was assigned an avatar and followed a logic path specified for that group. LITERATURE REVIEW Traditional Orientation Searches for literature on library orientation yield very broad and yet limited details about users of the traditional library orientation method. It is important to note that the terms “library tour” and “library orientation” can be somewhat vague, because this terminology is not interchangeable, yet is frequently treated as such in the literature.6 These terms are often included among library instruction materials which predominately influence undergraduate students.7 Kylie Bailin, Benjamin Jahre, and Sarah Morris define orientation as “any attempt to reduce library anxiety by introducing students to what a college/university library is, what it contains, and where to find information while also showing how helpful librarians can be.”8 Their book is a culmination of case studies of academic library orientation in various forms worldwide where the common theme across most chapters is the need to assess, revise, and change library orientation models as needed, especially in response to feedback, staff demands, and the evolving trend of libraries and technology.9 Furthermore, the majority of these studies are undergraduate-focused, and often freshman-focused, while only a few studies are geared towards graduate students. Other traditional orientation problems discussed in the literature include students lacking intrinsic motivation to attend library orientation, library staff time required to execute the orientation, and lack of attendance.10 Additionally, among librarians there seems to be consensus that the traditional library tours are the least effective means of orientation, yet they are the most highly used and with attention predominately focused on the undergraduate population alone. 11 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 4 In 1997, Pixey Anne Mosely described the traditional guided library tour as ineffective, and documented the trend of libraries discontinuing it in favor of more active learning options.12 Her study surveyed 44 students who took a redesigned library tour, all of whom were undergraduates (with freshmen as the target population). Although Mosely’s study only addressed one group of library users, it does attempt to answer a question on library perception whereby 93 percent of surveyed students indicated feeling more comfortable in using the library after the more active learning approach.13 A comparison study by Marcus and Beck looked at traditional vs treasure hunt orientations, and ultimately discovered that perception of the traditional method is limited by the selective user population and lack of effective measurements. They cited the need for continued study of alternative approaches to academic library orientation.14 A study by Kenneth Burhanna, Tammy Eschedor Voelker, and Julie Gedeon looked at the traditional library tour from the physical and virtual perspective. Confronted with a lack of access to the physical library, these researchers at Kent State University decided to add an online option for the required traditional freshman library tour.15 Their study compared the efficacy of learning and affective outcomes between face-to-face library tours and those of online library tours. Of the 3,610 students who took the required library tour assignment, 3,567 chose the online tour method and 63 opted or were required to take the in-person, librarian-led tour. Surveys were later sent to a random list of 250 students who did not take the in-person tour and the 63 students who did take the in-person tour. Of the 46 usable responses all but one were undergraduates and 39 (85 percent) of them were freshman.16 This is a small sample size with a ratio of slightly greater than 2:1 for online versus in-person tour participation. Although results showed that an instructor’s recommendation on format selection was the strongest influencing factor, convenience was also significant for those who selected the online option (81.5 percent). In contrast, only 18.5 percent of the students who took the face-to- face tour rated it as convenient. The authors found that regardless of tour type, students were more comfortable using the library (85 percent) and more likely to use library resources (80 percent) after having taken a library tour. Interestingly, students who took the online tour seemed slightly more likely to visit the physical library than those who took the in-person tour. Ultimately the analysis of both tours showed this method of library orientation encourages library resource use, and the “online tour seems to perform as well, if not slightly better than the in-person tour.”17 Gamification Use in Libraries An alternative format to the traditional method is gamification. Gamification has become a familiar trend within academic libraries in recent years, and most often refers to the use of a technology - based game delivery within an instructional setting. Some users find gamified library instruction to be more enjoyable than traditional methods. For these people, gamification can potentially increase student engagement as well as retention of information.18 The goal of gamification is to create a simplified reality with a defined user experience. Kyle Felker and Eric Phetteplace emphasized the importance of user interaction over “specific mechanics or technologies” in thinking about the gamification design process.19 Proponents of gamification of library instructional content indicate that it connects to the broader mission of library discovery and exploration as exemplified through collaboration and the stimulation of learning.20 Additional benefits of gamification are its teaching, outreach and engagement functions.21 Many researchers have documented specific applications of online gaming as a means of imparting library instruction. Mary J. Broussard and Jessica Urick Oberlin described the work of librarians at Lycoming College in developing an online game as one approach to teaching about INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 5 plagiarism.22 Melissa Mallon offered summaries of nine games produced for higher education, several of which were specifically created for use by academic libraries.23 Many of these online library games reviewed used Flash, or required players to download the game before playing. By contrast, J. Long detailed an initiative at Miami University to integrate gamification into the library instruction, a project which utilized Twine.24 Twine is an in-browser method and therefore avoids the problem of requiring users to download additional software prior to playing the game. Other libraries have used online gamification specifically as a tool for library orientations. Although researchers have demonstrated that the library orientation is an important practice in establishing positive first impressions of the library and counteracting library anxiety among new users, the differences between in-person versus online delivery formats are unclear.25 Several successful instances have been documented in which the orientation was moved to an online game format. Nancy O’Hanlon, Karen Diaz, and Fred Roecker described a collaboration at Ohio State University Libraries between librarians and the Office of First Year Experience; for this project, they created a game to orient all new students to the library prior to arrival on campus.26 The game was called “Head Hunt,” and was cited among those games listed in the article by Mallon. 27 Anna-Lise Smith and Lesli Baker reported the “Get a Clue” game at Utah Valley University which oriented new students over two semesters.28 Another orientation game developed at California State University-Fresno was noteworthy for its placement in the university’s learning management system (LMS).29 In reviewing the literature regarding online library gamification efforts, there appear to be several best practices. Several studies cite initial student assessment to understand student knowledge and/or perceptions of the content, followed by an iterative design process with a team of librarians and computer programmers.30 Felker and Phetteplace reinforced the need for this iterative process of prototyping, testing, deployment, and assessment as one key to success; however they also stated that the most prevalent reason for failure is that the games are not fun for users.31 Librarians are information experts, and are not necessarily trained in fun game design. Some libraries have solved this problem by partnering with or hiring professional designers; however for many under-resourced libraries, this is not an option.32 Taking advantage of open- source tools, as well as the documented trial-and-error practices of others, can be helpful to newcomers who wish to break into new library engagement methods utilizing gamification. As literature has shown, a traditional library tour may have a place in the list of library services, but for whom and at what cost are questions with limited answers in studies done to date. Gamification has offered an alternative perspective but with narrow accounts of its success in the online storytelling format and for users outside of the heavily focused freshman group. Across all literature of library orientation studies, there is little reference to other library user populations such as faculty, staff, community users, distance students, or students not formally part of a class that requires library orientation. DEVELOPMENT OF THE LIBRARY GAME ORIENTATION (LIBGO) LibGO was developed by the authors with not only a consideration for the Walker Library user experience, but also with a specific attention to the differing needs of the multiple user groups served by the library. This user-focused concern led to exploring creative methodologies such as user experience research and human-centered design thinking, a process of overlapping phases that produces a creative and meaningful solution in a non-linear way. The three pillars of design INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 6 thinking are inspiration, ideation, and iteration.33 Defining the problem and empathizing with the users (inspiration) led into the ideation phase, whereby the authors created low- and high-fidelity prototypes. The prototypes were tested and improved (iteration) through the use of beta testing in which playtesters interacted with the gamified orientation. The authors were novice developers of the gamified orientation, and this entailed a learning curve for not only the design thinking mindset but also the technical achievability. The development started with design thinking conversations and quickly turned to low-fidelity prototypes designed on paper. The development soon advanced to the actual coding so that the authors could get early designs tested before launching the final version. Prior to deployment on the library’s website, LibGO underwent a series of playtesting by library faculty, staff, and student employees. This testing was invaluable and led to such improvements as streamlining of processes and less ambiguity of text. LibGO was developed with the Twine open-source software (https://twinery.org), a product which is primarily used for telling interactive, non-linear stories with HTML. Twine was an excellent application for this project as it allowed the creation of an online and interactive “choose your own adventure” styled library orientation game, in which users could explore the library based upon their selection of one of multiple available plot directions. With a modest learning curve and as an open source software, Twine is highly accessible for those who are not accustomed to coding. For those who know HTML, CSS, JavaScript, variables, and conditional logic, Twine’s capabilities can be extended. The library’s interactive orientation adventure requires users to select one of the four available personas: undergraduate student, graduate student, faculty, or community member. Users subsequently follow that persona through a non-linear series of places, resources and points of interest built with the HTML output of using Twee (Twine’s programming language). See figure 2 for an example point of interest page and figure 3 for an example of a user’s final score after completing the gamified experience. Once the Twine story went through several iterations of design and testing, the HTML file was placed on the library’s website for the gamified orientation to be implemented with actual users. https://twinery.org/ INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 7 Figure 2. This instructional page within LibGO explains how to reserve different library spaces online. Upon reading this content, the user will progress by clicking on one of the hypertext lines in blue font at the bottom. Figure 3. Based upon the displayed avatar, this LibGO page is representative of a graduate student’s completion of LibGO. The page indicates the player’s final score and gives additional options to return to the home page or complete the survey. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 8 Purpose of Study LibGO utilized the common "choose your own adventure" format whereby players progress through a storyline based upon their selection of one of multiple available plot directions. Although the literature suggests that other technology-based methods are an engaging and instructive mode of content delivery, little prior research exists regarding this specific approach to library outreach. Furthermore, no previous research appears to have been conducted to understand the different online, gamified orientation needs of various library groups. The researchers wanted to understand the potential of interactive storytelling as a means to educate a range of users on library services as well as make the library more approachable from a user perspective. The study was designed to understand the user experience of each of the four groups. The researchers hoped to discern which users, if any, found the gamified experience to be a helpful method of orientation to the library’s physical and electronic services. Another area of inquiry was to determine whether this might be an effective delivery method by which to target certain segments of the campus for outreach. Finally, the study intended to determine whether this method of orientation might incline participants toward future use of the library. METHODOLOGY Overview The authors selected an embedded mixed methods design approach in which quantitative and qualitative data were collected concurrently through the same assessment instrument.34 The survey instrument primarily collected quantitative data, however a qualitative open-response question was embedded at the end of the survey: this question gathered additional data by which to answer the research questions. Each data set (one quantitative and one qualitative) was analyzed separately for each participant group, and then the groups were compared to develop a richer understanding of participant behavior. Research Questions The data collection and subsequent analysis attempted to answer the following questions: 1. Which group(s) of library users prefer to be oriented to library services and resources through the interactive storytelling format, as compared to other formats? 2. Which group(s) of library users are more likely to use library services and resources after participating in the interactive storytelling format of orientation? 3. What are user impressions of LibGO, and are there any differences in impression based on the characteristics of the unique user group? Participants Participants for the study were recruited in-person and via the library website. In-person recruitment entailed the distribution of flyers and use of signage to recruit participants to play LibGO in a library computer lab during a one-day event. Online recruitment lasted approximately ten weeks and simply involved the placement of a link to LibGO on the home page of th e library’s website. A total of 167 responses were gathered through both methods and participants were distributed as shown in table 1. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 9 Table 1. Composition of Study’s Participants Group number Affiliation Number of responses 1 Undergraduate students 55 2 Graduate students 62 3 Faculty 13 4 Staff 28 5 Community members 9 TOTAL 167 For the purposes of statistical data analysis, groups 3 and 4 were combined to produce a single group of 41 university employee respondents; also, group 5’s data was not included in the statistical analysis due to the low number of participants. Qualitative data for all groups, however, was included in the non-statistical analysis. Survey Instrument A survey with twelve total questions was developed for this study and was administered online through Qualtrics. After playing LibGO, participants were asked to voluntarily complete the survey; if they agreed, they were redirected to the survey’s website. Before answering any survey questions, the instrument administered an informed consent statement to participants . All aspects of the research, including the survey instrument, were approved through the university’s Institutional Review Board (protocol number 18-1293). The first part of the survey (see appendix A) consisted of ten questions, each with a ten-point Likert scaled response. The first five questions were each designed to measure a Preference construct, and the next five questions each measured a Likelihood construct. The Pref erence construct referred to participant’s preference for a library orientation: did they prefer LibGO’s online interactive storytelling format, or did they prefer another format such as in-person talks? The Likelihood construct referred to the participant’s self-perceived likelihood of more readily engaging with the library in the future (both in-person and online) after playing LibGO. The second part of the survey gathered the participant’s self-reported affiliation (see table 1 for the list of possible group affiliations) as well as offered participants an open-ended response area for optional qualitative feedback. Data Collection The study’s data was collected in two stages. In stage one, LibGO was unveiled to library visitors during a special campus-wide week of student programming events. On the library’s designated event day, the researchers held a drop-in event at one of the library’s computer labs (see figure 4 for an example of event advertisement). Library visitors were offered a prize bag and snacks if they agreed to play LibGO and complete the survey. During the three-hour-long drop-in session, 58 individual responses were collected: the vast majority of these came from undergraduate students (51 responses), with additional responses from graduate students (n = 4), university staff employees (n = 2), and one community member responding. Community members were defined as anyone not currently directly affiliated with the university; this group may have included prospective students or alumni. Stage 2 began the following day after the library drop-in event, and simply involved the placement of a link to LibGO on the home page of the library’s website. Any visitor to the library’s website could click on the advertisement to be taken to LibGO. This link INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 10 remained active on the library website for ten weeks, at which point the final data was gathered. A total of 167 responses were gathered during both stages and participants were distributed as previously shown in table 1. Figure 4. Example of Student LibGO Event Advertisement RESULTS Quantitative Findings Statistical analysis of each of the ten quantitative questions required the use of one-way ANOVA in SPSS. A post hoc test (Hochberg’s GT2) was run in each instance to account for the different sample sizes. For all statistical analysis, only the data from undergraduates, graduate students, and university employees (a group which combined both faculty and staff results) were utilized. A listing of mean comparisons by group, for each of the ten survey questions, may be found in table 2. The analysis of the one-way ANOVAs yielded statistically significant results for three of the ten individual questions in the first part of the survey: questions 2, 3, and 6 (see table 3). Table 2. Descriptive Statistics for Survey Results (10-point scale, with 10 as most likely) Survey Question Mean for Undergraduate Students Mean for Graduate Students Mean for University Employees 1. In considering the different ways to learn about Walker Library, do you find this library orientation game to be more or less preferable as compared to other orientation options (such as in-person tours, speaking with a librarian, or clicking through the library website on your own)? 7.02 6.39 6.02 2. In your opinion, was the library orientation game a useful way to get introduced to the library’s services and resources? 8.13 6.94 7.12 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 11 3. If your friend needed a library orientation, how likely would you be to recommend the game over other orientation options (such as in-person tours, speaking with a librarian, or clicking through the library website on your own?) 7.38 5.94 5.98 4. Please indicate your level of agreement with the following statement: “As compared to playing the game, I would have preferred to learn about the library’s resources and services by my own exploration of the library website?” 6.11 6.50 5.88 5. Please indicate your level of agreement with the following statement: “As compared to playing the game, I would have preferred to learn about the library’s resources and services through an in- person orientation tour.” 6.11 5.08 5.76 6. After playing this orientation game, are you more or less likely to visit Walker Library in person? 8.27 6.94 6.90 7. After playing this library orientation game, are you more or less likely to use the Walker Library website to find out about the library (such as hours of operation, where to go to get different materials/services, etc.)? 7.82 6.97 7.20 8. After playing this library orientation game, are you more or less likely to seek help from a librarian at Walker Library? 6.95 6.58 6.63 9. After playing this library orientation game, are you more or less likely to use the library’s online resources (such as databases, journals, e-books)? 7.67 7.15 6.90 10. After playing this library orientation game, are you more or less likely to attend a library workshop, training, or event? 6.96 6.73 6.24 TABLE 3. Overall Statistically Significant Group Differences df F p w2 Question 2 2 3.714 .027 .03 Question 3 2 4.508 .012 .04 Question 6 2 7.178 .001 .07 Question 2 asked “In your opinion, was the library orientation game a useful way to get introduced to the library’s services and resources?” The one-way ANOVA found that there was a statistically significant difference between groups (F(2,155) = 3.714, p = .027, ω2 = .03). The post hoc comparison using the Hochberg’s GT2 test revealed that undergraduates were statistically significantly more likely to prefer LibGO in this manner (M = 8.13, SD = 1.94, p = .031) as INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 12 compared to the graduate students (M = 6.94, SD = 2.72). There was no statistically significant difference between undergraduates and the university employees (p = .145). According to criteria suggested by Roger Kirk, the effect size of .03 indicates a small effect in perceived usefulness of LibGO as an introduction among undergraduates.35 Question 3 asked “If your friend needed a library orientation, how likely would you be to recommend the game over other orientation options (such as in-person tours, speaking with a librarian, or clicking through the library website on your own)?” The one-way ANOVA found that there was a statistically significant difference between groups (F(2, 155) = 4.508, p = .012, ω2 = .04). The post hoc comparison using the Hochberg’s GT2 test found that undergraduates were statistically significantly more likely to prefer LibGO over other orientation options (M = 7.38, SD = 2.49, p = .021) as compared to graduate students (M = 5.94, SD = 3.06). There was no statistically significant difference between undergraduates and university employees (p = .053). The effect size of .04 indicates a small effect regarding undergraduate preference for LibGO versus other orientation options. Question 6 asked “After playing this library orientation game, are you more or less likely to visit Walker Library in person?” The one-way ANOVA found that there was a statistically significant difference between groups (F(2,155) = 7.178, p = .001, ω2 = .07). The post hoc comparison using the Hochberg’s GT2 test revealed that undergraduates were statistically significantly more likely to visit the library after playing LibGO (M = 8.27, SD = 2.09, p = .003) as compared to graduate students (M = 6.94, SD = 2.20). Additionally, the test found that undergraduates were statistically significantly more likely to visit the library after playing LibGO (p = .007) as compared to university employees (M = 6.90, SD = 2.08). According to criteria suggested by Kirk, the effect size of .07 indicates a medium effect regarding undergraduate potential to visit the library in person after playing LibGO. 36 In addition to testing each individual survey question, tests were run to understand the possible group differences by construct (Preference and Likelihood). The Preference construct was an aggregate of survey questions 1-5, and the Likelihood construct was an aggregate of survey questions 6-10. For both constructs, the one-way ANOVA found results which were not statistically significant. In all, the quantitative findings indicated three areas by which the experience of playing LibGO was more helpful for the surveyed undergraduates than the other surveyed groups (i.e., graduate students or university employees). At this point, the analysis turned to the qualitative data so as to better understand participant views of LibGO. Qualitative Findings Analysis of the qualitative results was limited to the data collected in the survey’s final question. Question 12 was an open-response area, and was intentionally prefaced with a vague prompt: “Do you have any final thoughts for the library (suggestions, additions, modification, comments, criticisms, praise, etc.)?” Of the 167 total survey responses, 67 individuals chose to answer this question. Preliminary analysis showed that the feedback derived from this question covered a spectrum of topics, ranging from remarks on the LibGO experience itself to broader concerns regarding other library services. Open coding strategies were utilized to interpret the content of participant responses. Under this methodology, the responses were evaluated for general themes and then coded and grouped INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 13 under a constant comparative approach.37 NVivo 12 software was used to code all 67 participant responses. Initial coding yielded eight open codes, but these were later consolidated into six final codes (see table 4). One code (LibGO Improvement Tip) was rather nuanced and yielded five axial codes (see table 5). Axial codes denoted secondary concerns which fell under a larger category of interest. Although some participants gave longer feedback which addressed multiple concerns, care was taken to segregate each distinct concern to a specific code. Therefore, it is important to note that some comments addressed multiple concerns, and so the total number of concerns (n = 76) is greater than the total number of individuals responding to the prompt (n = 67). TABLE 4. Distribution of Qualitative Codes by User Group Code Undergraduate Graduate Faculty Staff Community member Total # concerns Positive feedback 7 7 1 4 2 21 Negative feedback 1 2 0 3 0 6 In-person tour preference 2 3 0 1 0 6 LibGO improvement tip 5 11 1 3 3 23 Library services feedback 2 4 3 0 0 9 Library building feedback 1 7 1 2 0 11 Total: 18 34 6 13 5 76 Discussion of Qualitative Themes Positive Feedback (21 separate concerns). Affirmative comments regarding LibGO were primarily split between undergraduate and graduate students, with a small number of comments coming from the other groups. Although all groups stated that the game was helpful, one undergraduate wrote “I wish I would’ve received this orientation at the very beginning of the year!” A graduate student declared “This was a creative way to engage students, and I think it should be included on the website for fun.” Both community members commented on the utility of LibGO in providing an orientation without having to physically come to the library; for example, “Interactive without having to actually attend the library in person which I liked.” Additionally, a community member pointed out the instructional capability of LibGO, writing “I think I learned more from the game than walking around in the library.” Negative Feedback (6 separate concerns). Unfavorable comments regarding LibGO primarily challenged the orientation’s characterization as a “game” in terms of its lack of fun. One graduate student wrote a comment representative of this concern by stating, “The game didn’t really seem like a game at all.” A particularly searing comment came from a university staff member who INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 14 wrote, “Calling this collection of web pages an ‘interactive game’ is a stretch, which is a generous way of stating it.” In-person Tour Preference (6 separate concerns). A small number of concerns indicated a preference for in-person orientations versus online. One undergraduate cited the ability to ask questions during an in-person tour as an advantage of that delivery medium. A graduate student mentioned their desire for kinesthetic learning over an online approach, writing, “I prefer hands - on exploration of the library.” LibGO Improvement Tip (23 separate concerns). Suggested improvements to LibGO were the largest area of qualitative feedback and produced five axial themes (subthemes); see table 5 for a breakdown of the five axial themes by group. 1. Design issues were the largest cited area of improvement, and the most commonly mentioned design problem was the inability of the user to go back to previously seen content. Although this functionality did in fact exist, it was apparently not intuitive to users; design modifications in future iterations are therefore critical. Other users made suggestions as to the color scheme used and the ability to magnify image sizes. 2. User experience was another area of feedback, and primarily included suggestions on how to make LibGO a more fun experience. One graduate student offered a role-playing game alternative. Another graduate student expressed an interest in a game with side missions, in addition to the overall goals, where tokens could be earned for completed missions; the student justified these changes by stating “I feel that incorporating these types of idea will make the game more enjoyable.” In suggesting similar improvements, one undergraduate stated that LibGO “felt more like a quiz than a game.” 3. Technology issues primarily addressed two related issues: images not loading and broken links. Images not loading could be dependent on many factors, including the user’s browser settings, internet traffic (volume) delaying load time, or broken image links, among others. Broken links could be the root issue since the images used in LibGO were taken from other areas of the library website. This method of gathering content pointed out a design vulnerability of using existing image locations (controlled by non-LibGO developers) rather than images exclusively for LibGO. 4. Content issues were raised exclusively by graduate students. One student felt that LibGO placed an emphasis on physical spaces in the library and did not give a deep enough treatment to library services. Another graduate student asked for “an interactive map to click on so that we physically see the areas” of the library, thus making the interaction more user-friendly with a visual. 5. Didn’t understand purpose is a subtheme where improvement is needed and is based on two comments made by the two university staff members. One wrote that “An online tour would have been better and just as informative,” although LibGO was not only designed to be an online tour of the library, but also an orientation of the library’s services. The other staff member wrote, “I read the rules but it was still unclear what the objective was.” In all, it is clear that LibGO’s purpose was confusing for some. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 15 Table 5. LibGO Improvement Tip Axial Codes by User Group Axial Code Undergraduate Graduate Faculty Staff Community member Total # concerns Design 4 3 0 0 1 8 User experience 1 2 1 0 1 5 Tech issue 0 1 0 1 0 2 Content 0 5 0 0 1 6 Didn’t understand purpose 0 0 0 2 0 2 Total: 5 11 1 3 3 23 Library Services Feedback (9 separate concerns). Several participants took the opportunity to provide feedback on general library services rather than on LibGO itself. Undergraduates simply gave general positive feedback about the value of the library, but many graduate students gave recommendations regarding specific electronic resource improvements. Additionally, one graduate student wrote, “I think it is critical to meet with new graduate students before they start their program,” something the library used to do but had not pursued in recent years. Although these comments did not directly pertain to LibGO, the authors accepted all of them as valuable feedback to the library. Library Building Feedback (11 separate concerns). This was another theme in which graduate students dominated the comments. Feedback ranging from requests for microwave use, additional study tables and better temperature control in the building appeared. Several participants asked for greater enforcement of quiet zones. Like the Library Services Feedback, the authors again took these comments as helpful to the overall library rather than LibGO. DISCUSSION The results of this study indicated that some groups of library visitors better received the gamified library orientation experience than other groups. Undergraduate students indicated the largest appreciation for a library orientation via LibGO. Specifically, they demonstrated a statistically significant difference over the other groups in supporting LibGO’s usefulness as an orientation tool, a preference for LibGO over other orientation formats, and a likelihood of future use of the physical library after playing LibGO. These very encouraging results provide evidence for the efficacy of alternative means of library orientation. The qualitative results provided additional helpful insight regarding the user impressions from each of the five surveyed groups. This feedback demonstrated that a variety of groups benefited from the experience of playing LibGO, including some community members who appreciated LibGO as a means of becoming acclimated to the library without having to enter the building. A virtual orientation format was not ideal for a few players who indicated a preference for a face-to- face orientation due to the ability to ask questions. Many people identified areas of improvement for LibGO. Graduate students in particular offered a disproportionate number of suggestions as compared to the other groups. While they provided a great deal of helpful feedback, it is possible that graduate students were so distracted by the perceived problems that they could not fully take in the experience or gain value from LibGO’s orientation purpose. It is also very likely that LibGO INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 16 simply was not very fun for these players: several players noted that it did not feel like a game but rather a collection of content. The review of literature indicated that this amusement issue is a common pitfall of educational games. Although the authors tried to design an enjoyable orientation experience, it is possible that more work is needed to satisfy user expectations. The mixed-methods design of this study was instrumental in providing a richer understanding of user perceptions. While the statistical analysis of participant survey responses was very helpful in identifying clear trends between groups, the qualitative analysis helped the authors draw valuable conclusions. Specifically, the open-response data demonstrated that additional groups such as graduate students and community members appreciated the experience of playing LibGO; this information was not readily apparent through the statistical analysis. Additionally, the qualitative analysis demonstrated that many groups had concerns regarding areas of improvement that may have impaired their user experience. These important findings could help guide future directions of the research. In all, the authors concluded this phase of the research feeling satisfied that LibGO showed great promise for library orientation delivery but could benefit from continued development and future user assessment. Although undergraduate students seemed most receptive overall to a virtual orientation experience, other groups appeared to have benefited from the resource. STUDY LIMITATIONS A primary limitation of this study was its small sample size. As the entire university campus was targeted for participation in the study, the number of respondents was far too small to generalize the results. Despite this limitation however, the study’s population reflected many different groups of library patrons on campus. The findings are therefore valuable as a means of stimulating future discussion regarding the value of alternative library orientation methods utilizing gamification. Another limitation is that the authors did not pre-assess the targeted groups for their prior knowledge of Walker Library services and building layout, nor for their interest in learning about these topics. It is possible that various groups did not see the value in learning about the library for a variety of reasons. Faculty members, in particular, may have considered their prior knowledge adequate for navigating the electronic holdings or building layout without recognizing the value of the other many services offered physically and electronically by the library. All groups may have experienced a level of “library anxiety” that prevented them from being motivated to learn more about the library.38 It is difficult to understand the range of covariate factors without a pre-assessment. Finally, there was qualitative evidence supporting the limitation that LibGO did not properly convey its stated purpose of orientation rather than imparting research skills. Without understanding LibGO’s focus on library orientation, users could have been confused or disappointed by the experience. Although care was taken to make this purpose explicit, some users indicated their confusion in the qualitative data. This observed problem points to a design flaw that undoubtedly had some bearing on the study’s results. CONCLUSION & FUTURE RESEARCH Convinced of the importance of the library orientation, the authors sought to move this traditional in-person experience to a virtual one. The quantitative results indicated that the gamified INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 17 orientation experience was useful to undergraduate students in its intended purpose of acclimating users to the library, as well as encouraging their future use of the physical library. At a time in which physical traffic to the library has shown a marked decline, new outreach strategies should be considered.39 The results were also helpful in showing that this particular iteration of the gamified orientation was preferred over other delivery methods by undergraduate students, as compared to other groups, to a statistically significant level. This is an important finding as it demonstrates that a diversified outreach strategy is necessary: different groups of library patrons desire their orientation information in different formats. The next logical question to ask however is: Why did the other groups examined through the statistical data analysis (graduate students and faculty) not appreciate the gamified orientation to the same level as undergraduates? The answers to this question are complicated and may be explained in part by the qualitative analysis. Based upon those findings, it is possible that the game did not appeal to these groups on the basis of fun or enjoyment; this concern was specifically mentioned by graduate students. Faculty members, including staff, provided a smaller level of qualitative feedback; it is therefore difficult to speculate as to their exact reasons for disengagement with LibGO. With this concern in mind, the authors would like to concentrate their next iteration of research on the specific library orientation needs of graduate students and faculty. Both groups present different, but critical, needs for outreach. Graduate students were the largest group of survey respondents, presumably indicating a high level of interest in learning more about the library. Many graduate programs at MTSU are delivered partially or entirely online; as a result, these students may be less likely to come to campus. Due to graduate students’ relatively infrequent visits to campus, a virtual library orientation could be even more meaningful for them in meeting their need for library services information. Faculty are another important group to target because if they lack a full understanding of the library’s offerings, they are unlikely to assign assignments that wholly utilize the library’s services. Although it is possible that faculty prefer an in-person orientation, many new faculty have indicated limited availability for such events. A virtual orientation seems conducive to busy schedules. However, it is possible that the issue is simply a matter of marketing: faculty may not know that a virtual option is available, nor do they necessarily understand all that the library has to offer. In all, future research should begin with a survey to understand what both groups already know about the library, as well as the library services they desire. Another necessary step in future research would be the expansion of the development team to include computer programmers. Although the authors feel that LibGO holds great promise as a virtual orientation tool, more needs to be done to enhance the user’s enjoyment of the experience. Twine is a user-friendly software that other librarians could pick up without having to be computer programmers; however, programmers (professional or student) could bring a design expertise to the project. Future iterations of this project should incorporate the skills of multiple groups, including expertise in libraries, user research, visual design, interaction design, programming, marketing, and testers from each type of intended audience. Collectively, this group will have the greatest impact on improving the user experience and ultimately the usefulness of a gamified orientation experience. This experience with gamification, and specifically interactive storytelling, was a valuable experience for Walker Library. These results should encourage other libraries seeking an alternate INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 18 delivery method for orientations. The authors hope to build upon the lessons learned from this mixed methods research study of LibGO to find the correct outreach medium for their range of library users. ACKNOWLEDGMENTS Special thanks to our beta playtesters and student assistants who worked the LibGO Event, which was funded, in part, by MT Engage and Walker Library at Middle Tennessee State University. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 19 APPENDIX A: SURVEY INSTRUMENT INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 20 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 21 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 22 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 23 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 24 ENDNOTES 1 Sandra Calemme McCarthy, “At Issue: Exploring Library Usage by Online Learners with Student Success,” Community College Enterprise 23, no. 2 (January 2017): 27–31; Angie Thorpe et al., “The Impact of the Academic Library on Student Success: Connecting the Dots,” Portal: Libraries and the Academy 16, no. 2 (2016): 373–92, https://doi.org/10.1353/pla.20160027. 2 Steven Ovadia, “How Does Tenure Status Impact Library Usage: A Study of LaGuardia Community College,” Journal of Academic Librarianship 35, no. 4 (January 2009): 332–40, https://doi.org/10.1016/j.acalib.2009.04.022. 3 Chris Leeder and Steven Lonn, “Faculty Usage of Library Tools in a Learning Management System,” College & Research Libraries, 75, no. 5 (September 2014): 641–63, https://doi.org/10.5860/crl.75.5.641. 4 Kyle Felker and Eric Phetteplace, “Gamification in Libraries: The State of the Art,” Reference and User Services Quarterly 54, no. 2 (2014): 19-23, https://doi.org/10.5860/rusq.54n2.19; Nancy O’Hanlon, Karen Diaz, and Fred Roecker, “A Game-Based Multimedia Approach to Library Orientation,” (paper, 35th National LOEX Library Instruction Conference, San Diego, May 2007), https://commons.emich.edu/loexconf2007/19/; Leila June Rod-Welch, “Let’s Get Oriented: Getting Intimate with the Library, Small Group Sessions for Library Orientation,” (paper, Association of College and Research Libraries Conference, Baltimore, March 2017), http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/201 7/LetsGetOriented.pdf. 5 Kelly Czarnecki, “Chapter 4: Digital Storytelling in Different Library Settings,” Library Technology Reports, no. 7 (2009): 20-30; Rebecca J. Morris, “Creating, Viewing, and Assessing: Fluid Roles of the Student Self in Digital Storytelling,” School Libraries Worldwide, no. 2 (2013): 54–68. 6 Sandra Marcus and Sheila Beck, “A Library Adventure: Comparing a Treasure Hunt with a Traditional Freshman Orientation Tour,” College & Research Libraries 64, no. 1 (January 2003): 23–44, https://doi.org/10.5860/crl.64.1.23. 7 Lori Oling and Michelle Mach, “Tour Trends in Academic ARL Libraries,” College & Research Libraries, 63, no. 1 (January 2002): 13-23, https://doi.org/10.5860/crl.63.1.13. 8 Kylie Bailin, Benjamin Jahre, and Sarah Morriss, “Planning Academic Library Orientations: Case Studies from Around the World,” (Oxford, UK: Chandos Publishing, 2018): xvi. 9 Bailin, Jahre, and Morriss, “Planning Academic Library Orientations.” 10 Marcus and Beck, “A Library Adventure”; A. Carolyn Miller, “The Round Robin Library Tour,” Journal of Academic Librarianship 6, no. 4 (1980): 215–18; Michael Simmons, “Evaluation of Library Tours,” EDRS, ED 331513 (1990): 1-24. 11 Marcus and Beck, “A Library Adventure”; Oling and Mach, “Tour Trends”; Rod-Welch, “Let’s Get Oriented.” https://doi.org/10.1353/pla.20160027 https://doi.org/10.1016/j.acalib.2009.04.022 https://doi.org/10.5860/crl.75.5.641 https://commons.emich.edu/loexconf2007/19/ http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LetsGetOriented.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LetsGetOriented.pdf https://doi.org/10.5860/crl.64.1.23 https://doi.org/10.5860/crl.63.1.13 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 25 12 Pixey Anne Mosley, “Assessing the Comfort Level Impact and Perceptual Value of Library Tours,” Research Strategies 15, no. 4 (1997): 261–70, https://doi.org/10.1016/S0734- 3310(97)90013-6. 13 Mosley, “Assessing the Comfort Level Impact and Perceptual Value of Library Tours.” 14 Marcus and Beck, “A Library Adventure,” 27. 15 Kenneth J. Burhanna, Tammy J. Eschedor Voelker, and Jule A. Gedeon, “Virtually the Same: Comparing the Effectiveness of Online Versus In-Person Library Tours,” Public Services Quarterly 4, no. 4(2008): 317–38, https://doi.org/10.1080/15228950802461616. 16 Burhanna, Voelker, and Gedeon, “Virtually the Same,” 326. 17 Burhanna, Voelker, and Gedeon, “Virtually the Same,” 329. 18 Felker and Phetteplace, “Gamification in Libraries.” 19 Felker and Phetteplace, “Gamification in Libraries,”20. 20 Felker and Phetteplace, “Gamification in Libraries.” 21 Felker and Phetteplace, “Gamification in Libraries”; O’Hanlon et al., “A Game-Based Multimedia Approach.” 22 Mary J. Broussard and Jessica Urick Oberlin, “Using Online Games to Fight Plagiarism: A Spoonful of Sugar Helps the Medicine Go Down,” Indiana Libraries 30, no. 1 (January 2011): 28–39. 23 Melissa Mallon, “Gaming and Gamification,” Public Services Quarterly 9, no. 3 (2013): 210–21, https://doi.org/10.1080/15228959.2013.815502. 24 J. Long, “Chapter 21: Gaming Library Instruction: Using Interactive Play to Promote Research as a Process,” Distributed Learning (January 1, 2017), 385–401, https://doi.org/10.1016/B978-0- 08-100598-9.00021-0. 25 Rod-Welch, “Let’s Get Oriented.” 26 O’Hanlon et al., “A Game-Based Multimedia Approach.” 27 Mallon, “Gaming and Gamification.” 28 Anna-Lise Smith and Lesli Baker, “Getting a Clue: Creating Student Detectives and Dragon Slayers in Your Library,” Reference Services Review 39, no. 4 (November 2011): 628–42, https://doi.org/10.1108/00907321111186659. 29 Monica Fusich et al., “HML-IQ: Frenso State’s Online Library Orientation Game,” College & Research Libraries News 72, no. 11 (December 2011): 626–30, https://doi.org/10.5860/crln.72.11.8667. https://doi.org/10.1016/S0734-3310(97)90013-6 https://doi.org/10.1016/S0734-3310(97)90013-6 https://doi.org/10.1080/15228950802461616 https://doi.org/10.1080/15228959.2013.815502 https://doi.org/10.1016/B978-0-08-100598-9.00021-0 https://doi.org/10.1016/B978-0-08-100598-9.00021-0 https://doi.org/10.1108/00907321111186659 https://doi.org/10.5860/crln.72.11.8667 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2020 APPLYING GAMIFICATION TO THE LIBRARY ORIENTATION | REED AND MILLER 26 30 Broussard and Oberlin, “Using Online Games”; Fusich et al., “HML-IQ”; O’Hanlon et al., “A Game- Based Multimedia Approach.” 31 Felker and Phetteplace, “Gamification in Libraries.” 32 Felker and Phetteplace, “Gamification in Libraries”; Fusich et al., “HML-IQ.” 33 “Design Thinking for Libraries: A Toolkit for Patron-Centered Design,” Ideo (2015), http://designthinkingforlibraries.com. 34 John W. Creswell and Vicki L. Plano Clark, Designing and Conducting Mixed Methods Research (Thousand Oaks, CA: Sage Publications, 2007). 35 Roger Kirk, “Practical Significance: A Concept Whose Time Has Come,” Educational and Psychological Measurement, no. 5 (1996). 36 Kirk, “Practical Significance.” 37 Sandra Mathison, “Encyclopedia of Evaluation,” SAGE, 2005, https://doi.org/10.4135/9781412950558. 38 Rod-Welch, “Let’s Get Oriented.” 39 Felker and Phetteplace, “Gamification in Libraries.” http://designthinkingforlibraries.com/ https://doi.org/10.4135/9781412950558 ABSTRACT INTRODUCTION Background Literature Review Traditional Orientation Gamification Use in Libraries Development of the Library Game Orientation (LibGO) Purpose of Study Methodology Overview Research Questions Participants Survey Instrument Data Collection Results Quantitative Findings Qualitative Findings Discussion of Qualitative Themes Discussion Study Limitations Conclusion & Future Research Acknowledgments Appendix A: Survey Instrument ENDNOTES