Tech Tools in Pandemic-Transformed Information Literacy Instruction: Pushing for Digital Accessibility ARTICLE Tech Tools in Pandemic-Transformed Information Literacy Instruction Pushing for Digital Accessibility Amanda Rybin Koob, Kathia Salomé Ibacache Oliva, Michael Williamson, Marisha Lamont-Manfre, Addie Hugen, and Amelia Dickerson INFORMATION TECHNOLOGY AND LIBRARIES | DECEMBER 2022 https://doi.org/10.6017/ital.v41i4.15383 Amanda Rybin Koob (amanda.rybinkoob@colorado.edu) is Assistant Professor, Literature and Humanities Librarian, University of Colorado. Kathia Salomé Ibacache Oliva (kathia.ibacache@colorado.edu) is Assistant Professor, Romance Languages Librarian, University of Colorado. Michael Williamson (michael.d.williamson@colorado.edu) is Assistant Director, Assessment and Usability, Digital Accessibility Office, University of Colorado. Marisha Lamont-Manfre (marisha.manfre@colorado.edu) is Accessibility and Usability Assessment Coordinator, Digital Accessibility Office, University of Colorado. Addie Hugen (addison.hugen@colorado.edu) is Senior Accessibility Tester, Digital Accessibility Office, University of Colorado. Amelia Dickerson (amelia.dickerson@Colorado.edu) is Accessibility Professional, Digital Accessibility Office, University of Colorado. © 2022. ABSTRACT Inspired by pandemic-transformed instruction, this paper examines the digital accessibility of five tech tools used in information literacy sessions, specifically for students who use assistive technologies such as screen readers. The tools are Kahoot!, Mentimeter, Padlet, Jamboard, and Poll Everywhere. First, we provide an overview of the Americans with Disabilities Act (ADA) and digital accessibility definitions, descriptions of screen reading assistive technology, and the current use of tech tools in information literacy instruction for student engagement. Second, we examine accessibility testing assessments of the five tech tools selected for this paper. Our data show that the tools had severe, significant, and minor levels of digital accessibility problems, and while there were some shared issues, most problems were unique to the individual tools. We explore the implications of tech tools’ unique environments as well as the importance of best practices and shared vocabularies. We also argue that digital accessibility benefits all users. Finally, we provide recommendations for teaching librarians to collaborate with campus offices to assess and advance the use of accessible tech tools in information literacy instruction, thereby enhancing an equitable learning environment for all students. INTRODUCTION The last fifteen years have seen the rise of collaborative and interactive web platforms and whiteboards, game-based learning technologies, audience polls, and other tools that contribute to student engagement in higher education classrooms. These educational tech tools have supported one-time library information literacy (IL) sessions by enabling student participation in real time. Still, knowing that tech tools may enhance engagement is not enough; we should also be asking whether these tech tools are accessible for all students and, if not, what can be done to make them more accessible. This paper examines the digital accessibility of five tech tools specifically for students who use assistive technologies such as screen readers. The tools are Kahoot!, Mentimeter, Padlet, mailto:amanda.rybinkoob@colorado.edu mailto:kathia.ibacache@colorado.edu mailto:michael.d.williamson@colorado.edu mailto:marisha.manfre@colorado.edu mailto:addison.hugen@colorado.edu mailto:amelia.dickerson@Colorado.edu INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 2 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Jamboard, and Poll Everywhere. These tech tools were identified in a 2021 paper inquiring which tech tools librarians used in emergency remote IL instruction during the COVID-19 pandemic along with their perceptions of the weaknesses and strengths of these tech tools.1 Although there are guidelines aiding librarians in assessing ADA accessibility around library spaces, there are no disability-related recommendations for specific tech tools used in IL instruction or studies examining tech tools’ digital accessibility features. 2 There is also a lack of documentation regarding librarians’ outreach to ADA-related academic offices and tech companies regarding tech tools. We argue that collaboration between libraries and ADA-related offices at the campus level increases awareness of digital accessibility issues and requirements and could ultimately advance digital accessibility in educational tech tools used in IL instruction. We place our paper within the context of other pandemic-responsive digital pedagogy research. We acknowledge that technology needs for student engagement are evolving in new face-to-face, hybrid, and remote instruction environments; thus, we hope to impact the way tech tools are assessed for digital accessibility and to promote the use of accessibility-tested tech tools in library instruction. First, we provide an overview of ADA and digital accessibility definitions, descriptions of screen reading assistive technology, and the current use of tech tools in instruction for student engagement. Secondly, we examine accessibility testing reports for the five tech tools selected for this paper. Then, we discuss two trends found in the reports: shared issues between the tools and the implications of unique environments. We also argue that digital accessibility benefits all users. Finally, we provide recommendations for teaching librarians to collaborate with campus offices to assess and advance the use of accessible tech tools in IL instruction, thereby enhancing an equitable learning environment for all students. OVERVIEW ADA Accessibility The Americans with Disabilities Act (ADA) was made law in 1990, signaling an initiative to protect people with disabilities from discrimination in employment opportunities, when purchasing goods, and when participating in state and local government services. 3 The idea behind the ADA law was to provide equal opportunity.4 However, as health sciences librarian Ariel Pomputius notes, ADA law protects people from discrimination, but it does not guarantee a right to accessibility beyond the legal requirements granted by this act.5 As higher education advances through the COVID-19 pandemic, digital accessibility has become more essential than ever in IL instruction as it takes place in hybrid, remote, and in -person environments. To ensure the digital accessibility of tech tools for all students, we should first understand its meaning. What is Digital Accessibility? The COVID-19 pandemic brought digital accessibility to the forefront as universities navigated complex remote and hybrid learning environments. Fernando H. F. Botelho, a scholar with expertise in technology and disability, explains digital accessibility as the interconnection of “hardware design, software development, content production, and standards definition.”6 For Botelho, accessibility is “an ongoing and dynamic process” rather than an immobilized state, where standards work together as a part of a ubiquitous process.7 As information studies INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 3 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON professor Jonathan Lazar notes, “Digital accessibility means providing an equal user experience for people with disabilities, and it never happens by accident.” 8 Georgetown Law also defines digital accessibility from a perspective that may resonate with instructors who seek technologies that are accessible to all students. They define digital accessibility as “the inclusive practice of removing barriers that prevent interaction with, or access to websites, digital tools, and technologies.”9 However, it is Lazar who moves the topic forward when referring to digital accessibility in research libraries, arguing that although accessibility laws protect people with disabilities, digital accessibility also benefits the whole population.10 Lazar made this assertion after capturing the challenges and lessons learned related to digital accessibility during COVID-19.11 The most salient lesson is that research libraries should create an infrastructure that supports digital accessibility, especially now that the COVID-19 pandemic has driven universities to provide instruction in multiple formats.12 We argue that this infrastructure should also include digital accessibility evaluation of tech tools used in the classroom. Assistive Technology for Blind Users Congress defined assistive technology in the Disabilities Act of 1988 as “any item, piece of equipment, or product system, whether acquired commercially off the shelf, modified, or customized, that is used to increase, maintain, or improve functional capabilities of individuals with disabilities.”13 Furthermore, special education professors Kathleen Puckett and Kim W. Fisher state, “Technology becomes assistive when it supports an individual … to accomplish tasks that would otherwise be difficult or impossible.”14 As scholars of occupational therapy Claire Kearney-Volpe and Amy Hurst note, screen readers assist people with no or low vision by presenting web information on “a non-visual interface” via Braille or speech synthesis.15 Screen readers’ purpose is important because all people should have the opportunity to access the same information and services in the digital environment without facing undue barriers or burdens. The Digital Accessibility Office’s (DAO) assessment and usability team at University of Colorado Boulder (CU Boulder) primarily tests tools for accessibility by utilizing screen reader assistive technology for both computers and mobile devices. Assessment and usability staff rely on screen readers for testing because this assistive technology uses and responds to the underlying code of each webpage, application, and environment. This in-depth output makes screen readers good tools for overall accessibility testing, even though they are generally for people with no vision. However, we found no studies on tech tools and classroom engagement that consider assistive technology such as screen readers. Classroom Engagement with Tech Tools Academic librarians Emily Chan and Lorrie Knight state in a 2010 study that library instruction risks being anachronistic if it does not include an engaging technology-based activity.16 With this in mind, there is ample literature documenting the impact and benefits of tech tools in the classroom. For example, authors highlight tech tools’ anonymous environment, categorized as free of judgment, noting that it is student-centered and enhances student participation.17 Moreover, anonymity provides a means for students to answer honestly, fostering classroom discussion that includes introverted students.18 On the other hand, some authors argue that anonymous participation does not enhance critical thinking. Ann Rosnida Md Deni and Zainor Izat Zainal, referring to Padlet as an educational tool, INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 4 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON argue that one challenge of using tech tools to advance student engagement is that they do not, on their own, enhance criticality or discussion because students may not want to oppose their peers’ opinions.19 As with other pedagogical techniques, intentional facilitation with tech tools is necessary to enhance criticality. Many authors regard the use of tech tools in the classroom positively.20 Examining Kahoot! to test students’ performance, Darren H. Iwamoto et al. state that students valued receiving immediate feedback on their answers after taking a high-stakes examination.21 Carolyn M. Plump and Julia LaRosa also appreciate the use of instructional games to provide immediate feedback to students, noting that this feedback warned faculty instructors against making assumptions about how much students understand in class.22 Similarly, librarian Maureen Knapp, referring to online tools for active learning, notes that instant feedback drives classroom discussions forward.23 Liya Deng, a librarian with a focus on disability studies, notes in a 2019 study that using Poll Everywhere in library instruction provides an opportunity to build rapport with students and a strategy to keep students focused and away from non-instruction-related internet distractions.24 Engineering classes have also used tech tools to enhance teaching and learning. A 2021 case study addressing online education due to COVID-19 reports that students found Kahoot! to be a useful online tool that helped them reflect, apply knowledge, and receive feedback. 25 Similarly, engineering educator Razzaqul Ashahn advocates for incorporating tech tools like Jamboard for active “think-pair-share” activities, noting that it enables instructors to connect with students as they do small group work.26 These studies suggest that tech tools continue to be relevant and beneficial during the pandemic, though again, they do not consider whether they are digitally accessible. For this reason, the continued use of tech tools in various modalities (in-person, hybrid, and remote) attests to their relevance, which may continue to grow as instructors transition to pandemic-transformed pedagogy. Pandemic-Transformed Pedagogy In a 2020 publication exploring COVID-19 impacts on teaching, learning, and technology use, scholars Jillianne Code, Rachel Ralph, and Kieran Forde coined the phrase “pandemic-transformed pedagogy.”27 As they state, educators find themselves “on the cusp of a rapid change that is compelling them to re-think their worldview in both how they teach and how their students learn, calling for their transformation as educators.”28 A review of the recent literature available through Google Scholar on “pandemic-transformed pedagogy” shows expanding adoption of this phrase, including academics publishing on a range of interdisciplinary subjects and in international contexts, with implications for both K–12 and post-secondary education.29 As we reflect on this transformation and call for responsiveness to rapid change, we emphasize the need for support, planning, and advocacy for digital accessibility and tech tools. Before the COVID-19 pandemic, scholars at the University of Sydney found in 2018 that the most significant factor driving the choice to use technology was whether it was immediately available. 30 These scholars emphasized the “just in time” use, noting that ready access to the technology required “actions, expenditure, support, and commitment from policymakers and administrators.”31 At the beginning of the pandemic, teachers, librarians, and students had a matter of days to pivot to remote work, and as Ibacache, Rybin Koob, and Vance found in a 2021 study, “availability” was a consideration for librarians in selecting tech tools for engagement and content delivery.32 This “just in time” consideration is even more important in the aftermath of COVID-19, which prompted emergency remote learning. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 5 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Yet, teaching librarians also ought to go beyond what is easily available and move towards what is digitally accessible. Part of the transformation we envision is to extend the concept of “pandemic- transformed pedagogy” to include digital accessibility and thus push for the tech tools we use in IL instruction to be readily available and digitally accessible. METHODOLOGY As previously mentioned, this study examines the digital accessibility of five educational tech tools used in IL instruction. To initiate a formal accessibility test, we created scripts detailing how to interact with the samples we provided for the five tools.33 These scripts were then used to manually test each tech tool for its digital accessibility using a variety of screen readers on both computers and mobile devices. About the Testers The testers are native users of screen reading assistive technology and are blind. They test each tech tool first, with additional staff in the DAO reviewing and validating results. About the Test Scripts Process Each test script contained the following parameters: 1. Basic information about the tool. 2. Contact information for access issues and technical questions, such as the tools’ customer support email and librarians’ emails for follow-up questions. 3. Access points to the software and websites (URLs). 4. Step-by-step instructions for testers to impersonate a student engaging in an IL task. As a part of these test scripts, we created short sample quizzes and activities for each of the five tools considered in this paper. In addition, the test scripts provided step-by-step descriptions to help the testers interact with the tools. The testers then tried each tool, focusing on functionality and whether they could complete the tasks in the script. The reports describe three levels of problems: severe, significant, and minor. The results section of this paper reports on these problems as found with the five tools tested. The testers also assessed general user experience (usability). The testers used a holistic approach, engaging with the entire virtual environment of the tool rather than looking only at isolated functions. Assistive Technology The testers utilized four types of screen reader software: VoiceOver, TalkBack, NVDA, and JAWS. VoiceOver, developed by Apple, is a screen reader for mobile devices and computers that “comes standard on every iPhone, Mac, Apple Watch, and Apple TV. It is gesture-controlled, so by touching, dragging, or swiping[,] users can hear what’s happening on screen and navigate between elements, pages, and apps.”34 TalkBack is a Google-based screen reader included in Android mobile devices that functions similarly to VoiceOver.35 NVDA is a Microsoft Windows-only free open access screen reader supporting people who are blind or have vision impairment.36 JAWS, also compatible with Microsoft Windows, allows people with visual impairment to read the PC screens with a text-to-speech output or via Braille display.37 We also tested for visual usability issues using a free web-based color contrast analyzer.38 The testers provided thorough reports detailing the results of their testing, including exact versions used. The tests were conducted between February 27 and May 1, 2022. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 6 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Tools Evaluated The educational technology tools in this study are web-based and have free options, allowing students to engage in activities using their computers or their phones. We identified these tools based on a survey about tech tool use during the COVID-19 pandemic and from our own experiences.39 The tools are Jamboard, Kahoot!, Mentimeter, Padlet, and Poll Everywhere. Jamboard is a Google-powered virtual whiteboard tool. Kahoot! is a quiz/game platform offering multiple styles; we tested the standard quiz question format and utilized one of the vendor - provided sample quizzes. Mentimeter is another quiz-making platform; we created the sample quiz utilizing multiple choice and short answer question formats. Padlet is a collaborative bulletin board platform with various formats (including the three we tested: Padlet Maps, Padlet Shelf, and Padlet Wall). Padlet includes options for users to add text and multimedia in response to question prompts or to post their own questions and other content in a collaborative virtual space. Finally, Poll Everywhere is a polling/survey platform. Limitations Although digital accessibility offices at different universities commonly rely on shared standards for technology evaluation, such as Web Content Accessibility Guidelines (WCAG) 2.1, we acknowledge that the assessment approach will vary from office to office. Overall, there is much debate on which practices and standards for evaluating tech tools yield the best results. Not all higher education institutions have digital accessibility offices, let alone accessibility and usability labs and testers. Some institutions may rely on automated checkers or a mix of automated and manual testing. Approaches to testing differ, and there is disagreement among digital accessibility practitioners about whether a fully automated, fully manual, or hybrid approach is best. Regardless, we expect that manual testing of these educational tech tools using similar assistive technologies would have similar results during the timeframe these tools were tested. The testing reports capture a moment in time, and it’s important to note that web-based tools are frequently updated. We only tested the free versions of these tools. There may be differences in accessibility between free and paid versions. We tested only the browser versions of these tools on computers and mobile devices and did not test mobile applications, which may or may not be more accessible. This decision was made due to the probability that most IL librarians and other instructors would not regularly ask students to download applications to their personal devices for in-class engagement. Kahoot!, Mentimeter, Padlet, and Poll Everywhere were tested on Windows, iOS, and Android platforms. Jamboard was tested only on Windows, because the browser version would not open on a mobile device using assistive technology. Instead, it attempted to force an app download. We also tested each tool using sample environments and functions that we hope captured some ways the tools would be used in a typical IL classroom. Due to the nature of the tools and the many options available for question and collaboration formats in each tool, these samples were not exhaustive of all options available. These testing results are meant to be illustrative rather than comprehensive. Finally, this study evaluated tech tools only for digital accessibility using the specific assistive technology of screen readers. Further research is needed regarding how students with a rang e of different disabilities may interact with the technology tools examined here. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 7 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON RESULTS This section reports the three levels of problems (severe, significant, and minor) that DAO testers found in Jamboard, Kahoot!, Mentimeter, Padlet (Shelf, Map, and Wall), and Poll Everywhere. The testers also assessed user experience (“usability”). Issues may be present in multiple categories based on how they impact the user’s ability to complete actions. Severe Issues Table 1 shows the severe issues found in the tools tested. Severe issues create access barriers that prevent assistive technology users from completing tasks and are issues that need to be remediated. The testers consider these issues prohibitive for many individuals with disabilities and for those who use assistive technologies. The DAO identified ten severe issues in Padlet Shelf; five severe issues each in Jamboard, Padlet Wall, and Poll Everywhere; four severe issues in Kahoot!, and three severe issues in Padlet Map. The testers did not find severe issues in Mentimeter. Table 1 shows that the most common severe issue corresponds to elements that are unlabeled or inappropriately labeled. In the case of Padlet Map and Jamboard, the testers found buttons that were unlabeled or labeled with irrelevant numbers. Testers felt unclear as to what the buttons were or what their functionality was. Padlet Shelf contained the most unlabeled buttons, including the buttons to add posts and the three vertical dots menu to edit or delete. This issue is highly relevant since users need these buttons to navigate and contribute to the Padlet. The testers observed a similar problem when using the screen reader TalkBack to engage with Padlet Shelf. TalkBack found unknown or unlabeled buttons, which impede users’ ability to navigate or interact with videos they submit to the Padlet. Figure 1 illustrates the play button located at the center of a video. In the screen reader, this button is unlabeled and appears after the video, preventing the screen reader from understanding its function and leaving users unclear whether this button is connected to the video. Figure 1. The play button at the center of the video is unlabeled. In the reading order, this unlabeled button appears after the video; therefore, it is unclear what it does or how it relates to the video. The second most prevalent severe issue is elements that are not accessible to screen readers. This issue affected Padlet Shelf and Padlet Wall. In the case of Padlet Shelf, the testers utilizing the INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 8 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON VoiceOver screen reader were unable to interact with or locate GIFs and graphics. When the testers utilized TalkBack, they would hear the GIF but could not find the graphics because they were marked as links. In addition, the drawing feature was also not accessible to screen readers, including the visual elements that control colors, which appear as clickable links instead of the visual elements associated with colors. These elements were unavailable for users utilizing VoiceOver and JAWS. The testers found a similar problem with the visual elements in Padlet Wall, especially when they tried to edit a post (see fig. 2). Figure 2. When users want to edit a post in Padlet Wall, there are visual elements that are available to change the color of the post. These elements are not available to screen readers. Figure 3. When images are not programmed to be read as graphics, screen readers are not able to gather information related to the GIF. This image was read as “jAF3mI0ja5hUk/giphy.” INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 9 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Figure 4. While using NVDA, the user hears links for images that do not make sense. The third most frequent severe issue relates to graphics and GIFs that are not appropriately programmed. This issue affected Padlet Shelf and Padlet Wall. When the testers were using JAWS in Padlet Shelf, the GIFs read as links with the following text: “jAF3ml0ja5hUk/giphy.” When the testers utilized NVDA, the GIFs read as “giphy,” conveying no information describing the GIF and hindering navigation (fig. 3). Similarly, graphics and GIFs in Padlet Wall are programmed as links rather than graphics. When the testers used JAWS to understand graphics and GIFs, they heard long links such as: “eb351cc20e6bfda76d443f1e93ad7963/pumpkin_seedling_3.” Long links like this are useless to people using screen readers and disrupt people’s ability to search for graphics. When the testers used NVDA, they also heard links for the images, but without the other series of characters included in JAWS (fig. 4). The testers also found severe issues with elements not available by keyboard or screen reader (Jamboard and Poll Everywhere) and timer features (Kahoot!). For example, the pen, eraser, laser, shapes, and text box elements in Jamboard can only be utilized or placed on the screen by a mouse, making them inaccessible to blind learners. Another issue is the lack of alternative text. Since Jamboard offers a collaborative multi-user space, some users may post images. However, there is no way to input alternative text to an image. In the case of Kahoot!, when the timer is activated, the countdown plays as the screen reader tries to read the page, confusing the screen reader and the user, who will hear the timer with random numbers and not the question. The timer feature also affects the user when starting a quiz or moving between questions. It is unclear whether the screen reader is unable to read the questions due to the short timeframe or whether the questions are truly unavailable to the screen reader. The instructor may extend the timer for quizzes in Kahoot!, but it is impossible to turn it off altogether when using the Kahoot! quiz question format. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 10 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Table 1. Number of occurrences of severe issues found during screen reader testing for Kahoot!, Jamboard, Mentimeter, Padlet (three formats tested), and Poll Everywhere. (Jamboard was tested on a Windows computer only; the other tools were tested on Windows, iOS, and Android.) Severe Issue Jamboard Kahoot! Mentimeter Padlet: Map Padlet: Shelf Padlet: Wall Poll Every- where Total occurrences Element not available by keyboard or screen reader 2 0 0 0 0 0 1 3 Element presents gesture/navigation traps 0 0 0 0 0 0 1 1 Elements are not keyboard accessible 0 0 0 1 0 0 0 1 Elements are unlabeled or inappropriately labeled 2 0 0 1 4 2 0 9 Elements not accessible to screen reader 0 0 0 0 3 2 0 5 Errors do not get focus 0 0 0 0 0 0 1 1 Graphics and GIFs are not programmed appropriately 0 0 0 0 3 1 0 4 Graphics are unlabeled or inappropriately labeled 0 0 0 0 0 0 2 2 Graphics lack alternative text 1 0 0 0 0 0 0 1 Lack of alert 0 0 0 1 0 0 0 1 Text not read by screen reader 0 1 0 0 0 0 0 1 Timed pages disrupt the ability to read the page 0 3 0 0 0 0 0 3 Tool Totals 5 4 0 3 10 5 5 32 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 11 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON The testers found other severe issues such as text not being read by screen readers, missing notifications, elements not accessible to screen readers, unlabeled graphics, and lack of focus on images. For example, in the case of Kahoot!, the screen reader could not read the answer notification text. This issue meant that while the tool offered visual indicators for correct and incorrect answers, the screen reader did not read these indicators and it remained unclear to testers whether their answers were correct or not. Finally, “lack of focus” or challenges with “focus handling” indicate that the assistive technology’s attention was not where it should be. This problem happens because tool developers do not set the appropriate code for screen readers. Significant Issues Table 2 shows the significant issues found in the tools. Significant issues represent items that create great difficulty for people who use assistive technologies, but they do not necessarily prevent the tool from being used. Significant issues are recommended for remediation. Interestingly, most significant issues were not shared across the five tools; out of seventeen problems, only one was shared by four tools (“inconsistent focus handling”), and three were shared by two tools each (“graphics are inappropriately labeled,” “reading order can be confusing to users,” and “state is not indicated”). Because of this lack of overlap, brief descriptions of how frequent issues affected specific tech tools are warranted, focusing on those issues that affected multiple tools, recurred most frequently, or both. The significant issue that recurred most frequently was “reading order can be confusing to users,” affecting Jamboard as well as all three Padlet styles. In Jamboard, when creating a sticky note, the focus of the assistive technology went into the edit field but ignored the color options. This meant that users were unable to switch between colors when making a post. Reading order also caused difficulties. Reading order is the way elements are tagged and read by screen readers. This may not be the same order most sighted users experience when reading elements on the page from top to bottom, though it should closely reflect the visual layout of the page. It determines what a blind learner will understand about the digital environment and in what order. In Padlet Map, the screen reader went through irrelevant content, including the terms and conditions, before reading the “new post” button. Padlet Shelf had three instances of confusing reading order; for example, the “publish” and “update” options were in the reading order above the “edit” field. The user would have to know to navigate back to finalize their post (this issue is repeated in Padlet Wall as well). Further, if a user leaves the new post dialog box, it is difficult to return due to the reading order. The “more buttons” element was also read before the heading of a new post, and those additional buttons are unlabeled. Finally, in Padlet Wall, the tester utilizing VoiceOver could not discard a post (fig. 5). A dialog opened asking for discard confirmation, but this dialog was buried in the reading order and challenging to locate. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 12 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Figure 5. A dialog box appears visually in Padlet Wall when a user attempts to discard a post, but it is buried in the reading order of the VoiceOver screen reader, making it difficult to locate and complete the task. The next most frequent significant issue was “inconsistent focus handling,” which occurred six times. Focus handling directs the attention of the user and facilitates various actions in a given environment. Inconsistent focus handling emerged in four out of the five tools: Jamboard, Kahoot!, all three Padlet styles, and Poll Everywhere. This issue often appeared when a new element on the screen was opened, but the “focus” (what the screen reader was paying attention to at any given time) remained on the previous panel or element, causing confusion and difficulty. For example, in Jamboard, when selecting the “Open a Jamboard” button, the panel opened visually, but the screen reader’s focus remained on the button behind the open panel. To get to the new Jamboard, the tester had to navigate the other page content first. Focus handling was inconsistent across many activation buttons and interactions in all three Padlet styles. In Kahoot!, focus handling was inconsistent across screen readers, with the focus going to different places, such as after answering a question. In Poll Everywhere, the focus traveled to other areas of the page after answering a question, returning to previous ques tions, or refreshing the page. These inconsistencies varied among screen readers. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 13 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Table 2. Number of occurrences of significant issues found during screen reader testing for Jamboard, Kahoot!, Mentimeter, Padlet (three formats tested), and Poll Everywhere. (Jamboard was tested on a Windows computer only; the other tools were tested on Windows, iOS, and Android.) Significant Issue Jamboard Kahoot! Mentimeter Padlet: Map Padlet: Shelf Padlet: Wall Poll Every- where Total occurrences Difficult combination/list box 0 3 0 0 0 0 0 3 Element difficult to access 1 0 0 0 0 0 0 1 Element state not indicated 1 0 0 0 0 0 2 3 Error does not get focus 0 0 1 0 0 0 0 1 Extensive load times create difficulties 0 1 0 0 0 0 0 1 Graphics are inappropriately labeled 0 1 1 0 0 0 0 2 Graphics not programmed appropriately 1 0 0 0 0 0 0 1 Headings are not used 1 0 0 0 0 0 0 1 Inconsistent focus handling 1 1 0 1 1 1 1 6 Lack of alert 0 0 0 1 0 2 0 3 Lack of contextual text/information 0 1 0 0 0 0 0 1 Lack of focus indicators 0 0 0 1 0 1 0 2 Lack of notification 3 0 0 0 0 0 0 3 Object placement 1 0 0 0 0 0 0 1 Reading order can be confusing to users 1 0 0 1 3 2 0 7 User-created objects initially lack markup 0 0 0 0 0 1 0 1 Tool Totals 10 7 2 4 4 7 3 37 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 14 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Another issue that recurred across two tools (Kahoot!, Mentimeter) was “graphics are inappropriately labeled.” In Kahoot!, a graphic showing the final scoreboard from the quiz and a podium were hidden from screen readers. In Mentimeter, the logo for the tool is labeled as a “logotype” in the alt text. While these inappropriate labels for graphics may seem minor, they leave players using assistive technology out of celebratory or fun elements and may be confusing. Inappropriate labels cannot be corrected by instructors, who are unable to adjust the alt text for elements that are built into the software. Another issue that recurred was “state is not indicated” (Jamboard, Poll Everywhere). Here, “state” refers to any change or option for an element—the state of it in the digital environment. In Jamboard, there is no indication of what color is selected for sticky notes, for example, which can be problematic if instructors use color to convey meaning (fig. 6). In one test question on Poll Everywhere, the unlabeled image reads as clickable to NVDA, and visually, the image becomes larger when clicked. But this change is not announced and again may be confusing. Figure 6. For screen readers, there is not an indication of what color has been selected for sticky notes, though this is available visually. With ten issues listed, Jamboard was the tool with the greatest number of significant problems. This was true even though Jamboard was tested only on Windows and not on mobile devices. Padlet Wall and Kahoot! had seven significant issues each. This is a slight departure from the data in table 1, where Padlet Shelf had the most severe issues. In general, tools with severe issues consistently exhibited some significant issues as well. Figure 6. Users can enter text on Option 1 and Option 2, but these options do not generate a heading. Minor Issues Table 3 shows the minor issues found in the five technology tools. Minor issues represent items that are inconvenient or annoying, but do not necessarily create barriers to accessibility, e.g., repetitiveness, unclear text, etc. The testers found that each tool had between one and four minor issues of their own but did not share any of the minor issues listed. Kahoot! had three issues related to confusing elements: gibberish text heard on screen readers, blanks in the statement not read by screen readers, and an icon that shows the total number of users who finished a test, which the screen reader could not read. Other minor issues include instructions, questions, and INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 15 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Table 3. Number of occurrences of minor issues found during screen reader testing for Jamboard, Kahoot!, Mentimeter, Padlet (three formats tested), and Poll Everywhere. (Jamboard was tested on a Windows computer only; the other tools were tested on Windows, iOS, and Android.) Minor Issue Jamboard Kahoot! Mentimeter Padlet: Map Padlet: Shelf Padlet: Wall Poll Everywhere Total occurrences Element is inappropriately labeled 0 0 1 0 0 0 0 1 Elements confusing to users 0 3 0 0 0 0 0 3 Elements read twice 0 0 0 0 1 0 0 1 Heading level not concise 0 0 0 0 0 1 0 1 Headings are not used to provide structure 0 0 0 1 0 0 0 1 Headings used too often 0 0 0 0 1 0 0 1 Inconsistent focus handling 0 0 1 0 0 0 0 1 Labels are inconsistent 1 0 0 0 0 0 0 1 Lack of a programmatic list creates confusion 0 0 0 0 0 1 0 1 Lack of notification 0 0 0 0 0 1 0 1 Same information is presented to screen reader multiple times 0 0 0 0 0 0 1 1 Sound effect portray meaning 0 1 0 0 0 0 0 1 Submenu item count not provided 1 0 0 0 0 0 0 1 Unclear text is confusing to user 0 0 0 0 1 1 0 2 Tool Totals 2 4 2 1 2 3 1 17 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 16 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON answers announced multiple times (Poll Everywhere), long heading text (Padlet Wall), lack of notification when the user adds an image (Padlet Wall), excessive use of headings in a page forcing users to go through the entire page to find a heading (Padlet Shelf), and headings not used to provide structure and facilitate navigation. The lack of heading structure complicates the ability of users who desire to add a post with a heading and text as seen in Figure 6 (Padlet Map). Usability Issues The accessibility assessment reports also included usability evaluation. Usability issues may impact users of any ability. The testers noticed insufficient color contrast in three tools (Poll Everywhere, Padlet Wall and Map, and Kahoot!). For example, Figure 7 illustrates a Poll Everywhere sample question where the color of the text does not have enough contrast between the text and the background. The evaluators also found a lack of color contrast in instructions and captions. In some Padlet formats, the instructor can change color contrast by choosing a different template. Figure 7. An example of Poll Everywhere answer options that do not have sufficient color contrast between the text and background. Conveying meaning using colors is another issue. In the case of Jamboard, the sticky note (fig. 8) has a blue bar at the bottom that appears to be loading. This bar is connected to a character count that is not noted by screen readers. In addition, the testers could continue typing past the character limit when the loading bar turned red. The testers also noticed layered elements that caused usability problems. Figure 9 illustrates how the preview panel in Padlet Map visually blocks the post and the button to close the preview panel. Padlet Shelf and Mentimeter did not have usability issues. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 17 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Figure 8. In Jamboard, the blue bar (below the yellow box) is used as a visual indication of character limit that is not available to screen readers. Figure 9. When the user has the “preview panel” in Padlet Map open and starts a new post, the preview panel blocks the post. Summary of Results The reports showed that Mentimeter was the most digitally accessible tool of those considered in this study. It is important to note that Kahoot! and Poll Everywhere were judged as relatively accessible with caveats. Both Jamboard and all three types of Padlet tested were found to be inaccessible for many individuals who use assistive technologies. In any case, all tools included either severe or significant issues, creating a great deal of difficulty for users. Most issues were unique to individual tools. Of twelve severe issues, only two were shared across two tools each. Of sixteen significant issues, only four were shared across tools, and only one was shared across more than two tools (“inconsistent focus handling” was a problem in all tools except Mentimeter). All fourteen minor issues were unique. Mentimeter, with very few issues, and Padlet, with the most problems in aggregate, were outliers. Because Padlet offers so many different format options, we tested three, which affected the findings. Still, Padlet Shelf had the most severe issues (ten). Jamboard had the most significant issues (ten) of any tool aside from Padlet. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 18 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON DISCUSSION We hypothesized that the tools selected for this study shared similar digital accessibility issues. To our surprise, the reports showed that these tech tools had few shared problems. We will thus consider two trends in the reports worthy of further examination: shared issues among the tools and unique environments. We will also discuss how digital accessibility can benefit all users of tech tools, not only people with disabilities. Shared Issues among the Tools As previously mentioned, many issues were unique to individual tech tools covered in this study, while a few problems were shared among the tools. When a particular issue was shared among different tools, the level of severity determined whether a person using assistive technology could have a successful interaction with a tool or not. Tracking shared issues and their severity may help developers and digital accessibility staff create a shared vocabulary for discussing user experience. It may also help both parties recognize when issues are common and relatively easy to remediate (e.g., labeling, heading, and alt text problems). There are other shared issues, such as focus handling inconsistencies, that are more difficult to resolve even though they are at the heart of screen reading assistive technology. Tracking focus handling problems may allow developers and digital accessibility advocates to share possible solutions with one another. Moreover, if tech tool developers and digital accessibility staff both understand the importance of a factor like focus handling, any difficult and severe problem can be prioritized when creating and fixing tech tools. It is also important for instruction librarians to have a basic grasp of this shared vocabulary so that they can anticipate the needs and experiences of the learners in their classrooms. Looking at each tech tool in isolation offers only a tiny glimpse into the possibilities of what might happen when students connect to engagement technologies. Evaluating multiple tools allowed us to better understand recurring problems and the barriers they create. Unique Environments Though tracking shared issues is important for these reasons, by the end of our testing, we found that the tools were not similar and that even when they had shared issues, these problems had unique characteristics. For this study, we selected tools that have similar functionality (for example, both Kahoot! and Mentimeter can function as quiz platforms) and others that are distinctive (such as Padlet Maps, which incorporates GPS data to allow users to interact with maps). These tools offer students real-time engagement, which helps foster a collaborative learning environment. As mentioned above, most severe issues (ten out of twelve), most significant issues (twelve out of sixteen), and all minor issues were unique—in other words, they were not shared across tools. From the testers’ point of view, the presence of unique problems is understood by the fact that the elements of each tool combine to create unique environments. For example, some tech tools are more image based, while others are text based.40 Our study shows that even tools that initially appear similar are revealed as unique through assistive technology testing. An interesting finding concerned Padlet. When tools have problems, these issues usually exist across all of the screen readers used for testing. Padlet, however, caused inconsistencies across INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 19 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON different screen readers. For example, Padlet Shelf had many unlabeled or inappropriately labeled elements that created different experiences between VoiceOver, NVDA, and JAWS. Moreover, when irregularities appear across similar assistive technologies, this might mean that developers created unusual code in order to facilitate specific visual elements or other aspects of the technology. Developers should consider testing on multiple platforms and with two or more screen readers to catch these inconsistencies and should also consider whether simple HTML alternatives are possible in place of more complicated code. Regarding Padlet, it may be possible that software developers used Accessible Rich Internet Application code (ARIA-code), which is known to cause inconsistencies for assistive technology. Whenever possible, user experience should be consistent across screen readers. Users should never be asked to switch assistive technologies in order to adapt to a tech tool. Although we sampled only five tech tools, when considering the breadth of other tools in the market and those that may not yet be developed, we wonder whether our results could indicate an abundance of unique environments with unique digital accessibility problems. This inference suggests that software developers may not be creating tech tools with digital accessibility in mind or may be testing with only one type of screen reader. It also speaks to the lack of digital accessibility best practices in software development for educational tech tools. If anything, our results also illustrate the complexity of tech tool environments and the nuances of assistive technology. Digital Accessibility Benefits Users with Different Abilities Digital accessibility is valuable for everyone, not just people with disabilities. Two specific values illustrate this comprehensive benefit. First, if standards for digital accessibility are followed, digital content will be more “portable across platforms, browsers, and operating systems.”41 This interoperability could mean that learning content and properly formatted tech tools will be easy to use across assistive technologies and devices such as smartphones. Secondly, accessible features benefit people who do not see themselves as having a disability. 42 For example, COVID-19 amplified the benefits of using captioning for all learners, even when these learners did not have a specific disability.43 A 2004 Microsoft survey also inferred that accessibility features benefit a wide variety of people.44 While a person with a disability benefits from clear organization, headings, labels, and color contrast, those aspects are also helpful for all users. RECOMMENDATIONS AND NEXT STEPS Planning with Intention Teaching librarians need to invest time learning about the environment of a tech tool they decide to use in IL instruction. Sometimes, tech tools that are digitally accessible are not easy or intuitive for instructors to use. We experienced this “easy to use” versus digital accessibility conflict when preparing the scripts for Mentimeter and Padlet. Padlet is used extensively at our institution due perhaps to its instructor-friendly platform. However, Padlet’s Wall, Shelf, and Map assessments revealed many problems with digital accessibility. Additionally, we had a difficult time creating a quiz in Mentimeter, finding this platform unfriendly for the instructor; yet this tool had the fewest digital accessibility problems. This tension between ease of use and digital accessibility illustrates the importance of taking time to read and understand documentation and training materials before creating engagement activities for IL sessions. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 20 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON We encourage teaching librarians to work with their local digital accessibility offices to evaluate the technology used most frequently in IL classrooms. If a digital accessibility office does not exist on your campus, you may wish to advocate for your administration to create one. Even if your institution does not yet have a digital accessibility office, there are ways for librarians to plan their IL sessions with accessibility in mind. Librarians may do basic assessments of tech tools without access to assistive technology by testing whether it is possible to access all features in a given tool using the Tab keyboard key. If there is a function or action that you cannot access with Tab or that you must use a mouse to navigate to, then that part of the tool will not be accessible to someone using a screen reader. You can also unplug or turn off the mouse and attempt using a tech tool. Librarians can approach each tech tool by asking: Is there anything in the tool that uses only images or colors? Do sounds convey a meaning that is not otherwise communicated on the screen? If there is anything in the tool that relies on a single form of sensory feedback, it may be unperceivable to people using assistive technology. Finally, we strongly suggest considering whether these tools add value to IL instruction. If you like a certain tool but know it is inaccessible (or you are unsure), consider trying a different way of involving students in the same kind of engagement. Think about simplifying the tech tools that you do use. Extend or turn off timers where possible if you choose to use Kahoot!, Mentimeter, or other quiz-making tools. Avoid using questions on any platform that require users to engage with images, even if alt text is provided, because they tend to be more difficult for screen readers. Pursue documentation and take time to understand various options for each tool and each question, then weigh which option will be most accessible for most students. It takes time and energy to plan ahead with intention but increasing the ability of all students to engage in learning makes the planning process worthwhile. Collaboration If collaboration between librarians and digital accessibility experts is possible on your campus, take the time to talk to one another about learning outcomes and reasons for using specific tech tools. Consult with experts in digital accessibility who can also help you advocate for accessibility clauses in purchase contracts before agreeing to subscribe to a given tool or service. You may also foster collaboration with an inclusive community of practice if you have one at your library. Further, the teaching and learning unit on your campus may offer support for integrating technology with pedagogy to promote the engagement and learning experience of all students attending IL instruction. This collaboration may be impactful for the library and the campus teaching community. As librarians with teaching responsibilities, we usually do not work in isolation. Instruction librarians can also serve as a resource for teaching faculty who may want to incorporate accessible tech tools into their instruction. In addition, librarians could investigate professional organizations that provide support and development in understanding digital accessibility. While a framework for assessing tech tools for accessibility does not currently exist, the development of standards and best practices would be beneficial for librarians, software developers, and accessibility professionals alike. We hope to undertake future research and consultation to develop such frameworks with colleagues, possibly through ALA Round Tables or ACRL Sections focused on instruction and accessibility. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 21 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Next Steps Our next steps include sharing these reports with the companies who created the five tools we tested. We will ask them to prioritize both the most severe issues and those issues that are easy to fix and that impact user experience. We will also underscore those areas that surprised us, such as inconsistencies between screen readers for the same issue in a given environment. The goal of this outreach will be to build relationships with tech tool developers so that continued dialogue and testing can occur. The ultimate goal is a more accessible learning environment for everyone with technology vendors as partners in this journey. CONCLUSION Advocating for digital accessibility in research libraries requires relationship and capacity building. The challenges faced during emergency remote learning illustrate the necessity of campus units working together to ensure student inclusion and success. Increased collaborations between academic libraries, tech tool developers, and digital accessibility offices mean that all parties can benefit from mutual expertise. Librarians may share the kinds of tech tools being used in IL sessions, while accessibility offices may test those tools and provide recommendations for improvement, which may then be leveraged when working with software companies to advocate for positive change. If more people are aware of digital accessibility vocabulary, needs, and resources across campus, that can also augment the number of people available to respond to and triage needs when future emergencies arise. ACKNOWLEDGMENT We would like to thank Scott Holman and Eric Klinger from the CU Boulder Writing Center for their help revising this manuscript. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 22 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON APPENDIX A: TESTER INSTRUCTION AND SCRIPT Background Poll Everywhere is an online platform used in classrooms to engage with students through questions, surveys, and polls. People can sign up for a free account or for one of four subscription - based account options. The free account allows users to create unlimited questions, have access to webinar tutorials, and upload images as question choices. This tool also allows people to respond via browser, SMS, or app; to export data and screenshots; and to share to social networks, though some of these features are limited with a free account. Poll Everywhere Script 1. Type in your browser or click on the link provided. A pop-up might show on your screen. Agree to the cookie policy if it does. 2. You may be prompted to introduce yourself and enter the screen name you would like to appear alongside your responses. 3. Click CONTINUE. The survey will let you know that there are six questions. Click START SURVEY. 4. The first question is multiple choice. Select your favorite sport. 5. Click NEXT on the upper right-hand corner. 6. The second question is a short response. Type your favorite ice cream flavor. Click SUBMIT. You can enter as many answers as you want. When you are ready to go to the next question click NEXT. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 23 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON 7. The third question is also a short response. Type in your favorite food. You can enter as many answers as you want. When you are ready to go to the next question click NEXT. 8. The fourth question is also a short response. Type what you are looking forward to this semester. You can enter as many answers as you want. When you are ready to go to the next question click NEXT. 9. The fifth question is a clickable image question. Click on the face that describes how you are feeling today. For this question, if you want to clear your response and enter a new one, you may do so by clicking “Clear last response.” When you are ready, click NEXT. 10. The sixth question is a ranking question. You need to use the arrow feature, which appears when you click next to the image. Move images up and down organizing them from favorite to least favorite. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 24 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON For this question, if you want to clear your response and enter a new one, you may do so by clicking “Clear last response.” When you are ready, click SUBMIT. 11. Click FINISH in the upper right-hand corner. A screen will appear that says “All done!” The results of the survey are only available when the creator of the survey presents them in class. We were not able to figure out a way for respondents to access group responses asynchronously. Notes • We noticed that when preparing questions 4 and 6, we were not prompted to enter alt-text by default. • The creator of the poll must enable alt-text for clickable image questions (such as question 4) by going to the user profile and selecting “Features Lab.” • Alt-text did not seem to be available for question 6. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 25 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON APPENDIX B: DIGITAL ACCESSIBILITY ASSESSMENT REPORT FOR POLL EVERYWHERE Information • Testing tools: o Windows 10 / JAWS 2021/ NVDA 2021.3.1 / Google Chrome (most recent version) o Pixel 3A, Android 12 / TalkBack / last updated 9/30/21 o iPhone 12 mini / Voiceover / Safari IOS 14.3 • Testing dates: February 27, 28, and March 3, 2022 Summary This document provides an overview of the issues the Digital Accessibility Office (DAO) identified on the Poll Everywhere platform. Overall, we found the site to be relatively accessible for many individuals with disabilities or who use assistive technologies or alternative forms of access depending on the question type. The questions with images—to rank or select—were inaccessible. That said, through our testing, we found five severe issues, two significant issues, one minor issue, and one usability issue. Severe issues represent items that create access barriers and need to be remediated, significant issues represent items that create a great deal of difficulty and should be remediated, and minor issues represent items that are the lowest priority but would be good to remediate. Usability issues can impact users of any ability. If there are questions, concerns, or the desire to see demos of the issues presented in this report, please reach out to the Assessment & Usability Testing Team. Please also consider filling out the Assessment & Usability Testing Feedback Form to help us improve our testing protocols. Issues Severe Graphics are unlabeled or inappropriately labeled • In question 6, there are four pictures of animals. The screen readers read all four images as “unlabeled images.” There is no differentiation between the four images. Appropriate image descriptions are needed. o Additionally, while reviewing the history of submissions, the answers are a list that read “(an image), (an image), (an image), (an image)” INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 26 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON • There were several elements that have dots in the label name. When using VoiceOver, these elements were read as “Unpronounceable. [braille dots] ...” followed by numbers. o These elements included the marker on the emoji image, the up and down arrow buttons on question 6, and the finished icon. Element presents gesture/navigation traps • On question 6, while using VoiceOver and TalkBack, the user could not swipe between the answer options. This made the buttons, links, and text before and after the options inaccessible. o A tester was able to leave the trap, but they had to use direct touch and focus landed outside the answers. o Additionally, while using TalkBack, there was not any indication that the image was being moved up or down. Element not available by keyboard or screen reader • Question 5 is an emoji question where the user would need a mouse or direct touch (while not using a screen reader) to answer successfully. The alternative text says there are emojis, but the user does not know what five emotes or different colors are presented. To activate, the user selects “enter” or double taps (mobile screen reader). This makes a random selection and places the marker in the middle of the image without a way to move the marker to the appropriate emoji. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 27 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Errors do not get focus • During one instance, a user received an error that the response was not submitted. During this one instance, the focus was not pushed to the error message. The user would have to know it was there. Ideally, the focus would be pushed to the error so all users would be aware that an error had occurred. Significant Element state not indicated • In question 6, the unlabeled image reads as “clickable” to NVDA. When selecting enter, the state of the element is not announced. Visually, the image gets larger. Inconsistent focus handling • Focus handling for all tools could be improved. Focus goes to different areas of the page after responses, returning to previous questions, or refreshing the page. o Focus inconsistencies depended on the screen reader. While going through the questions, focus would go to the top of the page, “Close app download offer” button, “Submit,” or “Next.” Ideally, focus would be on the heading 1 for each question. Minor Same information is presented to screen reader multiple times • While using VoiceOver, the instructions, questions, and answers were announced multiple times. This was noted on several occasions. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 28 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON Usability Insufficient color contrast (4.5:1) • In the multiple-choice question, after selecting an answer, the question’s color becomes lighter. The lighter color has insufficient color contrast for both the answer selected (2:1) and the answers not selected (1.8:1). INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 29 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON ENDNOTES 1 Kathia Ibacache et al., “Emergency Remote Library Instruction and Tech Tools: A Matter of Equity During a Pandemic,” Information Technology and Libraries 40, no. 2 (2021): 8, https://doi.org/10.6017/ital.v40i2.12751. 2 There are many examples of guides which include ADA compliance and library spaces, including but not limited to William W. Sannwald, Checklist of Library Building Design Considerations, 6th ed. (Chicago: ALA Editions, an imprint of the American Library Association, 2016); and Carrie Scott Banks et al., Including Families of Children with Special Needs: A How-To-Do-It Manual for Librarians (Chicago: American Library Association, 2014). 3 “Introduction to the ADA,” ADA.gov: United States Department of Justice Civil Rights Division, accessed June 15, 2022, https://www.ada.gov/ada_intro.htm. 4 “Introduction to the ADA.” 5 Ariel Pomputius, “Assistive Technology and Software to Support Accessibility,” Medical Reference Services Quarterly 39, no. 2 (2020): 203, https://doi.org/10.1080/02763869.2020.1744380. 6 Fernando H. F. Botelho, “Accessibility to Digital Technology: Virtual Barriers, Real Opportunities,” Assistive Technology 33, no. S1 (2021): S31, https://doi.org/10.1080/10400435.2021.1945705. 7 Botelho, “Accessibility to Digital Technology,” S27. 8 Jonathan Lazar, “Planning for Digital Accessibility in Research Libraries,” Research Libraries Issues, no. 302 (2021): 20, https://doi.org/10.29242/rli.302.3. 9 “Digital Accessibility,” Georgetown Law, accessed June 15, 2022, https://www.law.georgetown.edu/your-life-career/campus-services/information-systems- technology/digital-accessibility/. 10 Lazar, “Planning for Digital Accessibility,” 21. 11 Lazar, “Planning for Digital Accessibility,” 19. 12 Lazar, “Planning for Digital Accessibility,” 26–28. 13 Education of Individuals with Disabilities, 20 U.S.C. §§ 1400–1485 (Suppl. 2 1988), https://tile.loc.gov/storage-services/service/ll/uscode/uscode1988-03202/uscode1988- 032020033/uscode1988-032020033.pdf; see also Kathleen Puckett and Kim W. Fisher, “Assistive Technology,” in The SAGE Encyclopedia of Intellectual and Developmental Disorders, ed. Ellen B. Braaten (Thousand Oaks, CA: SAGE Publications, Inc., 2018), 100 –101. 14 Puckett and Fisher, “Assistive Technology,” 100. 15 Claire Kearney-Volpe and Amy Hurst, “Accessible Web Development: Opportunities to Improve the Education and Practice of Web Development with a Screen Reader,” ACM Transactions on Accessible Computing 14, no. 2 (July 21, 2021): 8:2. https://doi.org/10.6017/ital.v40i2.12751 https://www.ada.gov/ada_intro.htm https://doi.org/10.1080/02763869.2020.1744380 https://doi.org/10.1080/10400435.2021.1945705 https://doi.org/10.29242/rli.302.3 https://www.law.georgetown.edu/your-life-career/campus-services/information-systems-technology/digital-accessibility/ https://www.law.georgetown.edu/your-life-career/campus-services/information-systems-technology/digital-accessibility/ https://tile.loc.gov/storage-services/service/ll/uscode/uscode1988-03202/uscode1988-032020033/uscode1988-032020033.pdf https://tile.loc.gov/storage-services/service/ll/uscode/uscode1988-03202/uscode1988-032020033/uscode1988-032020033.pdf INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 30 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON 16 Emily K. Chan and Lorrie A. Knight, “‘Clicking’ with Your Audience: Evaluating the Use of Personal Response Systems in Library Instruction,” Communications in Information Literacy 4, no. 2 (March 1, 2011): 192–201, https://doi.org/10.15760/comminfolit.2011.4.2.96. 17 Referring to the use of Padlet to foster collaboration in a statistics course, Henrik Skaug Saetra suggests that students feel more welcome to ask basic questions in an anonymous environment, in “Using Padlet to Enable Online Collaboration Mediation and Scaffolding in a Statistics Course,” Education Sciences 11, no. 5 (2021), 6, https://eric.ed.gov/?id=EJ1297247. Christopher J. E. Anderson notes that anonymity invites classroom discussion participation for introverted students and states that answers can be reviewed without “requiring participants to reveal their choice, thus removing stigmas that keep many introverted students from orally participating,” in “Repurposing Digital Devices: Using Poll Everywhere as a Vehicle for Classroom Participation,” Journal of Teaching and Learning with Technology 7, (2018): 154, https://eric.ed.gov/?id=EJ1307006. 18 Citing a 2010 article by B. Jean Mandernach and Jana Hackatborn, Jared Hoppenfeld states that anonymity provides a means for students to answer honestly, in “Keeping Students Engaged with Web-Based Polling in the Library Instruction Session,” Library Hi Tech 30, no. 2 (2012): 238, https://doi.org/10.1108/07378831211239933. See also Anderson, “Repurposing Digital Devices,” 154. 19 This paper considered pedagogical approaches when using Padlet in the classroom, noting that this tool did not enhance criticality or students’ desire to counter a post by a classmate; see Ann Rosnida Md Deni and Zainor Izat Zainal, “Padlet as an Educational Tool: Pedagogical Considerations and Lessons Learnt,” Proceedings of the 10th International Conference on Education Technology and Computers (October 2018), 157, https://doi.org/10.1145/3290511.3290512. 20 Some authors surmise that an instructional game could be used to prepare students for exams, for example, Patricia A. Baszuk and Michele L. Heath, “Using Kahoot! to Increase Exam Scores and Engagement,” Journal of Education for Business 95, no. 8. (2020): 550, https://doi.org/10.1080/08832323.2019.1707752. Examining technology as a tool to enhance teaching and learning in engineering classes, Vian Ahmed and Alex Opuku mentioned that students found Kahoot! a useful online tool that helped them reflect, apply knowledge, and receive feedback, in “Technology Supported Learning and Pedagogy in Times of Crisis: The Case of COVID‐19 Pandemic,” Education and Information Technologies 27 (2021), https://doi.org/10.1007/s10639-021-10706-w. Darren H. Iwamoto et al. assert that Kahoot! provided students with a fun activity that helped them memorize important concepts, in Darren H. Iwamoto, Jace Hargis, Erik Jon Taitano, and Kv Vuong, “Analyzing the Efficacy of the Testing Effect Using Kahoot! on Student Performance,” Turkish Online Journal of Distance Education 18, no. 2 (2017): 83, 89, https://eric.ed.gov/?id=EJ1145220. 21 Iwamoto et al., “Analyzing the Efficacy,” 83, 89. 22 Carolyn M. Plump and Julia LaRosa, “Using Kahoot! in the Classroom to Create Engagement and Active Learning: A Game-Based Technology Solution for eLearning Novices,” Management Teaching Review 2, no. 2 (2017): 156, https://doi.org/10.1177/2379298116689783. https://doi.org/10.15760/comminfolit.2011.4.2.96 https://eric.ed.gov/?id=EJ1297247 https://eric.ed.gov/?id=EJ1307006 https://doi.org/10.1108/07378831211239933 https://doi.org/10.1145/3290511.3290512 https://doi.org/10.1080/08832323.2019.1707752 https://doi.org/10.1007/s10639-021-10706-w https://eric.ed.gov/?id=EJ1145220 https://doi.org/10.1177/2379298116689783 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 31 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON 23 Maureen Knapp, “Technology for One-Shot Instruction and Beyond,” Journal of Electronic Resources in Medical Libraries (2014): 224, https://doi.org/10.1080/15424065.2014.969969. 24 Liya Deng, “Assess and Engage: How Poll Everywhere Can Make Learning Meaningful Again for Millennial Library Users,” Journal of Electronic Resources Librarianship 31, no. 2 (2019): 63, https://doi.org/10.1080/1941126X.2019.1597437. 25 A survey- and interview-based study by engineering faculty members Ahmed and Opoku examined both instructors’ and students’ perceptions of technology-supported learning during times of crisis. With regard to technological and pedagogical best practices, student participants noted that interactive feedback tools such as Kahoot! helped them synthesize and apply their knowledge. As one student said, “Kahoot! was a fun and interactive application and engaging.” See Ahmed and Opuku, “Technology Supported Learning and Pedagogy,” 381. 26 Razzaqul Ahshan, “A Framework of Implementing Strategies for Active Student Engagement in Remote/Online Teaching and Learning during the COVID-19 Pandemic,” Education Sciences 11, no. 9 (2021): 487, https://doi.org/10.3390/educsci11090483. 27 Jillianne Code, Rachel Ralph, and Kieran Forde, “Pandemic Designs for the Future: Perspectives of Technology Education Teachers during COVID-19,” Information and Learning Sciences 121, no. 5/6 (January 1, 2020): 419–31, https://doi.org/10.1108/ILS-04-2020-0112. 28 Code, Ralph, and Forde, “Pandemic Designs,” 426. 29 One such study examines the impact of COVID-19 on higher education in Ethiopia; see Berhanu Abera, “The Effects of COVID-19 on Ethiopian Higher Education and Their Implication for the Use of Pandemic-Transformed Pedagogy: ‘Corona Batches’ of Addis Ababa University in Focus,” Journal of International Cooperation in Education 24, no. 2 (2021): 3–25. Another study focuses on Polish primary school integration of iPads; see Lucyna Kopciewicz and Hussein Bougsiaa, “Understanding Emergent Teaching and Learning Practices: iPad Integration in Polish School,” Education and Information Technologies 26, no. 3 (2021): 2916, https://doi.org/10.1007/s10639-020-10383-1. A third article explores pandemic-transformed pedagogy from the perspectives of early childhood instructors in the Caribbean; see Sabeerah Abdul-Majied, Zoyah Kinkead-Clark, and Sheron C. Burns, “Understanding Caribbean Early Childhood Teachers’ Professional Experiences During the COVID-19 School Disruption,” Early Childhood Education Journal (2022), https://doi.org/10.1007/s10643-022-01320-7. 30 Paul F. Burke et al., “Exploring Teacher Pedagogy, Stages of Concern and Accessibility as Determinants of Technology Adoption,” Technology, Pedagogy and Education 27, no. 2 (2018): 149–63, https://doi.org/10.1080/1475939X.2017.1387602. 31 Burke et al., “Exploring Teacher Pedagogy,” 158–59. 32 Ibacache, Rybin Koob, and Vance, “Emergency Remote Library Instruction,” 9. 33 The authors of this study hold roles as academic subject specialist librarians and Digital Accessibility Office staff, including accessibility and usability team testers. https://doi.org/10.1080/15424065.2014.969969 https://doi.org/10.1080/1941126X.2019.1597437 https://doi.org/10.3390/educsci11090483 https://doi.org/10.1108/ILS-04-2020-0112 https://doi.org/10.1007/s10639-020-10383-1 https://doi.org/10.1007/s10643-022-01320-7 https://doi.org/10.1080/1475939X.2017.1387602 INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2022 TECH TOOLS IN PANDEMIC-TRANSFORMED INFORMATION LITERACY INSTRUCTION 32 RYBIN KOOB, IBACACHE, WILLIAMSON, LAMONT-MANFRE, HUGEN, AND DICKERSON 34 “Free Accessibility Tools and Assistive Technology You Can Use Today,” Bureau of Internet Accessibility (blog), October 26, 2018, https://www.boia.org/blog/free-accessibility-tools-and- assistive-technology-you-can-use-today; see also “Chapter 1, Introducing VoiceOver,” in VoiceOver Getting Started Guide, Apple, Inc., accessed June 16, 2022, https://www.apple.com/voiceover/info/guide/_1121.html. 35 “Get Started on Android with TalkBack,” Android Accessibility Help, accessed June 16, 2022, https://support.google.com/accessibility/android/answer/6283677?hl=en. 36 “About NVDA,” NV Access, accessed June 17, 2022, https://www.nvaccess.org/about-nvda/. 37 JAWS was developed by Freedom Scientific members with vision loss; see “JAWS®—Freedom Scientific,” accessed June 16, 2022, https://www.freedomscientific.com/products/software/jaws/. 38 “Colour Contrast Analyser (CCA),” TPGi, accessed June 16, 2022, https://www.tpgi.com/color- contrast-checker/. 39 Ibacache, Rybin Koob, and Vance, “Emergency Remote Library Instruction,” 9. 40 Jamboard is very visual, with multiple options such as sticky notes, drawing pens, and image searching. Other tools such as Kahoot! and Mentimeter are not solely visual; they also include additional moving parts, such as sounds and other notifications. 41 Lazar, “Planning for Digital Accessibility,” 21. 42 Lazar, “Planning for Digital Accessibility,” 21. 43 Lazar indicated that captioning benefits people who process information in different ways, who are learning the language being used, or who may otherwise struggle to understand a dialect, in “Planning for Digital Accessibility,” 21. 44 Forrester Research, Inc., Accessible Technology in Computing: Examining Awareness, Use, and Future Potential, Redmond, WA: Microsoft Corporation (2004): 9, http://download.microsoft.com/download/0/1/f/01f506eb-2d1e-42a6-bc7b- 1f33d25fd40f/researchreport-phase2.doc. https://www.boia.org/blog/free-accessibility-tools-and-assistive-technology-you-can-use-today https://www.boia.org/blog/free-accessibility-tools-and-assistive-technology-you-can-use-today https://www.apple.com/voiceover/info/guide/_1121.html https://support.google.com/accessibility/android/answer/6283677?hl=en https://www.nvaccess.org/about-nvda/ https://www.freedomscientific.com/products/software/jaws/ https://www.tpgi.com/color-contrast-checker/ https://www.tpgi.com/color-contrast-checker/ http://download.microsoft.com/download/0/1/f/01f506eb-2d1e-42a6-bc7b-1f33d25fd40f/researchreport-phase2.doc http://download.microsoft.com/download/0/1/f/01f506eb-2d1e-42a6-bc7b-1f33d25fd40f/researchreport-phase2.doc Abstract Introduction Overview ADA Accessibility What is Digital Accessibility? Assistive Technology for Blind Users Classroom Engagement with Tech Tools Pandemic-Transformed Pedagogy Methodology About the Testers About the Test Scripts Process Assistive Technology Tools Evaluated Limitations Results Severe Issues Significant Issues Minor Issues Summary of Results Discussion Shared Issues among the Tools Unique Environments Digital Accessibility Benefits Users with Different Abilities Recommendations and Next Steps Planning with Intention Collaboration Next Steps Conclusion Acknowledgment Appendix A: Tester Instruction and Script Background Poll Everywhere Script Notes Appendix B: Digital Accessibility Assessment Report for Poll Everywhere Information Summary Issues Severe Graphics are unlabeled or inappropriately labeled Element presents gesture/navigation traps Element not available by keyboard or screen reader Errors do not get focus Significant Element state not indicated Inconsistent focus handling Minor Same information is presented to screen reader multiple times Usability Insufficient color contrast (4.5:1) Endnotes