152 Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools in U.S. Academic Libraries: A Qualitative Study Adina Mulliken* Telephone interviews were conducted with 18 blind academic library users around the United States about their experiences using their library and its website. The study uses the perspective that blind users’ insights are fundamental. A common theme was that navigating a webpage is time consuming on the first visit. Issues identified include the need for “databases” to be defined on the homepage, accessibly coded search boxes, logical heading structure, and several problems to be resolved on result pages. Variations in needs depending on users’ screen-reader expertise were also raised. Suggestions for libraries to address these issues are offered. Introduction Equal accessibility to websites and digital content for people with disabilities is increasingly recognized as an important obligation of libraries. Campus disability offices traditionally provided academic materials for people with disabilities after converting the materials to an accessible format, but the internet and online technologies have made it possible to provide full access to digital resources for people with disabilities at the same time as for everyone else. Blind students and professionals at academic institutions in the United States, with minor exceptions, are expected to use computers independently without institutionally provided hu- man assistance. Blind individuals use screen-reader software to read computer screens aloud and navigate and interact with websites and applications. For websites and digital content to function adequately with screen readers, websites and content must adhere to accessibility and usability standards. Not only is it now possible to make websites accessible to screen-reader users and to those without disabilities at the same time, but it is also much more technologically efficient to do so rather than recreating or retrofitting websites later. Moreover, “equally effective” and “timely” access are required, according to the Office for Civil Rights and Department of Justice’s guidance on disability law.1 These two agencies enforced accessibility requirements through resolution agreements with a number of schools after disabled students brought complaints against those * Adina Mulliken is a Librarian and Assistant Professor at Hunter College, City University of New York (CUNY); email: am2621@hunter.cuny.edu. The author would like to thank the late Dan Cherubin, former Chief Librarian at Hunter College, for his strong support for this project and for library accessibility in general. She would also like to thank Laura DeLancey for her help with the paper, and the participants for their effort to share their experiences. ©2019 Adina Mulliken, Attribution-NonCommercial (http://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC. mailto:am2621@hunter.cuny.edu http://creativecommons.org/licenses/by-nc/4.0/ Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 153 institutions. Additionally, court cases filed by students with disabilities regarding inacces- sible websites and digital content have been settled in the students’ favor.2 Schools are held responsible for accessibility of internally created websites as well as for digital content used by the school that is sourced from outside providers.3 The Office for Civil Rights and Department of Justice specify that “all faculty and staff,” rather than only a disability office, are responsible for ensuring digital accessibility. A school’s federal funding could be at risk if faculty and staff do not comply; therefore, federal agencies recommend that schools provide their employees with professional development regarding accessibility.4 Fulfilling these requirements involves a learning curve for administrators, IT professionals, and employees who procure or create digital content, including librarians. Ad- ministrators need an understanding of the amount of time it takes IT staff and other employees to implement and maintain these changes. Federal agencies that resolve digital accessibility complaints typically require steps such as creating positions for full-time web accessibility experts and training high-level school administrators.5 Many schools have yet to carry out all these steps; and, specifically, adequate professional development and support regarding digital accessibility is not always provided to librarians. These deficits provide context for understanding blind users’ experiences of academic library websites and search tools and also illustrate the level of institutional change needed to imple- ment accessible websites. In this study, 18 blind academic library users were interviewed about their experiences using academic libraries and library websites. This article focuses on their experiences using library websites and search tools. The purpose of this study is to help elucidate blind screen- reader users’ experiences, especially for librarians, library administrators, web developers, and others who have not had significant experience working with blind users. This study is not an accessibility review of websites, which would require a web developer experienced with accessibility standards to examine code and perform tests with adaptive technology, nor is it a user study that directly observes screen-reader users’ actions while navigating. However, participants’ reports of their experiences can provide web developers with clues regarding accessibility standards where more attention should be focused for basic compliance. Such reports might also provide clues about usability issues that may be especially relevant to library websites and search tools. Additionally, participants’ experiences may help librarians, library administrators, and others to further understand the purpose of certain accessibility standards and accessibility’s general impact on blind users in a way that is more readily understandable to many professionals than, for example, a spreadsheet listing accessibility errors. Issues identified in this study include problems arising from noncompliance with basic accessibility standards—namely, Web Content Accessibility Guidelines 2.0 AA—as well as usability issues unique to libraries. Specific issues identified include needs for defining da- tabases on library homepages, accessibility of search boxes on homepages, logical heading structure on library webpages, several improvements regarding screen-reader accessibility on various tools’ results pages, result titles to be highlighted as headings, and recognition that screen-reader users have varying levels of expertise. Literature Review Web Content Accessibility Guidelines (WCAG) 2.0 level AA6 contains the most commonly used set of technical criteria for web accessibility in U.S. higher education. It is the standard typically 154 College & Research Libraries March 2019 required in accessibility complaint resolutions and settlements. WCAG 2.0 includes clear-cut “success criteria” such as “keyboard accessibility,” meaning all parts of a page can be reached with a keyboard instead of a mouse, since screen readers operate with a keyboard, not a mouse. WCAG 2.0 also includes criteria that require more interpretation about a webpage’s information infrastructure. For example, there is a “success criterion” to “provide ways to help users navigate, find content, and determine where they are.” For instance, a page can be organized by heading tags in the webpage code.7 These heading tags can be identified and jumped to with a screen reader, similar to the way large, visually noticeable text is perceived by sighted users to identify headings that organize content on a webpage. Headings are one of several techniques developers can use to make a page navigable for screen reader users. Additionally, the webpage code must make it possible for a screen reader to determine a logical reading order.8 For example, in a set of tabbed search boxes on a library homepage (such as Discovery Tool/Search All, Catalog, Databases and Journals), which may have im- portant search hints written above or below the search boxes and drop-down menus associ- ated with the search boxes, the code must make it possible for a screen reader to be able to tell that the hints, search boxes, and drop-downs are associated with each other and with the tab underneath which they visually appear. Webpage design should take into consideration the various ways screen-reader users may navigate, such as reading line by line, tabbing, or moving through form fields. Research continues to show that web accessibility and usability problems in library web- sites and vendor-provided resources are common.9 Sahib, Tombros, and Stockman10 report that, while using search engines, screen-reader users tend to run significantly fewer searches and explore significantly fewer results than sighted users in their study because of the time it takes screen readers to process information sequentially. The time-consuming nature of learning to navigate new pages and explore results implies, at minimum, that meeting basic accessibility standards is critical for blind users to have “equally effective” access to digital content (as federally required). Dermody and Majekodunmi asked 10 students with print disabilities—eight with either low vision or vision loss—to use three library databases. The success rate in locating two scholarly articles was 55 percent. The authors state “the largest reason for students not com- pleting the task (32 percent) was due to the accessible barriers they encountered.”11 Dermody and Majekodunmi’s study is important as it is among the very few that include blind users. However, participants with low vision and other disabilities navigate fairly differently from blind screen-reader users, which would yield different results from studying blind students exclusively. Yoon, Dols, Hulscher, and Newberry conducted a usability test on three library websites with six experienced screen-reader users who are blind. The researchers also used the AChecker automated accessibility checker on the library sites. The website tasks tested included catalog searches, making research appointments, and signing up for events. The researchers found this: About half the tasks were completed successfully; of those, only half were com- pleted without the researchers’ intervention. No single participant was able to complete all of the tasks successfully. A task scenario that required finding a known item from a library catalog was completed by only one participant without intervention, taking 7 min.12 Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 155 Oswal13 superbly describes many issues experienced by blind library users, including dif- ficulties screen-reader users experience on pages not properly coded for accessibility. Specific issues include the following: the cursor landing anywhere on the page when it opens, some- times making it difficult to understand where to go to start; the need to guess unpredictable keystrokes, such as whether to use enter or the spacebar to activate buttons and drop-down menus; situations where webpage elements cannot be activated with the keyboard at all; the need to print out and rescan e-books one page at a time because the e-book does not allow screen-reader access; and inaccessibility of all major citation managers. Oswal further illus- trates a screen-reader user’s perspective regarding problems such as a lack of descriptive page titles and pages with a large number of links: The presence of identical page headers throughout the database website requires the blind screen reader user to remember their movements from one page to an- other. Keeping track of such movements becomes particularly problematic when certain links don’t function and the blind user might assume that they have actu- ally traveled on to the desired page and not know the truth until they totally fail to find the desired content…. Many times, blind screen reader users …are …busy looking for the right link or button among the clutter of numerous other links and buttons, many of which are simply promoting other products by the publisher.14 Though not explicitly stated in the article, Oswal’s “descriptive ethnography” appears to be based on his own experience. The present study describes additional difficulties with incorrectly coded pages and corroborates some of Oswal’s explanations with additional users. Technological solutions to many of these problems are well established, but the impact on users if they are not implemented is not widely recognized by library administrators, systems staff, and librarians. The present study adds to the literature in that it includes a larger number of screen- reader users, focuses specifically on blind users, describes some issues not mentioned in other studies of library users, and emphasizes explanations in blind users’ own words. Research Methods This study’s methods were first described in another paper based on the same study and are summarized here.15 This study approaches its topic from the perspective that blind users can provide especially cogent insight into issues that are critical to them. This is not to ignore the important role of web accessibility and usability standards, both of which are essential. Unlike usability tests, actual user behavior while navigating websites were not observed for this study, nor was this study intended to resolve specific web design or coding issues. Rather, the intent was to gain a deeper understanding of general challenges that blind users recall having experienced with library websites and e-resources, which can help web developers, library administrators, and other librarians involved with library websites to understand their needs. This study used qualitative methodology and an open-ended questionnaire. Qualitative research allows in-depth exploration of issues as opposed to predetermined interview ques- tions, which limit responses.16 156 College & Research Libraries March 2019 Procedure This study used telephone interviews, which allowed easier access to blind academic library users dispersed throughout the United States. Eighteen interviews were completed from June 2015 to April 2016. The interviews were recorded, transcribed by a transcription service, and then coded and analyzed by the researcher for patterns. This methodology has limits in that it is not statistically significant, so results cannot be proven to be generalizable. Open- ended qualitative research can be useful, particularly in areas that have not been extensively researched, in that it can point out questions or problems, as well as hypothesized solutions, that need more attention. Prior to the interview, interviewees received a list of possible questions to get a general idea of topics that would be discussed. Participants were encouraged to talk freely about rel- evant issues without feeling limited by specific questions. The interviewer gave participants a brief explanation of her background as a sighted person, librarian, and novice screen-reader user, but not a website coder. She explained that she had used the Jaws screen reader enough to remember keystrokes—for example, to list links and navigate by headings—but did not have a strong understanding of other aspects, such as when to change cursors. The interviews were scheduled for one hour each, and most lasted approximately that long. The researcher took brief notes that she used as a reference for follow-up questions during the interviews. In qualitative research, it is typically acknowledged that the researcher’s background affects the interpretation of the results. Beyond this study, the researcher has more than 11 years of experience working with or observing screen-reader users as they navigate web- pages, primarily in her role of providing reference services but also in other professional and personal contexts. These experiences influenced her understanding of the conversations with the participants. All participants gave verbal consent to participate in the study, which was reviewed and approved by the Internal Review Board for research on human subjects at Hunter College, City University of New York. Participants The participants were described in an earlier paper as follows:17 Study participants were selected based on the following criteria: all participants 1) must state that they meet criteria for legal blindness in the United States or comparable criteria; 2) must have experience relying on a screen reader to access computing devices and the Internet; 3) must have used an academic library, ei- ther online or in person, in the United States within the two years preceding the interview at least several times per semester. [Participants] included six graduate students, eight undergraduate students, and four professionals who were current users and have significant academic library experience. Two of the professionals also discussed recent use in a student role. Interviewees were recruited via the researcher’s personal contacts as well as via library listservs focused on disability topics. Potential participants known closely by the researcher were not recruited or included in order to avoid conflict of interest. Interviewees were offered a 20-dollar gift card for their time. Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 157 All participants were from different schools, except for two, who came from the same university. Four of the undergraduate students attended community colleges, three of whom expected to go on to further schooling, while the other had already obtained an advanced degree. Data Analysis Hill et al.18 describe reasons for developing themes or codes, which they call “domains,” after collecting data rather than using researchers’ preconceived beliefs about what would come up during interviews. In this study, themes were developed from interview transcripts using inductive analysis, as Hill et al. describe. Transcripts were read a minimum of three times and recoded several times. As themes were collected, preliminary coding categories were considered by the researcher and adjusted as needed. Transcripts of the interviews were divided into sections by topic and pasted into a final 43 codes in an Excel spreadsheet. These codes were continually expanded and revised while going through the transcripts. Transcripts that had already been coded were checked for content relevant to new codes as they were added. Codes relevant to this paper include time required to learn a new page, explanation of what participant used via the library website, library homepage, usability magnified for blind users, discovery tool, databases, results pages in general, level of expectation/comparisons to older technologies, headings, and different degrees of complexity for different users. The contents of the codes were then read as a group. This paper does not include all the categories. Some were merged into other codes, some were determined to be less relevant to libraries, and in some categories there were not enough data to be meaningful. The final identi- fied themes include: time to learn new pages, experiences with library homepages, awareness of databases, results pages, headings, and variations in users’ needs. Other articles based on the same study have focused on reference services and on full text. The pronouns “he” and “she” are sometimes changed in this article to protect partici- pants’ identities. Findings Time to Learn New Pages Many participants’ comments reflect a common theme, that it takes longer to learn to navigate a webpage the first time they use it, and then it becomes easier. It may be helpful, particularly for librarians and administrators who have not had the opportunity to work with a screen- reader user, to read participants’ explanations to get a sense of the cognitive load involved in learning to navigate each page, particularly webpages that are not designed with a logical navigational path for screen readers. For example, one student said, The first time I navigated it, it was a little difficult—not difficult, it was slower just because I didn’t know it was a table format, so I went through all the headings and reading everything. There was a lot on the page, so it took me a while to get there [to the results list within the results page]. So, the first time blind and visu- ally impaired users have used JSTOR or any other database, it’s a little bit slower just because they don’t know the shortcuts to getting the results quicker. 158 College & Research Libraries March 2019 Later, she added, I was trying to read everything that was on the page and looking for the result list, too. So, I was just exploring the page. I was kind of skimming and trying to register what else is on that page. A second student explained, We have to scan line by line or heading by heading or link by link. It just becomes so tedious that trying to figure out how the website works interferes with the workflow in our task. Although these are not the only ways to navigate, slowly reading line by line is more common when trying to become familiar with a new page. A third student said, It doesn’t take me that long [to get to know a new page like a library homepage or a new database homepage or something like that], maybe a half hour if I looked at the whole page—20 minutes to one-half an hour, I’d say. The student seemed to be accustomed to and accepting of that amount of time. A recent graduate explained, So as a blind person, often, when I approach a site, I have to figure out, okay, [it’s] going to take me an extra second. Probably more important to me, …how is this site laid out? …You can only figure out by looking at the site. If …you start with [name of discovery tool] and you say, oh, well, I’m going to look at a database, all of a sudden, you’re in a different interface with different rules, and you have to figure out another …way of optimizing that. When asked how much time it would take to learn a new page, he said, It really would depend.…[T]he better a site is laid out, if it has proper headings and heading levels, and if everything is labeled well, and it’s laid out logically, it doesn’t take me long at all. I mean I don’t think it’d take me any longer than a typical person. But the more quirks a site has—that’s like improper headings, or like buttons that don’t work very well, or inconsistencies from one page to another …the longer it’ll take. …There were sites I had to deal with on a daily basis …that were really compli- cated and had some issues. With those I was continually finding ways of being more efficient. That’s something that I think, probably not every blind person, but in general that’s something you’re always working on. Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 159 Library Homepage Accessibility and Usability A total of 11 participants discussed their library homepage. Five participants reported finding the homepage accessible and usable, or at least not too difficult. Two of these users were from a university whose library homepage allowed users to skip navigation links, had a logical heading structure, included ARIA (Accessible Rich Internet Applications, a specification for attributes that can make webpages more accessible to assistive technology users), and had a logical tab order when the researcher reviewed the homepage after the interview. Upon a cursory check, accessibility seemed to have been attended to very well on this library’s homep- age. Another participant who reported no trouble attended a community college with a fairly simple library homepage. Although the homepage did not include heading tags for all items that visually appeared to be headings, it did have heading tags for some of them. The library homepage only had about 22 links, and most of them were arranged in tagged bullet lists, which screen readers can identify. The small number of links likely made it easier for blind users to locate relevant content, such as databases, which is what this participant used. The last participant who reported no trouble attended a school with a library homepage coded with a few headings, lists, and a form label, all of which increase accessibility. The homepage had only one search box for its discovery tool, which was what the participant said she used. The other six participants reported several common challenges with library homepages. One participant worked in a disability office and reported giving input to improve accessibil- ity problems on his library’s homepage, such as multiple miscoded search boxes and lack of headings. He explained that, after the improvements were made, “Students can navigate with rudimentary keystrokes like H [for headings] and links and tab and arrow.” He pointed out that this is important since students may become blind at any time before or during college, and so will not necessarily have the time or opportunity to develop high-level screen-reader skills. A participant at another school may have experienced a similar problem with miscoded search boxes and lack of headings. She said she was a fairly new screen-reader user and had done library research for several English courses. She reported general confusion about her library’s homepage, although she had clearly made an effort to work with it. She said, Yeah, there are [multiple search boxes]. There’s like a search box for—one for your subject. And I think there were some for if you had different certain things you wanted to search. And there was like an “and” in between them. And it got really confusing. Sometimes you could find one for subject. There were differ- ent databases you could search in. So, to find a database, a little place you could search, got really overwhelming sometimes. And then there was just a main place you could search without going into the databases. So, and then some teachers have certain databases they want you to search. So, it gets really crazy. I think I’ve gotten a little bit of a better handle on it. I think it would help if they—like I said earlier, you can hardly ever tell if you’re supposed to tab, or if you’re supposed to—what you’re supposed to do. It would be more helpful if they had a more clear-cut way of how to navigate through the site. It is difficult to tell specifically what technical problems she experienced. However, her level of confusion suggests that accessibility was not ideal. 160 College & Research Libraries March 2019 Another student emphasized difficulty with the lack of headings on his library’s website. He had experience testing websites for accessibility and usability as a student employee, which may have increased his confidence in identifying the page’s problems, rather than wondering if the problems were due to his screen-reader skills. He said, As far as the homepage, …I would have issues with searching for books and that kind of thing. But I think one of the most annoying things to me was just more basic organization of the sites …it’s a pretty vast site…. And the [name of univer- sity] homepage, from what I remember, I don’t think it had any headings at all. That is just like the number one thing I noticed as a blind person on any page. If it doesn’t have any headings, it just severely impacts my ability to get around it.… If I’m just going to …news sites or something, I see a site with no headings, I just avoid [it]. And obviously I can’t do that if it’s …the library site of your university. Two other users described their experiences of their library homepage as being “over- whelming” or being “flooded with information.” One of these users, who had about eight years of experience using a screen reader, explained that he tried to bookmark parts of the library website, so he would not have to navigate through the homepage to access them. Awareness of Databases Most public service librarians know that new library users are often unaware that databases are tools that allow one to locate journal articles. Designing library homepages to make it clear that databases are places one may search for articles has arguably been a longstanding usability issue. One blind participant expressed frustration related to this, and the library homepage’s lack of accessibility may have compounded the problem. He said, I spent a lot of time—I wasted a lot of time at first just doing the general search [name of discovery tool] …And that was because I didn’t know anything about using databases, specific like [name of discipline] databases. And I think that if the homepage had headings and was organized a bit better, I think I could have understood all the resources that were available, and I could have saved a lot of time. Later, he continued, From what I remember, [name of discovery tool] was the most general search thing that was available…. But it would just pull up—you could narrow the search with certain criteria, but I mean it would just float the most random things. I would spend so long trying to get it to give me useful results…. I almost never got it to really give me anything …particularly useful …and it’s an issue blind people I think have in general is you sometimes miss stuff that a sighted person might just see right there on the page. [T]rying to be quicker, trying to search for something and jump to a specific thing on a webpage, you might overlook something that ultimately could have made your job easier. I would imagine when I first went to Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 161 the homepage at the library… I probably checked for headings, and there [were] no headings, so I probably searched for the word “search.” And that probably jumped me down to [name of discovery tool]. And I said, well, this looks like a way to search. And did that. I probably jumped right over the thing that said databases…. if I had seen that I might have not wasted as much time. It is not entirely clear whether he would have known the purpose of these databases had he located the link to them. At one point, he said, “I didn’t know anything about us- ing databases,” but later he expressed familiarity with them. It is possible that he meant he did not know about databases at first but became aware of them at some point. If so, even if the homepage had headings to make it easier to locate databases early in his program, he might have experienced the same usability issue as many sighted users (that is, not realizing that databases are a place where one can search for articles). His library’s homepage did not provide any explanation of what databases are adjacent to its databases link at the time the investigator reviewed the homepage, although the content is likely to have changed since he started his program. Search Results Pages Participants experienced a variety of issues while navigating results pages. Several participants expressed general confusion, frustration, and difficulty with limiting and narrowing results to something relevant. Limiting and narrowing results is challenging for sighted users as well, but it seems these issues may have been magnified by accessibility problems on the results screen page. One participant said, There does seem to be some uniformity [on results pages]. Basically, the results come up, there’s all the filters right on top and then underneath the filters there’s all the hits. Then you could have …hits …in the thousands… Trying to filter those results down to the most relevant information—that’s what I have the most difficulty with …I’ve come across databases where the filters, I mean it’s …this massive series of checkboxes. And you think you’re clicking these boxes to …filter …the results…. okay, did anything happen when I clicked this? …Not being able to actually see the page as it refreshes while you’re clicking on the filters—it is probably a very difficult thing because you don’t know if the results are shrinking …[or] getting larger. Another participant similarly described difficulty understanding how to use limits on a result screen, because she could not tell whether limiters in a combo box (a drop-down menu) had been selected or not. She said she could discern whether limiters had been selected on most webpages, but she could not tell on the library e-resources result screens. In addition to general difficulties limiting results, other points include the need to use an unintuitive workaround to locate a link for the next page of results. One participant explained, When I get down to the end of this 10 [results], I’ll know when it says, “Refine results” [presumably because she has used the same database before and has 162 College & Research Libraries March 2019 discovered that text happens to be read at the end] …and then if I work up [use the up arrow key], I can find the “next” [the link for the next page of results]. The participant who recounted this issue had advanced experience and did not express that such a workaround was problematic. However, one can imagine that having to discover such workarounds for each database could be time consuming, and less technologically inclined or inexperienced screen-reader users could have more trouble figuring out and memorizing the process. Related to results pages, another participant commented, I’ve found that a lot of times the search results are not where I would normally think they would be. They’ll be at the very bottom of the page, after basically where you would think the page ends there’s your search results. It is unclear why the participant had this experience. Perhaps he was simply surprised by the extensive amount of content that often comes before the results, such as another set of search boxes and many ways to limit results. A concern that came up with several participants was inconsistency in the design of ci- tation information on results lists. There did not seem to be any understanding of common organization of citation information on results pages. One difficulty that came up in relation to this was in distinguishing whether a result was a book, a scholarly journal article, a maga- zine, or another format. One participant said that one database “broke the results down into categories” such as books and articles, which he emphasized was very helpful. It was not clear which search tool he was using, although some databases do separate results into lists with headings for books, journal articles, and so on. Another participant said, It’d be really great if they put at the beginning of the title what format it’s in like book, newspaper, journal, magazine, instead of putting it, like, below somewhere. Of course, many databases do visually identify the format in front of the title. The inter- viewer pointed this out, and the participant said she had not noticed such icons. The Discus- sion section offers one explanation for this observation. Headings A number of participants were asked what they thought about recommendations made in an article by Haanpera and Nieminen,19 such as highlighting each result with a heading tag. Headings allow screen-reader users to quickly jump from one result to the next intuitively, rather than having to explore each search tool’s results page for a common element that might allow the user to jump from one result to the next without having to listen to the description under each result. Nine users said yes, and none said no. Several respondents were familiar with other search tools that highlight each result with a heading, such as Google, Yahoo, Amazon, and The National Library Service’s BARD (Braille and Audio Reading Download) program. Other comments included the following: Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 163 Oh yeah, I think that’d be pretty useful, just because when a lot of visually impaired students are taught to use a screen reader …they’re taught to automatically look for the headings. That saves them a lot of time. Another participant said this: Otherwise, [without headings], it would take all day to try to get through all the information…. It’d be a lot quicker to do it [with headings]. Varying Levels of Complexity Several advanced participants also noted that there is variation in the complexity levels of pages that different levels of screen-reader users can realistically be expected to understand. For example, one participant, who is a professional, said, One interesting thing about this job is I’ve gotten to meet a lot of people that are kind of emergently information literate or emergently literate at all…. And the things that I really enjoy in terms of having, like, a really rich, dynamic page with a lot of different parts and information on it that I feel like I can just fly around, that’s the stuff that makes them crazy. Another professional pointed out that it is important for library homepages to allow us- ers to navigate with “rudimentary” keystrokes such as H for heading, jumping to links, and arrowing up and down, since some users have not had the opportunity to obtain high-level screen-reader skills, as mentioned earlier. The first of these professionals pointed out that, in addition to designating each result as a heading, it is helpful to properly nest heading levels one to six. The interviewer asked the first of these professionals what she thought about the common advice to limit a page to no more than three levels of headings. She responded, That sounds like a cognitive accessibility issue and that’s where I feel like I’m kind of weak in my practice. I’m still trying to learn more. It’s something that when you say it to me it sounds true, it sounds smart. But I’m not really aware of what the research or the best practice is for sure.… In other words, limiting to three levels of headings might be the best practice for blind users who experience some degree of cognitive limitation, but other blind users appreciate the use of additional levels of headings for complex content, where additional heading levels are relevant and may be necessary. Another user, on the other hand, said she did not usually pay attention to the level of headings. Therefore, properly nesting headings, or limiting headings to a certain number of levels, would not be useful for her. 164 College & Research Libraries March 2019 Discussion, Recommendations, and Conclusions The comments of five participants suggest that it takes more time for screen-reader users to learn to navigate a new page than it takes for sighted users, particularly if the page does not have headings or a logical navigational path for screen readers. Specifically, one participant suggested that, in her experience, 20–30 minutes was the average amount of time it took to fully learn a new page. Spending 20–30 minutes to learn each new type of webpage layout one needs to use to do research in multiple library search tools, or to obtain full-text articles via a link resolver, would make keeping up with academic work difficult. Despite these challenges, some blind students do complete degrees, although at a sig- nificantly lower rate than individuals who are not blind. Statistics for legally blind students, excluding students with other visual disabilities who may not use screen readers, could not be located. However, according to the Cornell Disability Statistics website, in the United States “31.3 percent (plus or minus 0.09 percentage points) of non-institutionalized persons aged 21 to 64 years without a visual disability in the United States have an educational attainment of a BA degree or higher,” while only “an estimated 14.9 percent (plus or minus 0.49 percentage points) of non-institutionalized persons aged 21 to 64 years with a visual disability have an educational attainment of a BA degree or higher.”20 Research tools should be reasonably accessible, which might help improve blind persons’ graduation rates. Oswal makes a similar point.21 Improved accessibility and usability might significantly reduce the amount of time required to learn a new page. One participant’s com- ment supports this: If …[a site is] laid out logically, …I don’t think it’d take me any longer than a typical person. Participants’ experience of their respective library homepages’ accessibility varied. Homep- ages that participants experienced as accessible and usable included headings and, sometimes, a limited number of links. In addition to headings, an explanation of the purpose of databases on the homepage might have been helpful in the case of one participant. Homepages considered to be accessible and usable contained search boxes coded to be accessible by screen readers, or; in another case, a simple, single search box was provided, which would not be as difficult to decipher with a screen reader as multiple tabbed search boxes, even if the search box was not perfectly coded for accessibility. In the author’s experience, some library homepages that use multiple tabbed search boxes are read by screen readers in an incoherent order. This is similar to the experience reported by the student who said this: You can hardly ever tell if you’re supposed to tab, or if you’re supposed to—what you’re supposed to do. It would be more helpful if they had a more clear-cut way of how to navigate through the site. The disability office professional interviewed for this study also experienced inaccessible search boxes on his library’s website, though the problem was corrected after he gave his input. Similarly, the user study by Yoon et al.22 found this: Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 165 A… problem occurred on pages containing scripting for a tab panel whose links dynamically changed the content within a single page rather than linking to a new page. All of the [blind] participants who encountered the tab panel (four out of the six) became very confused after clicking on the links and not hearing the expected screen reader prompt announcing a URL change. These participants commented that they had no idea whether the links worked, and when they tried navigating the new page content, it seemed as though they were on the same page they had been previously (255–56). Web developers can use techniques listed under WCAG 2.0 criterion 3.3, Input Assistance, to learn methods to offer instructions to screen-reader users about how to use forms, such as search boxes. Additionally, web developers should learn WCAG 2.0 standards in general to ensure it is possible to navigate forms coherently with a screen reader. (Web developers who are unfamiliar with screen readers or digital accessibility need to dive deep to understand such standards. They should be granted time and support to consult educational resources or take courses.) For tabbed search boxes, Yoon et al. recommend specifically using ARIA (Accessible Rich Internet Applications) roles for “tabpanel,” “tab,” and “tablist” on the page’s HTML to assist with making the page behave in a way users expect.23 If libraries are not immediately prepared to support their web developers in taking the time needed to learn complex coding, such as tabbed search boxes, compliant with basic ac- cessibility and usability standards, it would be better to provide homepages that are simpler to code and easier to make compliant. Harvard Library’s homepage, for example, has a single search box instead of multiple tabbed search boxes.24 David Comeaux’s 2017 study of 37 aca- demic libraries found that single search boxes, rather than tabbed search boxes, have gained traction, so a library’s choice of this design option would not be unusual.25 Several participants mention the advantage of sites using a similar navigational layout on webpages, and the dif- ficulty caused by many different layouts from one search tool to another. As discovery tools gain the functionality and content of library databases, perhaps their single layout will lessen the need to learn many database layouts. However, it seems unlikely that discovery tools will be an adequate replacement for databases in the immediate future, which may leave it incumbent on reference librarians to improve guidance regarding using various layouts with screen readers. For further discussion of possible roles for reference librarians, see Mulliken.26 Another part of the solution may be for more library search tool providers to follow trends with which screen-reader users are familiar, such as highlighting each result as a heading. Several library databases already do this. Another issue, identified by two participants, was difficulty locating the format indicator (such as journal article, magazine article, or book) for each result. Many databases include an icon identifying the format in front of each title in a result list. Screen-reader users often prefer to navigate by headings in the result list if the list highlights each title as a heading. Navigat- ing in this manner, a user may not be likely to land on or hear any icons placed in front of a heading. One participant found it helpful when databases separated results into categories by format, which is one solution web developers could consider. EBSCO offers another solu- tion, placing format indicators just below the heading (in the order a screen reader reads it). Additional issues identified by participants on results pages include difficulty locating the result list on a results page and an unintuitive path for navigating to the next page of results. 166 College & Research Libraries March 2019 Web developers can use WCAG 2.0 guidelines such as 1.3, which includes success criteria and techniques for ensuring that all information and its sequence can be determined by assistive technology. Guideline 2.4, which concerns making navigation within a page coherent and comprehensible with a screen reader, is also relevant.27 When trying to filter search results, participants reported being unable to tell if or when search results had changed. WCAG success criterion 3.2.2 offers techniques regarding how to avoid accessibility barriers caused by automatically submitted forms, and 3.3.1 offers techniques for giving screen-reader users feedback about whether they have successfully or unsuccessfully filled out a form. Additional WCAG criteria may be relevant as well.28 It may be useful for web developers to keep in mind, as two participants pointed out, that blind people have varying levels of expertise with screen readers. The WebAIM screen- reader survey reports on common methods, such as headings, that screen reader users use to navigate.29 Making it possible to navigate with features more commonly known by screen- reader users is likely to be helpful, particularly on essential pages such as library homepages. However, higher education audiences likely include screen-reader users who appreciate ad- vanced features, such as more than three levels of headings, as mentioned by one participant. In conclusion, it is critical that web developers at libraries and library vendors be provided the time and support necessary to learn and implement accessibility and usability standards. Implementation of web accessibility has real-world consequences for blind users. Participants in the current study sometimes struggled with basic tasks. The challenges that participants in this study describe should help library professionals understand the need to move forward with learning about and applying web accessibility standards, as well as to understand a few specific issues to consider when developing and selecting library websites and search tools. Notes 1. Department of Education Office for Civil Rights, “Frequently Asked Questions about the June 29, 2010, Dear Colleague Letter” (last modified May 26, 2011), available online at http://www2.ed.gov/about/offices/list/ ocr/docs/dcl-ebook-faq-201105.html [accessed 17 September 2016]. 2. Atlantic Cape Community College, “NFB Lanzailotti Atlantic Cape Consent Decree” (2015) (last modified July 7, 2015), available online at http://www.atlantic.edu/documents/nfb_lanzailotti_atlantic_cape_consent_de- cree.pdf [accessed 2 July 2017]; Pennsylvania State University, “Settlement between Penn State University and National Federation of the Blind” (last modified 2011), available online at http://accessibility.psu.edu/nfbpsusettle- ment/ [accessed 21 June 2017]; TRE Legal Practice, “Accessible Instructional Materials and the Siskiyou Joint Community College District Settlement” (last modified November 20, 2016), available online at www.trelegal. com/posts/accessible-instructional-materials-and-the-siskiyou-joint-community-college-district-settlement/ [accessed 11 December 2016]; “UM Accessibility Agreement,” Missoulian, Missoula, MT (last modified March 19, 2014), available online at http://missoulian.com/um-accessibility-agreement/pdf_e34b65de-afac-11e3-a740- 001a4bcf887a.html [accessed 17 September 2016]; U.S. Department of Justice, “Settlement Agreement between the United States and Louisiana Tech University, and the Board of Supervisors for the University of Louisiana System” (last modified July 23, 2013), available online at www.ada.gov/louisiana-tech.htm [accessed 25 Febru- ary 2015]; U.S. Department of Justice, “Miami University Agrees to Overhaul Critical Technologies to Settle Disability Discrimination Lawsuit” (last modified October 17, 2016), available online at https://www.justice. gov/opa/pr/miami-university-agrees-overhaul-critical-technologies-settle-disability-discrimination [accessed 11 December 2016]; U.S. Department of Education, Office for Civil Rights, “Civil Rights Agreement Reached with South Carolina Technical College System on Accessibility of Websites to People with Disabilities” (last modi- fied March 8, 2013), available online at https://www.ed.gov/news/press-releases/civil-rights-agreement-reached- south-carolina-technical-college-system-accessibi [accessed 21 June 2017]; U.S. Department of Education, Office for Civil Rights, “Agreement: University of Cincinnati, Ohio,” OCR Case #15-13-6001 (last modified December 8, 2014), available online at http://www2.ed.gov/documents/press-releases/university-cincinnati-agreement.pdf [accessed 25 February 2015]; U.S. Department of Education, Office for Civil Rights, “Agreement: Youngstown http://www2.ed.gov/about/offices/list/ocr/docs/dcl-ebook-faq-201105.html http://www2.ed.gov/about/offices/list/ocr/docs/dcl-ebook-faq-201105.html http://www.atlantic.edu/documents/nfb_lanzailotti_atlantic_cape_consent_decree.pdf http://www.atlantic.edu/documents/nfb_lanzailotti_atlantic_cape_consent_decree.pdf http://accessibility.psu.edu/nfbpsusettlement http://accessibility.psu.edu/nfbpsusettlement http://www.trelegal.com/posts/accessible-instructional-materials-and-the-siskiyou-joint-community-college-district-settlement http://www.trelegal.com/posts/accessible-instructional-materials-and-the-siskiyou-joint-community-college-district-settlement http://missoulian.com/um-accessibility-agreement/pdf_e34b65de-afac-11e3-a740-001a4bcf887a.html http://missoulian.com/um-accessibility-agreement/pdf_e34b65de-afac-11e3-a740-001a4bcf887a.html http://www.ada.gov/louisiana-tech.htm https://www.justice.gov/opa/pr/miami-university-agrees-overhaul-critical-technologies-settle-disability-discrimination https://www.justice.gov/opa/pr/miami-university-agrees-overhaul-critical-technologies-settle-disability-discrimination https://www.ed.gov/news/press-releases/civil-rights-agreement-reached-south-carolina-technical-college-system-accessibi https://www.ed.gov/news/press-releases/civil-rights-agreement-reached-south-carolina-technical-college-system-accessibi http://www2.ed.gov/documents/press-releases/university-cincinnati-agreement.pdf Eighteen Blind Library Users’ Experiences with Library Websites and Search Tools 167 State University,” OCR Case #15-13-6002 (last modified December 12, 2014), available online at http://www2. ed.gov/documents/press-releases/youngstown-state-university-agreement.pdf [accessed 25 February 2015]. 3. Ibid. 4. Department of Education Office for Civil Rights, “Frequently Asked Questions about the June 29, 2010, Dear Colleague Letter.” 5  . Atlantic Cape Community College, “NFB Lanzailotti Atlantic Cape Consent Decree” (2015); Pennsylvania State University, “Settlement between Penn State University and National Federation of the Blind”; TRE Legal Practice, “Accessible Instructional Materials and the Siskiyou Joint Community College District Settlement”; “UM Accessibility Agreement”; U.S. Department of Justice, “Settlement Agreement between the United States and Louisiana Tech University, and the Board of Supervisors for the University of Louisiana System”; U.S. Depart- ment of Justice, “Miami University Agrees to Overhaul Critical Technologies to Settle Disability Discrimination Lawsuit”; U.S. Department of Education, Office for Civil Rights, “Civil Rights Agreement Reached with South Carolina Technical College System on Accessibility of Websites to People with Disabilities”; U.S. Department of Education, Office for Civil Rights, “Agreement: University of Cincinnati, Ohio”; U.S. Department of Education, Office for Civil Rights, “Agreement: Youngstown State University.” 6. W3C, “Web Content Accessibility Guidelines 2.0” (last modified 5 June 2018), available online at https:// www.w3.org/TR/WCAG/ [accessed 2 July 2017]. 7. W3C, “How to Meet WCAG 2.0,” available online at https://www.w3.org/WAI/WCAG20/quickref/ [ac- cessed 19 June 2017]; W3C, “G130: Providing Descriptive Headings,” available online at https://www.w3.org/ TR/WCAG20-TECHS/G130.html [accessed 19 June 2017]. 8. W3C, “1.3.2 Meaningful Sequence- Level A,” available online at https://www.w3.org/WAI/WCAG20/ quickref/#content-structure-separation-sequence [accessed 19 June 2017]. 9. Suzanne L. Byerley and Mary Beth Chambers, “Accessibility and Usability of Web-based Library Data- bases for Non-visual Users,” Library Hi Tech 20, no. 2 (2002): 169–78; Suzanne L. Byerley, Mary Beth Chambers, and Mariyam Thohira, “Accessibility of Web-based Library Databases: The Vendors’ Perspectives in 2007,” Li- brary Hi Tech 25, no. 4 (2007): 509–27; David Comeaux and Axel Schmetzke, “Accessibility of Academic Library Web Sites in North America,” Library Hi Tech 31, no. 1 (2013): 8–33; David Comeaux and Axel Schmetzke, “Web Accessibility Trends in University Libraries and Library Schools,” Library Hi Tech 25, no. 4 (2007): 457–77; Axel Schmetzke, “Accessibility of Web-based Information Resources for People with Disabilities,” Library Hi Tech 20, no. 2 (2002): 135–36; Axel Schmetzke, “Web Accessibility at University Libraries and Library Schools,” Library Hi Tech 19, no. 1 (2001): 35–49; Ron Stewart, Narendra Vivek, and Axel Schmetzke, “Accessibility and Usability of Online Library Databases,” Library Hi Tech 23, no. 2 (2005): 265–86; Jennifer Tatomir and Joan C. Durrance, “Overcoming the Information Gap: Measuring the Accessibility of Library Databases to Adaptive Technology Users,” Library Hi Tech 28, no.4 (2010): 577–94; Laura DeLancey, “Assessing the Accuracy of Vendor-Supplied Accessibility Documentation,” Library Hi Tech 33, no. 1 (2015): 103–13; Sushil K. Oswal, “Access to Digital Library Databases in Higher Education: Design Problems and Infrastructural Gaps,” Work 48 (2014): 307–17; Sushil K. Oswal, “Institutional, Legal, and Attitudinal Barriers to the Accessibility of University Digital Libraries: Im- plications for Retention of Disabled Students,” in Disability and Equity in Higher Education Accessibility, eds. H.C. Alphin, J. Lavine, and R.Y. Chan (Hersey, PA: IGI Global, 2017); T. Haanperä and M. Nieminen, “Usability of Web Search Interfaces for Blind Users: A Review of Digital Academic Library User Interfaces,” Lecture Notes in Computer Science 8011 (2013); A.J. Blechner, “Improving Usability of Legal Research Databases for Users with Print Disabilities,” Legal Reference Services Quarterly 34, no. 2 (2015), 138–75; Debra A. Riley-Huff, “Supporting Web Accessibility through Rich Internet Applications: Insights for Libraries,” in Accessibility for Persons with Disabilities and the Inclusive Future of Libraries, eds. Anne Woodsworth and W. David Penniman (Bingley, UK: Emerald Group Publishing Limited, 2015). 10. Nuzhah Gooda Sahib, Anastasios Tombros, and Tony Stockman, “A Comparative Analysis of the Information-Seeking Behavior of Visually Impaired and Sighted Searchers,” Journal of the American Society for Information Science and Technology 63, no. 3 (2012): 377–91. 11. Kelly Dermody and Norda Majekodunmi, “Online Databases and the Research Experience for University Students with Print Disabilities,” Library Hi Tech 29, no. 1 (2011): 154. 12. Kyunghye Yoon, Rachel Dols, Laura Hulscher, and Tara Newberry, “An Exploratory Study of Library Website Accessibility for Visually Impaired Users,” Library and Information Science Research 38 (2016): 253. 13. Oswal, “Institutional, Legal, and Attitudinal Barriers to the Accessibility of University Digital Libraries,” (2017); Oswal, “Access to Digital Library Databases in Higher Education.” 14. Oswal, “Access to Digital Library Databases in Higher Education,” 311–12. 15. Adina Mulliken, “‘There Is Nothing Inherently Mysterious about Assistive Technology’: A Qualitative Study about Blind User Experiences and Reference Services in U.S. Academic Libraries,” Reference and User http://www2.ed.gov/documents/press-releases/youngstown-state-university-agreement.pdf http://www2.ed.gov/documents/press-releases/youngstown-state-university-agreement.pdf https://www.w3.org/TR/WCAG https://www.w3.org/TR/WCAG https://www.w3.org/WAI/WCAG20/quickref https://www.w3.org/TR/WCAG20-TECHS/G130.html https://www.w3.org/TR/WCAG20-TECHS/G130.html https://www.w3.org/WAI/WCAG20/quickref/#content-structure-separation-sequence https://www.w3.org/WAI/WCAG20/quickref/#content-structure-separation-sequence 168 College & Research Libraries March 2019 Services Quarterly 57, no. 2 (2017). 16. Robert C. Bogdan and Sari K. Biklen, Qualitative Research for Education: An Introduction to Theories and Methods (Boston, MA: Pearson, 2007). 17. Mulliken, “There Is Nothing Inherently Mysterious about Assistive Technology.” 18. C.E. Hill, S. Knox, B.J. Thompson, E.N. Williams, S.A. Hess, and N. Ladany, “Consensual Qualitative Research: An Update,” Journal of Counseling Psychology 52 (2005): 196. 19. T. Haanperä and M. Nieminen, “Usability of Web Search Interfaces for Blind Users.” 20. W. Erickson, C. Lee, and S. von Schrader, Disability Statistics from the 2012 American Community Survey (ACS) (Ithaca, NY: Cornell University Rehabilitation Research and Training Center on Disability Demographics and Statistics), available online at www.disabilitystatistics.org/reports/acs.cfm?statistic=1 [accessed 3 December 2014]. 21. Oswal, “Institutional, Legal, and Attitudinal Barriers to the Accessibility of University Digital Libraries.” 22. Yoon, Dols, Hulscher and Newberry, “An Exploratory Study of Library Website Accessibility for Visually Impaired Users.” 23. Ibid., 258. 24. Harvard Library website, available online at http://library.harvard.edu/ [accessed 26 June 2017]. 25. David Comeaux, “Web Design Trends in Academic Libraries: A Longitudinal Study,” Journal of Web Li- brarianship 11, no. 1 (2017). 26. Mulliken, “There Is Nothing Inherently Mysterious about Assistive Technology.” 27. W3C, “How to Meet WCAG 2.0.” 28. Ibid. 29. WebAIM, “Screen Reader User Survey #6” (2015), available online at http://webaim.org/projects/screen- readersurvey6/ [accessed 19 June 2017]. http://www.disabilitystatistics.org/reports/acs.cfm?statistic=1 http://library.harvard.edu http://webaim.org/projects/screenreadersurvey6 http://webaim.org/projects/screenreadersurvey6 _GoBack _Hlk509337300 _Hlk509429481 _Hlk509430072