USABILITY AS A METHOD FOR ASSESSING DISCOVERY | IPRI, YUNkIN, AND BROWN 181 Tom Ipri, Michael Yunkin, and Jeanne M. Brown Usability as a Method for Assessing Discovery The University of Nevada Las Vegas Libraries engaged in three projects that helped identify areas of its website that had inhibited discovery of services and resources. These projects also helped generate staff interest in the Usability Working Group, which led these endeavors. The first project studied student responses to the site. The second focused on a usability test with the Libraries’ peer research coaches and resulted in a presentation of those findings to the Libraries staff. The final project involved a specialized test, the results of which also were presented to staff. All three of these projects led to improvements to the website and will inform a larger redesign. U sability testing has been a component of the University of Nevada Las Vegas (UNLV) Libraries Web management since our first usability studies in 2000.1 Usability studies are a widely used and rela- tively standard set of tools for gaining insight into Web functionality. These tests can explore issues such as the effectiveness of interactive forms or the complexity of accessing full-text articles from third-party databases. They can explore aesthetic and other emotional responses to a site. In addition, they can provide an opportunity to collect input concerning satisfaction with the layout and logic of the site. They can reveal mistakes on the site, such as coding errors, incorrect or broken links, and problematic wording. They also allow us to engage in testing issues of discovery to isolate site elements that facilitate or hamper discovery of the Libraries’ resources and services. The Libraries’ Usability Working Group seized upon two library-wide opportunities to highlight findings of the past year’s studies. The first was the Discovery Summit, in which the staff viewed videos of staff attempting find- ing exercises on the homepage and discussed the finding process. The second was the Discovery Mini-Conference, an outgrowth of a new evaluation framework and the Libraries’ strategic plan. Through a poster display, the Working Group highlighted areas dealing with discovery of library resources. The Mini-Conference allowed us to leverage library-wide interest in the topic of effective information-finding on the Web to draw wider attention to usability’s importance in identifying the likelihood of our users discovering library resources independently. The Usability Working Group engaged in three projects to help identify areas of the website that inhibited discov- ery and to generate staff interest in the process of usability. All three of these projects led to improvements to the website and will inform a larger redesign. The first project is an ongoing effort to study student responses to the site. The second was to administer a usability test with the Libraries’ Peer Research Coaches and present those find- ings to the Libraries’ staff. The final project was requested by the dean of libraries and involved a specialized test, the results of which also were presented to staff. n Student studies The Usability Working Group began its ongoing evalu- ation of UNLV Libraries’ website by conducting two series of tests: one with five undergraduate students and one with five graduate students. Not surprisingly, most students self-reported that the main reason they come to the Libraries’ site is to find books and journal articles for assignments. The group created a set of fourteen tasks that were based on common needs for completing assignments: 1. Find a journal article on the death penalty. (Note: If students go somewhere other than the library, guide them back.) 2. Find what floor the book The Catcher in the Rye is on. 3. Find the most current issue of the journal Popular Mechanics. 4. Identify a way to ask a question from home. 5. Find a video on global warming. 6. You need to write a bibliography for a paper. Find something on the website that would help you. 7. Find out what Lied Library’s hours were for July 4. 8. Find the Libraries’ tutorial on finding books in the library. 9. The library offers workshops on how to use the library. Find one you can take. 10. Find a library-recommended website in business. 11. Find out what books are checked out on this card. 12. Find instructions for printing from your personal laptop. 13. Your sociology professor, Dr. Lampert, has placed something on reserve for your class. Please find the material. 14. Your professor wants you to read the book Efficiency and Complexity in Grammars by John A. Hawkins. Find a copy of the book for your assignment. (The Tom Ipri (tom.ipri@unlv.edu) is Head, Media and Computer Services; Michael Yunkin (michael.yunkin@unlv.edu) is Web Content Manager/Usability Specialist; and Jeanne M. Brown (jeanne.brown@unlv.edu) is Head, Architecture Studies Library and Assessment Librarian, University of Nevada Las Vegas Libraries. 182 INFORMATION TECHNOLOGY AND LIBRARIES | DECEMBER 2009 moderator will prompt if the person stops at the catalog.) The results of these tests revealed that the site was not as conducive to discovery as was hoped. The Libraries are planning on a complete redesign of the site in the near future; however, the results of these first two series of usability tests were compelling enough to prompt an intermediary redesign to improve some of the areas that were troublesome to students. That said, the tests also found certain parts of the old site (figure 1) to be very effective: 1. All participants used the tabbed box in the center of the page, which gives them access to the catalog, serials lists, databases, and reserves. 2. All students quickly found the “Ask a Librarian” link when prompted to find a way to ask a ques- tion from home. 3. Most students found the Libraries’ hours, partly because of the “Hours” tab at the top of the page and partly because of multiple access points. 4. Many participants used the “Site Search” tab to navigate to the search page, but few actually used it to conduct searches. They effectively used the site map information also included on the search page. The usability tests also revealed some variables that undermined the goal of discoverability. 1. Due to the various sources of library-related infor- mation (website, catalog, vendor databases) navi- gation posed problems for students. Although not a specific question in the usability tests, the results show students often struggled to get back to the Libraries’ home page to start a new question. 2. Students often expected to find different content under “Help and Instruction” than what was there. 3. Students used the drop down boxes as a last resort. Often, they would expand a drop down box and quickly navigate away without selecting anything from the list. 4. With some exceptions, students mainly ignored the tabs across the top of the home page. 5. Although students made good use of the tabbed box in the center of the page, many could not distinguish between “Journals” and “Articles & Databases.” 6. Similarly, students easily found the “Reserves” tab but could not make sense of the difference between “Electronic Reserves (E-Reserves)” and “Other Reserves.” 7. No student found business resources via the “Subject Guides” drop down menu at the bottom of the home page. n Peer-coach test and staff presentation UNLV Libraries employs peer research coaches, under- graduate students who serve as frontline research mentors to their peers. The Usability Working Group administered the same test they used with the first group of undergraduate and graduate students to the peer research coaches. Although these students are trained in library research, they still struggled with some of the usability tasks. The Usability Working Group presented the findings of the Peer Research Coach tests with staff. The Peer Research Coaches are highly regarded in the Libraries, so staff were surprised that they had so much difficulty navigating the site; this presentation was the first time many of the staff had seen the results of usability studies of the site. The shocking nature of these results generated a great deal of interest among the staff regarding the work of the Usability Working Group. n The dean’s project In January 2009, the dean of libraries asked the Usability Working Group for assistance in planning for the Discovery Summit. Initially, she requested to view Figure 1. UNLV Libraries’ original website design USABILITY AS A METHOD FOR ASSESSING DISCOVERY | IPRI, YUNkIN, AND BROWN 183 the video from some of the usability tests with the goal of identifying discovery-oriented problems on the Libraries’ website. Soon after, the Dean tasked the group with performing a new set of usability tests using three subjects: a librarian, a library employee with little research or Web expertise, and a faculty researcher. Each participant was asked to complete three tasks, first using the Libraries’ website, then using Google. The tasks were based on items found in the Libraries’ Special Collections: 1. Find a photograph available in UNLV Libraries of the Basic Magnesium mine in Henderson, Nevada. 2. Find some information about the Baneberry Nuclear test. Are there any documents in UNLV Libraries about the lawsuit associated with the test? 3. Find some information about the local Greenpeace chapter. Are there any documents in UNLV Libraries about the Las Vegas chapter? The Dean viewed those videos and chose the most interesting clips for a presentation at the Discovery Summit. Prior to this meeting, The Libraries’ staff were instructed to try completing the tasks on their own so that they might see the potential difficulties users must overcome and to compare the user experience provided by our website with that provided by Google. At the Discovery Summit, the dean presented the staff a number of clips from these special usability tests, giv- ing the staff an opportunity to see where users familiar with the Libraries collections stumble. The staff also were shown several clips of undergraduates using the website to perform basic tasks, such as finding journal articles or videos in the Libraries, with varying degrees of success. These clips helped illustrate the various difficulties users encounter when attempting to discover library holdings, including unfamiliar search interfaces, library jargon, and a lack of clear relationships between the catalog and other databases. This discussion helped set the stage for the Discovery Mini-Conference. n Initial changes to the site UNLV Libraries’ website is in the process of being rede- signed, and the results of the usability studies are being used to inform that process. However, because of the seriousness of some of the issues, some changes are being implemented into an intermediary design (figure 2). The new homepage n combines article and journal searching into one tab and removes the word “databases” from the page entirely; n adds a website search to the tabbed box; n adds a “Music & Video” search option; n makes better use of the picture on the page by incorporating rotating advertisements in that area; n widens the page, allowing more space on the rest of the site’s templates; n breaks the confusing “Help & Instruction” page into two more specific pages: “Help” and “Using the Libraries”; and n adds the main library and the branch library hours to the homepage. This new homepage is just the beginning of our efforts to improve discovery through the Libraries’ website. The Usability Working Group already has plans to do a card sort for the “Using the Library” category to further refine the content and language of that section. The group plans to test the initial changes to the site to ensure that they are improving discovery. Reference 1. Jennifer Church, Jeanne Brown, and Diane VanderPol, “Walking the Web: Usability Testing of Navigational Pathways at the University of Nevada Las Vegas Libraries,” in Usability Assessment of Library-Related Web Sites: Methods and Case Studies, ed. Nicole Campbell (Chicago: ALA, 2001). Figure 2. UNLV Libraries’ new website design