Introduction

In an effort to meet patrons’ needs, academic libraries continue to develop a strong virtual presence to provide online library services. These efforts include granting access to traditional services (e.g., journal and book searches, room reservations, instruction, consultations) using a variety of mobile applications and Web 2.0 tools (e.g., social networking sites, blogs, wikis). The intent is to create and maintain intuitive and effective websites that meet users’ needs and preferences adapted to a wide variety of devices (Aldrich, 2010; Fang, 2007; George, 2005; Houghton, 2000; Kroski, 2008; Tullis & Stetson, 2004).

This study evaluates the release of a new website for a private, mid-western university library. The university has a student population of approximately 30,000 to 35,000 students and grants undergraduate and graduate degrees in a wide variety of disciplines (e.g. Engineering, Humanities, Education, and Nursing). While designated as a teaching university, the institution has a strong history of academic research. This study is a summative assessment of the new library website meant to meet the needs of university patrons (faculty, undergraduate and graduate students) on a variety of access devices (computer, tablet, and smart phone). It is summative in that it represents the point of release to patrons following development that included a needs assessment and multiple formative assessments to inform the redesign of the library’s webpage.

Literature Review

Some examples of new technologies adapted for academic libraries’ website design are Web 2.0 tools, built-in analytics software that tracks users’ behaviors, and a website layout that is consistent across mobile devices (i.e., tablets and smartphones). Iterative website evaluations and continuous improvement are essential to meet users’ needs in times of rapid changes in technology. Researchers describe the importance of identifying current patrons’ needs and providing responsive designs so evolving library websites are effective and relevant to patrons (Aldrich, 2010; George, 2005; Kroski, 2008). Iterative website upgrades and improvements require constant evaluation of patrons’ use patterns and needs (George, 2005). Evaluation tools used for such efforts include observations, built-in analytics, surveys, talk-along tasks, and usability testing. These tools provide data that indicate best practices for designing, developing, and improving library websites (Fang, 2007; Houghton, 2000; Tullis & Stetson, 2004; VandeCreek, 2005).

George (2005) discussed how to improve an academic library website’s navigation through the use of strategic use of color and graphics to better attract users’ attention. She also recommended improving visibility by adjusting fonts, labels, and placement. Readability is boosted by chunking and using keywords. Finally, she suggested increasing usability with a consistent design throughout the website.

Raward (2001) stated that libraries’ websites are designed with the goal of providing reliable content and an interface that is intuitive and easy for patrons to use. Using human-computer interface research, she examined the main challenges of designing an effective academic library website and established a list of best practices for website development. The checklist includes 100 questions from four main areas: supporting user tasks and finding, understanding, and presenting the information. She suggests that user-centered design principles are the most successful for user experiences and these areas have become the main foci of academic website developers.

Many researchers stress the importance of end user feedback during all stages of development, especially since traditional academic websites were based mostly on the experience and expectations of librarians and developers but not the users (Crowley, Leffel, Ramirez, Hart, & Armstrong, 2002). In their study, Crowley et al. (2002) conducted focus groups in order to gather users’ feedback on the library’s webpage. They were surprised to find out how many of the patrons were confused and frustrated with their website experience. They used this feedback to better understand user experiences to improve the website and make it more intuitive and user friendly for the patrons.

Duncan and Gerard (2011) described a process of integrating an academic library’s reference system into its website. One initial goal was to discover user needs and then rework the reference system to fit those needs. The process resulted in system-wide changes, not only to the virtual reference system, but also to all aspects of the reference system and customer service delivery. Understanding user needs first was the foundation of their success.

Evans (2012) similarly recommended using focus group panels of end users to make sure librarians receive users’ input about the new technologies to be implemented on the library’s website. Instead of asking librarians to discuss what they think patrons’ technological needs are, patrons were asked directly. Evans modified the focus group approach by creating a focus group panel and letting library staff sit in and observe patrons’ discussions. Making sure the information finds its way from users to librarians is the key to success in the library’s website development.

While the website development is important, another consideration is how patrons are accessing the website. Internet access via mobile devices is a growing trend as website users want to find something now instead of looking it up later (Kroski, 2008). This results in patrons accessing information as they are moving through their daily tasks or during a momentary break. However, it is difficult for libraries to adopt technology to access virtual services that may be the flavor of the month (Evan, 2012). While on-going evaluations during development are important, a summative evaluation is also important to know how the entire package works together. That is, how well do all changes made to the website integrate with access devices to meet patrons’ needs?

Aims

Following an extensive, iterative design and development process, the Harold B. Lee Library (HBLL) revised its website to better serve the needs of its patrons. A “liquid” design of the website that provides a same-look view across different devices (e.g., smartphone, tablet, and computer) was released for patron use. This study is a summative evaluation of the efficacy of HBLL’s new website in meeting the needs of all patrons (faculty and undergraduate and graduate students). It also seeks to determine how each patron group accesses the website via the mobile devices for which it was designed. Finally, the study seeks to establish a baseline for preference of device access by patron group use so that future assessments will indicate trends and patterns of library website access.

Methods

This study used a three-part approach to determine the website and device usage patterns of library patrons. The three parts in our evaluation included focus groups, a university-wide survey, and usability test interviews. All data collection methods were approved by the university’s institutional review board (IRB). Each step is described below.

Focus groups. A total of seven focus groups, consisting of male and female HBLL patrons and employees 18 years and older were conducted. Participants were recruited from a pool of library patrons used for library assessment purposes. An employee is any full or part time, non-student employee who works at BYU. This group includes faculty who work at the HBLL and faculty who work elsewhere on campus (non-library faculty), but both of which use library services. Focus groups lasted 20–50 minutes and focused primarily on employees’ and patrons’ experiences with the new website. Questions included demographic information (e.g., participants’ major or department affiliation, year in school or position) and experience with the new website (e.g., usage frequency, task preference, current and ideal device preference, problems encountered, and website satisfaction). Focus group questions are found in Appendix A. All focus groups were video recorded and transcribed. Data from the focus groups informed the online survey and usability study.

Survey. The second step of the evaluation included developing and distributing a survey to library patrons. Focus groups’ responses and website developers identified areas of interest that were used to create the survey questions. Survey questions are found in Appendix B. The survey sought to confirm the patron usage patterns, current and ideal device preferences, and website services satisfaction as identified by the information from focus groups and website developers. The survey was administered via an online survey tool (Qualtrics) to approximately 6,000 male and female participants 18 years of age and older who were students (undergraduate and graduate) and non-library faculty. Response time to the survey varied depending on individual responses, but average response time was approximately 10 minutes.

Usability test. As a final step of the study, a usability test was administered to 21 patrons (faculty, undergraduate and graduate students). Participants were asked to complete 14 website-related tasks on three different devices (computer, tablet, and smartphone). The top 14 uses of the website, discovered during previous two steps, were chosen as usability tasks. All participants completed each task, but the tasks were randomized by device. Each participant switched devices as he or she completed the 14 tasks. All usability tests were video recorded but not transcribed as comments generally referred to specific actions taken on each device. Transcripts would not have provided the context of the comment without the image of what the participant was doing. Participants for the usability test volunteered via the online survey and were provided with a gift card as compensation for the length of time (approximately 1 hour) for participation.

Table 1

List of Library Website Functions identified by website designers and patrons.

1	Use main search bar	8	Find an e-book
2	Access databases	9	Locate library floor maps
3	Find a journal	10	Reserve a group study room
4	Use library catalog	11	Find and reserve a movie
5	Logon to account/holds	12	Access course reserve
6	Request interlibrary loan	13	Add and delete a bookmark
7	Find library hours	14	Identify library events

Funding was provided for this effort was provided by the HBLL.

Results

The results focus on three areas: (1) how well the new website was received, (2) what device patrons used to access website functions, and (3) how access devices (e.g., computer, tablet, and smartphone) were used to access the new website. The results combine findings from the focus groups, survey (with a 23% response rate), and the usability study.

Response to the website. Three-quarters of all patrons responding to the survey were happy with the new website design. Undergraduate students were the most pleased and graduate students the least satisfied. Of all patrons:

77% agree or strongly agree that they can easily find what they need on the new website,
75% report that the HBLL’s new website is clear and understandable, and
78% state that the new website is visually appealing.

When disaggregated by patron group, faculty responses focused on learning how to use the new website for teaching and research responsibilities. Graduate students were solely focused on the website’s functionality in their specific area of research. Undergraduates welcomed the changes the most, viewing the website as more modern looking as they used it for course assignments (e.g., researching via databases, peer-reviewed journals, and books), collaboration activities (e.g., booking group study rooms), or recreational activities (e.g., finding a movie). The broader use of the website by a specific patron group, the more satisfied the patron was with the changes. Comments typical of patrons were:

I find it better than the old website. Fonts and colors are visually more appealing, and it’s easier to access the content you are looking for through the library. There is always a learning curve when using something new, but after exploring for a short while I figured things out easily enough. That tells me that it is a well laid out website.

I like the look of the new website. It is easier to search for articles and books by their category than to just search aimlessly.

Function access by device. A main goal of website developers was to create a fluid design with a same look across multiple devices.

During this evaluation, a set of 14 website functions were identified by website developers and users as primary website functions. A list of each of these functions is provided in Table 1.

Using this list of website functions, patrons were asked to use access devices (e.g., computer, tablet, and smartphone) to complete usability tasks for each function. This usability study provided further indication of how well the website is accessed via the device by each patron group. Results for the usability test are shown in Table 2.

These results, along with comments made by patrons, indicate that patrons are generally able to access functions on the website easily. Five of the functions were easily accessed regardless of patron or device. Another three functions were easily accessed using two of the three devices.

However, when viewed by device, there is an increasing trend in the number of tasks that were increasingly difficult for patrons to do (computer = 0, tablet = 12, smartphone = 21). In these cases, size does matter as patrons commented that doing some tasks on the smartphone became increasingly difficult because the font size got smaller or patrons were required to zoom in and out to complete the task. In addition to this, the tablet and smartphone orientation (portrait or landscape) required patrons to scroll down to find website features. Patrons reported that the need to scroll down to see features was not intuitive. While the look was the same, switching from portrait to landscape obscured obvious links, resulting in more difficulty for patrons to access functions.

Faculty and graduate students were able to access the website more easily, while undergraduate students demonstrated a third more instances where the functionality caused problems.

Other comments related to difficulty of access focused on the intuitiveness of the webpage link. As mentioned earlier, the mobile device orientation could obscure the link. However, in some cases, comments indicated that the link button simply did not look like a link or needed to be located in a more prominent place on the webpage. For example, several patrons did not find the link to their user account via their name (after log on) to be intuitive, regardless of device.

Table 2

Summary of successful use of website functionality by patron and device use

Website Function	Faculty			Graduate Students			Undergraduate Students
Website Function	C	T	S	C	T	S	C	T	S
1	+	+	=	+	+	=	+	+	=
2	+	+	=	+	+	=	+	+	=
3	+	–	=	+	+	=	+	+	=
4	–	–	–	–	–	–	–	–	–
5	+	+	+	+	+	+	+	+	+
6	=	=	=	=	=	=	=	=	=
7	+	+	+	+	+	+	+	+	+
8	+	=	=	+	=	=	+	=	=
9	+	+	+	+	+	+	+	+	+
10	+	+	+	+	+	+	+	+	+
11	+	=	+	+	=	+	+	=	+
12	+	=	=	+	=	=	+	=	=
13	=	+	=	=	+	=	=	+	=
14	+	+	+	+	+	+	+	+	+
C = computer, T = tablet, S = smartphone, + used with ease, = used with some difficulty, – did not use or difficulty using prevented success

Table 3

Preference of Patron Accessing Website Functions by Device

	Accessed by			Not accessed
Website Function (by importance)	Computer	Tablet	Smartphone	Not accessed
Main Search Bar	76%	6%	11%	6%
Databases	80%	5%	4%	11%
Journals	75%	4%	3%	19%
My Account (renewals, holds, etc.)	67%	6%	10%	18%
Interlibrary Loan	57%	3%	3%	36%
Special Collections	31%	2%	2%	66%
Course Reserve	31%	2%	2%	65%
Group Study Room Reservations	49%	4%	8%	40%
Library Chat	25%	1%	2%	72%
Bookmarks	23%	1%	4%	72%
Library Hours	55%	5%	11%	29%
Library Floor Maps	50%	4%	7%	39%
Library Events	28%	2%	4%	67%
WorldCat	23%	2%	1%	74%
Library Catalog	62%	5%	7%	26%
Physical Books	62%	3%	7%	28%
E-books	50%	5%	5%	40%
Movies	32%	2%	4%	62%
Audio books	20%	2%	3%	76%
Media Equipment (e.g. cameras, sound booth, etc.)	15%	1%	2%	82%
Boldface indicates a high rate of use or non-use (70% or greater).

Current website access. As this is the first website release that has the same look on multiple devices, one objective of the study was to determine current use patterns. This data serves two purposes. First, it indicates what devices patrons are currently using to access the website functions. Second, it determines a baseline of use patterns for future reference. Table 3 summarizes the percentage of all patrons accessing the website function by device. It also indicates the degree to which a website function is not accessed by any patrons.

While this information offers a better picture of how all patrons are accessing website functions, it is instructive to examine website function access by device and user group to establish trends and patterns among key user groups. Tables 4, 5, and 6 indicate the top five rank-ordered, website functions accessed by each patron group as indicated on the survey. In this case, graduate students were further disaggregated into master’s and doctorate students to determine whether differences existed in use patterns for this specific patron group.

Table 4

Top Five Accessed Services on a Computer by Patron Group

Undergraduate Students	Master’s Students	Doctorate Students	Faculty
Main Search Bar	Main Search Bar	Journals	Databases
Databases	Journals	Main Search Bar	Journals
Journals	Databases	Databases	Main Search Bar
My Account	My Account	My Account	Interlibrary Loan
Physical Books	Interlibrary loan	Interlibrary Loan	My Account

Table 5

Top Five Accessed Services on a Tablet by Patron Group

Undergraduate Students	Master’s Students	Doctorate Students	Faculty
Main Search Bar	My Account	Journals	Main Search Bar
My Account	Main Search Bar	Databases	Library Catalog
Library Hours	Databases	Main Search Bar	E-books
E-books	Journals	Interlibrary Loan	Databases
Library Catalog	Interlibrary Loan	My Account	My Account

Table 6

Top Five Accessed Services on a Smartphone by Patron Group

Undergraduate Students	Master’s Students	Doctorate Students	Faculty
Main Search Bar	Main Search Bar	E-books	Main Search Bar
Library Hours	My Account	My Account	Library Hours
My Account	E-books	Physical Books	My Account
GSR Reservations	Library Hours	Journals	Library Catalog
Library Floor Maps	GSR Reservations	Main Search Bar	Library Floor Maps

GSR=Group Study Room

While there is considerable overlap between patron groups, undergraduate students’ access has a strong relation to website access and the physical library. Each subsequent groups’ website access is more tied to virtual functions rather than to physical resources. The rank order also speaks to patron group preferences when accessing library resources. This listing provides the opportunity to examine use patterns of patrons and to determine how patrons can be directed to other complementary resources helpful for learning, instruction, and research needs.

Limitations of the Study

While a needs assessment and on-going formative assessment took place during the development of the new website and promotion of the website prior to its release occurred, two factors limited the assessment of the new website. While the limitations were of concern, we feel that the samples obtained were representative of the library website patrons and provided a strong baseline of the website’s utility and patron access.

Lack of experience with the website. Regardless of when the new website was released, there would be a period of adjustment to the new website features. One of the obvious findings of this study was users’ lack of experience with using the new website. Forty-seven percent of the students and faculty surveyed online reported that they did not have a chance to yet utilize the website. Several participants during the focus group used mobile devices and laptop computers to access the website to make comments on the website during the focus group. This first time access indicated the limited exposure patrons had with the new website. The lack of experience using the website was evident but expected as patrons learned the new functionality.

Lack of experience with access devices. Many participants did not have experience using some devices to access the website. For example, 65% of the survey respondents had not used a smartphone and 72% had not used a tablet to access the website. Faculty and graduate students reported rarely using a tablet or a smartphone to access the website.

Choice of device to access the website depended on patrons being familiar with the functionality of the device and having access to the device. For example, when asked if the website was accessed via smartphone, several undergraduate students expressed the sentiment, “I wish I had a smartphone.” Faculty also reported only rarely seeing some students using a tablet in class to access the library’s website. The lack of experience using a specific device may have influenced patrons’ access of specific website features.

Discussion

This study reiterates the importance of continuous feedback from patrons regarding library products and their delivery (Aldrich, 2010; George, 2005; Kroski, 2008). While a needs assessment and continuous patron feedback occurred during the development of the new website, a summative evaluation was required upon the website’s release. Such an evaluation provides a broader look at the library’s services or products and how they integrate into the overall organization. While this evaluation is primarily summative at this point in time, it inevitably uncovers additional information to inform future developments. It helps to embody the attitude that there is no limit to improvement.

In this specific case the look, feel, and access to the website is an improvement over the previous website. The changes in font, color, and link placement serve to make the website more functional, easier to navigate, and shortens the learning time, especially when using different devices. This finding is in line with previous suggestions from George (2005) on how to change fonts, colors, and placement to improve website usability. Comments from patrons throughout the data collection indicated that the website provided a cleaner, sharper appearance that facilitated use. The changes make the website use more reliable and intuitive to find library products (Raward, 2001).

The study also highlights several use patterns among patrons. Computers (both laptop and desktop) continue to be the dominant device for website access, but there is potential to use mobile devices more in accessing library services. The key is using the device as a tool that is best suited for accessing a specific website function. Mobile devices appear to be beneficial for on-the-go tasks related to attending the physical library (e.g., booking group study rooms, finding library hours) or housekeeping tasks (e.g., accessing one’s library account to renew books). While research tasks may be done on mobile devices, these tasks are better done on computer since the smaller text size makes mobile access more difficult. Tablets may become a happy mid-point between mobility and functionality of device access, as patrons gain increased access to and experience with them.

Having a fluid design enables patrons to learn only the website features and their location on one website once, but care needs to be taken with limitations of device use. For example, mobile device orientation may limit users’ views and patrons need to be aware of the need to scroll up or down or to zoom in or out to fully access the page views. The findings also indicate that device ownership does not translate to device use for accessing the library services via the specific device. It is important that librarians provide instruction using each device so patrons are better able to understand how each device may be effectively used.

Other findings indicate which device is commonly used to access website functions. The computer is the dominant device used by patrons for accessing website functions associated with research activities (e.g., main search bar, databases, and journals). The results also establish smartphones as a more dominant access device than tablets. However, this pattern may be affected by patron ownership of or access to a specific device. Greater numbers of patrons owned smartphones than owned tablets.

The dominant activities associated with smartphones indicate a pattern toward easily accessed functions. The on-the-go functions accessed include, but are not exclusive to, accessing patron accounts, reserving group study rooms, finding library hours, and viewing library floor maps. If access via smartphone was easier than using a computer, patrons used the smartphone. A common comment indicative of patron smartphone use was:

If I can access the service faster than it takes to get out my computer and look it up, I use my smartphone.

Use patterns also provide indications of how library services are used. For example, physical library services are accessed more by undergraduate students. Graduate students and faculty tend toward a greater use of virtual access of library services. While this is helpful for device access, it also provides insights into the type of patron services that should be developed for virtual use. The use patterns also indicate services that are rarely or underutilized (e.g., library chat, bookmarks, WorldCat, and media). This knowledge enables librarians to change instructional and promotional efforts to increase patron exposure to these services. It also enables librarians to examine the service to see if it is working as intended through additional input from patrons or if additional instruction is needed (Duncan & Gerard, 2011; Fang, 2007; Houghton, 2000; Tullis & Stetson, 2004; VandeCreek, 2005).

Instruction from librarians and faculty provides the opportunity to make patrons more aware of website services and how these services may be accessed on each device. The instruction may be imbedded in current discipline instruction, added on to library research instruction, or provided as stand-alone instruction. In the latter case, instruction may come in the form of short tutorial videos to help patrons learn or remember how to use the website better. This instruction helps patrons understand how services may be accessed via different devices and which services are best accessed with which device. It will help to shorten the learning curve on how to use different website functions and increase the usability of the website.

Future directions of research include a follow-up study to continue to revise and expand the website’s functionality (e.g., updating floor maps). As patrons gain more access to mobile devices, it will be of interest to determine if and how website access changes across devices and patron groups. Since one purpose of this study was to determine a baseline of use, determining future changes in website access will influence how the website adapts to those changes.

Conclusion

This assessment determined the usage patterns of patrons of the Harold B. Lee Library’s new website via a variety of devices. We conducted a three-step evaluation including focus groups, an online survey, and usability tests using computers, tablets, and smartphones. Each stage of the study helped inform the next stage, and the data gathered at each stage was used to triangulate the results and conclusions. About a half of our respondents did not have experience with the new website and were unable to provide us feedback because the website only became live at the beginning of the summer term when many students and faculty do not have classes.

The changes to the website were well-liked and well-received by a vast majority of the undergraduate students as well as most graduate students and faculty. The majority of the patrons with website experience reported high levels of satisfaction with the website’s look and aspects of its functions. Most areas of dislike were deemed to be issues of personal preference, issues that are easily fixed, or issues beyond the responsibilities of library website designers.

Another of our purposes was to explore patrons’ device preference for using the website. Computers are owned and used the most of the three devices, followed by smartphones and tablets. Patrons’ device preference for accessing website function was also determined and disaggregated by patron group and device. There are certain tasks patrons would perform on any device (e.g., search for materials or manage their account). There are tasks most participants would only perform on their computer (e.g., research or accessing course reserve). Finally, patrons prefer accessing specific functions of the website on their mobile devices (e.g., checking the library’s hours or reserving group study rooms) because of the ease and convenience.

The summative evaluation provides a broad view of the library’s new website. It increases the understanding of how well the website is working to meet patron needs. Finally, it adds to the feedback knowledge for future modifications to help the website be more functional, intuitive, and useful to patrons.

References

Aldrich, A. W. (2010, June). Universities and libraries move to the mobile web. EDUCAUSE Quarterly. Retrieved from http://www.educause.edu/ero/article/universities-and-libraries-move-mobile-web.

Crowley, G. H., Leffel, R., Ramirez, D., Hart, J. L., & Armstrong, T. S. I. (2002). User perceptions of the library’s web pages: A focus group study at Texas A&M University. Journal of Academic Librarianship, 28(4), 205–211.

Duncan, V., & Gerrard, A. (2011). All together now! Integrating virtual reference in the academic library. Reference and User Services Quarterly, 50(3), 280–292.

Evans, G. (2012). Emerging technologies: How do we know what’s happening “on the ground”? Public Services Quarterly, 8(2), 164–170. doi:10.1080/15228959.2012.675284

Fang, W. (2007). Using google analytics for improving library website content and design: A case study. Library Philosophy and Practice (e-Journal), Paper 121.

George, C. A. (2005). Usability testing and design of a library website: An iterative approach. OCLC Systems & Services, 21(3), 167–180. doi:10.1108/10650750510612371

Houghton, D. (2000). Building an academic library website: Experiences at De Montfort University program. Electronic Library and Information Systems, 34(3), 269–280. doi:10.1108/EUM0000000006934

Kroski, E. (2008). On the move with the mobile web: Libraries and mobile technologies. Library Technology Reports, 44(5), 5–9.

Raward, R. (2001). Academic library website design principles: Development of a checklist. Australian Academic & Research Libraries, 32(2), 123–136. doi:10.1080/00048623.2001.10755151

Tullis, T. S., & Stetson, J. N. (2004, June). A comparison of questionnaires for assessing website usability., Usability Professionals Association Conference (pp. 1–12). Retrieved from: http://home.comcast.net/~tomtullis/publications/UPA2004TullisStetson.pdf

VandeCreek, L. M. (2005). Usability analysis of Northern Illinois University Libraries’ website: A case study. OCLC Systems & Services, 21(3), 181–192. doi:10.1108/10650750510612380

Appendix A

Questions for the Focus Groups

1. Please tell us your major (students) or faculty position or staff position, and year in school (or position).

2. How frequently do you use the library website?

3. Describe your typical uses of the library website.

4. What are the most important features of the website?

5. What device do you usually use when accessing the website?

6. Given the choice, what device would you use for the website?

7. Why would you choose that device?

8. Have you tried the new version of the Library Website? What have you noticed is different?

9. How easy is navigation of the new website?

10. What problems or issues have you encountered in using the new website?

11. Given your list of typical uses, how do you feel about the new website meeting your expectations for achieving the tasks?

12. Please share any other thoughts or comments about the Library new website.

Please note that questions are numbered for convenience of reference. Depending on the course and comments of the focus group, questions were discussed in different order. Not all questions were addressed in some focus groups because patrons’ responses extended longer than the scheduled time of the focus group. Finally, additional questions were asked to help clarify or add explanation to given patrons responses, such as, “Would you please elaborate more on that?” and “Please explain that idea a bit further for me?”

Appendix B: Questions for the Online Survey

1. What is your affiliation with Brigham Young University?

· Undergraduate student

· Graduate student - Masters

· Graduate student - Doctoral

· Faculty member

· Staff

1A. How many years have you been in:

a. undergraduate studies?

b. your graduate program?

c. a BYU faculty position?

d. a BYU staff position?

Pull down menu (1-45)

1B. Are you/have you been a. HBLL student employee?

b. HBLL faculty/staff?

2. What is your major/department affiliation?

Pull down menu (list of departments)

3. In the last six months, approximately how frequently have you logged onto the Harold B. Lee Library website?

· About once a day

· About 2-3 times a week

· About once a week

· About 2-3 times a month

· About once a month

· Less than once a month

· Never

4. How proficient would you rate yourself when using the library website?

· Very proficient

· Proficient

· A little proficient

· Not at all proficient

· Other ______________

5. What device do you usually use to access the website? (Select all that apply)

· Personal computer

· Library’s desktop computers

· University (non-library) computer

· Tablet

· Cell phone

· Other ____________________

6. Please select the device you use to access the Harold B. Lee Library website to access:

	Computer	Tablet	Cell phone	I do not access this HBLL service
Databases
Journals
Physical books
E-books
Bookmarks
Interlibrary Loan
Media services (e.g. DVDs, audio books, video cameras, etc.)
Library Catalog
WorldCat
Special Collections
My Account
Library Chat
Course Reserve
Group Study Rooms Reservation
Search Bar
Library Hours
Library Maps
Library Events

7A. Do you use your smart phone to access HBLL website?

· Yes

· No

· I do not have a smart phone

If no or I do not have a smart phone, go to question 8

If yes, go to question 7B.

7B. To what degree do you access the following services on your smart phone?

1=very rarely, 5=all the time

	Very rarely	Rarely	Sometimes	Often	Very often
Study room reservations
Hours
Book check-out
Bookmarks/check-out list, call numbers
Course Reserve
My Account (e.g. book renewal)
Media services (e.g. DVDs or audio books)
Databases
Events
Search bar
Maps

8. Please rate the following statements about the Library website by choosing from 0 to 5 with 0 being no experience, 1 being strongly disagree, 5 being strongly agree

· The information offered on the website is clear and understandable

· The website is easy to navigate to find what I need

· The website is visually appealing to me

· My expectations of accessible information were met by the new website

· What is available on the mobile website right now is not what I would do on my smart phone

· The things I would like to do on my smart phone are difficult to access on the mobile website

· I never used the site on my phone because I didn't know it became more mobile friendly

· It is difficult to access my account-based things (e.g. bookmarks or renewals)

· I want to be able to customize the Library webpage links to meet my needs

· When reserving Group Study Rooms, I want them to be categorized by size and equipment, and not by location

· I want to know how search results are grouped and categorized

· Often when search results show the book is available, it is not found on the floor

· I would like to be able to access my check out history

· I would like to be able to access my past searches

· I do not love the new design – the colors and theme are not consistent with all BYU websites

· The color scheme and fonts make it hard for me to see things clearly on the new website

· What would you like to tell the Library about the new website? (Open-ended)

10. In our efforts to fully evaluate the library’s new website we are looking for participants for interviews and usability studies. Would you be willing to participate in an:

Interview (15-20 minutes, volunteer)? Yes No

Usability test (30-40 minutes, compensated, need access to smart phone)? Yes No

If no, end of survey.

If yes, please provide your name and contact information below.

Name:

Phone:

Email:

End of survey.