Evidence Based Library and Information Practice

Setting

Located in downtown Cleveland, Ohio, Cleveland State University (CSU) is comprised of 10 colleges and schools, which offer over

175 academic programs, including several doctoral programs. The university, which has a current enrollment of more than 17,000 students, is highly diverse with regard to age, ethnicity, and country of origin.

The Michael Schwartz Library (MSL) supports this diverse community with a collection of over 1 million titles, nearly 400,000 of which are electronic resources. The MSL subject librarians create and maintain 340 publicly viewable LibGuides, both general and course-specific, spanning 64 subjects. Research guides are online reference tools that librarians create to help students and faculty conduct research. Research guides can include lists of relevant sources, instructional content related to the research process, and contact information for library staff. LibGuides are research guides built on a web publishing and content management platform offered by SpringShare and used by libraries throughout the world. The MSL LibGuides are the focus of our research, which relies on a variety of evidence, including an extensive literature review of LibGuide design and user experience, data from our own users, and our librarians’ professional knowledge and experience.

Problem

To help frame the research and decide what evidence to obtain, the researchers used the PICO (Problem, Intervention, Comparison, Outcome) model to develop our research question. First, the problem was articulated: MSL librarians were unsure how useful our LibGuides were to CSU faculty, staff, and students on campus, and what impact the design of the guides had on their usefulness. We had encountered literature that suggested switching the layout of our guides from top to side navigation. When proposing this suggestion to our colleagues, it was met with some resistance, which was an additional problem. Without evidence, we didn’t know which design would be most effective. The intervention we chose was to develop research guide usability best practices using relevant literature, present the results to our colleagues, and observe if the presentation of evidence improved the librarians’ receptivity of our recommendations.

Evidence

Evidence based library and information practice (EBLIP), which relies on evidence rather than theory or previous precedent as a basis for practice (Hjorland, 2011), was used to structure the design of this study. We followed the EBLIP model of Koufogiannakis and Brettle (2016): we articulated our problem (described above), assembled relevant evidence, assessed the evidence for quality, and agreed to a course of action as a research team and department. As of this writing, we are still in the implementation stage, and will adapt our approach based on the outcomes of our intervention. In our case, the evidence assembled included local data from a community usability survey, the impressions and experiences of the researchers, and a thorough review of the relevant literature.

To determine the best intervention to address the problem, we conducted a thorough literature review. We gathered evidence by searching a variety of databases and platforms including Academic Research Complete; ACRL TechConnect; C&RL News; Digital Commons Network; Education Research Complete; ERIC; Google; Google Scholar; Library, Information Science & Technology Abstracts with Full Text; and Web of Science.

Results were excluded if they were published before 2013, not related to research guides (instead focusing on library websites or other online portals), or not related to user design. A variety of terms were considered acceptable to refer to user design, including design, layout, user experience, and others. Since there were too many LibGuides that describe best practices to make including them practical (over 2,000 in a LibGuide Community search), and because most of their evidence was anecdotal, these were also eliminated from the review results. We also investigated the citations in remaining resources and included them if they did not meet the exclusion criteria. Two articles were included, despite falling outside of the date parameters of the review, because they were cited so heavily in the literature and clearly remained relevant to the design of research guides.

Table 1

Search Strings Included

"best practices for libguides"	(libguide or "subject guide" or "research guide") AND use
"libguides best practices" filetype:pdf	ALAO AND libguides
"research guide" AND "user experience"	libguide
"research guide" AND "user experience"	libguide AND "best practices"
"research guides" AND "best practices"	research guide best practices
(libguide or "subject guide" or "research guide") AND (evidence based or best practice)

We assessed the gathered evidence by creating a list of codes for user experience and design best practices. To reduce bias in code creation, each of the three researchers developed codes separately and then the codes were compared and assembled into a master list. The literature sources were then coded by the researchers independently and results were analyzed and synthesized to create a list of best practices. Each best practice was accompanied by a list of all the relevant supporting literature, and the literature was color-coded to show what kind of evidence contributed to the authors’ conclusions (e.g., qualitative, quantitative, mixed methods, and anecdotal). The full color-coded list of best practices may be found here: https://researchguides.csuohio.edu/ld.php?content_id=47624389. Best practices from the literature that appeared to contradict one another were retained to reveal areas where more research is necessary.

Table 2 provides a summary of the suggestions we found via iterative literature searches, which was the primary basis of our research. However, in order to collect additional, local evidence and establish a benchmark for student, faculty, and staff satisfaction with the MSL’s LibGuides in our specific context, we also conducted a usability survey using LimeSurvey in February 2019. Undergraduate and graduate students, faculty, adjuncts, librarians, and library staff were included in the survey whether or not they had used research guides. The survey was available in the library’s voting booth (a publicly-accessible computer set up in a prominent location in the library lobby), as a link on the library website, and emailed directly to faculty by subject librarians. It was confidential, incentivized by a raffle, and solicited information such as what college the participant was from, whether they had used LibGuides before, what goals they had when visiting the site, and whether their goals were met. The survey was made available for two weeks and had 114 responses. The data from this survey were to be used to compare user satisfaction before and after the implementation of the literature-supported best practices to the library’s guides.

In the meantime, the researchers updated an existing research guide using the literature-based best practices list to demonstrate to our librarian colleagues how a guide might be modified to better match user experience standards. These guides and other relevant documents can be found here: https://researchguides.csuohio.edu/bestpractices.

Table 2

Literature-Based Best Practices with Conflicting Evidence in Brackets

Category	Best Practice	Details
Design / Organization / Layout	Template	Provide a guide template for all librarians [A template is only so useful - guides should be customized to their unique audiences in some cases, and authors should retain freedom over guide content and design]
	Policy	Create standards based on best practices or other criteria
	Uniformity / Consistency	Follow a unified, consistent format and design (fonts, background, color scheme) for subject guides and their content Make sure labels and language are consistent across guides Consistently name a core set of tabs by subject or format
	Key Resources / Best Bets Box	Provide a “key resources” or “best bets” box in a prominent location on the guide Use a large enough text size (larger than default for LibGuides 1.0)
	Hierarchy	List resources strategically or by importance, rather than alphabetically Sequence content in the order students would likely need to encounter it to accomplish their tasks Put the most important content on the left and/or top of the page in an F-pattern
	Integration	Use the main library or university website “frame” to visually integrate the guide with the rest of the website
	Personal Presence	Include a professional photo of one or more librarians on the guide Make guides more personal by providing librarian contact information and option to chat
	Chunking Content	Split up content into meaningful chunks
	Number of Columns	Use a two column layout [Use a three column layout] Don’t include important content in right column (users ignore this as it is commonly ad space on websites)
Navigation	Top vs. Side Navigation	Use side/left navigation to make menu more visible
	Tabs	Tabs tend to be unnoticed and large numbers of them confuse users and cause clutter, so use only most relevant ones, usually all in a single row
	Search Box	Include a search box as students prefer to be able to search the guide for content rather than browse/read [Don’t include a LibGuides search box on guides, as students often treat it as a discovery or Google search. If a search box is included, include a description of what can be searched.] Provide embedded search boxes for research tools (i.e. databases, catalog, etc.)
	Table of Contents	Do not provide a box on the guide that outlines its contents, while also providing tabs, as this is considered redundant by users [Provide a table of contents box on the homepage of each guide because students often overlook tabs, and/or to prevent users from having to scroll down]
Content	Jargon	Avoid the use of jargon throughout the guide or, if it’s necessary, provide clear explanations of unfamiliar language
	Labeling	Use short, clear, meaningful titles for guide names, boxes, menus, pages, and tabs If possible, include a description (annotations) for tools provided in the guide, especially if their titles are not self-explanatory or use jargon Name guides, tabs, and boxes the way students would search for them
	Writing for the Web	Write content using best practices for web writing Use bullet points and bolded or varied text sizes to make pages easier to read
	Content Maintenance	Regularly check for broken links, perhaps with a link check tool Make sure videos and screenshots are up-to-date Make sure terminology and content is current Develop a maintenance plan for guides Use the LibGuides asset manager to efficiently update links and reuse content across all guides
	Friendly Tone	Use a conversational tone in the text of guides
	Audio/Visual Material	Incorporate interactive and visual content to engage students [Use images sparingly, as they often add more clutter and waste space]
	Widgets	Include a chat widget allowing users to chat with the subject librarian when they are online
	Less Text / Content	Ensure amount of information on pages and in boxes is appropriate Include less content/fewer pages to avoid cognitive overload and encourage more usage Avoid long lists; if lists are used, create them such that users can skip to sections/content of interest
Accessibility		Ensure guide can be easily read by a screen reader Ensure all videos on guides are captioned Ensure all images have alt tags All “click here” links should instead have descriptive text for the link location Ensure the color of text and other elements contrasts enough Avoid relying solely on color for meaning Make guides ADA accessible (or meet other accessibility standard)
Purpose	Instruction vs. Reference	Consider the purpose of the guide (to teach or to provide curated resource lists) when designing it Provide instructional content in the guide that will help students complete the tasks that likely brought them there Build the guide around one or more student learning outcomes or other pedagogical goals Create course specific guides rather than broad subject guides
	Considering Audience	Think about how users will search for content in the guide, and in accessing the guide; let that govern your design The purpose of the guide should be made explicit to students
	Connect to Class / Assignment	Tie the content of guides to specific course research and assignments
External Factors	Guides Menu	Organize guides by how users would likely require access to meet an information need
	Promotion & Marketing	Librarians and, especially, instructors should promote guides Link to guides in the learning management system Email a link to the guide to students, provide the link in an in-class handout, and/or demonstrate how to access the guide in class
	Guide Access / Discovery	Provide a link to guides on the library’s homepage Provide links to guides in the learning management system Consider ways of improving findability of guides in an organic search
	Reduce Duplication / Stale Guides	Remove unused or stale guides
	Guide Assessment / Maintenance	Use guide usage data to regularly assess guides Use usability testing (focus groups, surveys, etc.) and outreach to regularly assess guides Guide authors should review guides regularly
	Guides Team / Administrator	Assemble an administrative team to maintain upkeep of guides and set guide standards for the institution

Implementation

After the evidence was assembled and analyzed, an intervention took place to apply the best practices for usability and improved design to our LibGuides. A 90-minute session was scheduled with guide creators to present the evidence, best practices, demo guide, and checklist (https://researchguides.csuohio.edu/ld.php?content_id=50666759) and to discuss implementation. Six out of twelve guide creators attended. Reception was much more favorable compared with previous discussions. Indeed, sharing our research encouraged guide creators to adapt the best practices where practical. It was determined that application of the best practices should be flexible to allow for different disciplines and specific guide uses. An optional follow-up meeting to work on the research guides (a hack-a-thon) was scheduled for about a month later. Four guide creators participated in the hack-a-thon, and others worked at their own desks.

Reflection

One thing we learned while working on our literature review was that there is still not enough rigorous evidence about best usability design practices for research guides, and much of what does exist is specific to one institution. We also found that some of the evidence was conflicting, so more research into those specific areas would be helpful.

An additional challenge we faced in gathering evidence was soliciting usable results to our survey. We lesarned that many participants in the survey did not know what a research guide was, or had never used one. These participants gave responses to the survey that did not provide relevant information about our research guides and, for this reason, many had to be removed from our analysis. We also found flaws in our survey questions. Rather than asking patrons how they used a research guide, we discovered that it would perhaps be more useful to ask patrons to show us in real-time how they would fulfill a need using a research guide.

Finally, we learned a great deal from the process of using evidence to recommend department-wide change in the library. We cannot force our library colleagues to change their user design decisions, nor would we necessarily want to. We found that doing the research and presenting a well-founded set of recommendations resulted in our colleagues sometimes choosing to make changes to their guides based on our best practices investigation. However, the process also helped us become aware of unique circumstances that may warrant ignoring our recommendations, and the discussion that this engendered helped us all feel more comfortable with the resulting decisions. We hope to conduct additional usability studies in the future to make a stronger case for applying research guide design best practices in a way that best helps our local community of library users.

References

Hjørland, B. (2011). Evidence‐based practice: An analysis based on the philosophy of science. Journal of the American Society for Information Science and Technology, 62(7), 1301-1310. https://doi.org/10.1002/asi.21523

Koufogiannkis, D. & Brettle, A. (eds). (2016). Being evidence based in library and information practice. London, UK: Facet Publishing.