Chat Reference Referral Strategies: Making a Connection, or Dropping the Ball?

Paula R. Dempsey*

The study analyzes how chat reference providers refer patrons to subject specialists in 467 interactions from two years at a university library. Qualitative analysis showed variation in how referrals are presented: as an option versus a recommendation, putting follow-up in the hands of patrons versus librarians, and apologizing versus promoting benefits. Professional librarians referred more questions to specialists, framed more referrals as benefits, and sent more transcripts (“tickets”) than graduate assistants or paraprofessionals did. Findings show correlations between patrons’ positive responses to referrals and the extent of the reference interview and positive framing of the referral, but not the attempt to assist before referring.

Introduction

Chat reference is a low-barrier entryway to library services and resources for students. Because chat reference is available at the point of need and is usually anonymous, it is an opportunity to reach users who might never ask a question in person. Because chat extends reference at a distance and often into late night and weekend hours, it draws a wide net of patrons at times when they are engaged in research. However, chat reference providers cannot fully respond to every question. In particular, subject research questions may be beyond the scope of providers staffing chat, especially student employees working after hours or on weekends. In addition, chat services staffed by cooperative librarians or librarians contracted by the software provider rely on referrals when specialized local knowledge is needed to fully answer the question. Referring patrons to a subject specialist may serve the user best in those cases. A referral puts the patron in touch with the librarian most capable of providing resources and teaching research strategies.

Referrals also benefit libraries with liaison programs by building awareness of liaison librarians’ services and developing a pipeline for collaborative relationships between subject specialists and their stakeholders. For the purposes of this study, librarians are subject specialists if they have formal responsibility for a specific discipline in terms of collections, instruction, research support, outreach, or some combination of these duties; special collections librarians are also considered subject specialists whose expertise crosses disciplinary boundaries. Liaison librarians with intimate knowledge of the collections, curriculum, and faculty research agendas have more background and context than others in the library to assist patrons in that discipline. Such liaison librarians may or may not have an academic background in the field, and their depth of experience working as a librarian within that discipline varies.

Professional guidelines for chat reference services acknowledge the necessity of referrals but do not flesh out best practices. The American Library Association’s Reference and User Services Association (RUSA) Guidelines for Implementing and Maintaining Virtual Reference Services stipulate that libraries should:

5.1.5 Establish procedures for referring a virtual patron (i.e., question) to another reference or public services point. Include procedures for both how the referral is presented to the patron and how information about the referral is communicated between the virtual reference desk and referral destination.1

It is not known how many virtual reference services adhere to RUSA Guideline 5.1.5, what referral procedures they have established, or how they carry out referrals in everyday practice. The current study provides empirical grounding for developing referral procedures by analyzing 467 subject research interactions at a large research university library over two academic years. This is important because, in order to develop effective procedures, chat providers and service administrators need evidence of the full range of variation in existing local practices.

The current study is grounded in two assumptions. The first, established in professional standards, is that reference work in any setting is a forum for teaching and learning.2 Moreover, chat reference should adhere to the same standards as in-person reference, including an effective reference interview. The second assumption is based on practical observation: referrals work best when the chat provider forwards the entire interaction to the referral destination by sending the chat transcript, which documents the original question and all details of the original response. In a term borrowed from computer help desks, transferring the transcript is often called “submitting a ticket.” Tickets can be created manually by copying the transcript into an email (as in the first year of data for this study) or automatically in a virtual reference system (as in the second year of data). Tickets have several benefits:

Note that the last three advantages are relevant even in cases where the patron’s immediate information need was fully resolved by the chat provider.

The study is exploratory and does not make a generalizable claim about patterns of referral, which are likely to vary based on factors such as the scope of virtual reference and the availability and background of subject specialists. Rather, this study aims to explore the range of variation in how providers manage referrals and to make preliminary judgments about best practice for further investigation. The value of the single-institution study is in-depth knowledge about the context; findings for this study are most applicable to academic institutions with subject liaison programs and broad expectations for chat reference service. Specific research questions follow:

Review of Selected Literature

This study fills a gap in the chat reference literature around how providers refer subject research questions to specialists. This research problem is rooted in studies investigating the purpose of chat reference service, its staffing, and the training of chat service providers. Previous studies question whether chat is a venue for only brief, straightforward questions or can be an effective mode for subject research questions. There is also ongoing discussion about whether student employees can be adequately trained to answer and properly refer chat questions. A third debate is whether a referral to an expert should be considered a signal of quality service or a disservice to the patron. No study was found that investigated the various ways in which referrals were presented to patrons, whether in face-to-face or chat reference.

Previous studies of chat reference have counted referrals, but the findings are not directly comparable because of differences in how researchers defined and sampled question types and referrals. Paul Neuhaus and Matthew Marsteller found that roughly 28 percent of chats were referred. Librarians quickly referred questions in subject areas they were not familiar with, and patrons were dissatisfied with the service. They concluded that, because a wide variety of staff provided chat service, “such occurrences were regrettable but probably unavoidable.”3 Nahyun Kwan and Vicki Gregory studied a consortial service that received very few subject research questions; 29 percent of all questions were referred back to the home library.4 Gang Wan et al. found that 10 percent of chat reference questions required a subject specialist, but the study did not report whether the referrals were actually made or how they were handled.5 Kelsey Keyes and Ellie Dworak found that 27 percent of the transcripts they studied were appropriate for referral, and 43 percent of those received a referral. The status of chat service providers made no significant difference.6 A study of the UIC Library virtual reference service in academic year 2004 found that 23.5 percent of chat reference inquiries were subject research questions, that collaborative staffing between a health sciences team and a general library team was “effective, efficient and desirable,” and that most questions were answered by someone with the appropriate subject expertise. Further, 8.6 percent of questions received via chat were reassigned or referred.7

Is chat a mode of reference service that is suited to answer subject research questions, and to what extent do patrons expect this kind of service? Professional guidelines require virtual reference services (VRS) to delineate a scope:

3.2.1 Define and make accessible the level of service to be provided so that staff and patrons will understand the mission of the service.
3.2.1.1 Level of service includes the types of questions the service will answer (or the mission and the scope), the response time, and the intended audience for the service.8

A survey of service policies by Maria Pinto and Ramon Manso found that a sample of 46 U.S. and European libraries were split on the types of questions the service will answer: “47.8 per cent of the services offer four variants: directional questions, quick reference, instructions, and research; the remainder only provide directional questions and quick reference in a more simplified form.”9 Libraries do not always provide an explicit policy, however, leaving the expectations informal or tacit. Researchers’ opinions have tended toward limiting the scope of chat service to “simple factual and directional but not reference questions.”10 Neuhaus and Marsteller argued that chat is more appropriate for “simpler types of questions than for providing in-depth research assistance. Librarians may be willing to spend the time needed for such research but many patrons are not.”11 David Ward agreed that lengthy questions are not suited for chat and suggested a policy limiting chat to short-answer inquiries: “If a question becomes lengthy or complex,” the provider should offer to follow up by phone or email.12 Studies of patron behavior, however, have found that they do expect in-depth reference assistance in chat. Vera Lux and Linda Rich found that chat reference received many more reference questions (91%) than did in-person reference (50%).13 The author’s previous study of two other chat services found that, at one library with a policy limiting chat to brief, factual questions, patrons asked subject research questions at the same rate as at another library with a broader service scope during times in the semester when many students are working on research projects.14 Furthermore, when libraries use a chat widget that appears across multiple web pages, it is difficult to convey a policy about the scope of the service, even if the library has defined it.

Expectations about the scope of chat reference service drive staffing decisions. The literature is divided about whether student employees can provide adequate service. Lux and Rich found that, although librarians provided a higher quality service, undergraduate students provided successful assistance with appropriate training and monitoring.15 Keyes and Dworak also found undergraduate assistants were effective providers of chat reference.16 In contrast, Maryvon Côté, Svetlana Kochkina, and Tara Mawhinney studied transcripts from a service staffed only with professional librarians and argued for continuing that staffing pattern.17

These studies used referrals to librarians as a measure of service quality. The first two considered a referral a sign of effective service and focused student training on this skill. However, the third looked at referral as a deficit for patrons expecting immediate assistance:

A high rate of referred questions could negatively affect user experience of the service and user perception of service quality, and signal a needed change in the staffing model or further training of the librarians providing the service.18

In an opposing perspective, Megan Oakleaf and Amy VanScoy argued that referrals were an instructional strategy rooted in social constructivist learning theory. They suggested training chat reference providers to make introductions to subject specialists as a way of engaging students: “Thus the goal of a referral to another library staff member is to ensure that the student is brought into the proper community of specialization.”19 Chat reference providers’ assumptions about whether a referral is regrettable or a strategy to further student learning may affect the way they present the referral to patrons.

The University and Library Context

The University of Illinois at Chicago (UIC) is a Carnegie Research 1 institution with 29,000 students in the period studied: academic years 2015–16 and 2016–17 (AY16 and AY17). Based on its full-time undergraduate enrollment, UIC is designated as a Minority Serving Institution (MSI) and as an Asian American and Native American Pacific Islander-Serving Institution (AANAPISI). First-generation college students are one-third of the student body. About 60 percent of students commute to campus from across the far-flung metropolitan area. The Ask a Librarian chat reference service is popular and growing. In the period studied, from fall 2015 through spring 2017, the number of chat interactions grew 39 percent, from 2,333 to 3,246 per semester. Most in-person and chat questions are directional/logistic or ready-reference questions (primarily the holdings of known article and book titles). However, chat received nearly three times as many substantive research questions in AY17 (17.4%) compared with in-person service points (6%).

Chat service providers include professional librarians, paraprofessional staff, and graduate assistants (GAs) at five library sites, four of which are dedicated to health sciences. Professional librarians cover chat from their own locations from 11:00 a.m. to 5:00 p.m., with a second librarian and GA at the general library reference desk also providing coverage. An incoming question gives an audio alert (“pings”), and the first provider to respond “claims” the chat and interacts with the patron in a synchronous exchange of typed responses. A provider who cannot fully answer the question can transfer to another chat provider or inform the patron that an answer can best be provided by a different librarian, in another venue, or at another time—this constitutes a referral.

The GAs who cover most service hours work in the general library and are trained to cover directional and ready-reference questions (primarily locating known article and book titles) and to refer subject research questions to a professional librarian while one is on call between 9:00 a.m. and 7:00 p.m. Outside of these hours, the GA was expected to provide starting points on research questions and make a referral for the inquirer to an appropriate librarian for follow-up. Some GAs expressed reluctance about “bothering” the librarians and the desire to respond to questions on their own.

Procedures, training, and feedback regarding referrals to subject specialists from in-person service points and chat reference are informal and vary by service unit, each with its own set of stakeholders and perspectives about the level of service provided in chat and the appropriate roles for librarians, staff, and GAs. To encourage referrals, in fall 2016 the chat reference service migrated from LibraryH3lp to LibChat (Springshare). In LibraryH3lp, providers who wanted to submit a ticket for follow-up had to copy and paste the transcript into the email reference system. LibChat allows automatic conversion of transcripts to an email reference queue. The migration posed a natural experiment to compare whether the availability of the automatic ticketing feature would prompt more referrals to subject specialists.

Data and Methods

This study analyzed transcripts of IM chat interactions between library patrons seeking subject research assistance and the chat service providers—librarians, GAs, and staff—who responded to their questions during AY16 and AY17 (fall and spring academic terms only). The transcripts recorded typed interactions in which a patron requested assistance identifying library resources relevant to a specific discipline, topic, or research question. The transcripts were captured by two library-specific IM chat platforms: LibraryH3lp (AY16) and Springshare LibChat (AY17). The data sets differ in the format of the transcripts, metadata attached to them, and ability to identify providers.

LibraryH3lp transcripts were downloaded by the host vendor and provided without transcript metadata (in other words, names of patrons and providers, IP addresses). The text of the transcripts did include some identifying data typed by the participants. The author and an assistant, who were also chat reference providers, scrubbed identifying details within the text of the chats: names, emails, phone numbers, locations. No one in the library has access to the original transcripts, because the LibraryH3lp subscription was cancelled in 2017. Therefore, the AY16 data are anonymous, and IRB Protocol #2018-0204 determined that they are not human subjects data.

The author downloaded the LibChat transcripts for AY17 and omitted identifying metadata and details in the text. However, a subset of library employees who have LibChat logins and are trained on the library privacy policy may read the original transcripts that identify providers and, in some cases, information about the patron. The data are retained for five years to allow for follow-up, quality assurance, and research. The data are professional work product and do not represent personal information about the providers. The research questions for this study do not address individual performance but rather comparisons between groups of providers (library faculty, staff, and GAs). Because these data are used in the everyday work and administration of the library, the research confers no more risk to the employees than they experience in the normal course of their work. For patrons, there is a risk that their research interests could be disclosed, but the study does not increase the risk that is already present in the everyday work of the library. Following an expedited review, the IRB approved research protocol #2018-0548 as minimal risk.

The author and an assistant reviewed 11,240 transcripts to identify the 1,956 interactions in which a patron requested assistance identifying library resources for a given discipline, topic, or assignment, referred to as subject research questions for the rest of this paper. The assistant was trained by first walking through interactions with the author and then independently coding a sample for the author to review. Regular meetings covered coding of ambiguous cases. The next step was to categorize the 1,956 subject research questions by domains that reflect the organization of subject librarian assignments to UIC colleges: Business, Engineering/technology, Health sciences, Humanities, Math/science, Social sciences, Other.

Next, the author and an assistant coded the 1,956 subject research questions for whether a referral occurred:

Note that interactions included a subject referral only if the librarian’s subject knowledge or responsibility for the discipline was explicit in the chat. A general referral could have landed with a subject specialist, but that is beyond the evidence in the data. In cases where a librarian offered to follow up with the patron personally, it was considered a general referral unless the librarian stated she or he was the liaison for that discipline.

In a first round of open coding, the author and an assistant examined 10 percent of the 467 transcripts that included a subject referral to identify whether a reference interview was conducted, the extent of the provider’s attempt to assist the patron, how the provider justified the referral, how the referral was handled logistically, and how the patron responded (see appendix). This was an exploratory, interpretive process yet one that was transparent, systematic, and low inference. That is, coding described what happened in the chat as made evident by what the other participant did next, and not by inferring the intention or motivation of either participant. The next step was to code 20 percent of the transcripts to confirm whether the action codes were inclusive of the full range of variation in the transcripts. Then coders reached agreement and completed coding of two semesters’ transcripts, so that all 1,956 subject research transcripts were coded.

Findings

Of the 1,956 subject research chat inquiries received in AY16 and AY17, 59 percent were not referred, 24 percent were referred to a subject specialist, 6 percent were referred to another mode of contact (phone, email, in person, by appointment), and 10 percent were dropped before a substantive response could be provided (with no way of knowing whether this was intentional or a technology glitch). Two-thirds of the subject research questions were asked Monday–Thursday 9:00 a.m.–7:00 p.m. and Friday 9:00 a.m.–5:00 p.m., and one-third were asked when no librarian was scheduled (after hours and on weekends). GAs working alone referred questions to subject specialists no more frequently than when professional librarians were scheduled to support them in a backup capacity. That is, the presence or absence of a professional librarian did not change the rate of subject referrals from GAs.

The distribution of subject referrals was roughly the same for the four semesters studied across the variables of chat provider status, subject domain, and means of referral. Librarians referred 28 percent of the subject research questions they claimed to specialists, GAs referred 21 percent, and staff referred 17 percent. Figure 1 compares the percentage of research inquiries and referrals across subject domains. Social sciences comprised the largest group of questions (52%), but were only 19 percent of the referrals. All subject domains other than the social sciences were referred at a disproportionate rate: health sciences comprised 24 percent of the questions and 37 percent of referrals; business, 7 percent of questions and 32 percent of referrals; engineering/technology, 3 percent of questions and 27 percent of referrals.

FIGURE 1

Chat Reference Research Inquiries (n=1,957) and Referrals (n=467) by Subject Domain (UIC Library Academic Years 2015–16 and 2016–17)

Figure 1. Chat Reference Research Inquiries (n=1,957) and Referrals (n=467) by Subject Domain (UIC Library Academic Years 2015–16 and 2016–17)

Figure 2 shows that the most common means of referring a patron in both AY16 and AY17 was to offer contact information for the subject specialist. In 50 percent of chats in AY16 and 46 percent in AY17, providers simply gave the name, email, and sometimes phone number for the specialist. An additional 29 percent of subject research inquiries in AY16 included a link to a research guide to provide contact information—that number shrank to 7 percent in AY17, when 27 percent of referrals were submitted as tickets. The most likely reason for the increase in tickets was migration to a chat platform with automatic ticketing (that is, direct conversion of chat transcripts to the email reference queue). Librarians were far more likely to submit a ticket than student employees or staff, increasing from 18 tickets in AY16 to 49 tickets in AY17; GAs increased from 1 to 9 tickets; and staff increased from 0 to 4 tickets.

FIGURE 2

Chat Reference Provider Means of Subject Referrals by Academic Year (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 2. Chat Reference Provider Means of Subject Referrals by Academic Year (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Textual analysis revealed a range of variables in how providers presented referrals. Examples that follow preserve spelling and grammar errors; identifying details are omitted. Variants included:

Figure 3 shows a range of variation in how chat providers present referrals to patrons as an option versus a direct recommendation and as a step for the patron to take versus something the librarian will take the initiative to follow up on.

FIGURE 3

Subject Referral Strategies in Chat Reference: Option vs. Recommendation and Responsibility to Follow Up

Option

Recommendation

Patron’s responsibility to follow up
  • Conditional: If you get stuck/have difficulty/run into further trouble/want targeted help
  • It might be a good idea/You may/might/will want to contact the subject librarian
  • I think a librarian could help you better.
  • I would suggest/encourage to you reach out
  • Please contact the subject librarian.
  • You need to talk to a subject librarian.
  • Your best bet/It would be best/The best option/The best course of action/You’d be well served

Librarian’s responsibility to follow up
  • Do you want me to submit a ticket?
  • I can have the subject librarian contact you.
  • Let me forward your inquiry to the subject librarian.
  • I will ask a librarian who specializes in [subject] to contact you.
  • I will forward your inquiry.
  • I will have to submit a ticket.
  • I will transfer you.

Figure 4 shows a range in variation in how chat providers “talk down” their own capabilities or apologize versus “talking up” the benefits of working with a subject specialist in terms of expertise, patron needs, or time constraints.

FIGURE 4

Framing Referrals in Chat Reference: Deficit vs. Benefit in Terms of Subject Expertise, Patron Information Need, or Time

Deficit (apologizing)

Benefit (promoting)

Subject expertise
  • “I don’t want to advise you incorrectly.”
  • “Rather than relying on my guesses, I’d recommend getting in touch”
  • “Since I am not specialized in this particular field, I am not able to help you further on this topic. I recommend you to contact one of our nursing librarians.”
  • “Did you need this information by tonight? Or do you have some time? The reason I ask is because I would love to give you the business librarian’s information since she might have the most useful information if you’d like to email her.”
  • “He should be much better with that, as it’s his specialty.”
  • “She specializes in those information resources and should have more ideas than I do as a generalist.”

Patron need and situation
  • “It sounds like this will take some digging.”
  • “It’s trickier looking for information about services than about products.”
  • “Since this is for a very in-depth project”
  • “This appears to be a little tricky and very specific”
  • “This is a very specialized question.”
  • “I want to make sure you get the best possible assistance”
  • “Just want to make sure you get some help”
  • “You could meet her whichever way is most conv[en]ient for you like chat, virtual conference, or coming in to meet her”
  • [if you switch to email] “you wouldn’t have to worry about your internet connection”

Time available
  • “I don’t want to keep you hostage to my learning curve.”
  • “in the interest of time, I will transfer this”
  • “They would be able to spend more time”
  • “They’d have some additional time to do some investigating”
  • “You would be able to ask questions and have one-on-one time to go over resources or review existing sources you might be using.”
  • “She’s really quick with email and comes to work early, so if you shoot her an email now, you should get a reply tomorrow morning.”

How did providers talk about the benefits of working with a subject specialist? They said that the subject specialist could offer “additional/more specific suggestions,” “more specialized/in-depth help,” “extra tips,” “an even more detailed answer,” “more effective strategy,” “additional research help,” “a more informed opinion,” or “even more in-depth advice.” The subject specialist would “point you towards more resources,” “help you better with finding keywords,” or “share some more insight.” The subject specialist had “more expertise,” “more grounding in your topic,” or “skills that are better suited to this type of request.” The subject specialist was “a lot better equipped to help you out,” “able to find better answers,” or “a lot more qualified/knowledgeable.” A common strategy for conveying expertise was to point out the contact details for the librarian who created or curated the research guide—the information in the guide served as verification of the librarian’s knowledge and/or formal responsibility for the discipline in the library.

At times, providers confidently recommended a referral; at other times, they were tentative. It is possible that providers were more confident in referring to librarians with whom they had stronger relationships.20 Examples of confident recommendations follow:

Some referrals struck a more tentative note:

How did chat providers talk down their skills and knowledge when referring a patron? They said they were “not well-equipped enough,” “not a medical librarian,” “not an expert,” “not the person to ask,” or “pretty new to this resource.” They said “My background is not [subject],” “My knowledge of this is very limited,” “My skills are limited,” “This is not a field that I am at all familiar with,” or “This is not my field of expertise.” Some providers announced a caveat about their ability to assist before getting into the reference interview or attempting to find information.

The transcripts offer hints about the relative importance to providers and patrons of subject knowledge. In one interaction, the patron resisted the provider’s claim that subject expertise was necessary for the information need, citing the more generalized search skill of narrowing a search, and the provider pushed back:

GA: As much as I’d like to help you this is far outside of my field of expertise
GA: I’d suggest getting in touch with the library of health sciences
Patron: my question is more on narrowing searches on CINAHL the terms dont matter
GA: as they are far better equipped to handle such inquiries
GA: Right, I’ve never used that database
Patron: ok
GA: I’m there right now
GA: trying to figure it out, which will most likely take me a while
GA: I don’t want to keep you hostage to my learning curve
Patron: its ok thanks tho
GA: of course

This interaction gives the impression that the GA is working to cut the chat short.

Rarely, a provider offered to work with a subject specialist on behalf of a patron, for example: “I might need to consult with a health sciences librarian,” “I would need to get some support from our business librarians to give you a solid answer,” and “I’m on the phone now with a health sciences expert.” It was also rare for a provider to suggest that chat as a mode of reference was the reason for a referral:

Patrons did not always respond to a referral, but positive responses correlated with the presence and extent of a reference interview and tickets as the means of referral. Examples of positive responses included “okay, thanks! I will connect with her”; “great thanks you so much i will be waiting to hear from her”; “I will contact her for help, thank you, otherwise I wouldn’t have known to contact her.” Note that there is currently no way to track whether the patron followed through and contacted the librarian, because contact is often made outside the virtual reference system. Any study to measure follow-ups would need to be designed carefully to protect patron privacy. Negative responses usually took the form of the patron declining the referral. Some stated the timeframe for the assignment or project made a referral impossible; for example, one patron was acquainted with the subject specialist but was completing a project at night when that librarian was not available.

Figure 5 shows that, in the small number of interactions in which the provider conducted an extensive reference interview to understand the information need in depth, 62 percent of patrons responded positively to a referral, 17 percent were neutral, 7 percent responded negatively, and 14 percent did not respond. Of the much larger number of interactions with a minimal reference interview, 51 percent responded positively. In interactions lacking a reference interview, 40 percent of patrons responded positively to a referral.

FIGURE 5

Patron Response to Referrals in Chat Reference by Presence and Extent of Reference Interview (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 5. Patron Response to Referrals in Chat Reference by Presence and Extent of Reference Interview (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Creating a ticket to have a librarian follow up with the patron also correlated with more positive responses to referrals (see figure 6). In interactions where providers put in a ticket (whether manual or automatic), 65 percent of patrons responded positively and 9 percent responded negatively. Fewer than half the patrons responded positively to other means of referral.

FIGURE 6

Patron Response to Referrals in Chat Reference by Means of Referral (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 6. Patron Response to Referrals in Chat Reference by Means of Referral (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 7 shows that the chat provider’s attempt to provide research assistance to the patron was not correlated with positive responses to the referral. A bit fewer than half the patrons responded positively to the referral regardless of the level of assistance the provider offered. Note that the study did not assess responses to the assistance provided, but only to the referral itself.

FIGURE 7

Patron Response to Referrals in Chat Reference by Provider’s Attempt to Assist (N=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 7. Patron Response to Referrals in Chat Reference by Provider’s Attempt to Assist (N=467) (UIC Library Academic Years 2015–16 And 2016–17)

In most chat interactions with all three types of providers, referrals were minimally justified, most often simply giving the subject specialization of the librarian (such as “our psychology librarian”). Figure 8 shows that librarians were the most likely to explain the benefits of working with a subject specialist, but this occurred in only a third of interactions.

FIGURE 8

Chat Reference Provider Justification of Referral by Status (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 8. Chat Reference Provider Justification of Referral by Status (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 9 shows that more patrons (55%) responded positively to referrals framed in terms of benefits. Patrons responded positively slightly more often to referrals framed as apologies (45%) than to minimal referrals (43%).

FIGURE 9

Chat Reference Patron Response to Referral by Justification Type (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Figure 9. Chat Reference Patron Response to Referral by Justification Type (n=467) (UIC Library Academic Years 2015–16 And 2016–17)

Discussion

In two years of chat reference service at a large R1 university library, providers referred 24 percent of subject research inquiries to a subject specialist. Compared with student employees and staff, professional librarians were more likely to refer a patron to a subject specialist, more likely to forward the transcript of the interaction (“submit a ticket”) for the specialist to answer the patron directly, and more likely to frame the referral as a benefit, rather than apologetically. Student employees were no more likely to refer a patron to a subject specialist when providing reference assistance after hours or on weekends, when no librarian was available to support them. One possible explanation for these findings is that librarians may be more aware of the scope and depth of colleagues’ backgrounds and so are more confident that the patron will benefit from working with a specialist. Another possible explanation is that librarians may feel more secure about their own skill levels and so are less concerned that a referral will reflect poorly on their own performance.

While exploratory, findings further suggest that patrons respond most positively to referrals that are framed as a benefit, put the initiative for follow-up in the hands of the subject specialist, and include a reference interview—regardless of the level of assistance the chat provider offered. This is evidence that a referral conducted in an optimal way can be considered a marker of quality service, rather than a regrettable necessity associated with nonprofessional staffing of chat reference service, especially because professionals in these data were most likely to provide such optimal referrals.

Even while recognizing the value of referrals, the disproportionate percentage of referrals in some disciplines requires thoughtful response. Efforts to increase referrals risk overloading subject specialists. In this study, more than twice as many subject research questions were not referred as those that were. Does it make sense to refer the transcript of a successful chat, one in which the patron was clearly satisfied? If the library’s goal is to raise awareness of subject librarians and increase student engagement with the library, the answer is yes. The subject specialist gains information about research interests to guide collection decisions, details of assignments to guide instructional efforts, and an opening for feedback about library resources and services. In the current case, if every one of the 1,160 subject questions not referred over two years had been ticketed, on average each of 32 subject librarians would have seen nine additional tickets per semester (varying by discipline). Responding to a successful interaction can be as simple as, “Sounds like an interesting project; let me know if there is anything more you need.”

In these data, reference interviews were conducted infrequently, but patrons were most likely to respond positively to a referral when the provider conducted a thorough reference interview. Reference interviews have always been best practice,21 but some chat providers and researchers are concerned that patrons in the chat environment will not have the patience to answer questions.22 This analysis suggests that patrons are willing to share information and will respond positively to a referral when the provider takes time to understand their information need.

Most chat providers in these data framed the referral minimally, simply stating the subject specialist’s area of responsibility. Librarians were more likely to frame the referral in terms of benefits to the patron than were GAs or staff and less likely than GAs to frame the referral as deficit in service by apologizing. This is important because patrons responded more positively to referrals when they were presented as a benefit. Responses from patrons to referrals framed as a deficit and minimal referrals were roughly the same. However, as seen in the conversation excerpt about searching CINAHL above, apologies might actually function as an attempt to curtail the chat, or they might be interpreted that way by the patron.

Limitations and Future Research

These data do not provide evidence about whether patrons (1) followed through with referrals by contacting the subject specialist, (2) perceived the referral to a subject specialist as a benefit or as a denial of immediate service, (3) ultimately received more complete and informed assistance because of the referral, or (4) learned better research skills from working with a subject specialist. The data also do not show how librarians responded to referrals. UIC Library does not track the origin of reference referrals, and doing so would require cooperation from subject specialists and close attention to the privacy rights of patrons.

The study compares across the two years when two different chat systems were used, and it compares across three groups of library employees, but it does not compare individuals or library sites, because the data are de-identified. Individual providers, training systems, and technology all changed in those years, and this study cannot isolate those changes from one another.

Questions were categorized differently for AY16 and AY17. For AY16, investigators manually coded the de-identified transcripts. For AY17, chat service providers categorized their own interactions. Some relevant transcripts could be missing if the provider neglected to add them to statistics or classified them incorrectly. Also, the count of transcripts may be inflated by interactions that were interrupted and picked up again by the same or different provider.

Working with data from only one institution limits the knowledge claims; many factors influence referral patterns. Further studies across institutions would shed further light, but researchers would need both flexible responses from multiple IRBs and data-sharing arrangements to engage in cross-institutional studies.

Future research would fruitfully address the following:

Conclusions and Best Practices

Chat reference has the potential for building awareness of specialized library resources and services. Chat is a low-barrier, semianonymous channel with the appeal of informality, and the service is available at times when many students are working on research outside of business hours. Students who are reluctant to walk up to a desk, because they feel they do not belong in the library or do not know what services are offered, might be especially attracted to chat reference. Students in this group of reluctant users likely include those with less exposure to libraries, such as underrepresented and first-generation students. The importance of effective referrals is heightened if they have the potential to support success in these groups.

A ticket is beneficial for the patron because it puts the initiative for follow-up in the hands of the librarian, and it is beneficial for the librarian because it preserves the complete conversation that shows the reference interview and what search strategies have already been tried. Anecdotal observation suggests that students are unlikely to reply to a follow-up from the subject specialist. Even so, the subject specialist stands to gain information about patron needs in their subject area to inform collections decisions, instructional content, and communication with faculty about assignments. Tickets also present an opportunity to correct, reinforce, or expand chat reference providers’ knowledge and skills if handled diplomatically and supportively. And even if the patron’s immediate need is resolved, an email from the subject specialist demonstrates the interest and service focus of the librarian to encourage direct contact on future research projects.

Interpretation of these findings is grounded in a professional commitment to the educational role of the academic library.23 That is, chat is assumed to be not only a convenient venue for quick questions but also a site for teaching and learning about subject research. Even in cases when a subject research question cannot be fully resolved in the chat, learning about additional resources and building networks of research assistance are worthy outcomes. In these data, there is evidence that chat providers who are socialized into the profession embraced this perspective more than GAs and staff. Librarians refer patrons to specialists more often than do GAs, and they are more likely to frame the referral as a benefit.

How patrons responded to a referral (when they did at all), serves as exploratory evidence for developing best practices for presenting referrals to patrons in libraries that seek to increase engagement between patrons and subject specialists:

Acknowledgments

The author is grateful for support from the UIC Library Research Fund, research assistants Alyne Connie and Supipi Weerasoriya, and internal and external reviewers who generously shared their feedback.

APPENDIX

Action Codes

The first round of coding for chats that included a referral to a subject specialist identified the following actions:

Notes

1. Reference & User Services Association, “Guidelines for Implementing and Maintaining Virtual Reference Services” (PDF, American Library Association, 2017), available online at www.ala.org/rusa/resources/guidelines/virtrefguidelines [accessed 15 May 2019].

2. Association of College & Research Libraries, “Standards for Libraries in Higher Education” (American Library Association, Oct. 2011), available online at www.ala.org/acrl/standards/standardslibraries [accessed 15 May 2019]; RUSA Professional Competencies for Reference and User Services Librarians Task Force and Professional Competencies for Reference and User Services Librarians Task Force, “Professional Competencies for Reference and User Services Librarians” (American Library Association, 2017), available online at www.ala.org/rusa/sites/ala.org.rusa/files/content/RUSA%20Professional%20Competencies%20Final%208-31-2017.pdf [accessed 15 May 2019]; Reference & User Services Association, “Guidelines for Behavioral Performance of Reference and Information Service Providers” (American Library Association, May 28, 2013), available online at www.ala.org/rusa/resources/guidelines/guidelinesbehavioral [accessed 15 May 2019].

3. Paul Neuhaus and Matthew R. Marsteller, “Chat Reference at Carnegie Mellon University,” Public Services Quarterly 1, no. 2 (2002): 38.

4. Nahyun Kwon and Vicki L. Gregory, “The Effects of Librarians’ Behavioral Performance on User Satisfaction in Chat Reference Services,” Reference & User Services Quarterly 47, no. 2 (Winter 2007): 137–48.

5. Gang (Gary) Wan et al., “Key Issues Surrounding Virtual Chat Reference Model: A Case Study,” Reference Services Review 37, no. 1 (2009): 73–82.

6. Kelsey Keyes and Ellie Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library: A Standards-Based Assessment,” Journal of Academic Librarianship 43, no. 6 (Nov. 2017): 475, https://doi.org/10.1016/j.acalib.2017.09.001.

7. Sandra L. De Groote et al., “Quantifying Cooperation: Collaborative Digital Reference Service in the Large Academic Library,” College & Research Libraries 66, no. 5 (Sept. 2005): 450.

8. Reference & User Services Association, “Guidelines for Implementing and Maintaining Virtual Reference Services.”

9. María Pinto and Ramón A. Manso, “Virtual References Services: Defining the Criteria and Indicators to Evaluate Them,” The Electronic Library 30, no. 1 (Feb. 10, 2012): 51–69, https://doi.org/10.1108/02640471211204060.

10. Anthony S. Chow and Rebecca A. Croxton, “A Usability Evaluation of Academic Virtual Reference Services,” College & Research Libraries 75, no. 3 (May 1, 2014): 312, https://doi.org/10.5860/crl13-408.

11. Neuhaus and Marsteller, “Chat Reference at Carnegie Mellon University,” 34.

12. David Ward, “Measuring the Completeness of Reference Transactions in Online Chats,” Reference & User Services Quarterly 44, no. 1 (Fall 2004): 46–56.

13. Vera J. Lux and Linda Rich, “Can Student Assistants Effectively Provide Chat Reference Services? Student Transcripts vs. Librarian Transcripts,” Internet Reference Services Quarterly 21, no. 3/4 (2016): 115–39.

14. Paula R. Dempsey, “Resource Delivery and Teaching in Live Chat Reference: Comparing Two Libraries,” College & Research Libraries 78, no. 7 (2017): 903–04, https://doi.org/10.5860/crl.78.7.898.

15. Lux and Rich, “Can Student Assistants Effectively Provide Chat Reference Services?”

16. Keyes and Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library.”

17. Maryvon Côté, Svetlana Kochkina, and Tara Mawhinney, “Do You Want to Chat? Reevaluating Organization of Virtual Reference Service at an Academic Library,” Reference & User Services Quarterly 56, no. 1 (Fall 2016): 36–46.

18. Côté, Kochkina, and Mawhinney, “Do You Want to Chat?” 40.

19. Megan Oakleaf and Amy VanScoy, “Instructional Strategies for Digital Reference,” Reference & User Services Quarterly 49, no. 4 (June 2010): 385, https://doi.org/10.5860/rusq.49n4.380.

20. Lux and Rich, “Can Student Assistants Effectively Provide Chat Reference Services?” 134.

21. Brenda Dervin and Patricia Dewdney, “Neutral Questioning: A New Approach to the Reference Interview,” RQ 25, no. 4 (July 1, 1986): 506–13; Marie L. Radford et al., “‘Are We Getting Warmer?’ Query Clarification in Live Chat Virtual Reference,” Reference & User Services Quarterly 50, no. 3 (Spring 2011): 259–79.

22. Neuhaus and Marsteller, “Chat Reference at Carnegie Mellon University.”

23. Association of College & Research Libraries, “Standards for Libraries in Higher Education.”

* Paula R. Dempsey is Assistant Professor & Head Librarian, Research Services and Resources in the Richard J. Daley Library at the University of Illinois at Chicago; email: dempseyp@uic.edu. ©2019 Paula R. Dempsey, Attribution-NonCommercial (http://creativecommons.org/licenses/by-nc/4.0/) CC BY-NC.