A Tale of Two Tools: Comparing LibKey Discovery to Quicklinks in Primo VE COMMUNICATION A Tale of Two Tools Comparing LibKey Discovery to Quicklinks in Primo VE Jill K. Locascio and Dejah Rubel INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2023 https://doi.org/10.6017/ital.v42i2.16253 Jill K. Locascio (jlocascio@sunyopt.edu) is Associate Librarian, SUNY College of Optometry. Dejah Rubel (dejahrubel@ferris.edu) is Metadata and Electronic Resources Management Librarian, Ferris State University. © 2023. INTRODUCTION Consistent delivery of full-text content has been a challenge for libraries since the development of online databases. Library systems have attempted to meet this challenge, but link resolvers and early direct linking tools often fell short of patron expectations. In the last several years, a new generation of direct linking tools has appeared, two of which will be discussed in this article: Third Iron’s LibKey Discovery and Quicklinks by Ex Libris, a Clarivate company. Figure 1 shows the “Download PDF” link added by LibKey. Figure 2 shows the “Get PDF” link provided by Quicklinks. The way we configured our discovery interface, a resource cannot receive both the LibKey and Quicklinks PDF links. These two direct linking tools were chosen because they were both relatively new to the market in April 2021 when this analysis took place and they can both be integrated into Primo VE, the library discovery system of choice at the authors’ home institutions of SUNY College of Optometry and Ferris State University. Through analysis of the frequency of direct links, link success rate, and number of clicks, this study may help determine which product is most likely to meet your patrons’ needs. Figure 1. Example of a LibKey Discovery Link in Primo VE. Figure 2. Example of a Quicklink in Primo VE. mailto:jlocascio@sunyopt.edu mailto:dejahrubel@ferris.edu INFORMATION TECHNOLOGY AND LIBRARIES JUNE 2023 A TALE OF TWO TOOLS 2 LOCASCIO AND RUBEL LITERATURE REVIEW Over the past 20 years link resolvers and direct linking have evolved in tandem. Early link generator tools, such as ProQuest’s SiteBuilder, often involved a process that “… proved too cumbersome for most end-users.”1 Five years later, tools from EBSCO, Gale, Ovid, and ProQuest had improved, but they were all proprietary. Bickford postulates that metadata-based standards, like OpenURL, may make linking as simple as copying and pasting from the address bar; however, they may be more likely to fail “… as long as vendors use incompatible, inaccurate, or incomplete metadata.”2 The first research was Wakimoto’s 2006 study of SFX, which relied on 224 test queries and 188,944 individual uses for its data set. 3 Of those queries, 39.7% of search results included a full-text link and that link was accessed 65.2% of the time. Unfortunately, Wakimoto also discovered that 22.2% of all full-text results failed and concluded that most complaints against SFX were problems with the systems it links to and not the link resolver itself. Alth ough intended to be provider-neutral, the OpenURL standard is, in fact, vulnerable to metadata omissions. Content providers, whether aggregators or publishers, have a vested interest in link stability and platform use and have therefore invested in building direct link generation tools. In 2006, Grogg examined EBSCO’s SmartLink, which checks access rights before generating the link; ProQuest’s CrossLinks, which was used to link from ProQuest to another vendor’s content; SilverPlatter and Links@Ovid, which relied on a knowledge base in the terabytes for static links.4 In 2008, Cecchino described the National Library of Medicine’s LinkOut tool for selected publishers within PubMed.5 They also described two Ovid products: Links@Ovid and LinkSolver, noting that the former is similar to LinkOut and the latter is similar to SFX. Most of the time these tools worked well, but their use was restricted to a particular platform or set of publishers. As online public catalogs became discovery layers, direct linking became a feature of the library management system. Two studies have been done thus far: Silton’s analysis of Summon and Stuart’s analysis of 360 Link. In 2014, Silton tested the percentage of full-text articles retrievable from Summon by running a test query and examining the first 100 results. Over a year, the total success rate for unfiltered queries rose from 61% to 76%. After direct linking was introduced, the success rate of link resolver links rose to 65.8–73% and direct links succeeded 90.48–100% of the time. Silton concluded, “While direct linking had some issues in its early months, it generally performs better than the link resolver.”6 In 2011, Stuart, Varnum, and Ahronheim began testing the 1-Click feature of 360 Link on 579 citations, 82.2% of which were successful. After Direct Linking became an option for Summon in 2012, 61–70% of their sample relied on it. “Between Direct Linking and 1-Click about 93 to 94% of the time an attempt was made to lead users directly to the full text of the article … [and] … we were able to reach full text … from 79% to about 84% of the time.”7 Direct Linking outperformed 1-Click with a 90% success rate compared to 58–67% for 1-Click. Stuart also compared the actual error rate with one based on user reports and discovered that “Relying solely on user reports of errors to judge the reliability of full-text links dramatically underreports true problems by a factor of 100.”8 OpenURL links were especially alarming with approximately 20% of them failing. Although direct linking is more reliable, Stuart closes by noting that direct linking binds libraries closer to vendors thereby decreasing institutional their flexibility. INFORMATION TECHNOLOGY AND LIBRARIES JUNE 2023 A TALE OF TWO TOOLS 3 LOCASCIO AND RUBEL METHODS The goal of this project was to assess two of the latest direct linking tools: Ex Libris’s native Quicklinks feature and Third Iron’s LibKey Discovery. We performed a side-by-side comparison of the two tools by searching for specific articles in Primo VE, the library discovery system used by the authors’ respective home institutions, SUNY College of Optometry and Ferris State University, and measuring • how often each vendor’s direct links appeared on the brief record; • success rate of the links; and • number of clicks it takes from each link to reach the PDF full text. Both SUNY College of Optometry and Ferris State University use Ex Libris’ Alma as their library services platform. Alma provides a number of usage reports in their Analytics module. We sourced the queries used in our analysis from the Alma Analytics Link Resolver Usage report. The report contains a field Number of Requests, which records the number of times an OpenURL request was sent to the link resolver. An OpenURL request is sent to the link resolver when the user clicks on a link to the link resolver from an outside source (such as Google Scholar), for example, when the user submits a request using Primo’s Citation Linker or when the user accesses the article’s full record in Primo by clicking on either the brief record’s title or availability statement. This means that results that have a direct link (whether a Quicklink or LibKey Discovery link) on the brief record will not appear in the report if the user clicked the direct link to the article. Thus, in order to create test searches that would be an accurate representation of articles being accessed, we used article titles taken from SUNY Optometry’s October 2019 Alma Link Resolver Usage report— a report that was generated prior to the implementation of both LibKey Discovery and Quicklinks. The report was filtered to include only articles with the source type of Primo/Primo Central to ensure that the initial search was taking place within the native Primo interface, as requests from outside sources like Google Scholar or from Primo’s Citation linker are irrelevant to this analysis. This filtering generated a total of 412 articles. After further removal of duplicates and non -article material, there were 386 article titles in our test query set. We created two separate Primo views as test environments: one with LibKey Discovery and the other with Quicklinks. We ran the test searches twice in each view. In the first round of testing, we recorded whether a direct link was present. We also recorded the name of the full-text provider (if present), as well as whether the article was Open Access. SUNY Optometry does not filter their Primo results by availability; therefore, many of the articles included in the initial search did not have any associated full-text activations. Since these articles are irrelevant to our assessment, we removed them before analyzing the first round of data and proceeding with the second search. The exception to these removals were articles identified as Open Access by Unpaywall, as the presence of Unpaywall links is independent of any activations in Alma. Furthermore, Third Iron’s LibKey Discovery and Ex Libris’ Quicklinks both incorporate Unpaywall’s API into their products to provide direct links to PDFs of Open Access articles. This functionality helps fill coverage gaps where institutions may not have activated a hybrid Open Access journal due to its paywalls. Therefore, we are including the presence of direct links resulting from the Unpaywall API when determining whether a LibKey Discovery link or Quicklink is present. After filtering for availability, we had 254 article titles for the first round of searching and analysis. The initial analysis revealed the need to further filter the INFORMATION TECHNOLOGY AND LIBRARIES JUNE 2023 A TALE OF TWO TOOLS 4 LOCASCIO AND RUBEL articles used for the second round of searching, which would provide a much closer comparison of the two direct linking tools as Third Iron had partnered with more content providers than Ex Libris. Controlling for shared providers would give a more accurate representation of how each direct linking tool performs in relation to the other. When controlling for shared providers and Open Access articles, we were left with 145 article titles for the second query set. During the second round of searching, we measured whether the direct link was successful in linking to the full text—meaning that the link was neither broken nor linked to an incorrect article—and how many clicks were necessary to get from the direct link to the article PDF. Along the way, additional qualitative measures were observed, such as document download time and metadata record quality. While not as easy to measure as the quantitative data, these observations provided additional insight into the strengths and weaknesses of each of these direct linking tools. Since April 2022, when our research was conducted, Ex Libris has added several Quicklinks providers, possibly increasing the current number of Quicklinks available. Additionally, both rounds of searching were conducted on campus, so our analysis excludes any consideration of authentication and/or proxy information. RESULTS Of the 254 articles searched, 208 (82%) had LibKey Discovery links present while 129 (52%) had Quicklinks present. While this seems like a large discrepancy between the two direct link providers, it can be explained by the fact that during the time of testing, Ex Libris was collaborating with fewer content providers than Third Iron. Ex Libris has since added more providers. While the provider discrepancy meant that there were many instances where a LibKey Discovery link was present where a Quicklink was not, there were 5 articles where a Quicklink was present while a LibKey Discovery link was not. As mentioned previously, the criterion for the 254 articles included in the second round of searching was that the articles must be activated in Alma or must be Open Access. Of these 254 articles, we identified 137 (54%) as Open Access. Of those Open Access articles, 132 (96%) had LibKey Discovery links present, and 118 (86%) had Quicklinks present. We found that 113 (82%) of the Open Access articles had both LibKey Discovery links and Quicklinks present. We also discovered within this set of 137 Open Access articles that 30 (22%) were from non-activated resources. Of those 30 Open Access articles from non-activated titles, all 30 (100%) had LibKey Discovery links appearing on the brief results and 24 (80%) had Quicklinks. To get a better idea of how LibKey Discovery links and Quicklinks compared in terms of linking success, we filtered to only those articles available from providers who were participating in both LibKey Discovery links as well as Quicklinks. Since both direct linking tools use Unpaywall integrations, we continued to include Open Access articles. This filtering resulted in 145 articles where LibKey Discovery links were present in 137 articles (94%) while Quicklinks were present in 129 articles (89%). We found that 123 (85%) of these 145 articles had both LibKey Discovery links and Quicklinks present. There were 2 (1%) articles that had neither LibKey Discovery links nor Quicklinks present despite being activated in a journal currently participating as a provider in both direct linking tools. There were also 14 articles (10%) that had LibKey Discovery links but INFORMATION TECHNOLOGY AND LIBRARIES JUNE 2023 A TALE OF TWO TOOLS 5 LOCASCIO AND RUBEL not Quicklinks; all of these articles were Open Access. In total, of the 145 articles searched, 128 (88%) were identified as Open Access. As for the 137 LibKey Discovery links, 130 (95%) of them successfully linked to the article. On average it took 1.07 clicks to get to the PDF of the article. Of the 129 Quicklinks, 126 (98%) of them successfully linked to the article. On average it took 1.07 clicks to get to the PDF of the article. We also attempted to measure the time it took for the pages to load after the initial click on the LibKey Discovery links and Quicklinks; however, the tools used to measure this, as well as the environments in which the links were being clicked, proved too varied to provide an appropriate comparison. Nevertheless, we noted observations such as the page load times after clicking on LibKey Discovery links and Quicklinks were generally consistent, but Quicklinks attempts to connect to the Wiley platform took a significant time (at least 10 seconds) to load. CONCLUSIONS With high article linking success rates, both Third Iron’s LibKey Discovery and Ex Libris’ Quicklinks deliver on the promise to provide fast and seamless access to full-text articles. However, the LibKey Discovery tool far outpaces Quicklinks when it comes to coverage. Both direct linking tools perform well with Open Access articles, supplying libraries with better options for full-text links to articles that may be in hybrid journals. As with any kind of full-text linking, both direct linking tools rely on metadata. In conclusion, while LibKey Discovery provides a more complete direct linking solution, both LibKey Discovery and Quicklinks are reliable tools that improve Primo’s discovery and delivery experience. ENDNOTES 1 David Bickford, “Using Direct Linking Capabilities in Aggregated Databases for E-Reserves,” Journal of Library Administration 41, no. 1/2 (2004): 31–45, https://doi.org/10.1300/J111v41n01_04. 2 Bickford, 45. 3 Wendy Furlan, “Library Users Expect Link Resolvers to Provide Full Text While Librarians Expect Accurate Results,” Evidence Based Library and Information Practice 1, no. 4 (2006): 60–63, https://doi.org/10.18438/B88C7P. 4 Jill E. Grogg, “Linking without a Stand-Alone Link Resolver,” Library Technology Reports 42, no. 1 (2006): 31–34. 5 Nicola J. Cecchino, “Full-Text Linking Demystified,” Journal of Electronic Resources in Medical Libraries 5, no. 1 (2008): 33–42, https://doi.org/10.1080/15424060802093377. 6 Kate Silton, “Assessment of Full-Text Linking in Summon: One Institution’s Approach,” Journal of Electronic Resources Librarianship 26, no. 3 (2014): 163–69, https://doi.org/10.1080/1941126X.2014.936767. https://doi.org/10.1300/J111v41n01_04 https://doi.org/10.18438/B88C7P https://doi.org/10.1080/15424060802093377 https://doi.org/10.1080/1941126X.2014.936767 INFORMATION TECHNOLOGY AND LIBRARIES JUNE 2023 A TALE OF TWO TOOLS 6 LOCASCIO AND RUBEL 7 Kenyon Stuart, Ken Varnum, and Judith Ahronheim, “Measuring Journal Linking Success from a Discovery Service,” Information Technology and Libraries 34, no. 1 (2015): 52–76, https://doi.org/10.6017/ital.v34i1.5607. 8 Stuart, Varnum, and Ahronheim, 74. https://doi.org/10.6017/ital.v34i1.5607 Introduction Literature Review Methods Results Conclusions