Research Article
Connecting Users to Articles: An Analysis of the
Impact of Article Level Linking on Journal Use Statistics
Michelle Swab
Public Services Librarian
Health Sciences Library
Memorial University
St. John’s, Newfoundland,
Canada
Email: mswab@mun.ca
Received: 29 July 2019 Accepted: 15 Oct. 2019
2019 Swab. This is an Open Access article
distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip29613
Abstract
Objective – Electronic
resource management challenges and “big deal” cancellations at one Canadian
university library contributed to a situation where a number of electronic
journal subscriptions at the university’s health sciences library lacked
article level linking. The aim of this study was to compare the usage of
journals with article level linking enabled to journals where only journal
level linking was available or enabled.
Methods – A list of electronic journal title
subscriptions was generated from vendor and subscription agent invoices.
Journal titles were eligible for inclusion if the subscription was available
throughout 2018 on the publisher’s platform, if the subscription costs were
fully funded by the health sciences library, and if management of the
subscription required title-by-title intervention by library staff. Of the 356
journal titles considered, 302 were included in the study. Negative binomial
regression was performed to determine the effect of journal vs. article level
linking on total COUNTER Journal Report 1 (JR1) successful full-text article
requests for 2018, controlling for journal publisher, subject area, journal
ranking, and alternate aggregator access.
Results –
The negative binomial regression model demonstrated that article level linking
had a significant, positive effect on total 2018 JR1 (coef:
0.645; p < 0.001). Article level
linking increased the expected total JR1 by 90.7% when compared to journals
where article level linking was not available or enabled. Differences in
predicted usage between journals with article level linking and those without
article level linking remained significant at various journal ranking levels.
This suggests that usage of both smaller, more specialized journals (e.g., Journal of Vascular Research) and
larger, general journals (e.g., New
England Journal of Medicine) increases when article level linking is
enabled.
Conclusions –
This study provides statistical evidence that enabling article level linking
has a positive impact on journal usage at one academic health sciences library.
Although further study is needed, academic libraries should consider enabling
article level linking wherever possible in order to facilitate user access,
maximize the value of journal subscriptions, and improve convenience for users.
Introduction
Library link resolver systems are designed to facilitate seamless
connections between full-text journal content and article databases and library
discovery layers. When working optimally, link resolvers connect users directly
to a full-text HTML or PDF version of a particular journal article. Linkage
failures remain common in libraries, however, despite attempts to make
improvements (Stuart, Varnum, & Ahronheim, 2015).
One form of suboptimal linking occurs when link resolvers connect at the
journal level rather than the article level. In journal level linking, a link
to a particular article resolves to the table of contents or homepage of a
journal, rather than the article itself. The user must then browse to the
volume and issue of interest, or search the journal platform for the article.
Article level linking functionality depends on the availability of accurate
linking parser information in an institution’s link resolver software, as well
as support for link resolvers from journal vendors.
At Memorial University’s
Health Sciences Library, many journal titles lacked article level linking
throughout 2018. A number of factors contributed to this situation, including
the cancellation of several “big deal” publisher packages during the preceding
years. Big deal journal packages are becoming financially unsustainable for
many institutions, but provide greater efficiencies with regard to electronic
resource management processes (Cleary, 2009). In some cases, big deal publisher
package linking can be activated with a few mouse clicks. In contrast, creating
and maintaining link resolver information for individual journal titles can be labour intensive. Library personnel must select individual
titles to activate, edit journal holdings information to match institutional
entitlements, and ensure that linking information is accurate.
Big deal cancellations increased the number of individual journal
subscriptions at Memorial University Libraries, thus creating additional
burdens on acquisitions personnel (Ambi, Morgan, Alcock, & Tiller-Hacket,
2006). Moreover, Memorial University Libraries had transitioned to a new
library services platform during this time period. The transition meant that
many electronic collections required cleaning and updates, and it took some
time to determine what electronic resource management workflows would work best
for the institution.
While journal level linking is not ideal for users, the situation at the
Health Sciences Library created a unique opportunity to study the effects of
article level linking on journal usage. Article level linking was unavailable
for many smaller publishers, and also for a variety of larger publishers
including Cambridge, Oxford, Wiley, Springer, and Elsevier. This resulted in a
more wide-ranging sample than would typically be possible in an observational
study of article level linking.
Literature Review
Initially developed in the late 1990s by Herbert Van de Sompel, link resolvers were quickly recognized as a “silver
bullet solution” to the problem of context sensitive linking (McDonald &
Van de Velde, 2004, p. 32). Link resolver tools provided seamless connections
between bibliographic databases, publisher websites, and library catalogue
holdings. In the following years, link resolver technologies were described as
“indispensable” (Singer, 2006, p. 15) and “essential” (Chisare,
Fagan, Gaines, & Trocchia, 2017, p. 93) for
academic libraries.
Early research into OpenURL tools and link
resolvers suggested that their implementation increased electronic journal
usage. Kraemer (2006) observed that electronic journals with advanced linking
features were more highly used at the Medical College of Wisconsin Libraries,
while McDonald (2007) reported that OpenURL resolver
availability was correlated with a large and significant increase in
publisher-reported electronic journal usage in a number of subject areas. Early
research also indicated that patrons exhibited positive attitudes towards link
resolver services (Eason, MacIntyre, & Apps,
2005).
Link resolvers are not the only way for library users to access journal
content, however. IP-authenticated users may click on publisher-direct article
links in databases such as PubMed to access content directly. IP-authenticated
users may also access articles through publisher-direct article links on search
engines such as Google. In contrast to the usage gains and favourable
patron attitudes reported during earlier research, more recent studies suggest
that many library users bypass library link resolvers for publisher-direct
linking. One study of health science journal usage found that publisher website
usage statistics were much higher than usage reported by link resolver
click-through statistics (De Groote, Blecic, &
Martin, 2013). This trend is even more apparent in a study from 2017, which
reports:
On average (for the 18 months in the sample) the
publishers’ full text request report was 45,512 per month. The average full
text requests registered from the library discovery tool (through its URL link
resolver service) in the same period was 14,612 per month (i.e. publishers
report 3.1 times more downloads than requested from the library discovery
tool). (Greenberg & Bar-Ilan, 2017, p. 460)
Creating and maintaining link resolver tools is a complex and
time-consuming process (Samples & Healey, 2014). Journals that require
title-by-title intervention demand even greater resources in terms of library
staff time, so it is useful to understand the relationship between article
level linking and journal usage by library patrons. If the effect size is small
or insignificant, it might indicate that libraries should allocate their
resources elsewhere. Alternatively, if the effect size is larger, it might
signal that libraries should prioritize link resolver maintenance and cleanup.
To date, there appear to be no published studies that examine the impact of
article level linking versus journal level linking on publisher reported usage
statistics.
Aims
This observational, cross-sectional study attempts to
understand the effects of article level linking at one academic health sciences
library. The study uses statistical modeling to compare the usage of journals
with article level linking enabled to journals where only journal level linking
was available. The study attempts to control for other factors which may affect
journal usage.
This study’s research question is: What is the effect of article level
linking vs. journal level linking on publisher reported successful full-text
article requests at one academic health sciences library, controlling for
journal subject area, journal ranking, publisher, and alternate aggregator
access?
Methods
Journals were eligible for inclusion in the study if they met the
following criteria: the journal subscription cost was fully funded by the
health sciences library, the journal was available via a publisher-direct
platform, access to the journal was available throughout 2018, and the journal
was part of a selective e-collection. Selective e-collections require at least
some amount of title-by-title intervention during electronic resource
management processes. Journals available through packages that did not require
title-by-title intervention were excluded from the study (e.g., LWW Nursing and
Health Professions Premier Collection). Journals partially funded by the health
sciences library were excluded because the size and composition of the user
group for partially funded titles (e.g., Nature)
varies significantly from the size and composition of the user group for fully
funded titles.
A list of selective journal subscriptions requiring title-by-title
intervention was developed from publisher and serial agent invoices. Print and
electronic ISSNs from this list were input into the Ex Libris Alma Overlap and
Collection Analysis module in order to generate a spreadsheet outlining
electronic resource portfolio availability. Duplicate journal titles, which
resulted from the use of both electronic and print ISSNs, were removed.
Journals missing one or more variables in the statistical model were excluded
from the study (e.g., Journal Report 1 [JR1] was unavailable). Journals that
changed publishers or platforms mid-year were also excluded from the analysis,
as these changes can cause significant variation in usage (Bucknell, 2012). A
total of 356 journal titles were considered for inclusion in this study; JR1
reporting was unavailable for 21 journal titles, Journal Impact Factor (JIF)
was unavailable for 41 journal titles, and 3 journal titles changed publishers
or platforms over the course of the year. The final dataset included 302
journal titles.
Dependent Variable: Total JR1 for 2018
This study used vendor-supplied COUNTER reports in order to measure
IP-authenticated, publisher-direct journal usage by library users at Memorial
University. Librarians frequently use COUNTER reports to evaluate electronic
resources and make subscription decisions (Baker & Read, 2008). Project
COUNTER (Counting Online Usage of Networked Electronic Resources) is a
non-profit member-based organization of libraries, publishers, and vendors that
have developed standards and definitions for electronic resource usage data.
The COUNTER Code of Practice improves comparability of electronic resource
usage data between vendors, although several studies suggest that vendor
platform design decisions may lead to inflated usage statistics for some
publishers (Davis & Price, 2006; Kohn, 2018; Wood-Doughty, Bergstrom, &
Steigerwald, 2019).
COUNTER Release 4 usage reports for January 1 through December 31, 2018
were obtained for all included journal titles. Usage reports were examined for
usage spikes and other indications of potential misuse (Bucknell, 2012). No
indications of misuse were observed.
Although a number of different types of COUNTER reports are available,
JR1 was selected for this study due to its ready availability and frequent use
by librarians in journal evaluation. Journal Report 1 indicates the number of
successful monthly full-text article requests in both HTML and PDF format on
the publisher’s website. All successful requests are included, regardless of
how the request originated (e.g., library link resolver, direct to publisher
links in databases or search engines, journal browsing). Journal Report 1
includes usage of backfile content, as well as gold open access articles.
Journal Report 1 excludes usage of journal content through other platforms such
as aggregators (Journal Usage Statistics Portal, 2013). Journal Report 5 (JR5),
which reports monthly requests by year of publication, was also considered as a
potential outcome measure. The availability of JR5 reports was more limited,
however, as fewer publishers were able to provide JR5 reports. Using JR1
allowed for the inclusion of more small publishers in the study.
Total JR1 is a count variable, and the shape of its frequency
distribution is long-tailed. Dependent variables that are not normally distributed
require specific considerations in statistical modelling, as described in
further detail below.
Independent Variable: Article Level Linking
In May 2018, journal title electronic portfolios were examined via Ex
Libris Alma to determine whether article level linking was enabled for the
journal. Article level link testing was also conducted in the user-facing Ex
Libris Primo interface. No substantive changes were made to portfolio linking
level until mid-December 2018, when article level linking was enabled wherever
possible. It was expected that the December update would not result in
substantial changes in usage due to academic holidays.
Control Variables
A number of other factors may impact the usage of journal titles. This
study attempts to control for these factors by including them in the
statistical model.
Journal publisher is included as a categorical variable because there is
evidence that publisher platform design decisions affect usage statistics
(Davis & Price, 2006; Kohn, 2018; Wood-Doughty et al., 2019). Publishers
are anonymized within the study due to license restrictions around the sharing
of usage data. It would take considerable effort to obtain permission to share
usage data from each publisher included in the study, and the effort did not
seem sufficiently beneficial given that publisher platform effects were not the
variable of primary interest. Publishers comprising less than 5% of the total
number of journals included in the dataset were grouped together in an “other”
category.
There is also evidence to suggest that usage statistics are impacted by
academic discipline or subject area (Gorraiz, Gumpenberger, & Schlögl,
2014; Mongeon, Archambault, & Larivière,
2018). Included journals were categorized by top-level Scopus subject area. In
cases where a journal was included in more than one top-level subject area, the
subject area where the journal had the highest Scimago
ranking was selected. Subject area categories comprising less than 5% of the
total number of journals included in the dataset were grouped together in an
“other” category.
Journal usage may also be influenced by the relative size and importance
of a journal. Here, journal size is defined by the number of articles that the
journal publishes in a given year. A 2004 study demonstrates that indicators of
journal quality are correlated with journal usage in one medical library (Wulff
& Nixon, 2004). Both Scimago’s Scientific Journal
Rank (SJR) and Clarivate’s Journal Impact Factor (JIF) were explored as proxy
measures of journal size and importance. These measures are strongly
correlated, suggesting that either measure may be appropriate (Elkins, Maher,
Herbert, Moseley, & Sherrington, 2010). Ultimately, JIF was selected
because it provided a slightly better model fit.
Finally, alternate access to journals via an
aggregator database may decrease usage on publisher websites (Bucknell, 2012).
Due to the importance of recency in medical libraries, journals were
categorized as having alternate aggregator access if the embargo period for the
journal title was six months or less.
Because there is some evidence of home country bias in
journal readership (Thelwall & Maflahi, 2015), a dummy variable representing Canadian
journal titles was initially considered for inclusion in the model. The number
of Canadian journal titles within the sample was small (n = 7), however, so ultimately this variable was not included.
Statistical Modelling
Like many outcomes of interest in library and
information science, the dependent variable in this study is not normally
distributed (Figure 1). In such cases, it is not generally appropriate to use
multiple linear regression, and generalized linear models should be considered.
Figure
1
Distribution of JR1 2018 (dependent variable). Created using 538 Schemes (Bischof, 2017).
A study analyzing statistical modelling of infometric
data suggests that the negative binomial regression model (NBRM) may be most
appropriate for infometric studies with count
response variables (Ajiferuke & Famoye, 2015). The study dataset was modelled in Stata
(Release 15) with both the Poisson regression model and the NBRM using methods
outlined by Long and Freese (2006). Ultimately, the NBRM with robust standard
errors was selected because there was significant evidence of overdispersion (G2 = 1.5e+05, p < 0.001).
Results
Tables 1 and 2 present summary statistics relating to
variables that were included in the NBRM. For the 302 journals included in the
study, the average total JR1 usage in 2018 was 554 per journal. The median JR1
usage was 234.5, with a range from 0 uses to 9202 uses. Article level linking
was enabled for 64% of included journals (n
= 193), and journal level linking was offered for 36% journal titles (n = 109).
Table
1
Descriptive Statistics for Categorical Variables
|
Variables |
Article Level Linking Enabled |
No Article Level Linking |
Total |
|||
|
n |
% |
n |
% |
n |
% |
|
|
Publisher |
||||||
|
Publisher A |
76 |
76.8 |
23 |
23.2 |
99 |
33 |
|
Publisher B |
32 |
100.0 |
0 |
0.0 |
32 |
11 |
|
Publisher C |
18 |
94.7 |
1 |
5.3 |
19 |
6 |
|
Publisher D |
21 |
100.0 |
0 |
0.0 |
21 |
7 |
|
Publisher E |
27 |
96.4 |
1 |
3.6 |
28 |
9 |
|
Other publisher (reference category) |
19 |
18.4 |
84 |
81.6 |
103 |
34 |
|
Aggregator Access Available to Most Recent 6 Months |
||||||
|
Yes |
40 |
65.6 |
21 |
34.4 |
61 |
20 |
|
No |
153 |
63.5 |
88 |
36.5 |
241 |
80 |
|
Scimago Subject Category |
||||||
|
Biochemistry, genetics, and molecular biology |
19 |
67.9 |
9 |
32.1 |
28 |
9 |
|
Nursing |
14 |
63.6 |
8 |
36.4 |
22 |
7 |
|
Other subject |
31 |
64.6 |
17 |
35.4 |
48 |
16 |
|
Medicine (reference category) |
129 |
63.2 |
75 |
36.8 |
204 |
68 |
|
Total |
193 |
63.9 |
109 |
36.1 |
302 |
100 |
Table
2
Descriptive Statistics for Continuous Variables
|
Variables |
Mean |
Median |
Std. Dev. |
Min. |
Max. |
|
JR1 2018 |
553.97 |
234.50 |
1039.94 |
0 |
9202 |
|
Subgroup - article level links enabled |
676.91 |
273 |
1228.04 |
16 |
9202 |
|
Subgroup - no article level
linking |
336.28 |
187 |
508.19 |
0 |
3271 |
|
JIF 2017 |
6.57 |
3.75 |
8.92 |
0.42 |
79.26 |
|
Subgroup - article level links enabled |
7.57 |
3.87 |
10.53 |
0.66 |
79.26 |
|
Subgroup - no article level linking |
4.8 |
3.63 |
4.43 |
0.42 |
23.43 |
Table 3 presents the results of the NBRM with robust
standard errors. Regression coefficients, z,
and p values are presented. The table
also includes the percent change in the expected count for each unit increase
in order to assist with interpretation of the coefficients. For categorical
variables that do not have defined units, the percent change in the expected
count for each unit increase indicates the change that would occur when
switching from one category (e.g., no article level linking enabled) to another
(e.g., article level linking enabled), controlling for other variables.
Table
3
Results of Negative Binomial Regression Model with
Robust Standard Errors
|
Variable |
Coefficient |
z |
p value |
% change in expected count for unit increase in X |
|
Independent
Variable |
|
|
|
|
|
Article level linking enabled |
0.645*** |
3.819 |
.000 |
90.7 |
|
Control
Variables |
|
|
|
|
|
Publisher |
||||
|
Publisher
A |
0.063 |
0.353 |
.724 |
6.5 |
|
Publisher
B |
-1.353*** |
-5.312 |
.000 |
-74.2 |
|
Publisher
C |
0.019 |
0.058 |
.954 |
1.9 |
|
Publisher
D |
-0.336 |
-1.109 |
.267 |
-28.5 |
|
Publisher
E |
-1.055*** |
-4.009 |
.000 |
-65.2 |
|
Other publisher (reference category) |
- |
- |
- |
- |
|
Aggregator
access available |
0.106 |
0.531 |
.595 |
11.2 |
|
JIF
2017 |
0.039*** |
5.031 |
.000 |
4.0 |
|
Scimago Subject Category |
||||
|
Biochemistry, genetics and
molecular biology |
0.005 |
0.021 |
.983 |
0.5 |
|
Nursing |
1.052*** |
3.995 |
.000 |
186.3 |
|
Other subject (non-medicine) |
0.355* |
2.106 |
.035 |
42.6 |
|
Medicine (reference
category) |
- |
- |
- |
- |
*p < .05; **p < .01; ***p < .001.
Article level linking is shown to have a positive
effect on total JR1, and the relationship is significant (p < .001). Article level linking increases expected total JR1 by
90.7%, when compared to titles with journal level linking and controlling for
other variables.
Several control variables were statistically
significant. As expected, JIF had a significant, positive effect on total JR1.
For every one-point increase in JIF, a 4% increase in total JR1 would be
expected. Interestingly, only two publisher dummy variables reached
significance when compared to the reference category (all other publishers). In
both cases, the relationship was negative. The nursing and “other” subject
category dummy variables also reached statistical significance when compared to
the reference category (medicine).
In order to clarify the effects of journal level vs.
article level linking at various journal ranking levels, the marginsplot Stata command was used to graph predicted total
JR1 over the 95% range of JIF 2017, holding other variables constant. Figure 2
presents the predicted results for JIF scores ranging from 0–20, and includes
the 95% confidence interval. Plotting the predicted margins results
demonstrates that the difference in usage between journal level and article
level linking remains significant at both lower and higher levels of journal
impact. This suggests that usage of both smaller, more specialized journals and
larger journals increases when article level linking is enabled.
