Article
Looking for Links: How Faculty Research Productivity
Correlates with Library Investment and Why Electronic Library Materials Matter
Most
Michael M. Rawls
Budget and Assessment Director
Virginia Commonwealth University
Richmond, Virginia, United States of America
Email: rawlsmm@vcu.edu
Received: 15 Feb. 2015 Accepted: 30 Apr. 2015
2015 Rawls. This is an Open Access article
distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective
– This
paper summarizes two studies that share the same research question: do
universities produce more scholarly research when they invest more in their
libraries? Research libraries spend a great deal of effort reporting their
expenditures, collections statistics, and other measures that serve as a basis
for interlibrary comparison and even rankings. The straightforward assumption
implied by this activity is that libraries better serve their student and
research communities when they are well-funded and well-resourced. The studies
examined here both ask if that notion can be validated empirically, not because
research libraries require some sort of justification, but because in an
environment of tough budget decisions and shifting opinions about the changing
role of libraries, it may be useful to demonstrate that sustained investment in
libraries offers tangible returns or that the failure to do so can result in
tangible costs.
Methods
– A
cross-sectional design featuring ordinary least squares regression analysis was
used in both studies to estimate the relationship between scholarly research
productivity at U.S. doctoral institutions and an array of institutional
characteristics presumed to influence that productivity. The concept of
research productivity is operationalized as the total number of scholarly
journal articles produced by each institution over a five year period – as
journal articles represent the most
common form of scholarly expression across the greatest number of academic
fields. Serving as the dependent variable, this data was regressed
against a variety of institutional characteristics including faculty size,
research expenditures, and grant awards, and several library variables centered
mostly on expenditures. The concept behind this design is that to realistically
explore the relationship between levels of library investment and research
productivity, all other institutional drivers of research productivity must
also be represented in the dataset. While the design was similar for both
studies, they each drew on different data sources and marginally different
populations.
Results
– Both
studies found that an institution’s research productivity is positively and
significantly correlated with the level of investment it makes in its
libraries. Furthermore, both studies found electronic library material
expenditures to be particularly associated with increased productivity. This
relationship was so strong that an institution’s level of research productivity
appears to be sensitive to how its library’s collection budget is allocated
between print and electronic materials. As the portion of the budget dedicated
to non-electronic material grew, research productivity decreased in
statistically significant fashion in both studies.
Conclusion
– While
both studies succeeded in demonstrating the existence of an empirical
relationship between library investment and research productivity, the most intriguing finding is that both
studies observed a decrease in number of journal articles being produced as
expenditures for non-electronic library materials increased. The conclusion is
that the efficiencies of electronic resources offer such advantages over the
use of traditional library materials in supporting scholarly research that
productivity suffers as institutions dedicate a greater portion of their
collection budgets to print materials at the expense of electronic materials.
Introduction
A
2009 membership survey conducted by the Association of College and Research
Libraries identified “concern about demonstrating library value and
effectiveness” as one of the most important considerations on the minds of
responding library directors. According to Michael Germano (2010), “the
ultimate goal is a demonstrable strengthening of support from user populations
that will translate into the avoidance of deeper or ongoing cuts during the
current economic climate.” Yet, the call to demonstrate library value can be a
gauntlet cast down more often than picked up, due to the difficulty in linking
a library’s contributions to campus-wide outcomes that are more manifold than
manifest. This paper summarizes two studies conducted by the author in 2012 and
2013 that were designed to overcome this challenge by incorporating
representative measures of as many of the drivers of scholarly
productivity
as practicable for more than 200 institutions. This approach allows for the
examination of how library characteristics relate to scholarly output while
also accounting for other relevant campus factors that are likely influences.
By using this type of design, both studies can offer insight into how libraries
contribute to scholarly productivity in an empirical sense – something that
cannot be achieved by examining any single institution. While finding a linkage
between library investment and scholarly productivity can only imply a return
on investment to the institution (no research design can prove causality so
long as we are unable to confine libraries and universities to a laboratory),
an empirically established relationship is still preferable to the absence of
evidence. Furthermore, if a reasonable theory can
establish a context for interpreting the correlation, it can provide a
reasonable basis for the claim that the correlation being measured represents
an actual impact of libraries’ services.
Literature Review
Many studies have
explored the relationship between library resources and faculty research
productivity. The two research projects featured in this paper are what Oakleaf
has categorized as “input/output assessments” of library impact on faculty
research productivity (ACRL, 2010, p. 48). Other examples of this type of
approach include Budd’s work in the 1990s that compared the number of journal
publications produced by institutions to their library’s volume count (1995,
1999). More recently, Wilson and Tenopir (2008) conducted local citation
analysis that compared library holdings to faculty member citations to
determine the percentage of referenced items that were available from the
faculty member’s library. Further examples of input/output assessment studies
related to research productivity can be found in The Value of Academic Libraries (ACRL, 2010, p. 48).
While these works
examine the relationship between library resources and faculty research
productivity, no U.S. studies have focused explicitly on how electronic library
material expenditures relate to research productivity or other institutional
outcomes. There are two groups in the United Kingdom, however, who have
launched empirical investigations analyzing the link between electronic
resources and higher education outcomes in that nation. CIBER Research Ltd
conducted a study that found a strong correlation between e-journal spending
and usage at U.K. universities (CIBER, 2008). The study found e-journal
spending was correlated with such “downstream” effects as the number scholarly
journal publications, PhDs awarded, and research grant awards at each institution.
These results were corroborated by another U.K. study conducted the following
year by the Research Information Network (RIN), a policy organization funded by
the U.K. Higher Education Funding Council (RIN, 2009). RIN later developed a
structural modelling technique to test the directionality of the relationship
between spending and use, determining that spending drove usage (RIN, 2011).
The studies presently
examined in this paper were largely influenced by Budd’s work linking research
productivity to volume counts, mentioned earlier, and Weiner’s work examining
the library’s impact on institutional reputation. Budd’s work relied on
citation indexes to attribute the number of journal articles produced by
individual research universities and then compared that total to each
institution’s volume count using ARL and ACRL library survey data (Budd, 1995,
1999). Budd also accounted for the effect of faculty size on productivity by
standardizing scholarly output on per-faculty basis. However, Budd did not
account for the effect that other institutional characteristics – such as
research expenditures, financial strength, and grant awards – might have on
research productivity. Weiner, on the other hand, employed a variety of
institutional characteristics to explore the relationship between libraries and
institutional reputation, as ranked by the U.S
News and World Report (2009). She used regression analysis whereby an
institution’s ordinal ranking served as the dependent variable and a variety of
library and non-library measures served as the independent variables. Her goal
was to determine if any library characteristics were positively correlated with
institutional reputation, but she also recognized that the prestige of a
university is not centered solely on the library. Therefore she included
expenditure data for instruction, research, and student services; levels of
alumni, corporate, and foundation giving; measures for graduate rate, retention
rate, and the number of grants received; as well as library expenditures,
staffing, and transactional data. The two studies explored in this paper
essentially amalgamate Budd’s comparison of library characteristics to
scholarly output with Weiner’s use of a regression model that features both
library and non-library institutional characteristics to determine their
relation to a campus-wide outcome.
This type of research
design has a precedent in the field of economics, where actual firm-level
output data for a particular industry is regressed against firm-level inputs to
form an industry-specific production function equation. Known as the
Cobb-Douglas model, this approach is used to study the relationship between a
set of inputs and the quantity of output produced, which in turn can be used to
measure production efficiency, including the impact of technological
improvements (Biddle, 2011). The two studies examined in this paper take a
similar approach by identifying the institutional inputs that go into producing
scholarly research and regressing those measures against actual scholarly
output – producing an industry production function of sorts for academic
scholarship. Furthermore, both studies’ findings regarding the potential
efficiencies that electronic library materials introduce into the scholarly
production process are consistent with the Cobb-Douglas model’s ability to
identify the impact of technological improvements on production.
Methodology and Results
First Study
The original study sought any evidence
suggesting that libraries confer value to the research mission of their host
institutions when properly resourced. At the time, there was no particular
focus on the role of electronic library materials. Instead, a wide array of
library measures was assembled to determine which aspects, if any, of libraries
are correlated with scholarly output. This array of library variables was drawn
from ACRL’s Annual Trends and Statistics
Survey using Counting Opinion’s ACRLMetrics
service (www.acrlmetrics.com) and included such
measures as total expenditures, library material expenditures, electronic
library material expenditures, volume counts, staffing levels, interlibrary
loan borrowing, and others. Non-library institutional characteristics that
might also influence scholarly productivity were collected using the U.S.
Department of Education’s Integrated Post-Secondary Education Data System
(IPEDS). These variables included such measures as research expenditures, grant
funding, faculty count, total university revenue, year-end value of the
endowment, the number of PhDs awarded, and others. All told, more than 25
different library and non-library institutional measures were represented in
the study as potential explanatory variables for scholarly research
productivity at each doctoral institution.
The concept of scholarly research productivity
was operationalized using the total number of scholarly journal articles
produced by each U.S. doctoral institution. Journal articles were selected over
other forms of scholarly expression because they are common to most academic
fields. The number of journal
articles attributable to each doctoral institution was established using Thomson Reuter’s ISI Web of Knowledge citation index. The
article count for each school could then be linked to that institution’s
library and non-library explanatory variables for analysis.
The choice was made to aggregate the data over a period
of five years, rather than relying on data from one particular year. The decision was
based on the rationale that it is too imprecise to tie a specific year’s inputs
to a specific year’s outputs. Instead, by examining a short range of years, it is possible to get a
more representative indication of the amount of resources that each institution
typically dedicates to scholarly research as well as the amount of productivity
that it typically achieves. For the IPEDS and ACRL data, this involved
collecting the reported figures for each measure from 2005 through 2009 and
then calculating an average (e.g., average library expenditures per year or
average number of faculty per year). This average was compared to the total
number of journal articles produced from 2006 to 2010. The range of years was
staggered between the explanatory variables and the dependent variable data
based on the assumption that inputs must necessarily precede outputs.
The Carnegie Classifications (2010) were used
as the basis for identifying doctoral institutions, though several were
excluded due to a lack of reported data. Ultimately, 234 institutions were
included in the study. A full discussion of this study, including an exhaustive
list of the variables, data limitations, and iterative details, can be found in
the Proceedings of the 2012 Library
Assessment Conference (Rawls, 2013).
Potential correlations were explored using
ordinary least squares regression analysis, where the number of journal
articles served as the dependent variable and the institutional characteristics
served as the explanatory variables. After exploring several different
combinations of explanatory variables in a number of iterations, the factors
deemed to be most strongly, consistently, and significantly related to journal
article output were as follows: total university revenue, number of faculty
members, research expenditures, the number of professional librarians, electronic
library material expenditures, and non-electronic library material
expenditures. Other explanatory variables also proved to be significantly
related to journal article output, but had to be excluded due to the issue of
multicollinearity. This occurs when two or more explanatory variables are so
highly related to each other that the scope of their relationship with the
dependent variable cannot be precisely measured. For example, both total
library material and electronic library material expenditures had statistically
significant relationships with journal article output. Both variables, however,
increased or decreased from one institution to the next in a very similar
manner. This similarity was so strong that when both variables were included
simultaneously in the same model, the analysis was unable to distinguish the
effect that one variable had from the other on the corresponding changes in
each institution’s article count. This development meant that some variables
needed to be excluded in order to gain an understanding of the degree to which
different characteristics related to scholarly productivity. Level and
consistency of statistical significance as well as size of standardized
coefficients were used as a basis for which significant variables were excluded
or retained.
Finally, it was necessary to include an
indicator variable for Harvard University to control for the outlier effects
that that institution’s unparalleled personnel expenditures and staffing levels
were exerting on the rest of the dataset. Prior to adding this “dummy”
variable, the regression results had mostly indicated that the library
variables were not significant. After it was introduced into the dataset,
however, most major library expenditures categories were consistently significant.
Another option would have been to exclude Harvard altogether, as both methods
would have reduced the residual effect of Harvard to zero. The decision was
made to retain Harvard, however, because it seemed appropriate to include the
highest-spending library, given the goals of the study.
The unstandardized coefficient for each
variable contained in the regression results represents its estimated
relationship to the number of journal articles produced by an institution (see
Table 1). For example, these results estimate that for each dollar dedicated to
electronic library materials, a U.S. doctoral institution is expected to
produce .00052 journal articles. Likewise, it estimates the publication of
.78292 journal articles per faculty member. When the coefficients for the
model’s variables are multiplied by the actual numbers belonging to a
particular institution and then added together, it provides an estimate for the
total number of journal articles that the institution is predicted to produce given
these inputs. The implication is that a change to any one of these variables
should result in a corresponding change to the number of journal articles that
an institution produces. For example, this model suggests that a $1,000,000
increase in electronic library materials spending should result in 520
additional articles.
The model produced an
adjusted r-squared value of .925, which was roughly consistent with other
iterations. Among the library-related measures, the number of professional
librarians had the largest standardized coefficient, suggesting that this
measure was more strongly associated with increased scholarly productivity than
electronic material expenditures (.218 to .184). While this finding was very
encouraging, and deserving of additional study, the second study was unable to
replicate a linkage between staffing levels and productivity.
Table 1
"Best fit" model
from first study
|
Independent Variables* |
Unstandardized
Coefficients |
|
Standardized Coefficients |
t |
Sig. |
|
|
B |
Std. Error |
Beta |
|
|
|
(Constant) |
-1401.94136 |
303.274 |
|
-4.623 |
.000 |
|
Total University Revenue |
0.00000212 |
.000 |
.21251 |
3.961 |
.000 |
|
Faculty FTE |
0.78292 |
.355 |
.10485 |
2.203 |
.029 |
|
Research Expenditures |
0.00002 |
.000 |
.33949 |
8.250 |
.000 |
|
Number of Professional
Librarians |
30.98683 |
7.519 |
.21828 |
4.121 |
.000 |
|
Electronic Library
Material Expenditures |
0.00052 |
.000 |
.18403 |
4.661 |
.000 |
|
Non-Electronic Library
Material Expenditures |
-0.00026 |
.000 |
-.09610 |
-2.739 |
.007 |
|
Harvard |
21924.60497 |
3282.390 |
.17972 |
6.679 |
.000 |
*Dependent variable: total
number of articles published by faculty and other researchers associated with
each US doctoral institution from 2006 to 2010 according to ISI Web of Knowledge.
Second Study
After the positive results of the first study, a
follow-up study was conducted to determine if similar results would be
replicated using a different data source. To achieve this, the new study relied
on the Academic Analytics (www.academicanalytics.com) database tool. Academic
Analytics (AA) is a subscription-based system that university administrators
can use to measure faculty scholarly productivity. It attempts to do this by
attributing scholarly works, citations, grants awards, and honorific awards to
individual faculty members and then aggregating that information at the PhD
program level and again at the institutional level. This allows administrators
to analyze the faculty scholarly productivity of each PhD program or the
overall university within the context of other programs and institutions around
the nation.
The general methodology of this study was very similar
to its predecessor. The main differences were that the AA system provided a
different source of journal count data (CrossRef), a slightly different time
frame (2008-2011), and it drew from a subpopulation of researchers at each
institution (only those faculty members associated with PhD programs are
tracked in AA) instead of the entire research community. The second study also
necessitated changes in the explanatory variable data. The IPEDS and ACRL data
used to represent library and other institutional characteristics were
re-collected for the years 2007 to 2010 to synchronize with the new time frame
of the dependent variable data.
Additional explanatory variable data from AA was also
introduced into the dataset. This included the system’s own count for faculty,
grants awards, and grant dollars – all of which were lower than similar
measures from IPEDS due to AA’s singular focus on just those professors
associated with doctoral programs. The reason for adding this additional data
from AA was that it was more proportionally scaled to the dependent variable
data. In other words, given that only journal articles published by faculty members
associated with a PhD program were being counted at each institution, it was
logical to count only those faculty members associated with such programs,
instead of the entire faculty, when measuring how faculty size relates to this
study’s measure of scholarly output. Likewise, the same logic applies for the
grant-related measures collected from AA over IPEDS grant and research
expenditures data. In this way, variations in the size of each university’s PhD
enterprise relative to the overall institution’s size would not skew results.
Again, ordinary least squares regression analysis was
used to test the relationship between journal output and the variety of
institutional and library characteristics represented in the dataset. The
results of the final model bore a resemblance to those of the first study,
particularly where electronic and non-electronic material expenditures were
concerned, though some notable differences occurred as well. The combination of
independent variables observed to most strongly correlate with journal article
output were: grant dollars, number of PhD faculty, number of PhDs awarded in
research fields, electronic library material expenditures, and non-electronic
library material expenditures. The model produced an adjusted r-squared value
of .969.
The grant dollars and
PhD faculty count variables in this model can be seen as more relevant
substitutes for the research expenditure and faculty count variables found in
the first study. The variables for total university revenue and the number of
professional librarians were not statistically significant. Both revenue and
the number of librarians are more realistically driven by overall institution
size than by the number of PhD programs, suggesting that these measures could
simply be out of synch with the dependent variable data used in the study.
Likewise, once professional librarians were no longer included in the model,
the indicator variable for Harvard proved unnecessary and was dropped.
Electronic and
non-electronic materials expenditures each had a similar relationship to
journal articles as in the first study, with the former being positively
correlated and the latter being negatively correlated, with both relationships
being statistically significant. The coefficients were lower, but this too could
be a result of scale, produced by comparing overall material expenditures to a
subset of each institution’s scholarly output, as opposed to all scholarly
output in the first study. A more detailed discussion of the second study is
available in the Proceedings of the 10th
Northumbria Conference on International Performance Measurement in Libraries
and Information Services (Rawls, 2014).
Discussion
The
inverse correlation between non-electronic material expenditures and journal
article output was unforeseen, in that the general expectation for explanatory
variables was that each one would have a relationship that was either
significantly positive or one that was not statistically significant at all.
But these results suggest that for each additional dollar invested in
traditional library materials, scholarly productivity decreases. How could this
be? It is not as though print materials offer no usefulness to researchers, let
alone serve as a hindrance. Furthermore, volume counts and other measures of
the physical collections did not register a significant or negative
correlation. Instead, a plausible interpretation is that electronic library
resources are more efficient in supporting research needs than print materials.
To illustrate the obvious, think of a researcher in her office conducting a
single, well-worded search on the library’s website and gaining instant access
to a dozen relevant titles for her literature search. Contrastingly, think of
her at a poorly resourced institution, finding only some of her needed articles
and having to work through interlibrary loan or make a trip to the library to
wade through the bound periodicals in order to access the remaining portion of
the same titles. The time difference between these two scenarios is likely measured
in hours or days. Likewise, access to digital archives, databases, and
secondary datasets may preclude a trip to far-flung archives or the need to
collect data, potentially speeding up a research project by days, weeks, or
months, or even allowing the research project to take place at all. When all of
these time savings, however great or small, are multiplied by each member of
the institution’s research community, it is not surprising that those
institutions that are better endowed with electronic materials are able to
produce more scholarship over a given period of time than those that are not.
Table
2
"Best
fit" model from Academic Analytics study
|
Independent
Variables* |
Unstandardized
Coefficients |
|
Standardized
Coefficients |
t |
Sig. |
|
|
B |
Std.
Error |
Beta |
|
|
|
(Constant) |
-317.09038 |
89.028 |
|
-3.562 |
.000 |
|
PhD
Faculty Count |
2.32040 |
.287 |
.29340 |
8.077 |
.000 |
|
Grant
Dollars |
.00002 |
.000 |
.57484 |
23.070 |
.000 |
|
PhDs
Awarded - Research Fields |
2.40900 |
.669 |
.11100 |
3.598 |
.000 |
|
Electronic
Library Material Expenditures |
.00011 |
.000 |
.07997 |
3.210 |
.002 |
|
Non-Electronic
Library Material Expenditures |
-.00005 |
.000 |
-.03317 |
-1.983 |
.049 |
*Dependent
variable: total number of journal articles published by faculty members
associated with a PhD program at US doctoral institutions from 2008 to 2011,
according to CrossRef.
Yet, the efficiency alone does not entirely
explain why print expenditures would be significantly negative. To illustrate
why this is the case, it is important to point out that non-electronic library
materials expenditures is a not a measure collected in the ARL or ACRL surveys.
Rather the variable was derived by subtracting each institution’s reported
electronic library material expenditures from their total library material
expenditures. This means that the non-electronic and electronic materials
variables serve as two components that comprise the library’s overall
collection budget. Therefore, as electronic material expenditures grew as a
total portion of the budget from one institution to the next, the
non-electronic material expenditures necessarily shrank. Conversely, as the
ratio of non-electronic library materials grew, it was at the expense of
electronic materials. The suggestion is that those institutions deciding to
invest more in non-electronic materials – or perhaps those that experienced a
slower transition from print to electronic during the span of this study – paid
an opportunity cost in terms of journal article production. Thus those
universities that spent more on non-electronic library materials experienced a
loss in scholarly productivity instead of realizing a potential gain. These
results are in line with the manner in which the Cobb-Douglas model detects
production efficiency in economic production theory. The model does this by
identifying firms that are producing more output than the sum of their inputs
suggest that they should be able to produce, when compared to an industry
average as established by a regression equation (Biddle, 2011). This suggests
that the excess production is attributable to a technological efficiency that
the highly productive firm is employing and that the average and lower
producing firms are not. In the case of these two studies, the institutions
allocating more of their collection budgets toward electronic resources
experienced greater productivity – presumably because they offered their
research communities more efficient inputs that reduced the time needed to
complete the research cycle.
The nature of the relationship between
non-electronic materials and scholarly output offers unique evidence in support
of the study’s original hypothesis. Recalling that the initial intent was to
demonstrate empirically whether well-supported libraries are generally
associated with higher levels of scholarly production, the strong positive
correlations that both electronic library materials and the number of
professional librarians exhibited with journal articles arguably achieved that
goal (total library material expenditures and total library expenditures were
also strongly related to journal articles, but again, were removed due to
multicollinearity). While these results realize the original objective of
detecting linkages between library inputs and scholarly output, they cannot
prove causality – as is the case with a quasi-scientific research design. In
fact, were it not for the negative coefficient associated with the print
materials, it would be simple to challenge these results with the argument that
the findings only prove that well-off doctoral institutions have more of
everything than less well-to-do universities. It follows that institutions of
greater prestige and deeper funding are simultaneously in a better position to
support research, to spend more lavishly in support of their libraries, and to
produce more scholarship. That all of these factors can be identified to
correlate with one another in a regression equation could be interpreted simply
as a rising tide that lifts all boats. The print material expenditure results,
however, confound this notion of a rising tide by going in the opposite
direction of every other statistically significant measure associated with
scholarly productivity. When coupled with the theory that print material
expenditures represent an opportunity cost to scholarly productivity, a basis
is provided for contending that some degree of causality is being measured
between electronic materials and scholarly output in this model.
Conclusion
The studies described
here each provide empirical evidence that scholarly research productivity
increases at U.S. doctoral institutions as they invest more in their libraries.
The primary finding both studies share in common is that growth in electronic
library material expenditures has an especially strong association with growth
in research productivity. These findings satisfy the original research question
and provide a credible argument that universities can realize a detectable
return on their investment in libraries, depending on how that investment is spent.
This argument would be less plausible if print materials had not proven to be
so spectacularly less productive than electronic resources. But because
scholarly productivity seems to ebb and flow so significantly based on how an
institution comprises its collections budget, the contention that scholarly
output is actually affected by library spending is much more persuasive.
Applying regression
analysis to the question of whether universities produce more scholarship when
they invest more in their libraries allowed both studies to control for other
important institutional characteristics that also drive scholarly productivity.
This means that the effects that an institution’s faculty size, research
expenditures, or grant awards might have on scholarly output were accounted for
and incorporated into the study alongside the library-related variables. This
approach makes the results more meaningful than simple correlations. As such,
it may have applications in other areas where libraries would like to demonstrate
their value, yet face the challenge of being one factor among many that
contribute to an important institutional outcome.
Because both studies
found such a sharp contrast between how electronic and print materials
expenditures each relate to scholarly research productivity, this topic merits
further inquiry. One approach may be to explore the relationship between
library investment and scholarly productivity at the discipline level, to
determine if these relationships persist across different subject areas. Such a
study might also benefit from substituting other forms of scholarly expression
in place of journal articles in order to further develop this line of inquiry.
References
Association of College and Research Libraries.
(2009). Building capability for
demonstrating value of academic libraries. Chicago, IL: American Library
Association.
Association of College and Research Libraries.
(2010). Value of academic libraries: A
comprehensive research review and report. Chicago, IL: American Library Association.
Biddle, J. E. (2011). The introduction of the
Cobb-Douglas regression and its adoption by agricultural economists. History of Political Economy, 43(Suppl.
1), 235-257. http://dx.doi.org/10.1215/00182702-1158745
Budd, J. M. (1995) Faculty publishing
productivity: An institutional analysis and comparison with library and other
measures. College and Research Libraries,
56(6), 547-554. http://dx.doi.org/10.5860/crl_56_06_547
Budd, J. M. (1999). Increases
in faculty publishing activity: An analysis of ARL and ACRL institutions. College
& Research Libraries, 60(4),
308-315. http://dx.doi.org/10.5860/crl.60.4.308
Budd, J. M. (2006). Faculty
publishing productivity: Comparisons over time. College & Research
Libraries, 67(3), 230-239. http://dx.doi.org/10.5860/crl.67.3.230
Carnegie Foundation for the Advancement of
Teaching (2010). Classification Downloads.
Retrieved from: http://carnegieclassifications.iu.edu/resources/
CIBER Research Ltd. (2008). Evaluating the
Usage and Impact of e-Journals in the UK.
Report. Retrieved from: http://ciber-research.eu/download/20110119-References.pdf
Counting Opinions. (2015). ACRLMetrics statistics portal. Retrieved
from http://www.acrlmetrics.com/
Germano, M. A. (2010). Narrative-based library
marketing: Selling your library’s value during tough economic times. The Bottom Line: Managing Library Finances,
23(1), 5-17. http://dx.doi.org/10.1108/08880451011049641
Rawls, M. (2013). Measuring the impact of
electronic library materials on the university’s research mission. In S.
Hiller, M. Kyrillidou, A. Pappalardo, J. Self, & A. Yeager. (Eds.), Proceedings of the 2012 Library Assessment
Conference: Building effective, sustainable, practical assessment. (pp.
67-76). Washington, D.C.: Association
of Research Libraries.
Rawls, M. (2014). Looking for links: Do
universities produce more scholarship when their libraries invest more in
electronic resources? In I. Hall, S. Thornton, & S. Town (Eds.), Proceedings of the 10th Northumbria
International Conference on Performance Measurement in Libraries and
Information Services. (pp. 253-257). York, UK: University of York.
Research Information Network. (2009).
E-Journals: Their Use, Value, and Impact.
Report available from: http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/e-journals-their-use-value-and-impact
Research Information Network. (2011).
E-Journals: Their Use, Value, and Impact.
Report available from: http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/e-journals-their-use-value-and-impact
Weiner, S. (2009). The contribution of the
library to the reputation of a university. Journal
of Academic Librarianship, 35(1), 3-13. http://dx.doi.org/10.1016/j.acalib.2008.10.003
Wilson, C. S., & Tenopir, C. (2008). Local citation
analysis, publishing and reading patterns: Using multiple methods to evaluate
faculty use of an academic library's research collection. Journal of the American Society for Information Science and Technology,
59(9), 1393-1408. http://dx.doi.org/10.1002/asi.20812