Commentary
Newcastle Libraries’ Evaluation Strategy: Evidence Based Practice in Challenging
Times
Becky Cole
Project
Officer: Digital Inclusion
Newcastle
Libraries
Newcastle
upon Tyne, United Kingdom
Email: rebecca.cole@newcastle.gov.uk
Received: 12 Nov. 2014 Accepted:
16 Nov. 2014
2014 Cole. This is an Open Access article
distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work is
properly attributed, not used for commercial purposes, and, if transformed, the
resulting work is redistributed under the same or similar license to this one.
Challenging Times
In the last twelve years, Newcastle Libraries has
undergone a significant period of organizational change. This commentary
describes how evidence based practice has informed and focused these
developments.
Newcastle Libraries is the statutory public library
service for the city of Newcastle upon Tyne in the northeast of England. In
2002, the service began a wide ranging modernization program, and by 2009, it
was recognized as a leading exemplar in its field with a large, newly built
destination City Library, seventeen satellite branches, a mobile library, and a
home delivery service. In 2010, the election of a new national government
transformed the political and financial landscape in which public libraries
operate, precipitating further reinvention and accelerating the need for
innovative service delivery models, multi-functional library spaces, and new
ways of working.
In 2014, Newcastle’s organizational structure reflects
these developments. Four years of public sector budget cuts and a revolutionary
shift in user behaviour brought about by the ascendancy of the networked
information landscape have challenged the service to reinvent itself as
invisible intermediary, memory institution, learning centre, and community
resource (Brophy, 2008, p. 8). Some core services have disappeared and been
replaced by new services, such as the Business and Intellectual Property
Centre. After the adoption of a hybrid model, library services in the City
Library and several branches share premises with complementary organizations,
such as social housing, adult education, and youth employment support
providers.
In brief summary, this may seem like a reversal of
fortunes, but in a period of widespread national library closures, it could
have been a great deal worse. The Chartered Institute of Public Finance and
Accountancy (CIPFA) have calculated that 201 U.K. library service points were
lost in financial year 2011–12 with a further 74 closing in 2012–13. Another
411 buildings and 80 mobiles are “currently reported as either likely to be
closed or passed to volunteers or have been closed/left council control from
1/4/13 to 31/3/14” (“Reasons”).
In Newcastle, the comparative preservation of services
has been achieved through resilient leadership, an energetic political and
public advocacy program, and a solid foundation of service delivery. This
relative survival is also due in part to the organization’s implementation of a
dedicated evaluation strategy, the proactive collection and use of evidence
demonstrating impact, and the libraries’ willingness to respond to this
evidence (within the constraints of budget and capacity) when determining how
the service can and should evolve.
Types of Evidence
Today, the types of evidence that Newcastle
collects—like the buildings it inhabits—reflect both the political and
financial demands on the sector and the wider societal and informational
zeitgeist. The way that Newcastle thinks about and uses evidence is innovative
and multi-faceted. Standard quantitative measures (% computer usage, library
memberships, books and e-book issues, and visits) still play a part, but these
are supplemented by client-specific statistics, such as numbers of people
attending work clubs or requesting assistance with online job search or social
housing platforms.
Recording the types of support patrons currently seek
in public libraries builds a picture of the social uses of library buildings,
which is of academic interest to information professionals. It also enables
library managers and advocacy groups to demonstrate to stakeholders (and in the
case of Newcastle’s close neighbours, Gateshead Libraries, to the local media;
see Proctor 2014 in the reference list) the economic value of libraries as
trusted information providers experiencing an increasing demand for assisted
services.
To complement this quantitative data, Newcastle
collects visual and multimedia evidence, including photographic records of
exhibitions, launches, installations, and celebratory events, that is used to
illustrate promotional materials, such as the quarterly “Page Turner” brochure
and the “Annual Guide.” The organization is active on social media, promoting
events and campaigns via blogs, Facebook, and twitter. As a result, it creates
a digital record of the breadth of its operations and the way that it interacts
with service users.
The libraries record in-depth, qualitative evidence by
means of a bespoke Evaluation Toolkit devised and implemented in 2011–12 and
used to collect, store, and present rich, explicit evidence of impact. This
evidence includes general trends and preferences, quotes and comments from
users, empirical evidence (where staff provide feedback of their observations),
and reactions and responses to library services. The evidence is discussed
further in the article “Rich Emotive, Evidence of Impact” (Cole, 2014).
Partnership work and the delivery of discrete,
externally funded projects demand an additional layer of rigorous data
collection and reporting. Here, the primary focus is the “use value” of the
resource, i.e., “the favourable consequences” of using the “information
products or service” (Tenopir, 2013, p. 271). In Newcastle, the European
Regional Development Fund-funded Business and Intellectual Property Centre and
the Chartered Institute of Library and Information Professionals Information
Literacy Group-backed Go Digital Newcastle digital inclusion project maintain
case studies, financial risk registers, and qualitative and quantitative
research outputs to demonstrate to stakeholders the benefits to small and
medium sized enterprises and digitally excluded residents of interacting with
their services.
The toolkit’s use of open-ended questions captures
evidence pertaining to the function and value of “library space as a service .
. . for cultural events (poetry readings, book launchings, displays of artwork
. . .)” (DeLong, 2008, para. 5). In 2013–14, users’ opinions, including “simple
praise or complaints” of events and the spaces in which they were delivered,
were used to support decision making by instigating specific courses of action
(e.g., adjusting the room temperature) and informing general approaches, such
as liaising with the library café to reduce noise pollution (Tenopir, 2013, p.
272).
Evidence in Practice
In times of change, the way an organization approaches
evidence collection and use becomes even more crucial. In the last five years,
evidence based practice has had radical implications for Newcastle Libraries by
informing decisions over areas of expansion (where evidence has been used to
secure external project funding that adds value to the service), and,
inevitably, contraction. Difficult decisions over library closures were based
on stark ROI calculations (visits and issues against building and staffing
costs) and the physical distance of one library from the next.
The Evaluation Toolkit has enabled Newcastle to take a
structured, integrated, and user-focused approach to evidence collection by
following the principles outlined by Brettle (2014): “specify the outcomes you
are measuring, so that you know what evidence you need to collect . . . [and]
be aware of your stakeholders so that you can ensure you collect evidence that
is important and relevant to each” (para. 2). Newcastle’s primary stakeholders
are its users (actual and potential), parent organization (Newcastle City
Council), funders, staff, staff representatives, managers, and national bodies
and advocacy groups—each of whom has a different perception of the libraries’
quality.
For the service user, quality often means exceeding
expectations and delivering satisfaction. To help achieve this outcome, the
toolkit asks questions that invite critical feedback from library users that is
utilised in future planning. In 2012–13, parents attending story time sessions
were asked to suggest improvements and indicate other activities they would
like to see. Responses were aggregated and analyzed as a word cloud that highlighted the terms toddlers and more. This data indicated a demand for increased provision for
under-five’s and led to the development of Little Bears story and rhyme
sessions that were rolled across the service.
The current volatility of the U.K. public library
sector means that effective practice is sometimes as rudimentary as survival,
and survival depends (at least in part) on the ability to prove value to the
parent organization. In practice, this means alignment with “wider
organisational objectives and priorities” and asserting the library’s
significance within the overall structure (Grant, Sen, & Spring, 2013,
para. 15). Newcastle’s toolkit achieves the former by linking harvested data
(at the collection stage) with the City Council’s four key performance priorities. The toolkit achieves the latter by collecting “‘explicit’ measures of
value that come directly from testimonials” and that are used to support
accreditation in areas such as Customer Service Excellence (Tenopir, 2013, p.
271).
With in-house resources scarce, public libraries are
increasingly reliant on external funding, and the relationship between evidence
and practice in this area is particularly complex. For the potential funder,
quality often translates as evidence of need, originality, and value for money
(Poll & te Boekhorst, 2008, p. 20). A library’s ability to demonstrate such
characteristics determines whether or not a grant or bursary is awarded. Thus,
practice (the types of projects or services the library is able to deliver) is
arguably determined by the cache of evidence it holds. Once in delivery, the
project itself is expected to “actively contribute to the building of an
evidence base that both supports decision-making and is actively deployed in
practice” (Hall, 2011, p. 12).
For the sponsoring government department for libraries
(currently Culture, Media and Sport), quality is measured in terms of value for
money and the extent to which an organization supports government policy and
meets the service standards it prescribes for the sector. In Newcastle, this
manifests as tailoring service delivery and collecting evidence pertaining to
“reading and informal learning,” “access to digital skills and services,” and
social inclusion (Bawden, Petuchovaite, and Vilar, 2005, p. 459). It also requires active and visible participation in
national initiativessuch as the Society of Chief Librarians’Universal Offers;
the Go ON UK campaign for digital skills; Access to Research and the Reading
Agency’s Books on Prescription scheme—all of which help to raise the public
profile of the service.
With 2015 approaching, U.K. public libraries are
neither comfortable nor secure, and each raft of budget cuts requires further
reinvention. In Newcastle, at least for the present, the library service
remains largely intact, even managing to maintain its Customer Service
Excellence accreditation due in part to its conscious placement of the “needs
of existing and potential customers . . . at the heart of planning” (D. Fay,
personal communication, October 28, 2014). In these immensely challenging
times, library services must be savvy and stakeholder conscious. They must
adapt to survive, and the ways in which they approach the collection and
proactive use of evidence to inform practice is key to this survival.
References
Bawden, D., Petuchovaite, R., & Vilar,
P. (2005). Are we effective? How would we know? New Library World, 106(9/10), 454-463. http://dx.doi.org/10.1108/03074800510623137
Brettle, A. (2014). Evidence of impact
[Editorial]. Evidence Based Library and
Information Practice, 9(2), 1-2. Retrieved from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/22182/16558
Brophy, P. (2008). Telling the story:
Qualitative approaches to measuring the performance of emerging library
services. Performance Measurement and
Metrics, 9(1), 7-17.
Cole, B. (2014, June). Rich, emotive
evidence of impact. CILIP Update. 42-44.
DeLong, L. (2008). Book review: The
evaluation and measurement of library services [Review of the book The evaluation and measurement of library
services, by J.R. Matthews]. The
Canadian Journal of Library and Information Practice and Research, 3(1).
Retrieved from https://journal.lib.uoguelph.ca/index.php/perj/article/view/447/737#.VF_NAPmsWSo
Grant, M. J., Sen, B., & Spring, H.
(2014, October 27). Demonstrating your value: 13 tips for library and
information professionals [Web log post]. Retrieved from http://www.cilip.org.uk/cilip/blog/demonstrating-your-value-13-tips-library-and-information-professionals
Hall, H. (2011). Project output versus
influence in practice: Impact as a dimension of research quality. Evidence Based Library and Information
Practice, 6(4), 12-14. Retrieved
from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/12138/9373
Poll, R., & te Boekhorst, P. (Ed.).
(2008). Measuring quality: Performance measurement in libraries (Rev. 2nd ed.).
Berlin, Boston: K. G. Saur. Retrieved from http://www.degruyter.com/view/product/37583
Proctor, K. (2014, October 8). Gateshead library staff shouldering the
burden of welfare changes. Newcastle Chronicle. Retrieved from http://www.chroniclelive.co.uk/news/north-east-news/gateshead-library-staff-shouldering-burden-7898865
Reasons for libraries: False economy. (n.d.). Public Libraries News. Retrieved from http://www.publiclibrariesnews.com/
Tenopir, C. (2013). Building evidence of
the value and impact of library and information services: Methods, metrics and
ROI. Evidence Based Library and
Information Practice, 8(2), 270-274. Retrieved from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/19527/15262