Using Evidence in Practice
Gathering Evidence for
Sustainable Development Goals: An Alignment Perspective
Paula
Ochôa
Integrated
Researcher and Assistant Professor
Centro de Humanidades (CHAM-NOVA FCSH)
Universidade NOVA de Lisboa
Lisbon, Portugal
Email: paulatelo@fcsh.unl.pt
Leonor Gaspar Pinto
Integrated
Research and Invited Assistant Professor
Centro de Humanidades (CHAM-NOVA FCSH)
Universidade NOVA de Lisboa
Lisbon, Portugal
Email: lgpinto@sapo.pt
Received: 1 Sept. 2019 Accepted:
11 Nov. 2019
2020 Ochôa and Pinto. This is an Open Access article
distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip29638
Setting
In
September 2015, the United Nations (UN) Member States adopted the 2030 Agenda for Sustainable Development
and set the global, national, and local framework for putting that
responsibility into action (United Nations, 2015). The new global agenda
includes 17 Sustainable Development Goals (SDGs) that must be met by all
countries by 2030 so that "no one will be left behind" (United
Nations, 2015, p. 6). The 169 associated targets are monitored and reviewed
using a set of global
indicators
(Sustainable Development Solutions Network, 2015).
Sectorial
organizations have worked to bring their potential contributions toward
sustainable development to the attention of Member States and the UN. In the
post-2015 process, the International Federation of Library Associations and
Institutions (IFLA) committed to promoting libraries within the the 2030 Agenda for
Sustainable Development as a key initiative under the strategic directions
set for 2016-2021 (IFLA, 2015). Access to information has been recognized as a
target under SDG16 (target 16.10), culture and information and communication
technology have also been included in the SDGs (target 11.4 and targets 5b, 9c,
17.8, respectively), and universal literacy is recognized in the vision for the 2030 Agenda for Sustainable Development
(IFLA, 2018).
Problem
In
Portugal, initiatives to promote library and information services within the
2030 Agenda are still modest. Research intersecting Information Science with
Performance Evaluation and Sustainable Development is not yet sufficiently
developed (Ochôa & Pinto, 2019).
Since
2012, researchers at the Centre for the Humanities (CHAM) in the Faculty of
Social Sciences and Humanities of Universidade NOVA
de Lisboa have been pursuing a line of research that
intersects performance evaluation of Library and Information Services (LIS)
with sustainability, transition management, and competence development. In
2013-2014, this intersection was put into practice through in co-creationworkshops directed at students of LIS courses that
focus on building a sustainability assessment framework. These
interdisciplinary experiences led to workshop participants outlining a draft of
a conceptual framework for assessing the sustainability and impacts of LIS. The conceptual framework was further
consolidated (Ochôa & Pinto, 2014) and expanded
by introducing the evaluation perspectives provided by: 1) ISO 16439 standard
(2014), namely impact evaluation areas/focus and methods for collecting
evidence of impact; 2) media and information literacy indicators (Ochôa & Pinto, 2015).
In
November 2016, a team of researchers at CHAM were willing to think globally but
act locally looked for an answer to the question How can public libraries
gather evidence and evaluate their contribution to the SDGs? The
researchers started an action research project named Bibliotecas
Públicas e Sustentabilidade:
Recolha de Evidências da Contribuição para os ODS (Public
Libraries and Sustainability: Gathering Evidence of Contribution to SDGs, aka
Project PLS). Project PLS aims to develop a framework (model) for evaluating
public libraries’ contribution to the SDGs, and to tailor the model to
Portuguese public libraries (Pinto & Ochôa,
2018a).
Evidence
Evaluation research
and evidence based theory provided the general
framework for Project PLS. The research question was addressed through an qualitative methodology supported by the combination of
two principal methods: 1) literature review and 2) construction and application
of conceptual models as well as tools for analysis. The literature
review process was complemented by a questionnaire that analyzed Portuguese
public libraries’ involvement in SDGs international, national, and local
initiatives.
An integrated and
holistic approach was used to co-create the Model for the Alignment and
Evidence Gathering of Libraries’ Contribution to Sustainable Development (see
figure 1). For the model, the concept of evidence can be any type of
information or data used to help answer a question. The information or data
collected becomes evidence when it is used to demonstrate a library’s
contribution(s) to sustainable development, when it is used to determine goal
or target achievement, when it is used to show the alignment initiatives
adopted, or when it is used to understand various impacts on stakeholders (Bill
& Melinda Gates Foundation, 2015).
In the alignment process,
the use and/or
adaptation of library assessment data is valued using any or a combination of
the following:
·
UN Global Indicators
·
Portuguese national indicators (whenever possible) that
correspond to the UN Global Indicators
·
ISO 11620 – library performance indicators
·
ISO 16439 – methods and procedures for assessing the impact of
libraries
Any of these may be
complemented by the Global Libraries Impact Planning and Assessment Guide (Bill
& Melinda Gates Foundation, 2015) or the Generic Learning Outcomes and
Generic Social Outcomes (Arts Council England).
Figure 1
Model for the alignment and evidence gathering
of libraries’ contribution to sustainable development (Pinto & Ochôa, 2018).
Implementation
The
cornerstone of the model is a roadmap that is anchored in the identification of
potential (inter)relations between the dimensions that frame the impacts of
library activities or projects and the SDGs. The roadmap supports the alignment
cycles in four steps:
1. Step
1 – Preplanning and learning. This step creates
the conditions for the implementation of the process that allows libraries to
obtain evidence of their contribution to drive forward the UN 2030
Agenda. Library and information professionals need to develop basic competences
(e.g., to understand the architecture and main concepts of the model).
2.
Step 2 – Planning in the face of alignment. Based
on the information collected in step 1, decisions must be made that will
determine what evidence will come out of the alignment process. This step
requires several actions such as defining
SMART indicators and choosing adequate methods to gather data or evidence.
Evidence can be structured according to
five levels. Level 1, the most basic, requires only a clear description
of the project or activity and its importance to the community and SDGs. In
Level 2, libraries gather data that shows some positive change (alignment), but
they may not be able to confirm what caused the modification. At Level 3,
libraries demonstrate causality between results and the alignment dimensions.
In Level 4, evidence is confirmed by independent evaluators, and a
determination is made as to whether the project shows potential for replication
in other libraries. Finally, Level 5, the highest level, implies the
recognition of libraries’ contributions to SDG by national and international
entities.
3.
Step 3 –
Implementing. The plans from prior steps need to be put
into practice. Data is gathered and analyzed, and then the information that
will be communicated is prepared.
4.
Step 4 – Using
evidence to advocate for libraries. This step
focuses on the effective use of a narrative tool that values the use of
anecdotal evidence for impact assessment obtained from users, staff, and other
stakeholders. The SDGs story must provide answers to three questions: Why? Who did
what, how, and when? So what?
There
have been several experiences of implementation focused on developing sharing
skills through participation at libraries’ workshops, at academic and
professional conferences, at academic classes, and through promotional
incentives, such as national awards.
Outcomes
The
Model for the Alignment and Evidence Gathering of Libraries’ Contribution to
Sustainable Development was consolidated and expanded within another project.
Under IFLA’s International Advocacy Programme, the Portuguese Association of
Librarians, Archivists and Documentalists implemented the Project Libraries for
Development and the UN 2030 Agenda (April to July 2018). To accelerate library
and information professionals’ involvement in the systematic process of
evaluating and promoting libraries’ contribution to SDGs, a step-by-step Guide
for Libraries (Pinto & Ochôa, 2018) was built on
this Model and made available through the website of the Portuguese Association
of Librarians, Archivists and Documentalists.
One
of the model’s tools, the Roadmap for the Alignment and Evidence Gathering of
Libraries’ Contributions to the 2030 Agenda, was tested in two workshops
targeted at Portuguese public libraries professionals in July 2018. It also
provided the framework for creating a national award in October 2018 related to
best practices in sustainable development in libraries of all types.
One
of the most lasting and consequential effects of the model may be the impact it
is having on LIS student engagement, especially among those who use it most as
a tool to reach out and identify new assessment profiles and roles in
libraries. The ability to extend evidence based
librarianship into assessment competencies demands the exploration of new
possibilities for understanding and sharing within curricular content.
Reflection
The
relationships between the 2030 Agenda for Sustainable Development and public
libraries’ outcomes can be relevant at different points to evidence
based librarianship. One of the early difficulties in exploring the
phenomenon of strategic alignment lies in the ability to capture that dimension
in a meaningful way. Social aspects, such as immediate personal outcomes,
health, and well-being, have been acknowledged as important traits for
acquiring new understanding and developing new knowledge through the model of
implementation.
Further,
evidence alignment can be significantly associated with other relevant
assessment constructs and tools as a management tool and as an agenda for joint
research with several stakeholders. In a future research agenda, it is our
belief that epistemic communities will develop transdisciplinary and evaluative
practices. In this scenario, co-evaluation models will have a pivotal role.
Different alignment strategies and evidence will also be valued to deal with
the complexity of new open evaluation processes.
The
implementation in various contexts of the Model for the Alignment and Evidence
Gathering of Libraries’ Contribution to Sustainable Development fosters a set
of evidence that values strategic alignment as well as impactful evaluation
practices and competencies that can be used toward libraries’ contribution to
the UN 2030 Agenda. The roadmap proved to be particularly useful in developing
library and information students’ and professionals’ alignment and assessment competencies.
It also supports libraries’ advocacy efforts toward their recognition as active
and valuable partners in the development and implementation of national,
regional, and local strategies that will help deliver on the SDGs.
References
Arts Council England. Generic
Learning Outcomes. Retrieved 10
June 2019 from https://www.artscouncil.org.uk/sites/default/files/S2D12_Detailed_framework.pdf
Arts Council
England (2016). Generic Social Outcomes: Indicator bank for museums,
libraries and archives. Retrieved 10 June 2019 from https://www.artscouncil.org.uk/sites/default/files/S3D29_GSO_Indicator_Bank.pdf
Bill
& Melinda Gates Foundation (2015). Global libraries impact planning and
assessment guide. http://www.publiclibraryadvocacy.org/wpcontent/uploads/2015/06/IPA-Guide-2015.pdf
IFLA (2015). IFLA strategic plan 2016-2021. https://www.ifla.org/files/assets/hq/gb/strategic-plan/2016-2021.pdf
IFLA
(2018). Libraries, development and the United Nations 2030 agenda. https://www.ifla.org/libraries-development
ISO
11620 (2014). Information and documentation: Library performance indicators.
Geneva: International Organization for Standardization.
ISO
16439 (2014). Information and documentation: Methods and procedures for
assessing the impact of libraries. Geneva: International
Organization for Standardization.
Ochôa, P., & Pinto, L.G (2014). Sustainability metrics in library and information
services: A quality management framework. In Proceedings of the 2014
Conference, Helsinki, Finland. Second plenary session (2014, June 3).
Retrieved 30 May 2019 from http://docs.lib.purdue.edu/iatul/2014/plenaries/5
Ochôa, P., & Pinto, L.G. (2015). Sustentabilidade e
medição de impactos em organizações: O papel dosindicadores de literacia
mediática, comunicação e cidadania. In S. Pereira, & M. Toscano (Eds.), Literacia,
Media e Cidadania. Livro de atas do 3. Congresso, (pp. 408-420). Braga: CECS. Retrieved from http://www.lasics.uminho.pt/ojs/index.php/cecs_ebooks/article/viewFile/2252/2169
Ochôa, P., & Pinto, L.G. (2019). Agenda 2030 e o
campo de intervenção da Ciência de Informação: Dinâmicas de aprendizagem,
envolvimento e desenvolvimento de competências. EDICIC 2019. Paper presented at EDICIC, Barcelona.
Retrieved from http://eprints.rclis.org/38546/
Pinto, L.G., & Ochôa, P. (2018). Alinhamento e
evidências do contributo para o desenvolvimento sustentável: Guia para
bibliotecas. June 2018. Retrieved from https://www.researchgate.net/publication/326581158_Alinhamento_e_Evidencias_do_Contributo_para_o_Desenvolvimento_Sustentavel_Guia_para_Bibliotecas
Pinto, L. G.,
& Ochôa, P. (2018a). Public libraries'
contribution to Sustainable Development Goals: Gathering evidences and
evaluating practices. In P. Hauke, M. Charney, & H. Sahavirta (Eds.), Going Green: Implementing sustainable
strategies in libraries around the world (pp.46-59). Berlin/Boston: De
Gruyer.
Sustainable Development Solutions Network (2015).
Indicators and a monitoring framework for the sustainable development goals:
Launching a data revolution for the SDGs.
https://irp-cdn.multiscreensite.com/be6d1d56/files/uploaded/150612-FINAL-SDSN-Indicator-Report1
United Nations (2015).
Transforming our world: The 2030 agenda for sustainable development. https://sustainabledevelopment.un.org/content/documents/21252030%20Agenda%20for%20Sustainable%20Development%20web.pdf