Evidence Summary

The 360-Degree Temporal Benefits Model Reimagines Value-Based Assessment of User-Centred Design Services

A Review of:

Kautonen, H., & Nieminen, M. (2018). Conceptualising benefits of user-centred design for digital library services. LIBER Quarterly, 28(1), 1-34. https://doi.org/10.18352/lq.10231

Reviewed by:

Melissa Goertzen

Consultant and Information Manager

Halifax, Nova Scotia, Canada

Email: goertzen.melissa@gmail.com

Received: 3 June 2018 Accepted: 12 Oct. 2018

2018 Goertzen. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

DOI: 10.18438/eblip29473

Abstract

Objectives – The study has two central objectives: to examine the conceptual elements of evaluating and managing user-centred design (UCD) performance in library settings; and to propose a new framework, the 360-Degree Temporal Benefits Model (360°TB Model), that assesses value-based evaluation of UCD performance in libraries.

Design – Data collection and analysis were conducted through literature reviews, case studies, semi-structured interviews, questionnaires, and reviews of digital library service documents.

Setting – Two digital library service environments in Finland that use UCD approaches: one located at the National Digital Library and the other at a medium-sized special library.

Subjects – There were 17 participants representing internal and external stakeholder groups such as digital service designers, end-users, and consumer organizations.

Method – Through a literature review, the authors studied several topics related to UCD services including digital services, design management, public value frameworks, and services. They examined literature from two theoretical perspectives: 1) performance management, which explains why and how performance evaluation is necessary for public services, and 2) temporality, the concept of time in relation to service provision. This lens allowed the authors to identify existing knowledge gaps in professional literature and define key concepts. The literature review informed the framework for the 360°TB Model.

Two digital library settings tested the model and served as case studies in the paper. Data collection activities in this phase included reviews of existing project documentation and semi-structured interviews with stakeholders, at which time participants were also asked to complete an online questionnaire. The authors recorded and transcribed the interviews and combined these results with comments derived from questionnaires. Finally, participants received the data collected from their interview sessions and were asked to review and validate their answers.

Main Results – The most significant result is the development of the 360°TB Model. The framework combines three components to evaluate UCD design: the identification of stakeholders; the benefits of UCD services; and the temporal phases (e.g., process-time, use time, and future service provisions) of UCD design efforts and outcomes. The authors summarize the relationship between the components of the framework as follows: “a Stakeholder anticipates Benefits of the design in different Phases” (p. 8).

Regarding the case studies, the authors captured a range of diverse opinions through semi-structured interviews and questionnaires. Participants in Case 1 selected a range of benefits and there was little consistency in responses. However, two-thirds of participants in Case 2 selected quality of services as the most desirable benefit of UCD, while the remaining one-third selected options such as process time and societal problem solving.

The participants stated that the 360°TB Model provided authority in matters of design goals. It was challenging to capture temporality in design performance because it is not easy to specify goals or state the anticipated benefits of design activities in library settings. This is because the impact of design is indirect and cannot be easily quantified or isolated from the larger context of the library environment. The model provides a method to justify managerial choices regarding UCD and frame service changes around phases of development (e.g., process-time, use-time, and future service provisions).

Conclusion – The 360°TB Model pushes assessment activities beyond organization-centric evaluations and into intra-organizational and polycentric perspectives. It reaches beyond the boundaries of the institution to capture diverse viewpoints and service needs of external stakeholders. Finally, the 360°TB Model bridges the theoretical gap between Public Value frameworks and real-world information environments through the use of three key concepts: stakeholders, benefits, and phases.

Commentary

For decades, librarians have experimented with and used performance indicators that provide evidence for the quality of library services. Many of these assessments justify the benefits of UCD activities by accounting for the impact and outcome of services (Best, 2010; Rosenberg, 2004; Wiebe, 2010). However, the profession has not developed a framework that evaluates the act of designing services. In order to fill this gap, the authors developed a study around one central question: what elements are essential when evaluating and managing UCD performance in libraries?

The paper covers the process of conceptualizing, developing, and testing the 360°TB Model, which is unique because it takes a 360° view of stakeholders’ opinions, perspectives, and needs. Essentially, the model captures the context surrounding UCD activities in order to support successful design plans in the present and allows services to evolve with the future needs of internal and external stakeholders. This characteristic is the framework’s greatest strength; by examining all stakeholder groups at a high level, UCD designers can identify conflicting interests or goals while services are still on the drawing board. Once the values of the user community are identified, librarians can adjust service models or locate opportunities for stakeholder buy-in before services are released to the public. Through this form of evaluation, libraries engage in proactive and strategic service management.

Based on the results of the case studies, the authors conclude that the 360°TB Model is not a mature tool for practitioners. Going forward, they wish to develop the framework and transform it into a practical tool that supports UCD service design. There are two areas that the authors did not account for that would strengthen the model in future development phases. The first is to account for the limitations of UCD design alongside the anticipated benefits. Since the framework predicts service evolution from the point of creation through future iterations, accounting for shortcomings places librarians in a proactive position: counteract the shortcomings or work with stakeholders to locate solutions. The second is to define the metrics and data sources that are appropriate for use with the 360°TB Model. Without this definition, it is difficult to standardize assessment practices and compare results of UCD services within the same institution or across the library profession.

Since the 360°TB Model is still in development, it would have been beneficial to librarians to learn about recruitment methods, interview questions, or the structure of the questionnaire in an appendix. As the paper currently stands, it would be difficult for others in the profession to replicate the study or explore how it can be applied to UCD service development at their institution.

Overall, the authors developed an interesting assessment framework that has great potential and fills an observed gap in professional knowledge: the evaluation of the act of service design. In the future, it would be of value to the profession if the authors continue to publish papers about the development of the 360°TB Model and discuss their methodology in greater detail.

References

Best, K. (2010). The fundamentals of design management. Lausanne, Switzerland: AVA Publishing.

Rosenberg, D. (2004). The myths of usability ROI. Interactions, 11(5), 22-29. https://doi.org/10.1145/1015530.1015541

Wiebe, E. (2010). Temporal sensemaking: Managers’ use of time to frame organizational change. In T. Hernes & S. Maitlis (Eds.), Process, sensemaking, and organizing (pp. 213-241). Oxford, UK: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199594566.003.0011