Design for data ethics: using service design approaches to operationalize ethical principles on four projects rsta.royalsocietypublishing.org Opinion piece Cite this article: Drew C. 2018 Design for data ethics: using service design approaches to operationalize ethical principles on four projects. Phil. Trans.R.Soc.A 376: 20170353. http://dx.doi.org/10.1098/rsta.2017.0353 Accepted: 6 July 2018 One contribution of 13 to a discussion meeting issue ‘The growing ubiquity of algorithms in society: implications, impacts and innovations’. Subject Areas: artificial intelligence Keywords: design, service design, user-centred design Author for correspondence: Cat Drew e-mail: cat.drew@uscreates.com Design for data ethics: using service design approaches to operationalize ethical principles on four projects Cat Drew Uscreates, Studio 103-4, Vox Studios, 1–45 Durham Street, London SE11 5JH, UK CD, 0000-0001-8485-7930 Ethical frameworks provide helpful guidance about what you should—and should not—do in relation to data projects. But they do not provide definitive yes/no answers about what an ethical data project is or is not. Indeed, research (Ipsos-MORI 2015 Public dialogue into the ethics of data science in government) conducted for the initial development of the Government’s Data Ethics Framework shows that the public does not hold any clear red lines; rather, they make nuanced assessments based on a number of variables, including public good and privacy. Ethical frameworks provide a list of these variables to consider in shaping the form of the work. Some are now starting to provide more practical tools and guidance to reshape data projects and push it along those variables into a more ethical space. Alongside technical tools, service design approaches can help enhance the degree to which a data project is ethical, and provides a toolkit for data scientists, analysts and policymakers to take projects from ‘what should we do’ to ‘how can we do it’. This paper sets out the emergence of data science ethical frameworks within the context of the use of data for social good, and— with the recent release of the updated UK Government Data Ethics Framework—shows the recognition more practical guidance needs to be provided. The author then argues that service design approaches provide a helpful ‘wrap around’ for data projects, and draws on experience in using service design tools on four projects, as well as wider examples. 2018 The Author(s) Published by the Royal Society. All rights reserved. D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 http://crossmark.crossref.org/dialog/?doi=10.1098/rsta.2017.0353&domain=pdf&date_stamp=2018-08-06 http://dx.doi.org/10.1098/rsta/376/2128 mailto:cat.drew@uscreates.com http://orcid.org/0000-0001-8485-7930 2 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ This article is part of a discussion meeting issue ‘The growing ubiquity of algorithms in society: implications, impacts and innovations’. 1. The emergence of data ethics frameworks Data science is a field which, while initially concerned with innovation and profits for companies, is increasingly concerned with social good (as referenced in paragraphs below). Governments, whose role it is to improve social outcomes for its citizens, are increasingly applying data science techniques to the development and implementation of public policy and services. With the ability to process larger and more complex datasets than ever before, data science can provide better insights for policymakers, make services more tailored and efficient, and democratize information by making it available to the public. In the UK, the Government’s Transformation Strategy [1] sets out how it will make better use of data by opening up, sharing and storing data, as well as improving its analytical skills and capabilities and the Government’s Industrial Strategy [2] provided £725 m of investment in innovation, including through a specific artificial intelligence (AI) sector deal £406 m in STEM training. This built on the progress of its open data agenda, which opened up large numbers of datasets for people to use, applying data science, other analytical methods or simply visualizing data to democratize information [3]. At the same time, there has been an increase in the number of private sector organizations working on data for social good by bringing together data scientists to do pro bono work for charities during weekend hackathons (e.g. DataKind, which has chapters in the UK and USA1). Organizations have also created fellowships where recent data science graduates are paired with public sector or charitable organizations to work on a social challenge, organized by non- profit arms of private sector technology firms on their own (e.g. IBM2) or with universities (e.g. Microsoft and the universities of British Columbia and Washington;3 the Data Science for Social Good programme at the University of Chicago, and this year in Europe4). Conferences have also been organized to move forward thinking, debate and practice around data for social good (e.g. Bloomberg’s Data for Good Exchange5). Ensuring that data science is ethical is an implicit part of using data science for social good. There is a large field of academic literature on data ethics [4–6] and about ethics within the field of innovation [7]. Within the field of data ethics, there is a particular focus on privacy, trust and consent, either directly relating to trust in technology [8] or within society more generally [9], algorithmic bias and discrimination [10,11], derived data [12], anonymity [13] and around the topic of health data which are seen as particularly sensitive [14]. Governments, think tanks and institutions have also published widely on emerging issues of data ethics in the UK within the context of its potential for delivering better social outcomes [15–18] and abroad [19,20]. Over the last 10 years, governments, not-for-profits and commercial organizations have been developing ethics frameworks (alongside other methods of ethical oversight, such as advisory bodies and organizations) to regulate how data science is used, provide confidence to those working with data, and reassure the public. As with all new technologies, a lack of confidence might mean that the opportunities presented by data science for social good are not seized (the ‘opportunity cost’), and a lack of awareness about ethical issues means that someone does experiment but in a way that makes the public feel uneasy and jeopardizes the technology’s wider use. In 2015, the UK Government Data Science Partnership started the process to create the UK Government’s/world’s first ethical framework for data science in government [21]. This article 1 http://www.datakind.org/ 2 https://www.ibm.com/ibm/responsibility/initiatives/IBMSocialGoodFellowship.html 3 https://dsi.ubc.ca/2018-dssg-program 4 https://dssg.uchicago.edu/ 5 https://www.techatbloomberg.com/blog/mining-data-public-good/ D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 http://www.datakind.org/ https://www.ibm.com/ibm/responsibility/initiatives/IBMSocialGoodFellowship.html https://dsi.ubc.ca/2018-dssg-program https://dssg.uchicago.edu/ https://www.techatbloomberg.com/blog/mining-data-public-good/ 3 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ sets out how it was developed through an open, evidence-based and service design/user-centred approach process, including doing user research internally to understand the ‘journey’ of a data science project and what data scientists and policymakers needed to make decisions about the work at different stages; running large-scale public dialogue, including deliberative workshops with 100+ people, an experiment conjoint survey to understand people’s latent opinions, and an online engagement tool (www.datadilemmas.com); and convening a series of expert roundtables and workshops, which included the Minister for the Cabinet Office’s Data Board. The framework was published in beta form in May 2016. It has since been tested, iterated and improved and was re-published in June 2018 [22]. It includes a set of principles, more detailed guidance and a workbook to support policymakers, operational staff and data scientists to work together to ensure they are using data appropriately. The principles are: (i) Start with clear user need and public benefit (ii) Be aware of relevant legislation and codes of practice (iii) Use data that are proportionate to the user need (iv) Understand the limitations of the data (v) Use robust practices and work within your skill set (vi) Make your work transparent and be accountable (vii) Embed data use responsibly. It is not just central government that has been using the framework. Feedback from organizations such as the Future Cities Catapult and local councils indicates that the framework is helping to shape ethical data science projects that are delivering benefits for the public. For example, Essex County Council has been using the framework to guide its work to link personal pseudonymized data to identify people at risk and target early intervention support [23]. There are other ethical codes or principles that cover data projects (e.g. the Open Data Institute’s Data Ethics Canvas6 or Defra’s data principles7, ethical technology (e.g. Ind.ie’s Ethical Design8) and ethical AI (e.g. Google DeepMind’s Ethics & Society principles and advisory board;9 the Association of Computer Machinery’s Code of Ethics10 and the IEEE’s Code of Ethics11). This list is not exhaustive, and even from a small sample it is clear that they differ in terms of the level of high-level principle versus practical advice; the focus on the outcome versus the process; and concern about data and privacy versus technology and harm. They all however start with a focus on using data and technology for good and are adding to the collective ethical practice around working with innovative uses of data and AI. 2. Iterating and improving ethical frameworks The initial Data Ethics Framework was published in beta form as it needs iteration: as technology advances and new ethical issues emerge, and as people use it and give feedback on what was useful and what could be improved. This section sets out two areas for improvement: how the updated framework has addressed them, and where there are still gaps in which a more coherent service design approach/wraparound could resolve (§3). 6 https://theodi.org/the-data-ethics-canvas 7 https://defradigital.blog.gov.uk/2017/04/04/defras-data-principles/ 8 https://ind.ie/ethical-design/ 9 https://deepmind.com/blog/why-we-launched-deepmind-ethics-society/ 10 https://ethics.acm.org/code-of-ethics/ 11 https://www.ieee.org/about/corporate/governance/p7-8.html D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 www.datadilemmas.com https://theodi.org/the-data-ethics-canvas https://defradigital.blog.gov.uk/2017/04/04/defras-data-principles/ https://ind.ie/ethical-design/ https://deepmind.com/blog/why-we-launched-deepmind-ethics-society/ https://ethics.acm.org/code-of-ethics/ https://www.ieee.org/about/corporate/governance/p7-8.html 4 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ (a) Need for wider communications and focus on public value Research with the public as part of the participatory approach to develop the initial ethics framework revealed how people make decisions about the acceptability of data projects [24]. It is a two-step process. First, participants engaged in a broad values assessment of the overall public benefit and the opportunity of using data science as opposed to other methods. They needed to support the overall policy objective and intended outcome (the user need and/or public benefit). In addition, they needed to understand the opportunities that data can provide, to understand what is possible and how data science could work better than traditional methods (which can be influenced by ideological views and values around technology, trust and reduced human interactions). If these conditions were not satisfied, or contentious, projects were often dismissed outright, before any consideration of how a data approach could work. This is a challenge when there is very little awareness of data science (only 15% of those surveyed12 had heard ‘a great deal’ or ‘a fair amount’ about it), despite people experiencing (consciously or not) its application every day. Once the public benefit and value of data science had been established, acceptance of opportunities for data science was based on a nuanced risk assessment of the entire project. This assessment balanced three further considerations: Is there a privacy concern? Is the approach effective in achieving the intended policy goal? What are the consequences of potential error, either intended or unintended? Therefore, there is a need to raise awareness about what data science is and can be, to make sure that there is a clear user need and/or public benefit, and to ensure the data used it proportionate to that user need. The updated framework reiterates the latter two points (relating to user needs) in its first and third principle, and provides detailed guidance on user needs, how to research them, and how to write them. It refers to guidance provided by the Government Digital Service, which since its introduction has impressively increased focus across all departments on user needs. However, the government’s use of data does not and cannot limit itself to focusing solely on individual needs or rights. There are many uses of data—enshrined in the Data Protection Act13 and now the GDPR14—where public good or a collective need trump individual user need, and individuals do not have rights over their data to create data ‘holes’. For example, to deliver justice effectively, the government needs to hold data on those committing crimes; to fund public services, the government needs to hold data on everyone’s tax records, and to conduct effective research into health conditions, identify groups of people most at health risk and direct interventions to support them, the government needs to use—at the very least—a representative (and properly pseudonymized) sample of people’s health data. Broadening out from data, others have argued that we need to think about collective systems’ needs, rather than just individual ones [25]. The role of the policymaker should be to absolutely understand individual needs and balance them with the collective, but as is—with guidance around user needs coming largely from understanding individuals’ accessing digital services, rather than living in complex places among others—this could be too simplified a method of understanding. In terms of raising public awareness of data science, the framework itself provides little guidance as it is intended to ensure appropriate use of data on individual projects. There are wider efforts to raise public understanding of data science and AI. The Royal Society’s 2017 literature review on public engagement [26] data builds on the initial research for the Data Ethics Framework and shows there is a spectrum of views and understanding, and the Society has a number of resources to explain AI (for example easily understable infographics and quizzes15). 12 An online survey of 2003 people aged 16–75 in Great Britain was conducted between 24 February and 7 March 2016 using the Ipsos MORI Access Panel. 13 https://www.gov.uk/data-protection 14 https://www.eugdpr.org 15 https://royalsociety.org/topics-policy/projects/machine-learning/what-is-machine-learning-infographic/ D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://www.gov.uk/data-protection https://www.eugdpr.org https://royalsociety.org/topics-policy/projects/machine-learning/what-is-machine-learning-infographic/ 5 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ The Partnership on Artificial Intelligence to Benefit People and Society16 is a global network of private and non-for-profit organizations promoting awareness and understanding about AI, and The Royal Statistical Society’s 2016 Big Data Manifesto prefers to call for more awareness through data literacy. While these are important efforts in shifting the overall public awareness on data science and AI, the author argues that it is also important to do so project by project, as part of general stakeholder or public involvement or engagement on the policy area that data science project is contributing to. (b) Need for a toolkit/methods to enact the principles and link to impact delivery Feedback from data scientists and policymakers using the framework indicated that it is a useful tool to raise awareness of the issues and variables which make a project ethical or not (for example, being as open as possible, having the minimum intrusion necessary), but further tools might be needed to implement this and to change the shape of the project. For example, the principle might be to be as open and accountable as possible, but a data scientist might not know how to share the data and codes they have used, or to create models to ensure algorithmic accountability. The updated framework provides detailed guidance for each of the principles so that policymakers, data scientists and operational staff can work together and operationalize the framework. However, the author argues that it is still fairly bounded around a data science project: it assumes that the data already exists and has been collected,17 and while it considers the project outcome, does not provide guidance on what form an activity that would use the data science insight, prediction or automation to achieve the outcome would take. Public acceptance of data projects may be as much based around the form of the action/decision taken as a result of the data science, as the rationale or problem which instigated it. For example, within the research, in response to hypothetical scenarios, some respondents could not understand why the government would need to understand a population’s sexual preferences (for example, to assess any discrimination in access to services or outcomes), and therefore did not think data science was acceptable here. Responding to another scenario—using data science to understand people’s eating habits to target healthy behaviour advice—some liberally minded respondents disagreed with the use of data science, because they did not feel that it was the government’s place to intervene in people’s lives in this way [24]. Therefore, data scientists and policymakers need to work together and think through public acceptance of both generating the insight and taking the specific form of action. Public engagement is better conducted through the context of the policy challenge, with both data science and delivery mechanisms up for discussion as two of many possible ways of addressing the challenge (rather than starting with the data work). 3. The value of a service design approach to ‘wrap around’ a data project A service design approach (to be defined in this section) could be considered useful here. There is much discussion in the literature about the value of design, rigorously summarized by Kimbell [28]. These range from design as finding form of new ideas [29], design as leading to a changed state [30], design as an approach to dealing with complex or ‘wicked’ problems [31,32] and ‘design thinking’ as a more inclusive mindset for designers and non-designers alike to look at problems and solutions from a human perspective [33]. ‘Service design’ is a practice that has emerged over the last 30 years from different fields (for example, management, engineering, product or technology design, graphic design). Kimbell’s [28] article goes on to conceptualize these in four different ways. One of those, ‘designing for services’ is defined as an enquiry-based practice which does not predetermine the form of the solution (e.g. goods or services). ‘Talking of designing for services rather than designing services recognizes that what is being designed 16 https://www.partnershiponai.org/ 17 Other writers or frameworks more explicitly consider data generation and collection, e.g. Ng [27]. D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://www.partnershiponai.org/ 6 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ is not an end result, but rather a platform for action with which diverse actors will engage over time’. It is this type of ‘service design’ that has been pioneered by organizations such as Participle, ThinkPublic and Uscreates, and has recently been adopted by governments to make Government services and policies more user-centred (for example, through MindLab in Denmark and Policy Lab in the UK). There is no one fixed method for this type of service design, but practices (Uscreates principles18, Kimbell [34], Design Council Double Diamond,19 GDS Design Standards20), commonly include the following design approaches or principles: — It understands the experience of the ‘service user’, or the person experiencing a policy, and reframes problems according to their need, rather than purely the ‘organizational’ need of the body delivering it. — It co-designs and prototypes solutions, creating buy-in from (and providing early training for) those who will need to deliver it, and tests out solutions in small-scale scenarios, spotting errors early on and making improvements, thus avoiding costly waste and increasing the chances of effectiveness. It is this definition that the author will use for ‘service design approaches’ throughout the paper. The discussion between user need and organizational need is an important one. As service design has been applied to policymaking and increasingly complex social challenges, it has had to find a way to marry these up. The overall focus for governments (national and local) needs to be on the overall social outcome. A sole focus on individual needs may not achieve such an outcome. For example, the ‘user need’ of someone who has offended might not be to be locked away from his/her family, but it might achieve the social outcome of making victims of crime feel safe. In order to achieve an outcome, the government needs to consider its politics, its delivery resources, organizations and people. Figure 1 shows how a design approach (here, advocated by IDEO) emphasizes human understanding. Whereas previously designing services might have started with politics (viability) or organizational capacity to deliver (feasibility), a service design approach asks it to start with people (desirability). Service design uses a number of tools and techniques to help design better services, which can be usefully applied to a data context. For example, user-journey mapping plots the experience of someone using a service from them becoming aware of a service, through to joining it, using it, and leaving it. It could be considered a helpful way of thinking about the activities that constitute a design process to provide a representation of what a data project is. Therefore, if this service design conceptualization method is applied to a data project, the stages might be ‘awareness of data opportunity’; ‘data generation and collection’; ‘data analysis’; ‘use of insight and action’ and ‘monitoring and accountability’. Figure 2 shows a map of the stages of the data journey, with the updated Government Data Ethics principles plotted underneath. As discussed in §2, in the alpha form, the principles did not stretch into insight and action, and monitoring and accountability, whereas they now do. However, as the author has also argued, more could be done to consider the form of the action taken as a result of the data insight and ensuring this has social impact, as well as increasing data awareness in the first place. The author will demonstrate how service design can support with this in the next section. 18 https://www.uscreates.com/our-thinking/ 19 Design Council. 20 https://www.gov.uk/service-manual D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://www.uscreates.com/our-thinking/ https://www.gov.uk/service-manual 7 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ viable politics desirable human feasible organizational capability not here start here Figure 1. Adaptation from IDEO [35] Human-centred design toolkit, IDEO. (Online version in colour.) 4. Reflections on the personal use of service design approaches in four data projects At Uscreates, a service design agency focusing on health, wellbeing and the public sector, we have been applying service design approaches to data projects to ensure that they deliver impact. We have been working with — The Open Data Institute, to explore with two local authorities how open data can drive public services [36]. — In Kent, open energy data have been used to help identify elderly households that might be in need of insulation services to prevent health problems. — In Doncaster, open data about the number and type of education and training institutions, the career advice they offer and the variance between learners’ destinations (into education, employment, training or NEET) between the school year end and eight months later has been used to democratize careers advice for learners. — nuron—a tech SME which has created a fibre-optic technology to provide real-time waste water monitoring to understand how different functions in water companies would use these data, and how the interface is best presented to allow them to take the decisions and actions needed. — The EU Joint Research Centre’s Blockchain4EU21 project, to design and prototype speculative uses of blockchain in order to engage the public and policymakers in how it could be used beyond the financial industry. We have been using service design approaches to make these projects more ethical by 21 https://blogs.ec.europa.eu/eupolicylab/portfolios/blockchain4eu/ D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://blogs.ec.europa.eu/eupolicylab/portfolios/blockchain4eu/ 8 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ data generation and collection data awareness 1. Start with clear user needs and public benefit 2. Be aware of relevant legislation and codes of practice 7. Embed data use responsibly 3. Use data that is proportionate to the user need 6. Make your work transparent and be accountable 4. Understand the limitations of the data 5. Use robust practices and work within your skill set Data Ethics Framework principles data science insight and action monitoring and accountability Figure 2. Plotting Data Ethics Framework against a ‘data project journey’. — Raising awareness about and communicating what data can do, for both those involved in delivering the projects (data scientists, policymakers) and those affected by it (citizens). — Ensuring that user need and/or public benefit sits at the heart of the work. — Designing good experiences throughout the data project (from data collection to insight use and round again), and prototyping, testing and iterating in order to increase its impact (and deliver the public benefit on which the public base their ethical acceptance). Figure 3 shows how design (in orange) can ‘wrap around’ and complement more technical work to ensure data projects are ethical. 5. Reflectionsonpersonaluseofservicedesignapproachesinfourdataprojects (a) Service design as a method of communicating and building understanding of data opportunities As discussed above, one of the key insights from the initial research that informed the creation of the beta Data Ethics Framework is that before the public can even make an ethical consideration about a project, they first need to understand what data science/open data are and what can be done with it. Service design takes a collaborative co-design approach to work, involving users, frontline staff and policymakers throughout. This achieves buy-in, levers in the need, knowledge and feedback of those who will experience the data’s application, and builds capability in them to do so. However, this will also involve non-data experts, so it is important to explain it to them. Service design approaches could be considered helpful in four ways through this co-design process and are visualized in figure 4: — Graphic design can visualize and simplify complex messages and information. Well-ordered information with reinforcing images can help non-data experts understand what data science is. For example, the datadilemmas.com opening pages visualized web-scraping D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 9 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ data generation and collection data awareness 1. Start with clear user needs and public benefit Raise awareness and communicate data possibilities and opportunities Consider and prototype the experience: generating data >Build trust >Nudge >Empower Consider and prototype the experience: using insight and taking action as a result >Data value chain >Co-design with frontline staff >Prototype and iterate 2. Be aware of relevant legislation and codes of practice 7. Embed data use responsibly 3. Use data that is proportionate to the user need 6. Make your work transparent and be accountable 4. Understand the limitations of the data 5. Use robust practices and work within your skill set Data Ethics Framework principles Design led approach data science insight and action monitoring and accountability Figure 3. Showing where service design approaches (orange) can enhance more technical guidance given within the Data Ethics Framework in enhancing the degree to which a data project is considered ethical. as a human-like computer (with a face) reading information from another computer, or big data being made up of icons representing numbers, photos, postcodes and words. — Lateral inspiration to show how it works in other sectors. A common innovation technique is to look out ‘horizontally’ at what other sectors are doing and apply it to your own industry. In place-based workshops in Doncaster where we were trying to increase understanding of open data within careers advice, we provided visual examples of how open data were being used to identify and create new bus routes or change behaviour so that residents put their bins out on time. — Provocations to push people into a transformative space. In co-design sessions, the challenge can often be around helping people to step out of the constraints of the present and imagine a different future. Futures-thinking techniques can be effective here. By asking people to imagine something in the distant future, a space in which to imagine alternatives can be opened up (without the worry of how such alternatives would work tomorrow). In the nuron co-design workshops, we gave participants provocative speculations such as ‘what if Google owned a wastewater business?’ and ‘what if a hurricane meant it was too dangerous for teams to go out and respond?’ to help them get into a different creative space. — Prototypes make things tangible. Prototypes—which can be a two-dimensional sketches, digital wireframe or physical three-dimensional model—have been used to get people to think through how they would interact with data use. A citizen might not need to see the data analysis itself, but would instead interact with an interface, service or object that is driven by the data. Prototypes can show people how data will be applied and, as discussed earlier, may be as important to how people perceive the ethics of the data project as the data collection and analysis itself. In the Blockchain project, we created a three-dimensional model of a blood transfusion drone enabled by blockchain together with provocation posters/scenarios to engage EU civil servants in how they might be able to use it in their policy areas. In the Doncaster project, we created wireframes of a digital platform showing the full breadth of institution options (sixth forms in schools D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 10 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ Figure 4. Four ways in which service design approaches can raise awareness and communicate of the opportunities of data to non-experts. Fromtop left: online data ethicsgame www.datadilemmas.com; case study ofCitymapper NightRider opendata example; prototype of a blood transfusion drone; provocation workshop card for sewage water monitoring technology. (Online version in colour.) and colleges, as well as apprenticeships) together with travel to learn time, which we tested with learners, using their feedback to improve the concept. Our experiences—as outlined above—demonstrate how necessary it is to continue to do this throughout the project. Data, open data and data science are all difficult concepts and co-design can help explain, re-explain and check heightened understanding of what is possible, helping people to move to a more nuanced discussion of how to make the work ethical. (b) Design as a way to focus on user need and/or public good Service design usually starts with a ‘discovery’ period of qualitative research into people’s lived experience of the social outcome the project is trying to achieve, as well as identifying user needs in achieving those outcomes. But data projects can often start with the data (and well- meaningfully, with the aim for social good). At worst, the process could involve simply playing with a dataset to see what can be found, without any purpose other than to work with the data. At best, the process would be motivated by a clear public benefit, and would involve a safe, controlled sandpit to experiment with the data to ensure that the level of intrusion is always justified by the user need. In line with Kimbell’s definition, the majority of service design projects that we work on are solution agnostic: establishing user need first and then creating whatever solutions are necessary to meet that. Service design projects from an engineering or technology field often start with a technology (or data) solution in mind and try to establish the user need and how the technology can support it [37]. There are advantages and disadvantages to both. Solution-agnostic projects create a real focus on user need, but the solution is either not always immediately feasible or even D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 www.datadilemmas.com 11 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ data data scan data feasibility assessment and value/openness segmentation co-design workshop data experiments and technical next steps concept prototypeslearner/advisor interviews analysis and opportunity briefs system needs user needs business case for systems board so lu ti o n deliverdevelop vision definition definediscover p ro bu lm co-design workshop co-design workshop system partner workshop and value exchange system partner interviews Figure 5. A ‘triple-track’ Double Diamond approach: simultaneously discovering, defining, developing and delivering around data opportunity, user need and public good/systems need. (Online version in colour.) known (which is why we often use lateral inspiration). Solution-determined projects will be able to move into delivery more quickly, but risk shoehorning a solution to fit a user need. The nuron and Blockchain projects both started with a technology (fibre-optic real-time waste water monitoring, or the blockchain for delivery chains) and our work focused on establishing user need through interviews with various roles within water companies, or through more hypothetical personas at a two-day workshop of policy, legal and technology experts in Brussels. Our open data work with Kent and Doncaster pursued a triple-track approach (outlined in figure 5) to discovery, scanning possible data sources to identify data opportunities at the same time as conducting interviews with learners, advisors and teachers (in the case of Doncaster) or health service commissioners (in Kent) to establish user need. (c) Thinking through and prototyping the experience of those involved If we return to thinking about the data project as a journey, we can think through how people (the public or people experiencing the problem, or policymakers or frontline staff trying to solve it) interact with the project, for example, generating data or using the insight. In service design, these moments are called ‘touchpoints’ and improving the experience of them can improve the likelihood that the service will achieve the intended impact. Service design approaches could be considered useful here, and are described below and visualized in figure 6. (i) Providing/generating data The law sets out clearly how organizations can legally process personal data, including the circumstances in which people can expect to be asked to provide their consent [38]. Article 4 of the GDPR clarifies this, including the need for consent to require an explicit, affirmative action. (Other conditions for processing personal data include meeting legal obligations, an individual’s vital interests (i.e. saving their life) and necessary obligations (e.g. fulfilling a contract); to deliver justice, government and other public functions; and to meet an organization’s legitimate interests, reflecting the points made earlier about data being a public good rather than a solely individual one.) This is reflected in the Ethical Framework under the principle ‘be be aware of relevant legislation and codes of practice. However, consent—as one of the conditions on which personal data processing can be based— is important, and as the GDPR came into force, many organizations have been considering how they can not only be compliant but go beyond a standard question with a tick-box answer. The act of generating data and providing consent could be seen as a ‘touchpoint’ between an organization D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 12 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................Figure 6. Top row: three ways design has improved the experience of generating data: Projects by If’s data permissions catalogue; Mappiness research and behaviour change tool; Alyssa Milano’s reignition of the #MeToo hashtag on 15 October 2017. Bottom row: three ways design has improved and prototyped the experience of using data: the data value chain used inthenuronproject;personacardstothinkthroughhowpeoplewillusedata;wireframestotestdata interface. (Onlineversion in colour.) and citizen that could achieve wider social outcomes than ensuring legal compliance. For example, it could be a useful way of — Building trust. There are many ways of recording explicit consent. Projects by If has drawn together a highly visual catalogue of these methods.22 Setting out clearly at the point of data collection how consent will be given and how the data are going to be used for social good should be considered an important way to raise public acceptance. Another ODI open data-driven and service design project—this time led by Waltham Forest— investigated how to provide free wifi as a means to collect GPS data to understand residents’ use of cultural institutions across the borough, with the aim of ensuring that culture was being taken up equally by all ethnic and socio-economic groups [39]. They created consent pages as people were logging in, and produced posters which clearly said how the data would be captured and used — Nudging behaviour. Service design often uses behaviour change theories to underpin how a service can lead to changes in user behaviour to meet a social outcome. Self-awareness is seen as an important behaviour change principle. Providing people with data they have generated—either consciously (e.g. responding to a question) or unconsciously (e.g. data generated through using Fitbits or accessing apps)—has the potential to make them aware of their behaviour and nudge them to change it. For example, Mappiness is a research tool that collects data on wellbeing at work, and through doing so is aiming to generate the largest dataset on work-based health at the same time as activating behaviour change in participants. — Engendering a sense of collective empowerment. Consider the experiences of reporting sexual assault to the police and writing a #MeToo social media post. Clearly these are different 22 https://catalogue.projectsbyif.com/ D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://catalogue.projectsbyif.com/ 13 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ reporting mechanisms which lead to different paths of action and legal remedies, but the former can often be a disempowering experience, leading to low prosecution rates by the criminal justice system, while the latter has been reported as a more empowering and collective experience, leading societal conversations about the need for change. (ii) Using data/insight As argued previously, the achievement of the user need and/or public benefit of the data project and the course of action taken as a result of the insight is an important consideration for deciding whether or not it could be considered ethical. Service design can help data scientists and policymakers think through how the data or insight will be used to achieve the intended impact. — Determine what type of data/information you provide. The value of a data project should be seen in the action—a decision, a change in behaviour, etc.—that will be taken as a result. This action could be taken by a policymaker, a frontline member of staff, a service user, or a citizen. Different people will need the data to be presented in different ways. In our nuron work, we called this the ‘data value chain’ [40]. Some might want to see it presented as raw data, others as information grouped into charts and graphs, others as purely insight, others as knowledge about the overall issue. Our work with nuron identified a number of ‘personas’ (pen portraits of people in different roles across a water network) and designed interfaces that provided them with data/information/insight in different ways. For example, a planning analyst might want to access the data to perform secondary analysis (raw data); a planning manager might want to know when water levels were rising to a critical level (information); a customer service manager might want to simply know when and where to send a repair team (insight); a customer might want knowledge that their water company’s track record of service was excellent (knowledge). — Co-design with frontline staff and users. As well as with users, we have found it extremely valuable to co-design how the insight will be used with those who might be delivering the service. In discussions around data (e.g. [41]), there is some concern being expressed regarding predictive modelling, computer-led decision-making and their impact on jobs—that these will automate jobs, or that open data will replace the need for intermediary advice. One of the aims of our ODI Doncaster project was to democratize careers advice, placing information directly into the hands of learners and parents. This does not equate to a reduced need for in-person careers advice. Through a co- design event, advisors helped us understand how they could also benefit from this insight to target their services, and make sure they spent more in-person time with those who were at ‘high’ and ‘mid’ level risk of falling out of employment, education or training. — Prototype and test. Service design uses prototyping as a way of mocking up or making tangible solutions in a low-cost, low-fidelity fashion firstly, in order to test ‘desirability’ (whether people accept the idea and are comfortable with it), as well as ‘feasibility’ and ‘deliverability’, spotting errors early, getting feedback from users and providers, and improving its chances of achieving the intended outcome. During our Doncaster project in conjunction with some data science work to test how it was technically possible to bring together and visualize datasets, we mocked up wireframes of what learners and advisors would see to get early feedback on the level of insight versus information they wanted (which then re-informed the data science). More speculative uses of prototyping can be used to help people imagine transformation solutions in a safe space. One of the risks around data science discussed by the government is ‘opportunity cost’ of not proceeding with work that could potentially deliver social good due to lack of confidence on the part of those working with data, or public concern. Our Blockchain project created D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 14 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ a prototype for a blood transfusion drone in order to give a tangible example of how distributed ledgers could provide a secure, automated blood transfusion service, and to create a proactive conversation about it. 6. Conclusion Ethical frameworks set out overarching principles for how to work with data in a way that the public would deem acceptable. The UK Government has recently updated its Data Ethics Framework to include more practical guidance for policymakers and data scientists on how to operationalize its seven principles, which is welcome. The author has argued that service design approaches could provide additional value in ‘wrapping itself’ around data projects, and using design approaches to — Raise awareness of and communicating of what data can do, for both those involved in delivering the projects (data scientists, policymakers) and those affected by it (citizens). — Ensure that user need and/or public benefit sits at the heart of the work. — Design good experiences throughout the data project (from data collection to insight use and round again). — Prototyping, testing and iterating in order to increase the impact of the project (and deliver the public benefit on which the public base their ethical acceptance). It will be important to test the stages of the data journey put forward in this article, as well as the ways in which a service design approach can add value and additional experiences and contributions to this hypothesis through practical applications of service design to data projects are welcome. More broadly, it requires continued connections to be made across the design, data and technology space. Data scientists and technologists need to understand users in order to humanize technology and ensure that it is ethical. Designers and those working in social innovation need to understand the potential of data, but also the ethical implications of working with it. Data and research organizations such as the Turing Institute, the Royal Academy and the Wellcome Trust have resources in place to explain AI, machine learning and data ethics to non-data experts which could be more widely used by those working in public sector services. Organizations such as Uscreates, Snook, FutureGov, IDEO and the Government Digital Service has resources on user-centred design, service design and prototyping. It is important for these elements of the data system to come together. The Ada Lovelace Institute23 has been created by a wide partnership of organizations in order convene diverse voices to build a shared understanding of the ethical questions raised by the application of data, algorithms and artificial intelligence (AI) and other collaborations between technology, ethics, data and design sectors are developing formally (e.g. through Doteveryone’s Society in the Loop24 conference) or informally (a network of women, trans and non-binary people working in the field of digital ethics and society meet for informal conversations). Data science and AI provide huge opportunities for human good. Ethics provides a framework for ensuring human acceptance. Service design approaches provide methods to ensure they are created with humans at their heart. Data accessibility. This article has no additional data. Competing interests. I declare I have no competing interests. Funding. I received no funding for this study. 23 http://www.nuffieldfoundation.org/ada-lovelace-institute 24 https://medium.com/doteveryone/society-in-the-loop-a-one-day-event-3d552ea9c029 D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 http://www.nuffieldfoundation.org/ada-lovelace-institute https://medium.com/doteveryone/society-in-the-loop-a-one-day-event-3d552ea9c029 15 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ References 1. HM Government. 2017 Government Transformation Strategy: Better Use of Data https:// www.gov.uk/government/publications/government-transformation-strategy-2017-to- 2020/government-transformation-strategy-better-use-of-data 2. HM Government. 2017 Industrial Strategy https://assets.publishing.service.gov.uk/ government/uploads/system/uploads/attachment_data/file/664563/industrial-strategy- white-paper-web-ready-version.pdf 3. HM Government. 2012 Open Data White Paper: Unleashing the Potential. See https://assets. publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/ 78946/CM8353_acc.pdf. 4. Floridi L, Taddeo M. 2016 What is data ethics? Phil. Trans. R. Soc. A 374, 20160360. (doi:10.1098/rsta.2016.0360) 5. Floridi L. 2013 The ethics of information. Oxford, UK: Oxford University Press. 6. Zwitter A. 2014 Big data ethics. Big Data Soc. 1, 2053951714559253. (doi:10.1177/205395171 4559253) 7. Stilgoe J, Owen R, Macnaghten P. 2013 Developing a framework for responsible innovation. Res. Policy 42, 1568–1580. (doi:10.1016/j.respol.2013.05.008) 8. Taddeo M. 2010 Trust in technology, a distinctive and problematic relation. Knowl. Technol. Policy 23, 283–286. (doi:10.1007/s12130-010-9113-9) 9. O’Neill O. 2002 A Question of Trust: The BBC Reith Lectures 2002. 10. Crawford K. 2013 The Hidden biases in big data. Harvard Business Review. 11. Barocas S et al. 2016 Accountable algorithms. University of Pennsylvania Law Rev. 165, 633–704. 12. Citron D, Pasquale F. The Scored Society: due process for automated predictions. Washington Law Rev. 89, 1. 13. Barocas S, Nissenbaum H. 2014 Big data’s end run around anonymity and consent. In Privacy, big data, and the public good: frameworks for engagement (eds Bender et al.), pp. 44–75. New York, NY: Cambridge University Press. 14. Wellcome Trust and Future Advocacy. 2018 AI in health: ethical, social and political challenges. 15. Royal Statistical Society. 2016 The Data Manifesto. See http://www.rss.org.uk/Images/PDF/ influencing-change/2016/RSS_Data%20Manifesto_2016_Online.pdf 16. Royal Society. 2017 Machine learning: the power and promise of computers that learn by example. 17. House of Commons Science and Technology Committee. 2016 Big data dilemma. See https:// www.parliament.uk/business/committees/committees-a-z/commons-select/science-and- technology-committee/inquiries/parliament-2015/big-data/. 18. House of Commons AI Committee. 2017 AI in the UK: ready, willing and able? 19. Executive Office of the President. 2014 Big Data: seizing opportunities, preserving values. 20. Villani C. 2018 For a meaningful AI, Parliamente de France. https://www.aiforhumanity.fr/ pdfs/MissionVillani_Report_ENG-VF.pdf. 21. Drew C. 2016 Data science ethics in government. Phil. Trans. R. Soc. A 374, 20160119. (doi:10.1098/rsta.2016.0119) 22. HM Government. 2018 Data ethics framework. See https://www.gov.uk/government/ publications/data-ethics-framework/data-ethics-framework. 23. Gates S. 2017 Updating the data science ethical framework. Government Digital Service blog. See https://gds.blog.gov.uk/2017/11/27/updating-the-data-science-ethical- framework/ (accessed 02 February 2018). 24. Ipsos-MORI. 2015 Public dialogue into the ethics of data science in government. London, UK: Ipsos- MORI. 25. Robinson C. 2017 Putting users first is not the answer to everything. Medium. See https://medium.com/doteveryone/putting-users-first-is-not-the-answer-to-everything- dd05b9f11b5 (accessed 02 February 2018). 26. Royal Society. 2017 Data governance: public engagement review. See https://royalsociety.org/~/ media/policy/projects/data-governance/data-governance-public-engagement-review.pdf? la=en-GB. 27. Ng A. 2017 Introducing data science in the public service: challenges and solutions. Tech and Public Good. See https://techandpublicgood.com/2017/09/26/introducing-data-science- in-the-public-service-challenges-and-solutions-annalyn-ng/ (accessed 02 February 2018). D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 https://www.gov.uk/government/publications/government-transformation-strategy-2017-to-2020/government-transformation-strategy-better-use-of-data https://www.gov.uk/government/publications/government-transformation-strategy-2017-to-2020/government-transformation-strategy-better-use-of-data https://www.gov.uk/government/publications/government-transformation-strategy-2017-to-2020/government-transformation-strategy-better-use-of-data https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664563/industrial-strategy-white-paper-web-ready-version.pdf https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664563/industrial-strategy-white-paper-web-ready-version.pdf https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664563/industrial-strategy-white-paper-web-ready-version.pdf https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/78946/CM8353_acc.pdf https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/78946/CM8353_acc.pdf https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/78946/CM8353_acc.pdf http://dx.doi.org/10.1098/rsta.2016.0360 http://dx.doi.org/10.1177/2053951714559253 http://dx.doi.org/10.1177/2053951714559253 http://dx.doi.org/10.1016/j.respol.2013.05.008 http://dx.doi.org/10.1007/s12130-010-9113-9 http://www.rss.org.uk/Images/PDF/influencing-change/2016/RSS_Data%20Manifesto_2016_Online.pdf http://www.rss.org.uk/Images/PDF/influencing-change/2016/RSS_Data%20Manifesto_2016_Online.pdf https://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/inquiries/parliament-2015/big-data/ https://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/inquiries/parliament-2015/big-data/ https://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/inquiries/parliament-2015/big-data/ https://www.aiforhumanity.fr/pdfs/MissionVillani_Report_ENG-VF.pdf https://www.aiforhumanity.fr/pdfs/MissionVillani_Report_ENG-VF.pdf http://dx.doi.org/10.1098/rsta.2016.0119 https://www.gov.uk/government/publications/data-ethics-framework/data-ethics-framework https://www.gov.uk/government/publications/data-ethics-framework/data-ethics-framework https://gds.blog.gov.uk/2017/11/27/updating-the-data-science-ethical-framework/ https://gds.blog.gov.uk/2017/11/27/updating-the-data-science-ethical-framework/ https://medium.com/doteveryone/putting-users-first-is-not-the-answer-to-everything-dd05b9f11b5 https://medium.com/doteveryone/putting-users-first-is-not-the-answer-to-everything-dd05b9f11b5 https://royalsociety.org/~/media/policy/projects/data-governance/data-governance-public-engagement-review.pdf?la=en-GB https://royalsociety.org/~/media/policy/projects/data-governance/data-governance-public-engagement-review.pdf?la=en-GB https://royalsociety.org/~/media/policy/projects/data-governance/data-governance-public-engagement-review.pdf?la=en-GB https://techandpublicgood.com/2017/09/26/introducing-data-science-in-the-public-service-challenges-and-solutions-annalyn-ng/ https://techandpublicgood.com/2017/09/26/introducing-data-science-in-the-public-service-challenges-and-solutions-annalyn-ng/ 16 rsta.royalsocietypublishing.org Phil.Trans.R.Soc.A 376:20170353 ........................................................ 28. Kimbell L. 2011 Designing for service as one way of designing services. Int. J. Des. 5, 41–52. 29. Alexander C. 1971 Notes on the synthesis of form. Cambridge, MA: Harvard University Press. 30. Simon HA. 1969 The science of the artificial, 1st edn. Cambridge, MA: MIT Press. 31. Buchanan R. 1992 Wicked problems in design thinking. Des. Issues 8, 5–21. (doi:10.2307/ 1511637) 32. Rittel H, Webber M. 1973 Dilemmas in a general theory of planning. Policy Sci. 4, 155–169. (doi:10.1007/BF01405730) 33. Brown T. 2008 Design thinking. Harvard Business Review, 86. 34. Kimball L. 2015 Applying Design Approaches to Policymaking: Discovering Policy Lab. See https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report. pdf. 35. IDEO 2015 Human-centred design toolkit. IDEO. See https://www.ideo.com/post/design-kit (accessed 02 February 2018). 36. ODI. 2018 Using open data for public services, ODI. See https://theodi.org/article/using- open-data-for-public-services-report-2/. 37. Wu C-S, Sung T-J. 2017 Applying Service design in R&D-based organisations. Touchpoint 9, 12–19. 38. Denham E. 2017 Consent is not the silver bullet for GDPR compliance. ICO Blog. See https:// iconewsblog.org.uk/2017/08/16/consent-is-not-the-silver-bullet-for-gdpr-compliance/ (accessed 24 February 2018). 39. ODI. 2016 New Service Delivery Models Case Studies: Four Councils using Open Data for Service Redesign. See https://theodi.org/article/new-service-delivery-models-case-studies- four-councils-using-open-data-for-service-redesign/. 40. Uscreates. 2018 A design wrap around for data projects. Medium. See https://medium.com/ @hello_83733/data-and-design-blog-3-a-design-wrap-around-for-data-projects-to-achieve- impact-35269184e512. 41. Frey C, Osborne M. 2013 The future of employment: how susceptible are jobs to computerisation. Oxford, UK: Oxford Martin School of Technology & Employment. D ow nl oa de d fr om h tt ps :/ /r oy al so ci et yp ub li sh in g. or g/ o n 05 A pr il 2 02 1 http://dx.doi.org/10.2307/1511637 http://dx.doi.org/10.2307/1511637 http://dx.doi.org/10.1007/BF01405730 https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf https://researchingdesignforpolicy.files.wordpress.com/2015/10/kimbell_policylab_report.pdf https://www.ideo.com/post/design-kit https://theodi.org/article/using-open-data-for-public-services-report-2/ https://theodi.org/article/using-open-data-for-public-services-report-2/ https://iconewsblog.org.uk/2017/08/16/consent-is-not-the-silver-bullet-for-gdpr-compliance/ https://iconewsblog.org.uk/2017/08/16/consent-is-not-the-silver-bullet-for-gdpr-compliance/ https://theodi.org/article/new-service-delivery-models-case-studies-four-councils-using-open-data-for-service-redesign/ https://theodi.org/article/new-service-delivery-models-case-studies-four-councils-using-open-data-for-service-redesign/ https://medium.com/@hello_83733/data-and-design-blog-3-a-design-wrap-around-for-data-projects-to-achieve-impact-35269184e512 https://medium.com/@hello_83733/data-and-design-blog-3-a-design-wrap-around-for-data-projects-to-achieve-impact-35269184e512 https://medium.com/@hello_83733/data-and-design-blog-3-a-design-wrap-around-for-data-projects-to-achieve-impact-35269184e512 The emergence of data ethics frameworks Iterating and improving ethical frameworks Need for wider communications and focus on public value Need for a toolkit/methods to enact the principles and link to impact delivery The value of a service design approach to `wrap around' a data project Reflections on the personal use of service design approaches in four data projects Reflections on personal use of service design approaches in four data projects Service design as a method of communicating and building understanding of data opportunities Design as a way to focus on user need and/or public good Thinking through and prototyping the experience of those involved Conclusion References