key: cord-0060150-sd4w5igb
authors: Allin, Paul; Hand, David J.
title: Using Statistics to Assess Progress
date: 2020-08-28
journal: From GDP to Sustainable Wellbeing
DOI: 10.1007/978-3-030-53085-3_2
sha: c39d36bfd5b405f28783719aba306effe76758df
doc_id: 60150
cord_uid: sd4w5igb

For statistics to be a useful way to assess progress, they must be trusted. Official statistics compete with other sources and often arrive at the consumer after various intermediaries have imposed their own angles. A key aspect is the question of who the statistics are for. The term “user” is not necessarily well-defined. Identifying users and indeed potential users may not be straightforward. But identifying the users is a pre-requisite for producing something useful. It is not enough to produce the statistics in the hope that, once they exist, they will become useful. The diversity of users has resulted in a lack of consistent process about what to include in a nation’s set of official statistics. We focus on an authoritative report that sought to align better the metrics measuring wellbeing with what actually contributed to quality of life. While the report attracted a great deal of attention, there is little evidence that the construction of alternative measures has had much practical impact. The UN’s Sustainable Development Goals are a further attempt to move in the same direction, although they are not without their critics, not least because of the sheer scale of the ambition implicit in them. At bottom, measures of social progress need to be anchored in the real practical world.

In this chapter, we explore the question of how is the country doing? In particular, what is the current level of wellbeing in the country? And then how is the level of wellbeing changing and the nation progressing? We are especially interested in how answers to these questions may, or not, be constructed in terms of official statistics.

For answers to those questions to be believed, for them to be trusted, they have to merit trust and be trustworthy. That is, the source of the data has to be trusted and the methods through which the answers are derived from the data have to have credibility and trustworthiness. In essence, this means that hearing a fact should prompt the listener to reflect on how the fact was derived and who is declaring it as a fact. If those declaring the fact obtained it from sources other than the primary source of their own data collection, then the chain of acquisition should, at least in principle, be traceable back to its origin.

Much has been written and discussed about the current information age: the vast amounts of data accumulating and the granularity of data describing our everyday lives so that all of us cast long data shadows. Asta Manninen, former director of City of Helsinki Urban Facts has put it dramatically, commenting that "[w]e live in information chaos" although adding that "official statistics brings clarity and continuity" (Condon 2017) . That, at least, should be the aspiration.

One of the challenges is that the data ecosystem is not only large and complex, but is itself changing rapidly. The growth of social media is a prime example, in which people obtain their information not from official sources but from other individuals-who may not have taken the trouble to establish the authenticity of the information they pass on, for example misrepresenting the risks of vaccination and downplaying its scientifically verifiable benefits. Roger Mosey, a former head of BBC News, observes that "styles of communication have changed, but it's the onslaught of social media that is proving a thorny challenge to the traditional broadcasters. The BBC in particular needs to be distinctive and focussed on the truth -a place of sanity amid the torrents of digital stuff" (Mosey 2018, p. 22) .

One would hope that official statistics should be a particularly trustworthy source of information about a nation's wellbeing. But even if the statistics that the national bodies produce are themselves trustworthy, this does not mean that the version of them that people see is equally trustworthy. The problem is that most people see official statistics through the filter of the media. This means, firstly, that the statistics they see have gone through the further interpretive hands of the journalists, bloggers, or pundits presenting them, rather than directly from the national statistical office. And, secondly, that they are competing with a welter of other statistics, much of which may arise from less trustworthy sources. The difficulty is how the public can distinguish between the sources. After all, at some level the public have to take the figures on trust, and while wider education in notions of why and when to trust figures would certainly be desirable, one cannot expect everyone to be an expert on statistical analysis.

A further complication is implicit in some of the discussion of Chapter 1. Many statistics are estimates, based on collecting data from a sample of the population being studied, with formal statistical inferences then being made to the whole population. While the notion of sampling was a major advance, it is inevitable that any inferences based on a sample are subject to a degree of uncertainty. Applying statistical theory and methods can help assess confidence in the statistical result, but they cannot remove this uncertainty. Worse still, there may be subtle biases and distortions in the data-of the kind discussed by Hand (2020) .

Failure to understand, or perhaps even record, the issues associated with estimation, and sampling in particular, can lead to two opposite kinds of misunderstandings. The first is that indicating the range of uncertainty can lead to suspicion and doubt about the numbers: the "so you mean you do not really know?" school of thought. A recent example of this arose in the UK, when the ONS announced revisions to its estimates of international migration, as part of a programme to improve the statistics by integrating a wider range of data sources. To reflect that the statistics were under development they were downgraded to "experimental statistics" rather than National Statistics (Humpherson 2019) . We observed that, while some commentators applauded the ONS for seeking to improve its estimates, others castigated it, apparently not understanding the complexity of measuring time-varying data with conflicting definitions. The second kind of misunderstanding is that failure to note the uncertainty in estimates can be very misleading. Here is a magazine report on the Chancellor of the Exchequer's Spring Statement in 2019: "Mr Hammond loves his numbers. MPs were therefore treated to the chancellor reeling off forecast after forecast, often with an absurd level of precision. By 2023, Mr Hammond claimed, the economy will have 600,000 more jobs. GDP growth that year will be 1.6% -no more, no less" (The Economist, March 16 2019).

The leader writer of The Economist magazine also makes it all sound quite easy to deal with. Commenting on the economics profession, but in terms that are applicable more widely to evidence and the use of evidence, they declared that "Instead of moving cautiously, the economics profession should do what it is best at: recognise there is a problem, measure it objectively and find solutions" (The Economist, March 23 2019).

All of this is an ideal. Unfortunately, one might suspect that much modern political discourse has moved beyond fidelity to the facts (but perhaps this is not new). For example, Peter Pomerantsev, a Kiev-born writer and TV producer living in London, suggests that political discourse in America-and we reckon elsewhere-is no longer "more or less" within the boundaries of reasoned argument: facts in the environment of political campaigning enhanced by social media and big data "become secondary. You are not, after all, trying to win an evidence-driven debate about ideological concepts in a public sphere of rational actors. Your aim as a propagandist is not deliberative democracy, but finding a discourse which seals in your audience … the only facts that matter are the ones that confirm existing biases … shows just how many American voters no longer felt invested in evidence-based, rational progress" (Pomerantsev 2019, p. 250) . Perhaps this might change now that the US federal government is bound by the Foundations for Evidence-Based Policymaking Act (Jolin 2020) .

Official statistics providers are also competing with many other providers of statistics, some of whom (e.g. estate agents) collect their own data, and some of whom are themselves using official data. These latter include academia and think tanks, increasing citizen use of open data, broadcasts, and articles from commentators, and a plethora of opinions on social media. Some of this has been categorised as post truth or alternative facts.

We introduced official statistics in Sect. 1.3 above, noting that they are underpinned by a set of fundamental principles. These principles are owned by the United Nations Statistical Commission, established in 1947. The Commission is the highest body of the global statistical system. It brings together the Chief Statisticians from member states from around the world. It is the highest decision-making body for international statistical activities especially the setting of statistical standards, the development of concepts and methods and their implementation at the national and international level. (United Nations 2019a) The remit of the Commission covers a huge sweep of topics and it seems hard to see it excluding statistics on anything. One might suppose that somewhere among the websites of the national statistics organisations, which are all accessible via the website of the UN's Statistics Division, must surely be numbers which could serve as the facts about wellbeing and progress. In practice, of course, the content of these websites is determined by a variety of factors, including meeting the needs of stakeholders, compliance with international agreements, and the availability of resources. All of these reflect decisions by statisticians curating their set of national statistics.

The role of official statistics in measuring the progress of nations was highlighted in the 2007 Istanbul Declaration. This identifies national statistical authorities as "key providers of relevant, reliable, timely and comparable data and the indicators required for national and international reporting" (OECD 2007, p. 1). While there have undoubtedly been advances in the measurement of progress (see Allin and Hand 2014) the fact that more remains to be done is apparent from a commitment in the United Nations agenda for sustainable development, agreed in 2015. Deep in the text, in paragraph 48, is a universal commitment to developing "broader measures of progress to complement GDP" (United Nations 2015a). The precise intent of this paragraph is not entirely clear since it is mostly about the many sustainable development indicators under development and starting to be published-to be discussed further below. Are these to be the facts about progress, broadly defined, or are there broader measures still to appear, in addition to the sustainable development indicators?

The challenge in moving beyond GDP as the main, or even sole, measure of progress is illustrated by commentators such as Fioramonti (2017, p. 30) , who calls GDP "the most powerful statistic ever invented. It is not just a number, but the ultimate objective of policy and a global benchmark for success". Indeed, until the recent international commitments, only a few governments wanted measures of progress in terms of wellbeing beyond GDP: Bhutan is the country widely quoted as "having rejected the concept of GDP growth since the early 1970s" (Fioramonti 2017, p. 171) although this is a country that still calculates traditional GDP as well as national wellbeing measures. (When we looked at the website of the Bhutan National Statistics Bureau in early 2020, GDP per person was the first of the list of key indicators: http://www.nsb.gov.bt/ main/main.php#&slider1=4).

There are many reasons for the lack of facts about progress more widely defined than by GDP growth. An obvious one is a lack of resources, especially where a statutory duty to provide GDP would steer priority-setting with limited budget. Moreover, while the UN Fundamental Principles may be anchored in the utility of official statistics (United Nations 2014, p. 2), in practice two things actually happen. First, the emphasis on national statistics offices as primarily the providers and publishers of trustworthy data tends to take precedence over engagement with users and potential users. But while the trustworthiness of the statistics is critical, it is pointless unless the statistics are answering the right questionsthose the users want to be answered. And, second, in cases where there is engagement, responding to emerging needs tends to major on addressing the needs of government, rather than the wider public.

Important as it is to meet the needs of government, this is only part of the public value of official statistics, as was confirmed by a UK parliamentary inquiry in 2019 into official statistics. Giving evidence to the inquiry, the regulator of UK official statistics spelled out how they serve the public good in two quite distinct ways, beyond "a traditional view that statistics serve the public good by providing policymakers with sound evidence on which to base their decisions". The first of these is to meet the needs of decision-makers not just in government or the central bank, but also "a wide range of other actors: businesses, trade unions, trade associations, community groups and charities, who all make active decisions about the things that they focus on using statistics-civil society". Second is "helping inform the perceptions that citizens have of the world and the society that they live in. These may not necessarily be the product of active study of a particular set of official statistics. They may be perceptions formed from hearing other people use statistics in the media or on social media, but they are very important, for example, in people's understanding of the nature of the economy, living standards, levels of crime and so on" (PACAC 2019, p. 2).

Official statisticians in the UK and elsewhere are responding to a vision of statistics to serve government, wider civil society organisations, and citizens. The United Nations Economic Commission for Europe (UNECE 2018) has set out the value of official statistics to society and has recommended ways for national statistical institutes to promote, measure, and communicate the value of their statistics. It subsequently reported that the recommendations were being piloted in seven countries, before further methodological guidance is prepared, noting that the "few" attempts at calculating the monetary value of official statistics to society "have demonstrated that Official Statistics bring net benefits" (UNECE 2019, p. 1).

We are not saying that national statistics offices fail to engage with users and potential users beyond government. There are formal processes and many informal occasions on which such engagement happens. To take an example, the system of national accounts (SNA), by which GDP is derived, is subject to strict version control, so that changes in methodology do not detract from its use to provide consistent statistics over time. When potential revisions are identified, the priority to be given to any revision is assessed against criteria including the urgency and importance of the topic "to ensure that the SNA continues to be relevant to the users" (United Nations 2009, p. 603). This phrasing, however, reduces many aspects of user engagement into a few words. How do national accountants know "the users", how frequent and how deep is the interaction between producers and users, and how are potential users identified and contacted? That last point is particularly important: a national statistics office will know who its current users are, but what about other potential users who perhaps are unaware of the value that official statistics could have to them?

We have elsewhere (Allin and Hand 2017) made the case that the development of a measure of national wellbeing and progress that is broader than GDP should build on the rigour of the SNA. Our proposal is to fit the measurement of economic performance, the focus of the SNA, within a broader assessment of national wellbeing and progress. There is already a proliferation of indicators and accounts, for example in the development of non-monetary measures of natural resources, on which to draw. Of course, there remain significant measurement challenges, and points of contention. Not the least of these is whether the "fact" that is GDP should be accompanied by (or replaced with, in the view of some) a single fact for wellbeing, say in the form of a single, overall measure or index of wellbeing. But the challenge of measurement, per se, is one thing: in our view, a more critical issue is whether the measures will actually be used. It seems to us that the starting point should be to identifying user requirements for wider measures more fully than hitherto. This would provide a firmer basis for national and crossnational developments in wellbeing accounting, rather than just national economic accounting. We envisage greater branding and marketing of national wellbeing concepts to promote measures, support their use, and stimulate further development. In Allin and Hand (2017) we called for outreach by producers, to stimulate a dialogue about the development and use of measures.

In connection with this, and to demonstrate that others are aware of the challenge, we draw attention to the Missing Numbers blog curated by data scientist Anna Powell-Smith. This is described as: about the data that the government should collect and measure in the UK, but doesn't. … areas where the government doesn't gather data at all, whether out of wilful ignorance, or just because it doesn't think it's a policy priority.… These tend not to get written about much, for the obvious reason that it's hard for journalists to report on numbers that don't exist.… But if we don't gather and report data, we can't spot where services are failing; we can't track improvement over time; and fundamentally, we can't improve people's lives. (Powell-Smith 2020) That official statistics might need to be marketed may not be a popular view among official statisticians, who can see their main function as making available statistics to a high level of technical quality. While this latter is clearly critically important, we seek to encourage official statisticians to go further, not just promoting their available products but also understanding the need for them and how they fit in. Indeed, they might even shape what could be described as a "market" for information. In short, we suggest that official statisticians can provide better answers to specific questions by finding out what the questions are beforehand, and collecting pertinent rather than general data.

One obvious objection to the market notion is that official statistics are not commercial products but are public goods, invariably published free of charge. We also recognise that users can sometimes make specific data requests and that these will be met free of charge if they involve marginal effort, or can otherwise be paid for. However, in our experience, such requests are limited in scope and only considered by a sub-group of expert users. We advocate a more open approach, based on marketing principles (Collins 2010) suitable for developers of any product or service who might otherwise believe that the world will beat a path to their door to acquire what has been produced. In the field of restorative ecology, and elsewhere, this has been referred to as the Field of Dreams hypothesis: "if you build it, they will come". While there is some support for this hypothesis, it is "generally assumed rather than tested" (Palmer et al. 1997, p. 295) , an observation we echo in the case of official statistics.

Taking more of a marketing approach in official statistics involves more interaction between producers and prospective users, to understand why statistics are needed, and how a prototype or experimental set of statistics might be used. Later there will be market research and advertising, to bring the statistic to the attention of people who might just be persuaded to be interested by it and to use it-tackling the point we made above, namely that today's users might not be the only potential users. As we noted, this may sit a little uneasily with official statisticians who are more accustomed to deciding how to measure, and to some extent what gets measured, and then publishing the resulting statistics.

In the approach that we advocate, official statisticians would need to engage with a wide range of users, certainly not merely those in government. That would conform to the apolitical role of official statisticians (and would also presumably have had some appeal to the founders of the Royal Statistical Society, producing statistics for others to interpret and to apply in decision-making, as we described in Chapter 1). But that is not the total of our recommendations. We would also expect official statisticians to be more engaged with all their users, inevitably taking statisticians out into an arena in which there is a diversity of political, commercial, societal and personal values and attitudes in play. It may well be challenging for official statisticians to steer between party-political positions while working in a way that is apolitical. We suggest that it is preferable to statisticians effectively playing a unique political role in deciding what gets included as measures of the state of the nation, its wellbeing and progress.

There does not seem to be a consistent process leading to decisions about what is included in a nation's set of official statistics, but there are sound historical reasons for this. One reason is that there are many interested parties. Of course, official statistics are not alone in lack of consistency arising for this reason. The International Classification of Diseases "defines the universe of diseases, disorders, injuries and other related health conditions, listed in a comprehensive, hierarchical fashion" and "The ICD is important because it provides a common language for reporting and monitoring diseases. This allows the world to compare and share data in a consistent and standard way -between hospitals, regions and countries and over periods of time. It facilitates the collection and storage of data for analysis and evidence-based decision-making". (ICD11 2019) However, Bowker and Star (1999, p. 21) , commenting on their investigation of the statistical principles underlying the ICD, said "… what we found was not a record of gradually increasing consensus, but a panoply of tangled and crisscrossing classification schemes held together by an increasingly harassed and sprawling international public health bureaucracy". The reason for this is that "[t]he ICD has been as heterogeneous as possible to enable the different groups to find their own concerns reflected. Because different models of medicine hold, they embody different rules for classifying. This has resulted in the fact that, although the list is in appearance homogeneous, there are at least four classificatory principles involved" (Bowker and Star 1999, p. 150) .

Recognising the multiplicity of parties interested in and producing official statistics, the UK Office for Statistics Regulation has implemented a successful programme of systemic reviews, each of which looks at a particular statistical domain (e.g. health, social care, data linkage, etc.), inviting parties to work together to better integrate their statistics, to avoid duplication, overlap, and conflicting and confusing definitions (UKSA 2019).

A second reason for lack of consistency about what has been included in official statistics is that many statistics are long-standing components of the information that government needs to govern, including population, housing, and health statistics, and that these requirements might be set at different levels: local government, national government, and the supranational level (for example in the European Union).

We remarked earlier that the system of national accounts, in the broad form it has today, was initially used during the Second World War to help plan industrial production and the war effort. The key headline measure of GDP in the national accounts, has emerged, as we have noted, as the dominant measure for assessing the progress of a country. This suggests one of two things happened. First, the national accounts have been used to manage the national economy by agreed objectives and key results, and so were an early example of the management approach now called "Measure What Matters" (Doerr 2018) . Second, society has been guided into focusing on GDP growth because that is the dominant measure of national progress and people might well assume that what matters must that which is measured (recall again that "what gets measured gets done"). Fioramonti, for one, notes that "Our entire development model rests on the way in which we measure prosperity, development and ultimately success". He continues, warning that a global order based on GDP growth is driving "a suicidal race to the cliff, which imposes stressful lifestyles, generates irrational desires and threatens to tear the world apart, while undermining the very social and natural foundations that make life possible" (Fioramonti 2017, p. 7) .

There have been many calls over recent years to go beyond GDP, not only in what we measure but more generally in how we live our lives. Not all of them appear to have started by asking what matters most, with the result that they come up with statistics describing the way that the organisation producing the new measure sees things, without at least confirming that these are the things that matter to others. When it comes to seeking, publishing, and using wider measures of progress than GDP, we have already welcomed (Allin and Hand 2014, p. 162) a practical guide to developing societal progress indicators prepared by the OECD Statistics Directorate (Trewin and Hall 2010) . This has, as the first step, to determine what matters most to a society.

For the remainder of this chapter we are particularly interested in how, or if, people, organisations and governments (outside of national statistical organisations) influence which statistics are collected. We will look at two developments that stand out as landmarks in the journey beyond GDP, and particularly how those developments tackle the choice of measures. First, the report in 2009 by the Commission on the Measurement of Economic Performance and Social Progress (often referred to by the names of the three lead authors, Stiglitz, Sen, and Fitoussi). Second, the UN's 2030 Agenda, agreed in 2015 and containing the Sustainable Development Goals. In the following two chapters of the book we will explore how, or even if, official statistics influence people, organisations, and governments.

The "Stiglitz Report", the report of the Commission on the Measurement of Economic Performance and Social Progress, was commissioned by the then President of France, Nicolas Sarkozy in early 2008, and was presented in September 2009. It had the objective of aligning "better the metrics of wellbeing with what actually contributes to quality of life, and in doing so, to help all of us to direct efforts to those things that really matter" (Stiglitz et al. 2010, p. xvii) .

In the foreword to the report, Sarkozy wrote: … we will not change our behaviour unless we change the ways we measure our economic performance … If we do not want our future and the future of our children and grandchildren to be riddled with financial, economic, social, and environmental disasters, which are ultimately human disasters, we must change the way we live, consume, and produce. We must change the criteria governing our social organisations and our public policies.… Our statistics and our accounts reflect our aspirations, the values that we assign things.… Treating these as objective data, as if they are external to us, beyond question or dispute, is undoubtedly reassuring and comfortable, but it's dangerous. It is dangerous because we get to the point where we stop asking ourselves about the purpose of what we are doing, what we are actually measuring, and what lessons we need to draw. (Stiglitz et al., 2010, p. vii) Sarkozy noted the gap between the rosy economic statistics (just before the financial crash) and the hidden truth of pollution, climate change, and other adverse consequences of measuring progress by narrow measures such as GDP. And he commented that the world had changed, and that the measures of progress had not kept pace. Of course, the report was not intended merely for France. As Sarkozy said, France will "put debate on this report's conclusions on the agenda of every international meeting …" and will "strive to get all the international organisations to modify their statistical systems in accordance with the commission's recommendations" (p. x).

In the preface to the report, its three authors, Joseph Stiglitz, Amartya Sen, and Jean-Paul Fitoussi stressed how we see the world through the statistics we use to measure it-with the obvious implications and consequences when the major such statistics are economic ones, and in particular GDP. They cited examples of countries with increasing GDP but decreasing life expectancy and declining personal income.

Statisticians go to great pains to ensure that their summary statistics are not biased, that they validly and reliably reflect the underlying reality being measured, and that their (inevitable) random measurement error is not too great. But none of this sophisticated technical expertise suffices if the statistics are fundamentally measuring the wrong thing.

It was certainly true that the Stiglitz report had impact, and contributed to the growth of interest in wider measures throughout the world. In the UK "[t]he recommendations of the Stiglitz Commission therefore fell on fertile soil …" with the then Prime Minister David Cameron "fulfilling a long-cherished promise when he launched a large-scale initiative in this domain in November 2010" (Kroll 2011) .

The report made twelve recommendations, five concerned with classical GDP issues, five with broader quality of life, and two with sustainable development and the environment.

It was ironic that, between the time at which the report was commissioned and the time it was presented the financial crisis occurreddescribed in the report as "one of the worst financial, economic and social crises in post-war history" (Stiglitz et al. 2010, p. 4) . (This was before the coronavirus pandemic). The report said:

The reforms in measurement recommended by the Commission would be highly desirable, even if we had not had the crisis. But some members of the Commission believe that the crisis provides heightened urgency to these reforms. They believe that one of the reasons why the crisis took many by surprise is that our measurement system failed us and/or market participants and government officials were not focusing on the right set of statistical indicators. (Stiglitz et al. 2010, p. 4) However, it is now over ten years since the report appeared, and while the construction and implementation of measures of national progress and wellbeing alternative to GDP has continued apace, there is little evidence of them yet having an impact. This could be simply a matter of the time required to break down the entrenched perspectives. If so, it is not the only domain where change is slow: the philosopher Thomas Kuhn wrote "[c]onversions [of scientists to new theories] will occur a few at a time until, after the last holdouts have died, the whole profession will again be practicing under a single, but now a different paradigm" (Kuhn 1962, p. 152 ). On the other hand, it is possible that the very crisis itself has slowed down the adoption of alternative measures, while governments have struggled to extricate their countries from the crisis.

The Sustainable Development Goals [SDGs] are the blueprint to achieve a better and more sustainable future for all. They address the global challenges we face, including those related to poverty, inequality, climate, environmental degradation, prosperity, and peace and justice. (United Nations, 2019b) The SDGs were introduced during the United Nations Conference on Sustainable Development in Brazil in 2012. They replaced and built on the earlier Millennium Development Goals (MDGs) which focused on global poverty. They were adopted by the General Assembly of the UN on 25 September 2015 with the title "Transforming our world: the 2030 Agenda for Sustainable Development".

There are 17 Sustainable Development Goals (SDGs) and 169 targets. Each target has up to 3 indicators to assess progress towards the target (up from 8 goals and 60 indicators for the MDGs). The indicators are described as "integrated and indivisible" (United Nations 2015a, para 5). The goals are certainly wide-ranging, going far beyond GDP, and including things as ambitious as ending poverty in all its forms everywhere, building resilient infrastructure, and reducing inequality within and among countries. Wellbeing is explicitly mentioned in Goal 3: Ensure healthy lives and promote wellbeing for all at all ages. Of course, macroeconomic considerations do appear: Goal 8 is to promote sustained, inclusive, and sustainable economic growth, full and productive employment, and decent work for all.

As one might expect, the goals are not without their critics. Apart from the scale of ambition (surely not all are achievable), some people see them as requiring difficult balances, possibly even having incoherencies (e.g. promoting economic growth and at the same time inequality reduction; increasing wages while reducing the cost of living and sustaining ecosystems). The obvious criticism that 169 targets were far too many has also been made. Certainly, a corporation which had that many "key performance indicators" would struggle to meet them all, and even to decide on a coherent direction in the face of such an abundance. They have also been criticised as ignoring local context-even though, as characterised by Felix Dodds (n.d., p. 1), "Unlike 2000 where the MDGs were predominately brought forward by developed countries, the UN has set up a process to enable all countries and stakeholders to participate".

Latent beneath the use of these goals and their associated indicators is the assumption that individual countries have sufficiently effective statistical offices to measure them, collecting accurate and reliable data, analysing it, and publishing the results. In some cases, substantial capacity building is needed.

Having said that, there does seem to be a strongly held view among statisticians working in international development that targets and indicators can have profound effects. For example, the Partnership in Statistics for Development in the twenty-first century (PARIS21) works to promote "the better use and production of statistics throughout the developing world. Since its establishment in 1999, PARIS21 has successfully developed a worldwide network of statisticians, policy makers, analysts, and development practitioners committed to evidence-based decision making" (PARIS21 2019, p. 3). Initiatives like this have as their main objective the achievement of national and international development goals. But it has to be said that it is not clear that these effects go beyond raising awareness, important though that is, and do actually help improve people's lives. Moving from data to action, in the form of changed behaviour by individuals and businesses, is assumed to involve the consideration of evidence and decisions informed by evidence. But how this actually happens, and the extent to which it happens, is not understood. The UN's 2030 agenda refers only in general terms to how the indicators are to be used, for example that "Quality, accessible, timely and reliable disaggregated data will be needed to help with the measurement of progress and to ensure that no one is left behind. Such data is key to decision-making" (United Nations 2015a, para 48).

While there has been an extensive and inclusive process to develop the indicators to support the SDGs, primarily through an inter-agency expert group, that group only started meeting in 2015 (United Nations 2015b), just ahead of the launch of the UN Agenda. In narrating how the SDGs were negotiated, Dodds et al. (2017, p. 125) show that statisticians were intentionally unable to influence the choice of the targets and goals, which was a political process played out at national, regional, and international levels. They recall that in early 2015, "Now that heads of state have adopted the SDGs, one of the significant issues that needed to be finally addressed was the issue of what indicators there should be for the targets … in March 2015, an open and transparent process to look for the relevant indicators for the targets was set out". Selection criteria and processes were soon determined. The challenge was in finding methods, new sources, and resources to define suitable indicators-and then to publish timely, sound and trusted data.

Work on the indicators in the UK is being led by the ONS. It ran a public consultation in the summer of 2017 on its approach to reporting and how to fill the gaps. Despite having an established and extensive system of official statistics, the UK is currently (early 2020) still exploring data sources for almost 1 in 4 (23%) of the 244 indicators (ONS n.d.) . Many other countries are finding the task even more difficult. We can see that there is a case for deciding political goals without worrying about the cost and feasibility of measuring progress towards them. On the other hand, there must be some scope for having statisticians involved in goal setting, helping inform decision-making and building a clearer understanding of how indicators are to be used. Without some practical input, the exercise could be markedly hampered.

The indicator data that is available around the world is being collated and published on UN websites and in UN reports (e.g. United Nations 2019c). Individual country data is also available from national statistics offices and from organisations collating global data (e.g. https://sdg-tra cker.org/).

As a stepping stone towards 2030, each country is expected to review progress towards the SDGs at least once, and to report this to the United Nations. The UK was one of 50 countries due to report in 2019see Voluntary Review (2019). It notes that one of the three principles adopted in delivering the review, and presumably the plan, is that it is "underpinned by data" (p. 7). It also records that information and data was collected from a range of sources, not only the global indicators data from ONS, but also other government and non-government sources, as well as government plans, annual reports, and accounts.

Following the global agreement on the SDGs, the UK government published a document (DfID 2017) to detail how it would support the delivery of the SDGs, both domestically and internationally. Two years later, the government reported that all of the SDGs are "now reflected throughout the UK government's [overall] programme of work collectively delivering activity on social, economic, and environmental issues. Each UK government department has embedded the Goals in its Single Departmental Plan -an established process to focus government efforts on important issues" (UK Voluntary Review 2019, p. 7). Thus, the ambitions that the UK government has for achieving the SDGs are now central to its work plan (UK Government 2019), which also embraces getting the best Brexit deal for Britain (the reader will be able to judge how that worked out), making the economy work for everyone (raising deep measurement issues), and five other points including tackling injustices, wherever they exist in our society, and keeping our families, communities, and country safe-all ambitious aspirations.

Delivering the SDGs is not just for governments and policy-makers. It is recognised that "we [must] act now and act together" (e.g. United Nations 2019c, p. 3), where "we" includes businesses, civil society, national and international organisations, and individuals. This is something that we (the authors) will expand on in the following chapters. One implication of this collective responsibility is that the audience for the SDG indicators is potentially huge and diverse. Nevertheless, there are ways for statistical offices to reach out, especially through an increasing number of communities of interest in the SDGs.

One such community within the UK, for example is found at https:// www.ukssd.co.uk/. This has a range of projects, undertakes reviews and encourages "partner" organisations to act in ways that should deliver the SDGs. The UKSSD network (2020) wrote to the UK Prime Minister, noting that there were now ten years left until the SDG deadline and calling on him to "Help the UK understand its performance on the SDGs by openly disclosing progress towards the targets".

Compiling and publishing statistics is all very well, but statistics are not an end in themselves. Statistics are intended to reflect underlying conditions, but then, hopefully, they are used to monitor and guide changes. So, we might ask, what consequences do official statistics have? For example, assuming that the 244 indicators for the SDGs are calculated for each country and published regularly, will that actually lead to sustainable development? Or is there a risk of meeting the abstract statistical SDGs, rather than establishing developments that are genuinely sustainable, in the Brundtland sense of respecting future generations? Moreover, the approach to using indicators that appears to be envisaged is primarily one of assessing progress towards the goals, but we must be aware of the temptation and opportunities for gaming and manipulation of statistics, subconscious though those might be.

In addition, different degrees of detail might be needed for different uses, such as describing the current state of a country, measuring a change of that state (requiring that change be detectable and measurable), and actionability in policy terms, as well as enabling comparative communication between countries.

We must also ask what is, or should be, in place to help ensure that the statistical indicators are accepted as an authoritative picture of the world. "Authoritative" must include notions of trustworthiness, as discussed above, as well as transparency, inclusivity, and, at bottom, utility; the ultimate measure of value.

While GDP is part of a complex system of national accounts, it also functions very much as the single headline measure, and we should likewise ask whether progress and sustainable development can be (or should be) expressed as a single index number. It is possible that the two opposing views expressed on this-"cannot be expressed as a single number" and "should be expressed as a single number"-might best be addressed by some sort of compromise. In their report, Stiglitz et al. (2010, p. xxv) wrote: "There is no single indicator that can capture something as complex as our society". While this is clearly an obvious truism, in that the multiple facets of a multivariate object cannot be captured by a single indicator, perhaps a single aggregate indicator might be derived which is useful for our purposes, albeit with acknowledged weaknesses. We discussed such matters in depth in the context of the multivariate nature of national wellbeing in Allin and Hand (2014) , contrasting a dashboard of indicators with a single derived indicator.

Since producing 244 indicators across the 196 countries in the UN is a non-trivial and costly process, such questions need to be asked-and answered-before full-scale production of new measures and indicators gets under way. It appears that this has been done only in high-level terms, leaving as yet unanswered questions about the full applicability and value of the indicators.

That these questions have force is demonstrated by the example of social progress. Although social progress is a dimension of sustainable development, there is already a long-standing tradition of publishing robust social statistics and social monitors that should provide examples and pointers to help reflect on the development and use of indicators for the SDGs. This stretches back to the origins of formal statistical methods with the work of Arthur Bowley, Adolphe Quetelet, William Farr, and others. The social indicator movement has been a consistent force for over 60 years now and has been fully embraced by the European Statistical System, comprising national statistical offices and the statistical office of the European Union. Cantillon (2017, p. 585 ) has commented on the significance of the developments in European social indicators "to understand and monitor social progress, put the indicators into perspective, and present innovative ways for their improvement and enrichment". The indicators and the surveys providing data are sophisticated and accurate, but the considerable effort made in compiling social indicators across Europe has in fact not been rewarded with progress towards meeting poverty and social exclusion goals. Moreover, new aspects of poverty, such as an increase in rough sleeping, are reported in the media (and are apparent on walking through many towns and cities).

The picture is complicated because of the disruption caused by the financial crisis. Nevertheless, Cantillon (2017, p. 593) concludes that little if any progress has been made in combating relative financial poverty and social exclusion, despite "remarkable improvements in European output governance, including measurement, goal setting, and monitoring" and increasing use of the social indicators by researchers and by European Commission policy-makers.

There seems to be a gap that is hard to bridge, between using indicators to observe and using indicators actually to make a difference. Most work is done in observation mode, monitoring levels and distributions of poverty, and evaluating policy measures that have been put in place to tackle poverty. Perhaps we are expecting too much of indicators and the clue is in the name: are indicators simply to be indicative, helping to steer fuller investigations of affluence and poverty, and of social progress more generally? Clearly there is a literature of such scholarly studies (e.g. Bosco and Poggi 2020), although these are characterised by Cantillon (2017, p. 593) as "complicated and time-consuming micro-simulation modelling", and the results need to be translated into material of use to policy-makers and for those designing and delivering programmes to reduce poverty.

One weakness of social indicators is that they have tended to be about outcomes, such as the number of people in employment. Cantillon (2017, p. 594) concludes "This has meant that the link between 'goals' -as measured by the portfolio of social indicators -and 'policies' has remained vague and unarticulated while difficult trade-offs (e.g. between work and poverty reduction) have not been made explicit". Reassuringly, there are signs that new, broader indicators are in development and are being crafted with a view to their use in policy.

For example, an independent commission has come up with a new poverty measurement framework for the UK. It is seeking broad, longterm political support for the measurement framework and its use in policy-making. In its 2019 report, the Social Metrics Commission (2019, p. 4) welcomed an announcement from the government that "the Department for Work and Pensions would be developing experimental statistics based on the Commission's measurement approach". The Commission also recognised that "its work is only the start of what needs to happen" with further development of the framework, collection of new data based on improved survey and administrative data. The Commission is committed "to work with the widest range of stakeholders possible to ensure that, once fully developed, the Commission's measurement framework can form the basis of a consensus view on poverty measurement across the Government, the ONS, policymakers and those researching and working with people in poverty" (p. 5).

Without detracting from the value of reaching consensus on the measurement of poverty, we should recognise that measuring social conditions is complex, detailed, and demanding. As Hills et al. (2009, p. 358) concluded, for example, answering the question of whether the UK had become a more equal society since 1997 "depends on which inequalities are being examined, between whom, and over which time period". They looked across a range of available official indicators, finding that "trends improved after 1997 compared with the period before for nearly half of them, but they deteriorated for a quarter". While there were some significant policy initiatives from 1997 onwards, "the scale of the action was often small in relation to the underlying inequalities, and the momentum gained by the middle of the period had often been lost by the end of it. Problems were often harder to tackle than the government appears originally to have assumed, and less amenable to a one-off fix".

The conclusion we draw from this review of social indicators is that the development of any system of indicators needs to be based on a full understanding of what users and potential users need and how the indicators are to be used in a sustained way. This may well include a dialogue about how new indicators can and should be used, and about things that might be problematic in using them. We are back to our point about the critical nature of extensive user engagement at an early stage of development. As part of this, more emphasis needs to be placed on targets and on reporting the gap between the measured value and the target. Of course, necessary statistical capacity needs to be in place, along with good communication skills, but that is not sufficient. There is also the issue of analytical skills among users, which needs to be assessed, perhaps enhanced, so that statistical outputs can be designed to meet a range of needs and capabilities. We have set out how this might work in terms of new measures of progress and development, proposing that "the system of national accounts should evolve into a process of national wellbeing accounting" (Allin and Hand 2017) .

Two final points about usage should be made. The first is that usage can be prescribed in legislation. The Wellbeing of Future Generations (Wales) Act (Wales 2019) is an example of this, which we will return to in Chapter 3. The second point is that a rich body of comparable and regularly produced statistics has intrinsic value. While naturally we are concerned with the usability and usefulness of the statistics for particular purposes and the purposes for which they are created, we should not lose sight of the wider picture they give of the world in the Twenty-First Century-see, for example, Our World in Data (https://ourworldindata. org/).

Measures of social progress and of sustainable wellbeing continue to appear, most recently driven by the imperative to meet the SDGs by 2030. We have drawn on the experience of the development and use of European social indicators over the last half century to enter a note of caution about the apparent assumption that introducing new measures is all that is needed. As Cantillon (2017, p. 594) concluded, "although in some countries the 'numbers' became part of the public debate, their impact in national politics and social dialogue remained limited".

It is also clear that there is no consensus on how to measure wellbeing and progress among official statisticians. We have briefly referred to some of the emerging guidelines and recommendations and we acknowledge that international organisations such as OECD are facilitating and promoting new standards. However, the take-away message from this chapter is that measures of progress need to be anchored in their use in the real world. Considerable effort is going into the construction of a large set of indicators and finding the numbers for each indicator in each country. But building such an edifice does not mean that people will use it, or even find it. We stress the importance of assessing the quality of indicators in a way that emphasises fitness for purpose as well as technical accuracy, trustworthiness, and reliability. Indicators need to be relevant and meaningful to individuals and businesses as well as to governments. They also need to be designed in a way that recognises the challenge that progress is about where we are headed to, not just where we are today.

The key issue is how the statistics are to be used, and who the intended users are. We see the role of national statistics offices not just as providers of data, but also interpreters and promoters. This is what we will explore further in the next two chapters, looking first at the use of wider measures of wellbeing, progress, and sustainable development in public policy and then at their use by companies, individuals, and civil society.

The Wellbeing of Nations: Meaning, Motive, and Measurement

From a System of National Accounts to a Process of National Wellbeing Accounting

Middle Class, Government Effectiveness and Poverty in the EU: A Dynamic Multilevel Analysis

Sorting Things Out: Classification and Its Consequences

Monitoring Social Inclusion in Europe

Building a Better Mousetrap Isn't Enough

Interview with Asta Manninen

Agenda 2030: The UK Government's approach to delivering the Global Goals for Sustainable Development -At Home and Around the World

Negotiating the Sustainable Development Goals

Measure What Matters

Dark Data: Why What You Don't Know Matters

Towards a More Equal Society? Poverty, Inequality and Policy Since

Response on Reclassification of Migration Statistics

Yes, There's Hope for Good Government in the U

Measuring Progress and Well-being: Achievements and Challenges of a New Global Movement. Friedrich-Ebert-Stiftung, International Policy Analysis

The Structure of Scientific Revolutions

The Istanbul Declaration

Ecological Theory and Community Restoration Ecology

Partner Report on Support to Statistics

Measuring Poverty

Mismeasuring Our Lives: Why GDP doesn't Add Up

Developing Societal Progress Indicators: A Practical Guide

Recommendations for Promoting, Measuring and Communicating the Value of Official Statistics

What are Statistics Worth? Determining the Value of Official Statistics

System of National Accounts

Transforming Our World: The 2030 Agenda for Sustainable Development

First Meeting of the IAEG-SDGs

The Sustainable Development Goals Report