key: cord-0982718-uu7s0bcf
authors: Zhao, Luhua; Cao, Bochen; Borghi, Elaine; Chatterji, Somnath; Garcia-Saiso, Sebastian; Rashidian, Arash; Doctor, Henry Victor; D'Agostino, Marcelo; Karamagi, Humphrey C; Novillo-Ortiz, David; Landry, Mark; Hosseinpoor, Ahmad Reza; Noor, Abdisalan; Riley, Leanne; Cox, Adrienne; Gao, Jun; Litavecz, Steve; Asma, Samira
title: Data gaps towards health development goals, 47 low- and middle-income countries
date: 2022-01-01
journal: Bull World Health Organ
DOI: 10.2471/blt.21.286254
sha: c5a743325f6fc2acbec3a08346b2e7c283eb45a7
doc_id: 982718
cord_uid: uu7s0bcf

OBJECTIVE: To assess the availability and gaps in data for measuring progress towards health-related sustainable development goals and other targets in selected low- and middle-income countries. METHODS: We used 14 international population surveys to evaluate the health data systems in the 47 least developed countries over the years 2015–2020. We reviewed the survey instruments to determine whether they contained tools that could be used to measure 46 health-related indicators defined by the World Health Organization. We recorded the number of countries with data available on the indicators from these surveys. FINDINGS: Twenty-seven indicators were measurable by the surveys we identified. The two health emergency indicators were not measurable by current surveys. The percentage of countries that used surveys to collect data over 2015–2020 were lowest for tuberculosis (2/47; 4.3%), hepatitis B (3/47; 6.4%), human immunodeficiency virus (11/47; 23.4%), child development status and child abuse (both 13/47; 27.7%), compared with safe drinking water (37/47; 78.7%) and births attended by skilled health personnel (36/47; 76.6%). Nineteen countries collected data on 21 or more indicators over 2015–2020 while nine collected data on no indicators; over 2018–2020 these numbers reduced to six and 20, respectively. CONCLUSION: Examining selected international surveys provided a quick summary of health data available in the 47 least developed countries. We found major gaps in health data due to long survey cycles and lack of appropriate survey instruments. Novel indicators and survey instruments would be needed to track the fast-changing situation of health emergencies.

The United Nations (UN) Transforming our world: the 2030 agenda for sustainable development is a global plan shared by Member States for a far healthier, more prosperous world. 1, 2 Fifty-seven of the 232 UN sustainable development goal (SDG) indicators were subsequently identified as health-related indicators by the World Health Organization (WHO) in 2019 with two more added afterwards. 3, 4 In alignment with the SDGs, WHO Member States approved the Thirteenth general programme of work 2019−2023 in 2018 and set the triple billion targets to accelerate delivering impact in countries. 5 The results framework of the general programme of work identified 46 health outcome indicators to track countries' progress towards the triple billion targets, 6 including 39 SDG indicators and seven non-SDG indicators. 7 The seven non-SDG indicators were approved in World Health Assembly (WHA) resolutions including two that relate to health emergencies. These 46 indicators assist WHO Member States to stay focused and accelerate their progress towards achieving the SDGs and the triple billion targets.

The recommendations of the thirteenth general programme of work would require gathering data in many dimensions, yet little is known about the efforts needed by countries to measure these 46 indicators. Researchers have estimated that 12 data systems would be required in each country to monitor 57 health-related SDG indicators. 3 A similar number of systems would likely be needed to measure the 46 indicators. Representative household surveys, civil registration and vital statistics, and other administrative health data systems are the primary and preferred data source for monitoring health indicators. 8 Regular population-based health surveys are a key component of well-functioning data systems for surveying populations and health risks. 9 Household surveys can provide data for an estimated 29 of the 57 health-related indicators. 3 Additionally, valid measurement of some indicators, such as out-of-pocket health expenditure, are mostly achieved through surveys. Many countries, however, especially lowand middle-income countries, lack the technical capacity and financial resources to perform these surveys, and have relied on development partners to implement surveys. 10 The WHO SCORE (survey, count, optimize, review, enable) global report in 2020 revealed that most low-income Health data gaps for SDGs Luhua Zhao et al.

countries relied heavily on external funding for surveys, with only 3% of the surveys fully funded by a government. 9 International surveys provided essential health data and this was sometimes the only data available in these countries to monitor some important health trends. Reviewing the survey methods and procedures in low-and middle-income countries would offer valuable insight into whether and to what extent key health data, measurable by the 46 indicators, were available in these countries. Our literature search showed that not much work has been done in this area. The UN Intersecretariat Working Group on Household Surveys conducted a review to map the SDG global indicators to household surveys. 11 The report found that 77 of the SDG indicators could be sourced from household surveys; however, the availability of the data was not assessed.

We therefore aimed to assess the availability and gaps in the relevant data and to explore new methods for improving health data in low-and middle-income countries. Our study focused on reviewing international surveys that have been implemented in these countries. We reviewed each survey to determine if it could measure any of the 46 health outcome indicators defined by the WHO. We also reviewed the implementation of the surveys between the years 2015 and 2020 in 47 low-and middle-income countries that are designated by the UN as the least developed countries. 12 These countries lack adequate local capacity, and depend heavily on surveys sponsored by international partners to provide essential health data. We therefore found it feasible to assess their health data systems through examining the international surveys used. The results provide evidence on countries' progress towards the health-related SDGs. We also propose exploring new methods to reduce the data gaps through surveys, particularly for timely and rapid data collection in health emergency settings.

We identified potential international population surveys based on a literature review and consultation with experts across WHO technical programmes. We included surveys in our study if they covered health topics; were coordinated or facilitated by international agencies or partners; had been implemented in multiple least developed countries since 2000 using nationally representative samples; and had a core questionnaire or instrument and a standard protocol. We included surveys concerning human immunodeficiency virus (HIV), tuberculosis, malaria and hepatitis B only if rapid diagnosis tests or laboratory testing were used in the surveys in addition to data collected through oral interviews.

For each type of survey, we conducted an extensive online review to identify standard and country-specific survey instruments. We examined the instruments to determine whether they contained tools (such as survey questions, health examinations or collection of biomarkers) that could potentially be used to measure any of the 46 health outcome indicators. When the country-specific survey instrument was not available, we used the standard international questionnaire for assessment. We categorized each indicator into one of three groups according to its measurability: (i) measurable (if the surveys we studied contained suitable questions, health examinations or biomarkers); (ii) potentially measurable (if the surveys we studied did not contain suitable questions, health examinations or biomarkers but such tools could be formulated and added); and (iii) not measurable (if suitable questions, health examinations or biomarkers could not be formulated for any surveys). If surveys containing tools to measure an indicator were conducted in a country between 2015 and 2020, we assumed that data were available for this indicator in this country for this period. We judged that an indicator was not measurable by surveys if its calculation involved important information unavailable from surveys. For SDG 3.4 (incidence of HIV, tuberculosis, malaria and hepatitis B infections), for example, we judged that surveys needed to include rapid diagnostic tests or laboratory testing to measure the indicators.

For all 47 least developed countries 12 we assessed the availability of data for all indicators that were deemed measurable by the surveys we studied. We did this by mapping the indicators each survey can measure. We first assessed data availability and the number of indicators measured by the surveys for the 47 countries for the period of 2015-2020. We conducted a second analysis using the period 2018-2020 to evaluate the impact of survey frequency on availability of health data in these countries. For each indicator, we recorded the number of countries that could measure it and then calculated the percentage out of the 47 countries.

We identified 14 different surveys that could measure at least one of the 46 health outcome indicators from 2015 to 2020 (Table 1 ). The surveys included comprehensive multi-topic surveys such as demographic and health surveys (DHS), labour force surveys, living standards measurement surveys and multiple indicator cluster surveys (MICS), as well as single-topic surveys such as household income and expenditure survey and reproductive health surveys. More details of these surveys are provided in the authors' data repository. 13 DHS were able to measure the greatest number of indicators (22 indicators), followed by MICS (14 indicators), living standards measurement surveys (11 indicators), reproductive health surveys (eight indicators), household income and expenditure surveys (five indicators) and labour force surveys (five indicators). There were major overlaps in survey contents among them. Nine indicators measured by DHS could also be measured by MICS and living standards measurement surveys. Out of 14 indicators measurable by MICS, 11 indicators could also be measured by DHS. Similarly, 10 of 11 indicators that are measured by living standards measurement surveys are also measured by DHS.

We found that 27 indicators were measurable by the selected surveys. The percentages of countries that used the surveys to measure each indicator between 2015 and 2020 are listed in Thirteen indicators were not measurable by surveys. These indicators fell into six groups: (i) indicators that concern health facility and health workers, including SDG 3.b.3 (health facility operation) and SDG 3.c.1 (health workforce); (ii) indicators that involve government policies, including WHA 66.10 (trans-fats regulation), SDG 3.d.1 (International Health Regulations compliance) and SDG 1.a.2 (government spending); (iii) indicators that involve sophisticated testing and diagnosis, including WHA 68.3 (poliomyelitis) and WHA 67.25 (antimicrobial resistance); (iv) indicators that are usually reported through civil registration and vital statistics and administrative data systems, including SDG 1.5.1 (death from disasters) and SDG 11.6.2 (fine particulate matter); (v) indicators that involve complex calculation algorithms, including SDG 3.9.1 (mortality rate attributable to household and ambient environment); and (vi) composite indicators that involve multiple tracer variables, including SDG 3.8.1 (coverage of universal health care) and the two indicators that are related to readiness and response to health emergencies.

The remaining six indicators were potentially measurable by surveys, but were not measured by the surveys we studied. These indicators were SDG 3.3.5 (intervention against neglected tropical diseases), SDG 3.5.1 (treatment intervention for substance abuse disorder), WHA 68.7 (antibiotic consumption level) and SDG 3.4.1, SDG 3.9.2 and SDG 3.9.3 (mortality indicators).

Our results also provided information to assess how the 47 countries fared overall in tracking public health trends. Over the period 2015-2020, 19 of the countries collected data on 21 or more of the 46 health outcome indicators and 16 countries had data on 11 to 20 indicators; nine countries collected no data, of which five countries were from WHO African Region, three from Western Pacific Region and one from Eastern Mediterranean Region (Table 2 ). When we analysed the time period 2018-2020, the number of countries with data on 21 or more indicators dropped to six, and countries with no data increased to 20, across five WHO regions.

As WHO Member States are mandated to track and report their progress at country level towards SDGs and triple billion targets, 5 it is essential that they collect and use health data to identify the key priority areas for improvement. Well-functioning health information systems, particularly civil registration and vital statistics and other administrative data systems, were lacking in the 47 least developed countries. 12 WHO's SCORE global report estimated that 40% of annual deaths went unregistered globally, particularly in Africa. 9 Many low-and middle-income countries, particularly the least developed countries, relied on international surveys as the primary source for many essential health data. We were therefore able to assess the key health data gaps in the studied countries by examining international surveys. In conjunction with the 46 health outcome indicators identified by WHO, international surveys can provide a relatively simple and quick way to assess gaps in essential health data in these countries.

We found data gaps in the 47 countries studied. None of the countries had data for all 27 indicators measurable by surveys between 2015 and 2020. The most monitored indicators were measured in about three quarters of countries. The least tracked indicators were those for infectious diseases, particularly tuberculosis and hepatitis B, each of which were reported by 4.3% and 6.4% of the countries, respectively. This result is expected given that the diagno-sis of these health conditions requires laboratory work that was not provided by most surveys. Furthermore, less than one third of the countries had data to monitor child development status and child abuse, a sign that more investment is needed to track children's welfare.

The pandemic of coronavirus disease 2019 (COVID-19) has highlighted the urgency of timely and accurate measurement of population mortality, as well as the need for rapid collection of key data for informing policies and actions. Household surveys are one of the major sources for mortality data in countries that do not have reliable civil registration and vital statistics systems. However, surveys often lack instruments to obtain mortality statistics. Five of the 10 mortality indicators identified by the WHO thirteenth general programme of work are either not measurable by surveys or potentially measurable but not measured by the surveys we studied. Sibling survival history -a method commonly used in surveys to measure mortality -only provides all-cause and pregnancy-related mortality, and tends to produce sparse data among older adults. Researchers have tested additional questions for sibling survival history to determine HIV/AIDS mortality, 15 while others have explored the idea of a social network survival method to capture death data more efficiently. 16 Further research is needed to develop more innovative methods to enhance the capacity of surveys for rapid measurement of population mortality.

The COVID-19 pandemic also showed the need to rapidly assess the local situation during health emergencies. Health emergency readiness is usually measured through reports based on the International Health Regulations, 17, 18 which relies on government reporting prepared in advance and does not provide up-to-date information. Surveys can be used to obtain quick results. However, out of the 46 indicators studied, only two indicators were directly related to health emergencies and neither were measurable directly by surveys. Developing suitable indicators for population-based surveys is needed to track and monitor the rapidly changing situation during health emergencies.

Heavy reliance on a few surveys can have negative consequences. Comprehensive population surveys such as DHS and MICS take significant time, manpower and financial invest-Health data gaps for SDGs Luhua Zhao et al.

ment to implement, and are thus usually conducted every 5 years or more. 11 Countries relying on these surveys may find themselves without data to monitor important health trends between survey cycles, which can be affected by unexpected events such as pandemics. In the current study, the number of countries with no data for the studied indicators rose from nine to 20 when the time period studied was shortened from 2015-2020 to 2018-2020, showing the risk of dependence on a small number of surveys. Our analyses revealed that the difference was primarily due to the scheduling of DHS and MICS. Greater efforts are needed to find new ways of filling the data gaps between these surveys. Moreover, most health surveys still rely on face-to-face interviews for data collection. As the COVID-19 pandemic has shown, this method is often not feasible due to safety concerns and travel restrictions imposed by authorities. Many countries have responded by adopting mobile phone surveys and web-based surveys as alternatives, 19 even though guidelines have been developed to restore the capacity to run in-person interviews. Adapting to the challenges posed by COVID-19 would help not only to collect data that is urgently needed for responding to the pandemic but also to prepare surveys for other emergencies and rapid responses. Mobile phone surveys are suitable for this purpose as they can collect data rapidly with lower costs. Such surveys can also be used to quickly fill the specific data gaps in combination with prior health data evaluations, such as the assessment introduced in our study and WHO SCORE technical package. 9, 20 International regulations need to be developed to reduce unnecessary ethical review requirements for non-sensitive surveys and minimize restrictions on mobile network access. Such barriers can offset the speediness of mobile phone surveys.

Data that are comparable across countries and consistent over time are important not only for national planning and evaluation of health systems but also for global assessment and progress tracking. Previous studies have indicated poor correlation of data on SDG indicators across various reports due to differences in indicator definitions, data sources, data processing and methods of synthesis. 21, 22 International surveys may adopt indicator definitions that are not comparable with those approved 23 whereas measuring progress on SDG 3.3.1 requires HIV indicators for broader populations disaggregated by sex, age group and other categories. This inconsistency would likely represent a challenge for tracking progress towards the SDG and triple billion targets in countries that rely on these data. Despite the efforts among international partners to coordinate the monitoring of health trends, greater collaboration is needed to increase data comparability and close data gaps.

This study has a few limitations. First, we did not include important data sources such as civil registration and vital statistics and administrative data sources. However, the impact on our findings should be minimal as most of the indicators examined rely on surveys as the primary data source, and civil registration and vital statistics systems are mostly too poorly developed in the 47 least developed countries to play a key role. Second, when details were not available online for some countries, we used core questionnaires from the same survey series for assessment. This approach could cause potential discrepancies for countries with tailored questionnaires. Last, low data quality can render some data unusable. However, as data quality varies across surveys and can be influenced by many factors that are difficult to measure, this issue, although important, was not included in the discussion.

Examining international surveys provided a quick summary of essential health data in the 47 least developed countries. Current population surveys, the bulk of data sources, are not frequent enough to provide timely monitoring for health-related indicators in these countries. Population surveys also lacked suitable instruments to collect data for informing actions during health emergencies. We propose exploring novel indicators and survey instruments to monitor and track health emergencies, along with rapid data collection methods such as mobile phone surveys. ■ Funding: This study was supported by WHO internal funding. . Девятнадцать стран собрали данные по 21 или более показателям за 2015-2020 годы, в то время как девять стран не собрали данные ни по каким показателям; за 2018-2020 годы их количество сократилось до шести и 20 соответственно. Вывод Изучение отдельных международных обследований позволило получить краткую характеристику данных о состоянии здоровья в 47 наименее развитых странах. Авторы обнаружили серьезные пробелы в данных о состоянии здоровья из-за длительных циклов обследований и отсутствия соответствующих инструментов обследований. Для отслеживания быстро меняющейся ситуации чрезвычайных ситуаций в области здравоохранения потребуются новые показатели и инструменты обследования.

Objetivo Evaluar la disponibilidad y las carencias de datos para medir el progreso hacia los objetivos de desarrollo sostenible relacionados con la salud y otras metas en países seleccionados de ingresos bajos y medios. Métodos Se utilizaron 14 encuestas internacionales de población para evaluar los sistemas de datos sanitarios en 47 países menos desarrollados durante los años 2015 a 2020. Se revisaron los instrumentos de las encuestas para determinar si incluían herramientas que se pudieran aplicar para medir 46 indicadores de salud que establece la Organización Mundial de la Salud. Se registró el número de países con datos disponibles sobre los indicadores de estas encuestas. Resultados Las encuestas que se seleccionaron permitieron medir 27 indicadores, pero no fue posible medir los dos indicadores de emergencias sanitarias mediante las encuestas actuales. El porcentaje de países que aplicaron encuestas para recopilar datos entre 2015 y 2020 fue el más bajo para la tuberculosis (2/47; 4,3 %), la hepatitis B (3/47; 6,4 %), el virus de la inmunodeficiencia humana (11/47; 23,4 %), el estado de desarrollo infantil y el maltrato infantil (ambos 13/47; 27,7 %), en comparación con el agua potable (37/47; 78,7 %) y los partos a cargo de personal sanitario cualificado (36/47; 76,6 %). Diecinueve países recopilaron datos sobre 21 o más indicadores entre 2015 y 2020, mientras que 9 no recopilaron datos sobre ningún indicador; entre 2018 y 2020 estas cifras disminuyeron a 6 y 20, respectivamente. Conclusión El análisis de encuestas internacionales seleccionadas proporcionó un breve resumen de los datos sanitarios disponibles en los Research Health data gaps for SDGs Luhua Zhao et al. 47 países menos desarrollados. Se encontraron importantes carencias en los datos sanitarios debido a los largos ciclos de las encuestas y a la falta de instrumentos de encuesta adecuados. Se necesitarían nuevos indicadores e instrumentos de encuesta para hacer un seguimiento de las situaciones de emergencia sanitaria, que cambian muy rápido.

Independent Expert Advisory Group on a Data Revolution for Sustainable Development. A world that counts: mobilising the data revolution for sustainable development

United Nations

Monitoring the health-related sustainable development goals: lessons learned and recommendations for improved measurement

Global indicator framework for the sustainable development goals and targets of the 2030 Agenda for Sustainable Development

Geneva: World Health Organization

Geneva: World Health Organization

Effective health information systems for delivering the sustainable development goals and the universal health coverage agenda. East Mediterr Health J

SCORE for health data technical package: global report on health data systems and capacity

Summary report on the expert consultative meeting to discuss priority national population-based surveys for better reporting of WHO regional core indicators and SDG health-related indicators. Cairo: World Health Organization Regional Office for the Eastern Mediterranean

United Nations; c2019

United Nations

Data gaps in population surveys of health-related sustainable development goals in 47 low-and middle-income countries: international surveys admitted in the study and the information sources

Data gaps towards health-related sustainable development goals in 47 low-and middle-income countries: data availability for 46 WHO GPW13 outcome indicators by selected international surveys in 47 low-and middle-income countries

Collecting data on HIV-related mortality during household surveys: a randomized validation study in Malawi

The network survival method for estimating adult mortality: evidence from a survey experiment in Rwanda

Geneva: World Health Organization

Joint external evaluation tool: International Health Regulations

United Nations

SCORE for health data technical package: essential interventions. Geneva: World Health Organization

Measuring progress from 1990 to 2017 and projecting attainment to 2030 of the health-related Sustainable Development Goals for 195 countries and territories: a systematic analysis for the Global Burden of Disease Study

World health statistics 2018: monitoring health for the SDGs. Geneva: World Health Organization

Rockville: DHS Program Office