key: cord-0694878-m5oxjrzd authors: King, Jessica; Sadique, Zia; Amara, Michael; Borghi, Josephine title: Has Ebola delayed progress on access to routine care and financial protection in Sierra Leone? Evidence from a difference-in-differences analysis with propensity score weighting date: 2022-04-30 journal: Soc Sci Med DOI: 10.1016/j.socscimed.2022.114995 sha: eaa0edc4d0cb8e6c578d38b04b95627bf06aca53 doc_id: 694878 cord_uid: m5oxjrzd INTRODUCTION: Covid-19 has highlighted the need to understand the long-term impact of epidemics on health systems. There is extensive evidence that the Ebola epidemic of 2014-16 dramatically reduced coverage of key reproductive, maternal, newborn and child health (RMNCH) indicators during the period of acute crisis in Sierra Leone. However, less is known about the longer lasting effects, and whether patients continue to be deterred from seeking care either through fear or cost some years after the end of the epidemic METHODS: We analysed nationally representative household surveys from before (2011) and after (2018) the Ebola epidemic to estimate the coverage of 11 indicators of access to RMNCH, and affordability of care. We used a differences-in-differences analysis, exploiting the variation in epidemic intensity across chiefdoms, to identify the effect of epidemic intensity on access and affordability outcomes, with propensity score weighting to adjust for differences in underlying characteristics between chiefdoms. RESULTS: 13537 households were included across both datasets. Epidemic intensity was associated with a significant stalling in progress (−12.2 percentage points, 95% CI: 23.2 to −1.3, p = 0.029) in the proportion of births attended by a skilled provider. Epidemic intensity did not have a significant impact on any other indicator. CONCLUSION: While there is evidence that chiefdoms which experienced worse Ebola outbreaks had poorer coverage of attendance of skilled providers at birth than would have otherwise been expected, more broadly the intensity of the epidemic did not impact on most indicators. This suggests the measures to restore both staffing and trust were effective in supporting the health system to recover from Ebola. 13537 households were included across both datasets. Epidemic intensity was associated with a significant stalling in progress (-12.2 percentage points, 95% CI: -23.2 to -1.3, p=0.029) in the proportion of births attended by a skilled provider. Epidemic intensity did not have a significant impact on any other indicator. While there is evidence that chiefdoms which experienced worse Ebola outbreaks had poorer coverage of attendance of skilled providers at birth than would have otherwise been expected, more broadly the intensity of the epidemic did not impact on most indicators. This suggests the measures to restore both staffing and trust were effective in supporting the health system to recover from Ebola. Covid-19 has highlighted the need to understand the long-term impact of epidemics on health systems. There is extensive evidence that the Ebola epidemic of 2014-16 dramatically reduced coverage of key reproductive, maternal, newborn and child health (RMNCH) indicators during the period of acute crisis in Sierra Leone. However, less is known about the longer lasting effects, and whether patients continue to be deterred from seeking care either through fear or cost some years after the end of the epidemic We analysed nationally representative household surveys from before (2011) and after (2018) the Ebola epidemic to estimate the coverage of 11 indicators of access to RMNCH, and affordability of care. We used a differences-in-differences analysis, exploiting the variation in epidemic intensity across chiefdoms, to identify the effect of epidemic intensity on access and affordability outcomes, with propensity score weighting to adjust for differences in underlying characteristics between chiefdoms. Results 13537 households were included across both datasets. Epidemic intensity was associated with a significant stalling in progress (-12.2 percentage points, 95% CI: -23.2 to -1.3, p=0.029) in the proportion of births attended by a skilled provider. Epidemic intensity did not have a significant impact on any other indicator. The Covid-19 pandemic has renewed interest and discussion around the long term impact of epidemics on access to and affordability of healthcare. There are concerns that epidemics can impact health systems both directly through undermining the health system's ability to deliver services and deterring patients from seeking care, [1 ,2] and indirectly through increasing the cost of services if the epidemic impacts negatively on the availability of funds [3] . Of particular concern is the risk of epidemics undoing recent gains in coverage of essential reproductive, maternal, newborn, child and adolescent healthcare (RMNCH) interventions [4] , which are critical to improving health outcomes [5] . One important example to learn from is the West African Ebola epidemic of 2014-16, with Sierra Leone providing a useful case study of the impact of epidemics on health systems. The first case of Ebola was recorded in Sierra Leone in May 2014, and there were a total of 14061 laboratory confirmed cases and 3956 Ebola deaths before the country was declared Ebola free in March 2016. The already weak health system was weakened further by the epidemic [6] ; maternal and child mortality rates in Sierra Leone were among the highest in the world, with an estimated 1120 maternal deaths per 100,000 live births in 2017 [7] and 109 deaths in under 5s per 1000 live births in 2019 [8] . Maternal mortality has shown a slight increase since the last pre-epidemic estimate of 1100 deaths per 100,000 in 2013 [9] , though under 5 mortality has improved on the 2013 figure of 161 deaths per 1000 [10] The Ebola epidemic had a major impact on the health workforce which might well be expected to hinder its ability to provide care in the years that followed. Approximately 28% of all healthcare workers (HCWs) in Sierra Leone were reported to have been infected with Ebola, and 72% of these died, an overall mortality risk of 20% [11] There is a wealth of evidence that uptake of RMNCH services was reduced during the active period of the Ebola epidemic in Sierra Leone. Country-level studies of utilisation, relying on data collected through health facility surveys [12 ,13] or reporting through health management information systems (HMIS) [14] , have found that uptake of antenatal and postnatal care, facility-based delivery and family planning services were reduced during the epidemic. Studies examining attendance patterns at particular facilities found decreased utilisation for antenatal, postnatal, and delivery care when the district in which the facility was located had more cases of Ebola compared to when it had fewer or none [13 ,15] . There is also concern that the epidemic had longer lasting effects on both health seeking behaviour, and the ability of health facilities to deliver care, which extend beyond the immediate crisis period. There is a suggestion that mistrust and fear of health services remain a barrier to care seeking [16] , and that the health system has been weakened permanently. However, evidence on the extent to which the epidemic had lasting effects on utilisation is limited. A study of health facilities in one district of Sierra Leone with a relatively low level epidemic found that utilisation of delivery, antenatal and family planning care had a decreasing trend immediately after the end of the Ebola epidemic (from March 2015 to December 2017), compared to the increasing trend seen before [17] . A study using facility data in the highly affected Forest region of Guinea, which experienced an Ebola epidemic at the same time as Sierra Leone, found that coverage of facility-based delivery, antenatal care and childhood vaccinations had not recovered to their pre-Ebola levels by February 2016, one year after the last case of Ebola was recorded in the region [18] . Existing literature documenting the effect of the Ebola crisis on RMNCH service access in Sierra Leone and the wider region examines utilisation trends at facility level, relying on routine health information systems which report on those seeking care. Such data does not offer a full picture of access due to the absence of patient characteristics and information on health expenditures. Data quality could potentially have been affected by the crisis itself. To date, there is no evidence on the epidemic impact on coverage of RMNCH interventions at the population level from household surveys, allowing an assessment of effects across population sub groups together with the cost and affordability of care. We estimate the impact of the epidemic and its severity on coverage of essential RMNCH services and the affordability of health care in Sierra Leone in January-December 2018, up to 33 months after the end of epidemic, using a difference-in-differences analysis with propensity score weights controlling for confounding factors associated with epidemic intensity, to allow for doubly robust estimates of impact. We use a large nationally representative household survey dataset, instead of relying on facility utilisation data as in previous studies, avoiding the potential bias associated with it. We examined the impact of the 2014-2016 Ebola epidemic with a differences-in-differences analysis, in which we compared changes in care seeking and health expenditure outcomes from households surveyed in 2011 (before the start of the epidemic) and 2018 (up to 33 months after the last case of Ebola) in chiefdoms where few or no Ebola cases were reported and in chiefdoms with larger outbreaks. J o u r n a l P r e -p r o o f Sierra Leone's 149 chiefdoms and two districts (referred to hereafter as chiefdoms) of Western Area (the location of Freetown) have been classified into seven patterns according to the size and length of outbreak experienced during the 2014-16 Ebola epidemic [19] , using a weighted-average linkageclustering method [20] . To produce a binary measure of epidemic intensity, we categorised the 151 chiefdoms into two groups: no/mild epidemic (those with no cases, sporadic cases, or a single smallscale outbreak in a short period) and moderate/severe epidemic (with multiple small-scale outbreaks, a continuous low-level epidemic over a long period, or larger or more prolonged outbreaks). 40 chiefdoms were classified into the moderate/severe epidemic (or exposed) group and 111 into the no/mild epidemic (or unexposed) group. Further details of classification are given in the appendix Data Data from the Sierra Leone Integrated Household Survey (SLIHS) in 2011 [21] and 2018 [22] were used to measure study outcomes and covariates before and after the Ebola epidemic. The SLIHS are cross-sectional surveys of a representative national sample of households in Sierra Leone, and were conducted in every chiefdom of the country to measure living standards and wellbeing. The sample was selected using a two-stage cluster design, sampled by enumeration areas (EAs) at the first level and households at the second level. In both years, 684 EAs were selected with probability proportional to size selection, stratified by rural or urban location and district, and ten households were randomly chosen to be surveyed in each selected EA, with a target sample size of 6840 households in both surveys. Fieldwork was conducted nationally in January-December 2011 and January-December 2018. The response rate for the 2011 SLIHS was 98.4%, and for 2018 was 100%. Response rates by chiefdom were unavailable. Outcomes to measure use of RMNCH services were chosen based on coverage indicators given in Sierra Leone's 2017-2021 RMNCAH strategy[23], and which could be estimated from both 2011 and 2018 surveys. Cost of care outcomes were total healthcare expenditure (comprising of expenditure on outpatient, inpatient and antenatal care). To measure the affordability of healthcare, two outcomes used to monitor universal health coverage were chosen: catastrophic spending on health and impoverishing spending on health [24] . Healthcare expenditures were winsorized at 99.9%. Outcomes and their measurement as implemented in this analysis are given in Table 1 . J o u r n a l P r e -p r o o f Difference-in-differences analysis with propensity score weighting [25] was used to estimate the effects of the Ebola epidemic, with households in the 40 chiefdoms experiencing a moderate/severe epidemic designated as the exposed group with those living in the remaining 111 chiefdoms with mild or no epidemic, serving as the comparison group. Significant differences at baseline and endline were observed in several household characteristics between exposed and comparison group households ( Table 2) . Propensity score methods were therefore used within the difference-indifferences analysis, to mitigate the concern that the groups may differ in ways that affect their trends over time and therefore violate the 'parallel trends' assumption. This method is tailored to repeated cross-sectional surveys and is described further below. Pre-epidemic trends for two outcomes, the proportion of births at a health facility and the proportion of births attended by a skilled provider, were examined for the five years before the baseline survey (January 2006-December 2010). No evidence of non-parallel trends was found, and further details are given in the appendix. We used a bootstrapping method to carry out post-facto calculations of the minimum detectable effect for each outcome, further details and results of which are given in the appendix. Observations with missing outcomes or covariates were excluding from all analyses, with no attempt to impute missing data. The extent of missingness is described in the appendix. The population was divided into four groups by year (2011 OR 2018) and exposure (no/mild epidemic or moderate/severe epidemic). A multinomial logistic regression model predicting the probability of being in each group as a function of the study covariates was run separately for each outcome, to account for the different populations included in the estimation of each outcome (household, woman of reproductive age, pregnant woman; child under five years). Each individual or household then has a propensity score, the probability of being in a given year and group, which is used to create weights such that each group is balanced in terms of its covariates. The propensity score weights were multiplied by the survey weights to account for the study sampling strategy. A multivariate linear (OLS) difference-in-differences regression model, with propensity score weighting and standard errors taking into account clustering by chiefdom, was estimated for each outcome as follows: A difference-in-differences model with a binary interaction term for wealth (above and below median per adult equivalent consumption, measured at the household level) and education (no vs some education of the household head for household level outcomes, no vs some maternal education for outcomes in children 0-5, and no vs some education for outcomes in women aged ) was used to test for differential effects of the epidemic intensity in these subgroups. If the interaction term was significant at the p<0.1 level, the overall result was presented alongside the result for each subgroup. Ethics approval was not sought for this study as it uses only anonymised data entirely in the public domain. 13537 households were included across both datasets. In 2011, 2854 households were surveyed in unexposed chiefdoms, and 3873 in exposed chiefdoms. In 2018, 2712 and 4098 households were surveyed in each group respectively. There were significant differences at baseline and endline in several household characteristics before propensity score weighting (Table 2) . Households in exposed chiefdoms were more likely to be in urban areas in both 2011 and 2018 compared to those in unexposed areas, and the household heads also had generally higher education levels in those chiefdoms in both years. Households were larger in exposed chiefdoms in 2018 than those in unexposed chiefdoms, after the epidemic, but not before in 2011. Households in exposed chiefdoms were more likely to be headed by a woman and were wealthier than those in unexposed areas. There were also differences in individual characteristics, with higher levels of women's and mother's education and lower parity in exposed chiefdoms. Maternal age was lower in exposed chiefdoms in 2018 but not in 2011. The balance of covariates after propensity score weighting was much improved. The only significant difference which remained was that exposed households were wealthier than those in the unexposed group in 2018. Missingness was below 4% for all outcomes and covariates; further details are given in the appendix. J o u r n a l P r e -p r o o f 8 After propensity score weighting, there was a decrease in coverage of postnatal care in the exposed group from 77.8% to 75.6% between 2011 and 2018, while the comparison group coverage increased from 78.2% to 80.5% (Table 3 ). The coverage of four or more ANC visits increased in in both unexposed (75.4% to 95.6%) and exposed chiefdoms (69.4% to 87.4%) ( Table 3 ). The proportion of births in a health facility also increased in both groups, from 66.0% to 89.2% in unexposed and from 58.7% to 79.5% in exposed. The increase between 2011 and 2018 in contraceptive coverage and fully vaccinated under ones was similar in both exposed and comparison households, while birth registration coverage, total annual equivalent healthcare expenditure, catastrophic expenditure and impoverishment levels decreased in both groups (Table 3) (Table 4 ). There was evidence of interaction between household wealth and epidemic intensity for all four cost and affordability outcomes, and between the education of the household head and the household becoming (appendix Table A7 ), and the results by subgroup (below and above median income) are therefore shown in Table 4 . Though the effect sizes varied across the subgroups, there was no evidence of an impact of epidemic intensity in any individual subgroup. We carried out a difference-in-differences analysis to examine the effects of Ebola up to two years after the end of the epidemic on access to RMNCH and affordability of care in Sierra Leone. Our findings show that both service coverage and affordability generally improved across the country between 2011 and 2018, and the change in coverage of only one intervention (skilled birth attendant at delivery) was differential with respect to epidemic severity. However, this is not an indication that coverage decreased in areas which experienced a worse epidemic, rather that the increase in coverage was smaller in those areas (from 66% in 2011 to 80% in 2018 after propensity score weighting) compared to the increase in areas with no or a mild epidemic (from 63% in 2011 to 90% in 2018 after propensity score weighting). The differences-in-differences estimate of -12.2% represents the decrease in coverage in 2018 in the severe epidemic areas which can be ascribed to the intensity of the epidemic, that is, had those areas experienced a mild or no epidemic, coverage would be expected to be 12 percentage points higher than is actually observed. This effect of epidemic severity on coverage did not differ across wealth and education groups. That the only significant impact of the epidemic was on a process of care outcome which required qualified staff may be explained by the high mortality of healthcare workers during the epidemic. It is important to note that we did not find an impact of epidemic severity on the other six RMNCH access outcomes examined, or on the three expenditure and affordability measures. Coverage of RMNCH interventions was generally higher in exposed chiefdoms in 2011 (though this difference was not observed after propensity score matching), but increased by a similar amount in exposed and unexposed households over the time period of the study. The most immediate conclusion is that the epidemic did not have a long term impact on the ability of the health system to provide accessible and affordable care. This may be in part because chiefdoms which experienced worse epidemics were more likely to be urban, and may have had stronger health systems at baseline, as well as more engrained norms and practices around health seeking. This may have facilitated the rebuilding of the health system and resumption of services after the epidemic. However, it may be J o u r n a l P r e -p r o o f the case that unexposed chiefdoms would have recovered equally well, with no true long term impact of the Ebola epidemic. Our findings also suggest that perhaps concerns that mistrust in the health systems would be long-lasting are unfounded, and the community were happy to return to health facilities after some time had passed. That birth registration coverage was the only access indicator where coverage decreased in both groups between 2011 and 2018 suggests more focus was on ensuring the resumption of health services than administrative functions. For facility-based delivery, antenatal care and postnatal care, our effect estimates were sizeable but not significant. Post-facto calculations of the minimum detectable effects given our sample size (see appendix) were much larger than these effect estimates: it is therefore also possible that there was a true impact of epidemic intensity on a wider set of outcomes but the analysis was not powered to detect it. In terms of Sierra Leone's RMNCAH strategy, our analysis suggests the health system is achieving most of its targets regardless of Ebola. Exceptions are postnatal care, where coverage is at 81% in unexposed chiefdoms and 76% in exposed chiefdoms, compared to the 2018 target of 88%, and birth registration coverage, at 53% and 50% respectively, compared to the target of 74%. The target of 85% of children being delivered in health facilities is achieved in the unexposed group (at 89%) but not in the exposed group (at 80%). [26] . Research in another district of Guinea also found that antenatal care and institutional deliveries did not recover to their pre-Ebola levels by March-July 2016 [27] , nor did childhood vaccinations [28] . Other studies have found evidence that health systems can be resilient and 'bounce back' after crises. A study comparing two areas in Guinea, one which experienced a severe Ebola outbreak and one which did not, found that in 2017, two years after the end of the epidemic, parents were more likely to access health services for an under 5 with fever in the Ebola-affected region than those in the non-affected region [29] . Guinea was a wealthier country than Sierra Leone at the end of the Ebola epidemic, with a GDP of As with all non-randomised studies, there are limitations which must be borne in mind when considering our conclusions. Foremost is that chiefdoms which experienced no or mild epidemics might reasonably be expected to differ from those which experienced severe epidemic in several ways, which could violate the 'parallel trends' assumption that, without the Ebola outbreak, the two groups would have had similar trajectories. Our exploration of pre-trends (shown in Appendix) goes some way to assuage this concern, but examines limited outcomes and only includes births in the five years preceding 2011. We also to deal with this by using propensity score weighting and including covariates in the difference-in-differences model to produce doubly robust estimates. Propensity score weighting was overall very effective, with only one variable (household consumption expenditure) imbalanced after weighting. Since this was at endline and not baseline, and consumption expenditure was also controlled for in the difference-in-differences model, we do not believe it has an important impact on the results or their interpretation. Unobserved timevarying confounders may also have had an impact on the results. For example, chiefdoms with worse epidemics are likely to have had higher population densities, and a greater number of health facilities, before the start of the outbreak, and the epidemic might have had a differential impact on factors which would be expected to affect care seeking and affordability. Another concern is the sample selection caused by the epidemic; it could be argued that the epidemic would have affected patterns of pregnancy and childbirth, and so the populations in 2011 and 2018 are not directly comparable. Borders between neighbouring chiefdoms are permeable, so it is realistic to expect people to move between them freely. While this is not a concern for our exposure of interest, since we measure the Ebola cases in each chiefdom directly, so can capture any spread of infection between chiefdoms, this may impact access to care. For example, if an individual lived in a high incidence chiefdom, but could travel to a low incidence chiefdom to access care, their access may be less constrained than someone living in a high incidence chiefdom which was surrounded by high incidence chiefdoms. Ideally, this analysis would be conducted using coverage estimates taken immediately before the start of the Ebola epidemic. As such data are not available, our results include the time between 2011 and 2014 which was not impacted by Ebola, but during which one might have expected a rapid J o u r n a l P r e -p r o o f increase in coverage and affordability due to ongoing health systems reforms [6] . We were unable to include several important indicators in the RMNCAH strategy, such as the treatment of children with pneumonia and coverage of malaria prevention interventions, and so cannot comment on the effect of Ebola on these. An important message for policy makers in light of this study, particularly when considering health systems post-Covid-19, is that it cannot be assumed that the regions within a country which experience the worst epidemic will be those which have the least resilient health systems and require most resources in order to recover. Rather, attention should be paid to the characteristics of individual regions and their needs. Further research is needed to understand that factors underpinning health system 'bounce back' following an epidemic or other shock, to facilitate system strengthening strategies that build resilience to future crises. Secondary analysis of publically available data Impact of COVID-19 on access to healthcare in low-and middle-income countries: Current evidence and future recommendations Impact of the societal response to COVID-19 on access to healthcare for non-COVID-19 health issues in slum communities of Bangladesh, Kenya, Nigeria and Pakistan: results of pre-COVID and COVID-19 lockdown stakeholder engagements Implications of COVID-19 Pandemic for Health Financing System in Ghana Why COVID-19 strengthens the case for a dedicated financing mechanism to scale up innovation in women's, children's, and adolescents' health. The Lancet Global Health Advances in the measurement of coverage for RMNCH and nutrition: from contact to effective coverage The free healthcare initiative in Sierra Leone: Evaluating a health system reform World Bank Group and the United Nations Population Division. Geneva: World Health Organization United Nations Inter-agency Group for Child Mortality Estimation (UN IGME). Levels & Trends in Child Mortality: Report 2020, Estimates developed by the United Nations Inter-agency Group for Child Mortality Estimation Trends in maternal mortality Estimates Developed by the UN Inter-agency Group for Child Mortality Estimation: United Nations Inter-agency Group for Child Mortality Estimation Impact of the Ebola outbreak on health systems and population health in Sierra Leone The Effect of Ebola Virus Disease on Maternal and Child Health Services and Child Mortality in Sierra Leone Women and babies are dying but not of Ebola': the effect of the Ebola virus epidemic on the availability, uptake and outcomes of maternal and newborn health services in Sierra Leone Counting indirect crisis-related deaths in the context of a lowresilience health system: the case of maternal and neonatal health during the Ebola epidemic in Sierra Leone The Impact of the West Africa Ebola Outbreak on Obstetric Health Care in Sierra Leone In the hospital, there will be nobody to pamper me": a qualitative assessment on barriers to facility-based delivery in post-Ebola Sierra Leone Impact of Ebola outbreak on reproductive health services in a rural district of Sierra Leone: a prospective observational study Effect of Ebola virus disease on maternal and child health services in Guinea: a retrospective observational cohort study Transmission dynamics of Ebola virus disease and intervention effectiveness in Sierra Leone Statistics with STATA (Cengage Learning, Boston) Sierra Leone Integrated Household Survey (SLIHS) Ministry of Health and Sanitation. Sierra Leone National Reproductive, Maternal, Newborn, Child and Adolescent Health Strategy Tracking universal health coverage: 2017 global monitoring report: World Health Organization and International Bank for Reconstruction and Development / The World Bank Using propensity scores in difference-in-differences models to estimate the effects of a policy change Effect of Ebola virus disease on maternal and child health services in Guinea: a retrospective observational cohort study Effect of the 2014/2015 Ebola outbreak on reproductive health services in a rural district of Guinea: an ecological study Ebola outbreak on the vaccination of children in a rural district of Guinea Do memories of the Ebola virus disease outbreak influence post-Ebola health seeking behaviour in Guéckédou district (epicentre) in Guinea? A cross-sectional study of children with febrile illness The 2014-2015 Ebola virus disease outbreak and primary healthcare delivery in Liberia: Time-series analyses for 2010-2016 We declare no competing interests