key: cord-0259826-11ib0geb
authors: White, E.; Mendin, S.; Kolubah, F. R.; Karlay, R.; Grant, B.; Jacobs, G. P.; Subah, M.; Siedner, M. J.; Kraemer, J. D.; Hirschhorn, L. R.
title: Impact of the Liberian National Community Health Assistant Program on Childhood Illness Treatment in Grand Bassa County, Liberia: A Difference-in-Differences Analysis of Population-Based Data
date: 2021-11-24
journal: nan
DOI: 10.1101/2021.11.22.21266582
sha: 5032208e04a7fae81b1a2a8a43dc06bda6b1e34a
doc_id: 259826
cord_uid: 11ib0geb

Liberia launched its National Community Health Assistant Program in 2016, which seeks to ensure that all people living 5 kilometers or farther from a health facility have access to trained, supplied, supervised, and paid community health workers (CHWs). This study aims to evaluate the impact of the national program following implementation in Grand Bassa County in 2018 using data from population-based surveys. We measured before-to-after changes in childhood treatment from qualified providers in a portion of the county that implemented in a first phase compared to those which had not yet implemented. We also assessed changes in whether children received oral rehydration therapy for diarrhea and malaria rapid diagnostic tests if they had a fever by a qualified provider (facility based or CHW). For these analyses, we used a difference-in-differences approach and adjusted for potential confounding using inverse probability of treatment weighting. We also assessed changes in the source from which care was received and examined changes by key dimensions of equity (distance from health facilities, maternal education, and household wealth). We found that treatment of childhood illness by a qualified provider increased by 60.3 percentage points (95%CI 44.7-76.0) more in intervention than comparison areas. Difference-in-differences for oral rehydration therapy and malaria rapid diagnostic tests were 37.6 (95%CI 19.5-55.8) and 38.5 (95%CI 19.9-57.0) percentage points, respectively. In intervention areas, treatment by a CHW increased from 0 to 81.6% and care from unqualified providers dropped. Increases in treatment by a qualified provider did not vary significantly by household wealth, remoteness, or maternal education. This evaluation found evidence that the Liberian National Community Health Assistant Program has increased access to effective treatment in rural Grand Bassa County. Improvements were approximately equal across three measured dimensions of marginalization.

CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 24, 2021. ; https://doi.org/10.1101/2021.11.22.21266582 doi: medRxiv preprint Based in part on the early success of these initial pilot programs and lessons from the 2014-2015 Ebola 101 epidemic response, Liberia adopted a National Community Health Assistant Program (NCHAP) in 2016 102 as part of a comprehensive strategy to extend the reach of the country's primary health care system into 103 areas more than 5km from a health facility. The program uses an integrated and standardized approach in 104 which CHWs, called community health assistants (CHAs), are trained and deliver a package of 105 interventions in their communities.(13,14) However, the impact of implementation under the full national 106 program on access and uptake of health services has not been assessed. Further, more data are needed to 107 assess whether the program has been implemented in a way that equitably includes the most vulnerable 108

populations. 109

Our study seeks to evaluate the national CHA program implementation in rural Grand Bassa County, 111

Liberia. The implementation was led by the Liberian Ministry of Health with technical and financial 112 support from Last Mile Health. Because program implementation was phased for operational reasons, we 113 can assess program impact by measuring changes in child health treatment in areas that have already 114 implemented compared to areas that have not implemented yet. We aim to answer three questions. First, 115 did the program achieve greater gains to child health treatment in intervention areas than seen in 116 comparison areas where implementation has not yet begun? Second, if treatment gains are observed, did 117 the source of care shift to CHAs and facility-based providers from less effective informal sector 118 providers? Third, how did changes in child health treatment outcomes in implementation areas vary 119 across three measures of inequity: relative poverty, geographic remoteness, and maternal education? CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint in June 2018 and provides services to households in communities more than 5 km from the nearest health 129 facility. Because of resource constraints, the program rollout was planned in four phases, with two health 130 districts per phase. In this paper, we report results from the first phase of implementation. included all eight districts in Grand Bassa but our analytical sample is limited to the four districts sampled 149 at follow-up; the follow-up survey included the two implementation districts and the two non-150 implementation districts that were most comparable on baseline child health outcomes. At baseline, we 151 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 24, 2021. ; https://doi.org/10.1101/2021.11.22.21266582 doi: medRxiv preprint mapped the county immediately before data collocation and then we took a simple random sample of 152 communities, stratified by district, and a census of households within each selected community. At 153 follow-up, we sampled communities with probability of selection proportional to size, stratified by 154 implementation phase, and then selected 24 households per selected community by a modified random 155 walk procedure. All women aged 18-49 in selected households were invited to complete the maternal and 156 child health modules of the survey. We provide more details on the sampling approach in the methods 157 supplement (supplement 1). Sample size was determined to estimate trends in programmatic indicators 158 with adequate precision over the planned series of surveys. Enumerators recorded responses on Android mobile phones and uploaded data regularly to facilitate data 168 quality assurance and enumerator supervision. We discarded data collected by one enumerator because 169 those surveys failed a quality assurance check designed to ensure that enumerators spent enough time 170

surveying each household to obtain accurate information. This resulted in data for 82 children (41 171 intervention and 41 comparison) being excluded from the follow-up analysis, which was 10.2% of 172 children in that round of surveys. We tested for sensitivity to excluding these observations by re-running 173 our primary analyses including them, and this check is provided in supplement 3. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 24, 2021. ; https://doi.org/10.1101/2021.11.22.21266582 doi: medRxiv preprint Our primary outcome of interest was treatment from a qualified provider if a child has suspected diarrhea, 178 malaria, or acute respiratory infection (ARI) in the two weeks preceding the survey. Suspected malaria 179 and diarrhea were defined by maternal report of fever and diarrhea, respectively. Suspected ARI was 180 defined as maternal report of cough plus fast or difficult breathing.(17) Qualified providers were defined 181 as facility-based providers, community health assistants, or general community health volunteers 182 (GCHVs, a cadre of community health workers that preceded the NCHAP). Other non-qualified provider 183 types included drugstores, black baggers (informal drug dispensers), and traditional providers. For our 184 primary outcome, we assessed care for any of the three conditions; we analyzed each condition separately 185 as secondary outcomes. As additional secondary outcomes, we assessed quality of care using maternal 186 report of oral rehydration therapy for diarrhea and rapid diagnostic testing for malaria. We also assessed 187 what providers were sources of care as secondary outcomes to determine whether care providers changed 188 when the community health assistant program was implemented and to directly assess causal 189 mechanisms. 190

Our exposure of interest was residence in implementation areas rather than comparison areas in the period 192 after implementation. We treated the following variables as potential confounders: child's gender, child's 193 age in months (continuous), maternal age in years (continuous), maternal education (categorized as none, 194 some primary school, or completed at least primary school), whether the mother preferred to complete the 195 survey in English or Bassa, whether the child was reported as having only one illness or more, the number We used a difference-in-differences approach to estimate whether before-to-after implementation period 205 changes were greater in the intervention than comparison areas. In the unadjusted analysis, we fit a linear 206 probability model with indicator variables for intervention versus comparison area, year, and their 207 interaction. We accounted for potential changes in the composition of the population over time by using 208 inverse probability of treatment weighting (IPTW) to balance all of the covariates listed above except for 209 residence in a mining community across all four time-by-intervention groups. We assessed balance using We assessed changes in provider types using survey design-adjusted tests for differences in proportions. 214

We assessed equity in program outcomes by examining before-to-after changes in childhood disease 215 treatment in the intervention areas, stratified by distance from the nearest health facility (dichotomized as 216 5-9.9km versus 10 or more km based on prior research on distance as a barrier in a similar Liberian 217 setting (22)), household wealth (above or below the median), and maternal education (none versus some). 218

We conducted several sensitivity analyses (see supplement 3). First, we could not include residence in 220 mining rather than agricultural communities in the IPTW-adjusted models due to the small number of 221 mining communities, but we conducted supplemental analyses restricted only to agricultural 222 communities. Second, we accounted for confounders using regression adjustment rather than IPTW. 223

Finally, we re-ran analysis including observations we excluded for data quality assurance, as described 224 Overall, samples were similar across years and areas prior to inverse probability of treatment weighting 250 (Table 1) ; however more households in the comparison group were in mining communities in both 251 surveys, fewer surveys were completed in English in 2019 for both intervention and control, fewer 252 illnesses were reported for children in 2019 for both intervention and control, and wealth was higher in 253 the control group in 2019. In all groups, IPT weighting produced approximate balance, as measured by 254 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint There was a significant increase in the proportion of children receiving care from community health 280 assistants (0% to 81.6%; p<0.001) from baseline to follow-up in the intervention areas (Figure 2 ). In the 281 comparison areas, care from a community health assistant also increased from 0% to 2.0% but this was 282 not significant (p=0.154). The increase in care seeking in the intervention area was accompanied by 283 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. This study finds evidence that Liberia's National CHA Program led to substantial improvements in 311 treatment coverage for childhood fever, diarrhea, and ARI using a research approach designed to infer 312 causality. Care sought from qualified providers increased by 60 percentage points more in intervention 313 areas than comparison areas. Essential elements of care--ORT for diarrhea and RDT-based diagnoses for 314 malaria-also increased by about 40 percentage points more than in comparison areas. In support of the 315 role for the CHA program in causing these gains, we also found that care shifted predominantly to CHAs To that end, our results join evidence from other countries that public sector-led national programs can 344 achieve can meaningfully improve care for childhood illnesses when they are able to induce demand for 345 community-based services.(29-33) 346

Our results also corroborate existing literature about the equity and reach of CHW programs. We found 348 improvements were similarly large among groups that have been traditionally disadvantaged as those that 349 are relatively advantaged -such as those further from health facilities, with lower wealth, and with lower 350 education. We interpret these results as suggesting equitable reach and acceptability of the program. Prior 351 research has generally found improvements in child health outcomes across maternal education, 352 household wealth, and rural versus urban residence.(34) However, so far as we could identify, prior 353 research has not examined program equity by distance within rural areas, even though distance is a well-354 known determinant of health outcomes that CHW programs are designed to address.(35) It is particularly 355 important to assess equity by distance because CHW supervision, supply chains, and referral networks all 356 become more difficult in more remote communities. 357

This study has limitations. We lacked statistical power to directly assess changes in childhood mortality 359 as a result of the community health assistant program, though we would expect that increases in coverage 360 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 24, 2021. ; https://doi.org/10.1101/2021.11.22.21266582 doi: medRxiv preprint of evidence-based interventions known to reduce amenable under-5 mortality would result in decreases in 361 mortality, based on prior literature.(36) We are executing a subsequent study in 2022 that is powered to 362 assess child mortality. We discarded one enumerator's data because of concerns about data quality. 363

However, because data were discarded evenly from both intervention and comparison areas, we do not 364 have reason to believe it caused any bias and, in a sensitivity analysis, results did not change 365 meaningfully when that enumerator's data were included. Finally, our study is subject to the recall, 366

reporting, and recording errors that can occur in survey research. We reduced these risks by using 367 standardized Demographic and Health Survey items, building reporting validation into the data collection 368 program, and through training and supervision of enumerators. Parental report of malaria diagnostic 369 testing, in particular, has previously been found to be less than 70% sensitive, which may explain 370 differences between improved fever care and RDT use.(37) Social desirability may have encouraged 371 respondents to decrease reporting of use of unqualified providers, but this would not explain the 372 magnitude of overall increases in coverage we observed. Data entry errors are inevitable, but in previous 373 assessment of our data systems, we found recording error rates below 2%. areas, but the quality of care also improved along with a shift from unqualified providers mainly to 380 community health assistants. All of these findings suggest that the Liberian National Community Health 381

Assistant Program as implemented has been successful in strengthening the systems of care and reinforces 382 the value of well-designed public-sector community health worker programs to expand the delivery of 383 equitable, quality care. 384 385 Acknowledgements 386 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted November 24, 2021. ; https://doi.org/10.1101/2021.11.22.21266582 doi: medRxiv preprint

We would like to thank the Last Mile Health Liberia programs and monitoring, evaluation and learning 387 teams for their commitment to supporting the Liberia MOH and Grand Bassa County Health Team in 388 implementing the program and in conduct of the household survey, and our External Advisory Committee 389 for their inputs into study design and interpretation. We also would like to thank the women who 390 responded to the questions, and the CHAs for their remarkable commitment to delivery of quality care to 391 the members of their community. 392 393 394

Liberia Institute of Statistics and Geo-Information Services, Ministry of Health

Demographic and Health Survey

Liberia Institute of 398 Statistics and Geo-Information Services

An analysis of Liberia's 401 2007 national health policy: lessons for health systems strengthening and chronic disease care in 402 poor, post-conflict countries

Can the health system deliver? Determinants of rural 404

Liberians' confidence in health care. Health Policy Plan

Community-based health care is an 406 essential component of a resilient health system: evidence from Ebola outbreak in Liberia

System Reconstruction in Rural Liberia Prior to the Ebola Epidemic: A Comparison of Three 410

Population-based Surveys of Maternal and Child Healthcare Utilization

Remoteness and 412 maternal and child health service utilization in rural Liberia: A population-based survey

Which intervention 417 design factors influence performance of community health workers in low-and middle-income 418 countries? A systematic review. Health Policy Plan

Comprehensive review 420 of the evidence regarding the effectiveness of community-based primary health care in improving 421 maternal, neonatal and child health: 8. summary and recommendations of the Expert Panel

Comprehensive review of the 424 evidence regarding the effectiveness of community-based primary health care in improving 425 maternal, neonatal and child health: 4. child health findings

Implementation research on 427 community health workers' provision of maternal and child health services in rural Liberia

World Health Organ

Worker Intervention to Increase Childhood Disease Treatment Coverage in Rural Liberia: A 431

Controlled Before-and-After Evaluation

Ministry of Health. Revised National Community Health Services Policy

Ministry of Health

Liberia's Community Health 437

Assistant Program: Scale, Quality, and Resilience. Glob Health Sci Pract

Community Health Workers in Liberia [Internet]. Exemplars in Global Health

MD: ICF

Using propensity 450 scores in difference-in-differences models to estimate the effects of a policy change

Moving towards best practice when using inverse probability of treatment 453

Lay health workers in primary and community health care for maternal and child health and the 474 management of infectious diseases. Cochrane Database Syst Rev

Comprehensive review of the evidence regarding 477 the effectiveness of community-based primary health care in improving maternal, neonatal and 478 child health: 7. shared characteristics of projects with evidence of long-term mortality impact

Community Health Workers Are Effective but How to Scale-Up?

2018: successes and challenges toward universal coverage for primary healthcare 484 services. Glob Health

Ethiopia's Health Extension Program on Maternal and Newborn Health Care Practices

Districts: A Dose-Response Study

Community Case Management of Childhood Illness Strategy on Child Mortality in Ethiopia: A 490

Cluster Randomized Trial

Evaluating the impact of community-based health interventions

from Brazil's Family Health Program

Institute for the Study of Labor

Case studies of large-scale 495 community health worker programs

Comprehensive review of 499 the evidence regarding the effectiveness of community-based primary health care in improving 500 maternal, neonatal and child health: 5. equity effects for neonates and children

Proximity to health services and child survival in low-and middle-503 income countries: a systematic review and meta-analysis

Assessing the impact of integrated community 505 case management (iCCM) programs on child mortality: Review of early results and lessons learned 506 in sub-Saharan Africa

Measuring Coverage in 508 MNCH: Accuracy of Measuring Diagnosis and Treatment of Childhood Malaria from Household 509 Surveys in Zambia. Osrin D, editor

Validation Relaxation: A Quality 511

Assurance Strategy for Electronic Data Collection

GBD Compare. 2021