key: cord-0016158-ut8fya45
authors: Lilford, Richard; Nepogodiev, Dmitri; Chilton, Peter J; Watson, Samuel I; Erlangga, Darius; Diggle, Peter; Girling, Alan J; Sculpher, Mark
title: Methodological issues in economic evaluations of emergency transport systems in low-income and middle-income countries
date: 2021-03-18
journal: BMJ Glob Health
DOI: 10.1136/bmjgh-2020-004723
sha: 07780101145bcc41341342f87a8c2098e54d77d2
doc_id: 16158
cord_uid: ut8fya45

A recent systematic review identified few papers on the economic evaluation of systems for emergency transport of acutely ill or injured patients. In addition, we found no articles dealing with the methodological challenges posed by such studies in low-income or middle-income countries. We therefore carried out an analysis of issues that are of particular salience to this important topic. This is an intellectual study in which we develop models, identify their limitations, suggest potential extensions to the models and discuss priorities for empirical studies to populate models. First, we develop a general model to calculate changes in survival contingent on the reduced time to treatment that an emergency transport system is designed to achieve. Second, we develop a model to estimate transfer times over an area that will be served by a proposed transfer system. Third, we discuss difficulties in obtaining parameters with which to populate the models. Fourth, we discuss costs, both direct and indirect, of an emergency transfer service. Fifth, we discuss the issue that outcomes other than survival should be considered and that the effects of a service are a weighted sum over all the conditions and severities for which the service caters. Lastly, based on the above work, we identify priorities for research. To our knowledge, this is the first study to identify and frame issues in the health economics of acute transfer systems and to develop models to calculate survival rates from basic parameters, such as time delay/survival relationships, that vary by intervention type and context.

Decisions about implementing new interventions need to reflect resource constraints; evidence-informed decisions balance the benefits of a potential intervention against the benefits that could be achieved in other ways with the same resources. Economic evaluation studies inform these decisions and generic guidelines have been drawn up to guide health economic evaluations. 1 2 Each type of evaluation creates particular challenges and a literature has developed to inform decision models for interventions covering surgery, [3] [4] [5] [6] treatments that not everyone will choose even when they are freely available, 7 8 and service/policy interventions, [9] [10] [11] [12] [13] [14] [15] for example. We have been working on health economic models to inform investment decisions concerning emergency transport systems, such as motorised ambulance systems, in low/middle-income countries (LMICs). During the course of this work, we encountered a number of methodological issues, and turned to the literature for help. We have updated a systematic review published in 2017, 16 but found only 12 articles concerned with the economic evaluation of emergency transport in LMICs, none of which dealt with methodological issues (Erlannga, Lilford. Article in preparation. 2021). Our aim in this article is to describe issues in health economic assessment of emergency transport systems and discuss potential solutions.

Estimates of intervention effectiveness are often obtained by means of studies with Summary box ► Despite the importance of the topic, there is very little literature available on economic evaluation of emergency transport in healthcare in low-income and middle-income countries, and none dealing with the methodological issues that this topic throws up. ► We develop a basic model for a decision maker to calculate cost per life saved for a particular condition from the correlation between survival and treatment delay, and the conditions affecting delay in a particular locality. ► We discuss how this model may be extended to deal with all emergencies, outcomes apart from survival and provision of prehospital care. ► We identify priorities for empirical studies to populate models.

BMJ Global Health contemporaneous controls. In policy and service delivery research, these studies often take the form of cluster studies. Ideally such studies would be randomised and would include baseline observations. 17 However, our systematic review (cited above) identified no controlled studies of the effectiveness of emergency transport systems in LMICs. Moreover, even if we had found a randomised controlled study of the effectiveness of a transport system in one place, the estimate could not be used directly in a cost-effectiveness analysis in another place. This is for two reasons. First, there is a range of service specifications: transport for all emergencies versus specific conditions (eg, road traffic incidents or maternity care); transport types (eg, motor vehicle vs motorcycle with side-car) and workforce configurations (eg, with or without paramedical support). Second, the effectiveness of a given emergency transport system depends on the variegated physical factors that affect transfer times-poor or nonexistent roads in Liberia, dense jungles in Papua New Guinea, swollen rivers in Mozambique, steep slippery terrain in Nepal, dense traffic in Lagos and so on. In short, a single effectiveness measure cannot suffice across the range of the different service configurations and geographical contexts. Models are therefore required to estimate the effectiveness of a particular transport system in a particular place.

The benefits of an emergency transport system are mediated by reducing the time delay between injury or the onset of illness and treatment. The model we propose is thus built around the relationship between time and survival. We are concerned here with delay in reaching the facility where treatment will be given. We are not concerned with any delay in receiving care once an appropriate facility has been reached (sometimes called the 'third delay') because this is not amenable to a transport system. However, we cannot ignore the question of whether, or when, a decision to seek care is made (the 'first delay'). This is because an emergency transport system may influence whether care is sought at all-a kind of supply-induced demand. 18 We will therefore 'start the clock' at the point where a patient or carer recognises an emergency health need. We will describe the delay between this point and arrival at the facility by the shorthand 'transfer delay' to signify the component delay that can be reduced by a transport system. We illustrate the model outputs with a theoretical population of 100 000 over a 1-year period. We propose a basic model in the first instance dealing with survival only and assuming no treatment 'en route', leaving permanent disability and paramedical services to future developments of the model. We will motivate our model through the lens of a singlepresenting condition-postpartum haemorrhage-as an example of an archetypal time-critical condition. We address the problem of consolidating costs and effects across all emergency conditions for later discussion.

We need to calculate the death rate for this population over 1 year, absent a formal transport system. Then, we will calculate the mortality given a transport system assuming, in the first instance, that it has sufficient coverage to transport all who need the service. Lives saved are based on subtracting the latter from the former. In figure 1 , curve A represents the probability, S(t), of survival as a function of transfer delay, t, where t=0 is the moment a care need is recognised. Curve A becomes 'flat' after a time in recognition of the fact that in most conditions some will survive even if they never reach an appropriate facility. The origin of curve A starts at a figure lower than 100% on the y axis in many (most) circumstances where some deaths are inevitable. Curve B is proportional to the probability distribution, f(t), of the time taken from recognising a care need to arriving at a care facility (transfer delay) and is intended to represent a plausible scenario in some part of the world. Curve C is the counterfactual probability distribution, f*(t), under a proposed new transport system.

In online supplemental appendix 1, we develop a mathematical model to calculate the survival gain (per 100 000 of population), contingent on reducing the distribution of transfer delays from curve B to curve C.

However, transfer times at baseline and the extent to which a transport service may reduce these times, will vary widely by region and country. These time estimates depend on local geographical variables, particularly population density and travel times. For example, population density is greatest in cities where people gain less in terms of transfer times than in more remote locations. Moreover, the propensity to seek care might be affected by location. Rather than specifying arbitrary probability distributions, such as curves B and C in figure 1 , it would be more accurate to average over the non-uniform distribution of population density and transfer times across the area of interest, in calculating expected gains in survival.

In online supplemental appendix 2, we develop a mathematical model to calculate a distribution of transfer delays from population densities, distances and 'transfer delays' by location.

Populating the basic model: parameters needed to calculate effectiveness The relationship between delay and survival A number of empirical studies concerned with the effect of delay on survival have used distance from facility as a proxy for delay. This method is flawed because such studies are subject to 'survival' bias: bias resulting from factors that prevent a person from being included in the denominator. 19 The extent of the bias might be sufficient to reverse the association between distance and outcome, BMJ Global Health such that those living further away have a lower observed rate of death than those living nearby-a phenomenon that has been observed in both high-income 20 and lowincome 21 settings. Distance as a proxy for survival cannot be recommended for use in decision models.

Studies that focus on the relationship between time and outcome are not without problems either. The literature is limited. In LMICs, we have found the largest literature on delay survival relationships in the context of snake bite, 22-27 followed by obstetrics. 28 There is a limited literature from high-income countries covering trauma [29] [30] [31] [32] and stroke. 33 The literature relating to paediatric emergencies and 'the acute abdomen' is sparse. Moreover, as stated, it is important to be aware that introduction of a new emergency transport system may be associated with people accessing a service who previously would not have done so, creating a kind of 'survivor bias'. This bias is unlikely to apply to major trauma or interfacility transfer, where there is likely complete enumeration of cases, but could be a significant bias in conditions such as acute childhood illness, where carers do not seek treatment in a substantial and variable proportion of cases. 34 Where the parameter estimates are not available, very sparse or potentially biased, then expert opinion should be used 35 in the form of probability densities. 36 The methods to elicit these 'Bayesian' probability densities have been described elsewhere. [37] [38] [39] [40] Probabilistic sensitivity analyses can then be used to consolidate the many uncertainties mentioned above. 41 By producing a range of estimates on the effect of reducing delay as a proxy for survival, local decision-makers can adjust the model to their context.

As stated above, two types of data are needed to calculate aggregate transfer delays under an existing and a proposed emergency transfer specification: local population densities and transfer times from different locations in an area. Evidence is available on population densities, since gridded (point) estimates of population density are available for the whole planet up to a resolution of 1 km 2 , for example, from WorldPop. 42 Estimating transfer times is more difficult. Travel times round the globe by the fastest mode of possible transport under normal (nonemergency) conditions, can be obtained from publicly available data. 43 However, these data do not represent transfer delays under either baseline or counterfactual circumstances. This is because transfer delays are affected Figure 1 Postpartum haemorrhage. Curve A is the survival curve by time from recognition of the care need to arrival at the facility that will provide care, curve B represents the counterfactual time distribution from recognition of the need for facility care to arrival at the facility (transfer delay) under baseline conditions and curve C is the transfer delay under a proposed transport system. In order to construct curves B and C, it is necessary only to specify the shape of the distribution (assumed to be normal above), the mean transfer delays (currently and after introduction of the service) and the SD (assumed above to be the same for both curves). The uncertainty in the estimated mean would also be specified for a probabilistic analysis.

by delays in mobilising vehicles, need for round trips, different speeds allowed in emergency vehicles, loading patients into vehicles and so on. And transfer delay also has to take induced supply demand into account. There is thus a great need for empirical investigation and development of exemplar models (discussed further later and in online supplemental appendix 2).

The rates per year for various emergency conditions (R in equations (3) and (4), online supplemental appendix 1) are published for populations of 100 000 by the Global Burden of Disease studies. 44 Modellers and users of models need to be wary of pitfalls in this step. First, definitions of conditions are applied very differently in different places. Second, and arguably more problematic, is that the definition may not fit the kind of person who is transferred. For example, the definition of postpartum haemorrhage would not necessarily correspond to the type of patient referred to hospital. In that case, the referral rates to which survival data apply may include people at higher or lower risk of survival than those to whom the survival data apply.

While production costs for vehicles and the costs of staff to operate them may be reasonably stable, the costs the service must pay are influenced by a range of issues. 45 For example, the price of an ambulance service is subject to local negotiation. The period over which fixed costs are amortised is also a local matter. Different terrain places different levels of wear and tear on vehicles. Peaks and troughs in demand must be considered, since the transport system must either tolerate clashes when not all needs can be served (thereby reducing the effectiveness of the system) or it must create redundancy (thereby driving up costs). These costs depend on the type of service contract negotiated. The notion that a given configuration of transport vehicles is or is not cost-effective is thus not tenable; just as benefits vary by context as described above, so do costs. The resources required can be described in published literature, as recommended in a recent guideline, 46 while local decision-makers are wellplaced position to translate these into costs that reflect local contexts.

In order to estimate net costs, it is necessary to include an estimate of the extent to which hospital costs may be affected by a change in the number of people reaching hospital when a service is provided versus when it is not-supply-induced demand. Like other community interventions, an emergency transport service may move bottlenecks 'upstream'. In the single study we located on this topic, a voucher scheme for round-trip transport to a hospital for obstetric care in Uganda resulted in a threefold increase in referral compared with a control district. 47 More studies of this type are required, as we discuss later, but examination of verbal autopsy reports, such as the Million Deaths Study in India, 34 can provide an indication of 'pent-up' demand.

Here we discuss two broader issues relating to the decision problem: the type of service (comprehensive or condition specific) and the issue of the broader system in which a transport system operates.

Ambulance services may provide open access across all emergency (time critical) conditions. Alternatively, an ambulance service may be hypothecated to a particular 'client group': road traffic incident victims or obstetric services, for example. 48 49 Low-income countries may implement condition-specific (hypothecated) services with a view to upgrading to comprehensive services as their economies develop. It is easier to compute the benefits and costs of an ambulance service over a specific client group than for a service designed to cater for all emergencies. When the service is comprehensive, covering all emergencies, then the costs and effects must be averaged over all the conditions for which it is deployed, weighted by patient volumes. This is an enormous and potentially open-ended task. An approach to this sort of problem is to select an 'inframarginal' set of presenting conditions where we expect the greatest value from provision of a service. In the context of emergency transport, the inframarginal group may be restricted to major categories of emergency condition, such as major trauma, obstetric emergency, life-threatening infection in a child and 'the acute abdomen'. The cost-effectiveness of the service is then aggregated to a population level. If the positive effect among the inframarginal group is sufficient when covering the fixed cost of the service, the marginal costs and benefits of adding other presenting conditions when spare capacity exists can be considered. That said, the issue of supply-induced demand described above must also be considered, and this issue is likely to be greater in a system catering for all perceived emergencies than a system targeted at a specific clinical groups, such as childbirth or road traffic incidents.

We have argued that emergency transport systems have knock-on effects on hospitals. In post-conflict situations, where there are no hospitals worthy of the name, it makes no sense to fund an emergency transport system. As the secondary care system of hospitals develops, so the value of transport systems will increase. There must be a theoretical optimal development pathway that could be defined in health economic (or indeed in broader economic) terms. Such a system-wide approach to service development is advocated by the WHO Choices Guidelines. 50 Such an approach, while perhaps optimal, does not vitiate the model proposed here; a model for reduced BMJ Global Health transfer time would be an essential part of a model that sought to optimise value for money by developing services iteratively to follow an optimised pathway. We also note, that optimal as such a system-wide model might be, decision-makers are often, or usually, presented with more specific investment opportunities.

The systematic review we quote above shows that there is little or no methodological literature in the field of economic evaluation of emergency transport. In addition, we do not find any comparative studies of the effectiveness of emergency transport systems in LMICs. Even if such studies existed, it would still be necessary to extrapolate beyond the observed effects, given the wide variety of system specifications and contexts in which they might be used. Modelling will therefore always be required in the field of emergency transport economics, although, as we discuss below, better data are needed to populate models. A crucial insight from this study is that the effects of an emergency transport system, and therefore the benefits that might derive, turn on reducing the delay to treatment (including infinite delay for the proportion of people who otherwise would not seek care at all). Thus, 'reduced delay' can be thought of as a type of proxy for the intervention; first estimate the effect a proposed intervention will have on delay and then use that estimate to calculate the contingent survival effect over the population of interest (equation (9), online supplemental appendix 2 refers).

The role of the decision-maker in framing and populating the model The high-context dependency of factors, such as reductions in transport time, means that some of the parameters or distributions required as inputs for the models simply cannot be taken from the literature and applied without modification to a new situation. This means that local decision-makers are not just asked to use judgement in interpreting the model, but must actually contribute to parameter estimates with which to populate the model. Estimating health benefit Delay and outcome More and better data on delay outcome correlations are needed but these have proven hard to come by, save for certain specific diseases such as thrombotic stroke.

The literature on delay and outcome focuses on death with some notable exceptions, such as stroke, in high-income countries. 52 Yet, the effects of delay may involve many outcomes, including many types of permanent and short-term physical and mental effects. Work is needed on prognosis for survivors to inform cohort or simulation models of future mortality, quality of life and costs.

Paramedical services Delay in getting to hospital and delay in the start of treatment differ according to the availability of people and facilities en route and how they are used. This topic has been studied in high-income countries, but remains a contentious issue in the medical literature. 53 54 Reducing delay by means of transport systems Current transfer times Data are needed over a range of 'specimen' geographical and service configuration scenarios to describe how people somehow manage to reach hospital even when there is no formal transport system in place.

Real-time studies of implementations of improved transport services are required across service specification and contextual variables. We need to better estimate how services change proportions who set out for a facility, the time taken to reach a decision to try to reach a facility and transfer times.

The above studies should document changes in pathways to care, since delay may result when a patient is taken to an inappropriate venue.

Building on the above, we need worked models of changes in call-out to arrival times over a range of different types of geographical area.

Implementation costs More and better information is needed on the cost of ambulance services by types and number of vehicles, human resources and type of procurement contract. We have found that policy leads are reluctant to divulge this information which is often locally negotiated.

Operational effects It would be useful to collect data on the costs of maintaining different types of vehicle in the field and how these costs are affected by the type of contract.

The size of vehicle fleet needed to cope with peaks and troughs and delays when repairs and spare parts are acquired should be explored using Queue theory.

Paramedical services Costs of paramedical services to offset against benefits above, by reason for call-out. Knock-on effects The effects of transport services on demand (both intended and unintended) are tractable and should be more thoroughly researched.

The corollary is that, where possible, decision-makers and modellers should work together. However, health economists are a scarce resource, and models are often published in the literature with the intention that they will inform practice. Given a range of published models over a wide range of scenario types, a decision-maker in a particular place could select and adapt the scenario most similar to the local context.

Pending further research, 'back-of-the-envelope calculations' or simple models to shed some light on a decision may help. For example, the cost-effectiveness equation model can be rearranged in such a way as to find out whether a local intervention of known cost is likely to fall below the cost-effectiveness threshold for the country concerned. Thus, given an appropriate threshold 51 and a net cost, the model could define how effective the intervention would have to be in order to be cost-effective in a given economy.

A non-exhaustive list of research imperatives is given in table 1.

We present some issues that should be considered by anyone attempting cost-effectiveness modelling of transport systems in LMICs. There is a shortfall in health economic studies of emergency transport systems in healthcare, and we are not aware of any methodological literature on this subject. Our model is basic but can be extended to include outcomes apart from survival, prehospital care and other considerations. We offer our analysis as an early contribution to an important, underresearched and hopefully burgeoning field of enquiry.

Contributors The initial draft was written by RL, DN, PJC and DE. Statistical support was provided by SIW, PD, AJG and MS. All authors contributed to revisions of the manuscript All authors read and approved the submitted manuscript.

Funding RL and PJC were funded by the National Institute of Health Research (NIHR) Applied Research Collaboration (ARC) West Midlands. RL and DN were also funded by the NIHR Global Health Research Units on Global Surgery and Improving Health in Slums using UK aid from the UK Government to support health research.

Disclaimer This paper presents independent research and the views expressed are those of the authors and not necessarily those of the NIHR or the UK Department of Health and Social Care.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement All data relevant to the study are included in the article, or uploaded as supplementary information.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Clinical guidelines: a NICE way to introduce costeffectiveness considerations?

Recommendations for conduct, methodological practices, and reporting of costeffectiveness analyses: second panel on cost-effectiveness in health and medicine

Evaluation and stages of surgical innovations

Challenges in evaluating surgical innovation

No surgical innovation without evaluation: the ideal recommendations

Underpowered studies

Cost-Utility analysis when not everyone wants the treatment: modeling split-choice bias

Assessing the long-term effectiveness of interferon-beta and glatiramer acetate in multiple sclerosis: final 10-year results from the UK multiple sclerosis risksharing scheme

Evaluating the health impact of largescale public policy changes: classical and novel approaches

Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues

The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting

Evaluating policy and service interventions: framework to guide selection and interpretation of study end points

Health services and delivery research. economic analysis of service and delivery interventions in health care

Challenges, solutions and future directions in the evaluation of service innovations in health care and public health

Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations

Ems systems in Lower-Middle income countries: a literature review

An epistemology of patient safety research: a framework for study design and interpretation. Part 2. study design

The demand for health: a theoretical and empirical investigation

A structural approach to selection bias

Are differences in travel time or distance to healthcare for adults in global North countries associated with an impact on health outcomes? A systematic review

Distance to care, facility delivery and early neonatal mortality in Malawi and Zambia

Profile of snakebite envenoming in Western Maharashtra

Snakebite envenoming in Kerala, South India: clinical profile and factors involved in adverse outcomes

Factors affecting snakebite mortality in north-eastern Nigeria

Predictive factors for death after snake envenomation in Myanmar

Older age and time to medical assistance are associated with severity and mortality of snakebites in the Brazilian Amazon: a case-control study

The epidemiology of snake bite in central Province and national capital district

): the link between travel time and in-hospital maternal mortality

Transfer delay and in-hospital mortality of trauma patients in Pakistan

Does out-of-hospital EMS time affect trauma survival?

Emergency medical services intervals and survival in trauma: assessment of the "golden hour" in a North American prospective cohort

Pre-Hospital transport times and survival for hypotensive patients with penetrating thoracic trauma

Reducing delay in seeking treatment by patients with acute coronary syndrome and stroke

Prospective study of one million deaths in India: rationale, design, and validation results

Bias modelling in evidence synthesis

For debate: the statistical basis of public policy: a paradigm shift is overdue

Bayesian cohort and crosssectional analyses of the pincer trial: a pharmacist-led intervention to reduce medication errors in primary care

Clinical trials and rare diseases: a way out of a conundrum

Uncertain judgements: eliciting experts' probabilities (statistics in practice)

Methods to elicit experts' beliefs over uncertain quantities: application to a cost effectiveness transition model of negative pressure wound therapy for severe pressure ulceration

Decision modelling for health economic evaluation

Population density

A global map of travel time to cities to assess inequalities in accessibility in 2015

Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the global burden of disease study

Pricing of medical devices under coverage uncertainty--a modelling approach

Methods for economic evaluation Project (MEEP. London: National Institute for Health and Care Excellence

Cost-effectiveness analysis of a voucher scheme combined with obstetrical quality improvements: Quasi experimental results from Uganda

An ambulance referral network improves access to emergency obstetric and neonatal care in a district of rural Burundi with high maternal mortality

Ambulance referral for emergency obstetric care in remote settings

Making choices in health: who guide to cost-effectiveness analysis

Country-Level costeffectiveness thresholds: initial estimates and the need for further research

Facility distance and child mortality: a multi-country study of health facility access, service utilization, and child health outcomes

Is it cost effective to introduce paramedic practitioners for older people to the ambulance service? results of a cluster randomised controlled trial

Emergency medical service: delays, response time and survival