key: cord-0957499-a13qmppb
authors: Broomhead, Sean C.; Mars, Maurice; Scott, Richard E.; Jones, Tom
title: EHealth Investment Appraisal in Africa: A Scoping Review
date: 2021-12-14
journal: Inquiry
DOI: 10.1177/00469580211059999
sha: fddcb155a23e8214bca36a4418d91041c51a4545
doc_id: 957499
cord_uid: a13qmppb

eHealth is an opportunity cost, competing for limited available funds with other health priorities such as clinics, vaccinations, medicines and even salaries. As such, it should be appraised for probable impact prior to allocation of funds. This is especially pertinent as recognition grows for the role of eHealth in attaining Universal Health Coverage. Despite optimism about eHealth’s potential role, in Africa there remain insufficient data and skills for adequate economic appraisals to select optimal investments from numerous competing initiatives. The aim of this review is to identify eHealth investment appraisal approaches and tools that have been used in African countries, describe their characteristics and make recommendations regarding African eHealth investment appraisal in the face of limited data and expertise. Methods: Literature on eHealth investment appraisals conducted in African countries and published between January 1, 2010 and June 30, 2020 was reviewed. Selected papers’ investment appraisal characteristics were assessed using the Joanna Briggs Institute checklist for economic evaluations and a newly developed Five-Case Model for Digital Health (FCM-DH) checklist for investment appraisal. 5 papers met inclusion criteria. Their assessments revealed important appraisal gaps. In particular, none of the papers addressed risk exposure, affordability, adjustment for optimism bias, clear delivery milestones, practical plans for implementation, change management or procurement, and only 1 paper described plans for building partnerships. Discussion: Using this insight, an extended 5-Case Model is proposed as the foundation of an African eHealth investment appraisal framework. This, combined with building local eHealth appraisal capabilities, may promote optimal eHealth investment decisions, strengthen implementations and improve the number and quality of related publications.

Ehealth is an opportunity cost that will utilise public funds that could otherwise have been used to fund alternate health priorities such as building clinics, rolling out vaccination programmes or hiring more healthcare staff. Since healthcare funds are limited, the case for investing in eHealth should be justified with evidence that the investment will produce sufficient net benefits over time, to satisfy its stakeholders. Consequently, eHealth should not attract public investment until its probable impacts, financial and otherwise, have been appraised. 1 This applies especially in lowerand middle-income countries (LMICs) such as those in sub-Saharan Africa, where annual health expenditure per capita ($83,25 in 2018) is less than 1 10th of the global average ($1111.08 in 2018), 2 and as recognition grows for the role of digital health in attaining Universal Health Coverage (UHC) in Africa. 3, 4 The World Health Organization (WHO) continues to encourage member states to adopt digital health 5 across a wide range of applications, 6 expanding more than a decade of effort underway through the World Health Assembly 7 and the WHO Regional Office for Africa. 8, 9 The principle remains relevant whether the term is eHealth or digital health 10 and despite any perceived differences in their meanings. 11 Countries strive to bridge the digital divide, expand connectivity and grow their digital economies, 12 now heightened by COVID-19 experiences, 12, 13 and highlighted by whether or not people have access to advanced technologies such as smartphones 14 and the internet-of-things. 15 Access to digital tools has begun to function as a health determinant 16 and there is an increased opportunity for eHealth to expand its role in health systems' strengthening. 17 While economic appraisal can estimate cost-effectiveness and distinguish between more or less favourable innovations, 18 there is a need for investment appraisal to go beyond this to help identify what is needed for a potentially cost-effective solution to provide the most value. 19 Experience during the COVID-19 pandemic has shown rapid uptake of digital health 13, 17 without clarity on whether there has been appraisal of the impact of these investments. Numerous digital initiatives are underway to help contain the virus, including tools for remote consultation, 20,21 contact tracing 22 and patient management. 23 While there is recognition that, ideally, these require investment appraisal, 1 the reality of the pressures of the pandemic needs agile and innovative responses. Nonetheless, responsible research and innovation must select the most appropriate digital health initiatives and requires investment appraisal techniques that can be applied with available but incomplete data and limited economic and investment appraisal skills. This need for pragmatism is consistent with conditions in many African countries.

Given above, it is more crucial than ever to understand and employ appraisal options to enable favourable investment decisions. Investment appraisal is a process 'to evaluate the worth, significance or status of 24 a financial investment. It has been described as 'a collection of techniques used to identify the attractiveness of an investment', with its goals being to assess the likelihood of achieving identified objectives and to support the production of a viable business case. 25 Methods include average rate of return, discounted cash flow techniques such as internal rate of return or net present value and payback period. 26 eHealth investment appraisal is therefore a process to evaluate which information and communication technology investment in health produces optimal net benefits. To arrive at these benefits it should consider outcomes across the health and social contexts in which eHealth is implemented, acknowledging the different stakeholders it is intended to benefit, each of which may have a different perspective on value and benefits. 27, 28 Recent initiatives such as the Digital Investment Implementation Guide (DIIG) 29 exemplify the feeling of urgency regarding understanding the eHealth investment process. However limitations to current tools exist, including DIIG, in terms of the breadth of issues considered, the depth of appraisal of each issue and the applicability and utility of tools for non-economist decision makers.

Two commonly recommended digital health appraisal techniques are Cost-effectiveness Analysis (CEA) and Costbenefit Analysis (CBA). The data and expertise required to perform them are demanding, and in short supply in LMICs, particularly at sub-national locales. 30, 31 Data issues include 'relevant data are not local; are incomplete, unreliable or imprecise; are challenged by experts or are completely absent and skill-related issues include the lack of 'high quality analytical capacity'. 32 Consequently, tools such as CBA and CEA may be impractical for general and regular use as decision making tools in resource constrained LMIC settings.

Few African eHealth initiatives undergo economic assessment 33, 34 with some countries reporting little prior appraisal of any kind for eHealth projects. 35, 36 It is not clear whether lack of analytical capacity is the main barrier, though it is likely to be an important factor. Nevertheless, it is unlikely that this expertise will become readily available across Africa's extensive health service delivery landscape in the short term. In the meantime, health-strengthening capabilities of digital health are no longer disputed and are deemed necessary to meet UHC objectives. 3 An eHealth investment appraisal approach is required that can be readily used by officials who have limited economics training or experience in identifying optimal investments, especially related to eHealth.

Internationally recognised metrics for eHealth investment appraisal have been proposed and used to demonstrate the applicability of the 5-Case Model (FCM) to African countries. 1 The aim of this scoping review is to identify and assess eHealth investment appraisal approaches and tools that have been used in African countries, describe their characteristics and assess their utility. Based on the findings, recommendations regarding eHealth investment appraisal are presented to inform development of an evidence-based 'eHealth investment appraisal framework' suitable for African settings.

Three electronic databases 'suitable for use as principal search systems' and 'well-suited to evidence synthesis' 37 (PubMed, Scopus and Science Direct) were searched for articles published between January 1, 2010 and June 30, 2020, since reviews had noted that there were few studies of good quality identified prior to that. [38] [39] [40] Inclusion criteria were that the paper addressed any form of eHealth or digital health investment appraisal in Africa and was published in English. Conference proceedings were excluded.

Search terms used MeSH indexing and free text terms 'electronic health records', 'telemedicine', 'e-health', 'ehealth', 'digital health', 'developing country', 'Africa', 'economic', 'business case' and 'finance'. One author (SB) conducted the searches and removed duplicates. Thereafter, all authors screened the abstracts and titles independently to determine inclusion status. Discrepancies were discussed, and consensus reached. Full texts of selected resources were retrieved and reviewed by 3 authors (SB, MM and RES) using the inclusion criteria. Discrepancies were again discussed and consensus reached to determine final selection of papers.

Data charting from eligible papers was conducted by 1 author (SB) and verified by 2 authors (MM and RES). Charted data included article title; first author name; publication date; African country or countries and African region; type of digital health initiative; clinical discipline; investment appraisal methods; investment appraisal characteristics and any recommendations made by authors regarding eHealth investment appraisal in African countries. Due to the absence of eHealth-specific tools for assessing the characteristics and quality of investment appraisal approaches used in each paper, 2 tools were used in combination: the Joanna Briggs Institute (JBI) critical appraisal checklist for economic evaluations 41 and the FCM for Digital Health (FCM-DH) checklist for investment appraisal, shown in Table 1 . These were chosen because the JBI checklist identifies key issues required in reviews of economic evaluations 42 and the FCM, considered applicable to African eHealth investment decisions, 1 could be adapted for digital health creating the FCM-DH checklist based on the FCM Checklist for Assessment of Business Cases. 43 Together, the tools combine economic evaluation and digital health perspectives to assess the quality of digital health investment appraisal.

A three-point Likert scale (yes, unclear or no) was used to indicate the level of agreement with each checklist question. While higher numbers of Likert scale response options can improve reliability and validity of psychometric studies, 44 the three-point scale was deemed appropriate for this study to identify whether or not each appraisal characteristic was utilised by an African eHealth investment appraisal. The assessment was conducted by 3 authors (SB, MM and RES) and discrepancies discussed until consensus was reached. A narrative approach was used to summarise findings and comment on the included studies' overall relevance to eHealth investment appraisal in Africa and any implications for developing an eHealth investment appraisal framework.

The search process identified 488 articles, reduced to 468 after removing duplicates and articles not in English. Screening of abstracts found 22 articles of which, after full paper review, 5 met inclusion criteria ( Figure 1 ). They are listed and categorised in Table 2 .

Checklist responses were either 'yes' or 'no'; no 'unclear' responses occurred. These findings are summarised in Table  3 . Mean 'yes' scores for the 5 selected papers were 26% for the FCM-DH checklist, 44% for the JBI checklist and 34% for the aggregate of both checklists.

All 5 papers provided a well-defined research question (JBI1). Four papers provided a case for change (FCM-DH1) and strategic fit between health strategy and the digital health initiative (FCM-DH2). [45] [46] [47] [48] The role of digital health in extending access to healthcare services was emphasised in 4 of the 5 studies. [45] [46] [47] [48] 4 studies reported on mobile health implementations; 1 was an initiative supporting AIDS care 49 and the other 3 varying aspects of Reproductive, Maternal, Neonatal and Child health (RMNCH). 45, 47, 48 The fifth study dealt with costing a hypothetical teleradiology initative. 46 While all but 1 study achieved the JBI criterion of a description of alternatives (JBI2), including comparison between an intervention and a comparator, 46 only 1 study met the more stringent criterion of analysis of intervention alternatives (FCM-DH3). 47 The investment analyses reported were simple costings 46,49 ; 1 break-even analysis, 47 1 CEA 45 and 1 cost-outcome analysis (COA). 48 All 5 papers identified costs. Three papers identified costs and benefits (JBI3, FCM-DH4), 45,48,49 2 of which identified all important and relevant costs and outcomes for each alternative and measured costs and outcomes accurately (JBI5), 45,48 for example, using 4 different costing tools to collect costs in 20 facilities. 45 2 papers measured costs and consequences in their control groups. 45, 49 Costs and outcomes were valued credibly (JBI6) in the only paper to establish clinical effectiveness (JBI4). 45 1 paper adjusted costs and outcomes for differential timing (JBI7), 48 no paper conducted an incremental analysis of costs and outcomes (JBI8) and 4 studies conducted a sensitivity analysis (JBI9, FCM-DH5) to investigate the uncertainty of estimates for costs and consequences. 45, [47] [48] [49] No paper addressed risk exposure (FCM-DH6) or adjusted for optimism bias (FCM-DH7). Neither did they provide clear practical plans for implementation (FCM-DH9) and delivery milestones (FCM-DH10) nor change management (FCM-DH11) or procurement (FCM-DH12). None of the papers addressed availability of data. One paper described plans for building partnerships (FCM-DH13). 47 Two papers reported results regarded as generalisable for the setting of interest in the review (JBI11), 46,47 and 1 addressed generalisability by substantially describing 4 different cost-recovery scenarios and using uncertainty analysis to demonstrate that break-even points were driven by user volume rather than variations in programme costs. 47 No paper assessed all issues identified as of concern to users (JBI10).

The range of scale of the reported projects varied from 20 facilities to 125,320 users. Affordability challenges for poor users were noted 45, 47 and none of the papers addressed affordability adequately (FCM-DH8).

One paper recognised connectivity as a pre-requisite for digital health and the need for minimum functional bandwidth to achieve large scale implementations. 46 Cloud computing was a suggested approach.

Collaboration with international non-governmental organisations was identified as helpful, citing WHO, International Atomic Energy Agency, International Telecommunicatuions Union and policy statements such as the Smart Africa Manifesto. 46 The importance of the global digital health community helping to create successful and socially desirable business models was also noted. 47 

This scoping review identifies the general absence of adequate investment appraisal of African digital health initiatives in studies published to June 2020, something specifically noted in 3 of the 5 papers, and emphasises the importance of the work presented here. 45, 47, 48 While they may have been fit for their original purpose, the 5 papers were consistent in their absence of important investment appraisal characteristics but inconsistent in their approaches, using simple costings, break-even analysis, CEA or COA. The choice of these assessment approaches may be explained by the predominance of mobile health initiatives (4 of the 5 studies identified) 45,47-49 since CEA and COA are well suited for appraisal of mobile health's relatively welldefined business cases and value chains. The relative simplicity of these investment appraisal techniques, which are easier to apply for users without advanced health economics expertise, may further explain the predominance of mobile health initiatives in the selected studies. More complex initiatives such as electronic patient records, hospital information systems and routine health information systems require more sophisticated approaches such as CBA. The predominance of mobile health projects (4 out of 5) is not surprising. The opportunity for mobile phones to contribute to advancing UHC has been recognised by WHO 50 and WHO Guidelines on Digital Health provide extensive mobile health use cases. 51 Furthermore, the WHO guidelines emphasise the role of mHealth in targeted client communication specifically addressing the use of mobile phones to support communication for sexual, reproductive, maternal, newborn, child and adolescent health interventions. This role of mHealth to support RMNCH is also well established [52] [53] [54] and consistent with the focus on maternal and child health strengthening between 1990 and 2015 in the Millennium Development Goals, 55 and now addressed in goals focussing on good health and well-being, and gender equality, in the Sustainable Development Goals. 56 This may explain the high proportion of papers on RMNCH projects (3 out of 5).

Experience and research has emphasised the importance of needs-based interventions that respond to identified health priorities. Thus, the momentum initiative in Europe noted 'Identify a compelling need' as a critical success factor for telemedicine interventions, 57 and prior research noted the need to find an 'optimal solution to the most pressing (existing or anticipated) health-related problems' for successful eHealth strategies. 58 The included papers align with health priorities with all but 1 study 49 articulating a case for change and strategic fit between the health strategy and digital health initiative. The issue is no longer whether a digital health initiative is aligned to health-strengthening needs, but rather whichfrom a number of optionsoffers the optimal solution and what investment appraisal approach should be used to make the case. There was a disjunction between the recognition of the importance of appraisal techniques, and the actual application of these techniques in the reviewed papers. For example, whilst 3 studies identified costs and benefts, 45, 48, 49 only 2 measured them, 45, 48 and only 1 of those valued them credibly. 45 Similarly, while 4 studies conducted sensitivity analysis, the absence of credible values assigned to costs and benefits made these analyses difficult to interpret. 45, [47] [48] [49] Noting that 40% of sub-Saharan Africans live on less than $1.90 a day, 59 sustainable digital health investments in Africa will need to be affordable and support other costcontained strategies enabled by digital transformation. 46 This reinforces the important role of investment appraisal to help select optimal investments from numerous competing initiatives.

The role of partnerships to reduce communications cost was emphasised, 47 as well as ethical issues arising from certain cost reduction strategies such as data mining and advertising, which need further exploration. One paper 49 noted how an initiative could be 'potentially affordable' pending addressing a number of considerations such as task shifting. The opportunity to transfer costs to users was noted, as well as the concerning potential exclusion of those who are 'too poor to pay'. 49 This highlights the need for a suitable business model, particularly to realise scale. 47 The inconsistent scale of the 5 studies' initiatives made further comparison of scale impractical.

The importance of making informed decisions about optimal allocation of scarce resources when considering investment in eHealth applications is clear. How to perform this task in a straightforward manner in resource-constrained local and national settings is not provided, 29 particularly given data availability and quality challenges and limited training and skills in economic analyses. Whilst 'appraisal checklists' or other reporting guideline tools are available to guide reviews of available evidence, 41,60-64 their applicability for digital health investment appraisal is uncertain. Some are narrow in their scope of application and can be complex, or require specific training or experience to apply correctly. 65, 66 Most are intended to enhance the quality of reported research. To address data and human capability challenges, it was necessary to find a compromise between sophistication and practical utility. To this end, the JBI tool was adopted, together with the adapted FCM tool.

The FCM-DH checklist was newly developed for this review and no evidence could be found of the JBI checklist being used in the manner described. Certainly, the 2 checklists applied have commonalities and differences, and when used in conjunction, they provided a novel, structured approach to the analysis of the investment appraisals reviewed. The FCM-DH checklist addresses characteristics not considered in the JBI checklist, such as exposure to risk, adjusting for optimism bias, having a practical plan for delivery, providing delivery milestones, addressing change management and building partnerships. Similarly, the JBI checklist addresses characteristics not considered by the FCM-DH checklist, such as a more thorough exploration of the extent to which costs and outcomes are evaluated, whether clinical effectiveness was established, whether the appraisals included all issues of concern to users and whether results are generalisable. Using them in conjunction may be an appropriate approach to compensate for the frequent absence of reliable data and lack of skills and ability when performing traditional economic assessments. 32 Neither checklist addresses 2 important factors, usability and sustainability, both necessary to secure eHealth's benefits. 67 None of the 5 papers addressed usability or the related aspects of user acceptance, nor the technical knowledge and support required for productive use of the digital health tools. These contribute to sustainability, as does the existence of a business model, which was also absent from the papers. The lack of business models limits sustainability, increasing reliance on additional funding streams, including donor funding, to ensure continued service.

If either checklist had been used alone to select economic evaluations, all 5 articles would have been excluded. Therefore, despite their individual limitations, inclusion of aspects from both would provide a useful foundation for the development of a strong African-specific eHealth investment framework. The FCM-DH checklist identified the most gaps, and its modification could address missing aspects such as more robust values for costs and benefits, whether a study establishes clinical effectiveness, application usability and sustainable business models. This will provide the foundation for an eHealth investment framework that is accessible to those without high levels of health economics expertise, to guide Africa's eHealth investment decisions.

Reports on the subsequent progress of the 5 initiatives could help to demonstrate the role, if any, that their investment appraisals played in their further development. While we could not find any formal evidence of further development, all 4 mobile health initiatives appear to have continued supported by grant funding, with information about their progress available on related project websites. This raises a sustainability issue that is beyond the scope of our paper; projects that are not able to transition from grant funding to a more sustainable funding model are at risk of not becoming sustainable interventions in the long term. We could not find any evidence of further development of the hypothetical teleradiology initiative described in the fifth paper.

The aim of this scoping review was to identify, characterise and assess eHealth investment appraisal methods used in African countries. The absence of appropriate methods for assessing this in resource-constrained settings led to selection of the JBI checklist and development of the FCM-DH checklist. Their application revealed key missing elements from the few publications found, such as affordability, risk, usability, sustainable business models, credible valuation of costs and benefits, planning and change management and addressing implementation issues.

While neither checklist was sufficient on its own, a combined JBI and FCM-DH checklist could form the foundation of an African eHealth investment appraisal framework. This, combined with modest building of eHealth appraisal capabilities among a cadre of African implementers and decision makers, may stimulate meaningful improvement of investment decisions and complement specialist health economics expertise, which is in short supply. It is unlikely that these concerns are confined to Africa and the findings may be pertinent to lower-and middle-income countries generally.

The findings of this study will facilitate the further development of an African eHealth investment framework.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclose receipt of the following financial support for the research, authorship and publication of this article: This work was supported by the Fogarty International Center of the National Institutes of Health [Award Number D43TW007004-13]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Applicability of the five case model to African eHealth investment decisions

How can digital health technologies contribute to sustainable attainment of universal health coverage in Africa? A perspective

The role of digital strategies in financing health care for universal health coverage in low-and middle-income countries. Glob Health: Science and Practice

Classification of Digital Health Interventions v1.0: A Shared Language to Describe the Uses of Digital Technology for Health

Resolution WHA.58.28. eHealth. Fifty-eighth World Health Assembly. Geneva: World Health Organization

Resolution: Ehealth Solutions in the African Region Current Context and Perspectives. Document AFR/RC60/5). Brazzaville: World Health Organization Regional Office for Africa

Utilizing eHealth Solutions to Improve National Health Systems in the African Region

Here we go again -'digital health

Would a rose by any other name -cause such confusion?

International Telecommunication Union and United Nations Educational, Scientific and Cultural Organization

Digital Technologies and the COVID19 Pandemic, Briefing & Learning Note. Barcelona: United Cities and Local Governments

Smartphone internet access and use: extending the digital divide and usage gap

Any thing for anyone? A new digital divide in internet-of-things skills

COVID-19 and digital inequalities: reciprocal impacts and mitigation strategies

COVID-19: a window of opportunity for positive healthcare reforms

Commercial viability of medical devices using Headroom and return on investment calculation

Problems and promises of health technologies: the role of early health economic modeling

Use of telemedicine and virtual care for remote treatment in response to COVID-19 pandemic

COVID-19 transforms health care through telemedicine: evidence from the field

COVID-19 Mobile positioning data contact tracing and patient privacy regulations: exploratory search of global response strategies and the use of digital tools in nigeria

The use of digital health in the detection and management of COVID-19

Business Dictionary

Identifying where the values come from ITinnovations in health and social care

A methodological and operativeframework for the evaluation of ane-health project

World Health Organization

Strengthening expertise for health technology assessment and priority-setting in Africa

Growth and capacity for cost-effectiveness analysis in

Economic evaluation for health investments en route to universal health coverage: cost-benefit analysis or cost-effectiveness analysis? Value Health

Economic analysis of sustainable ehealth implementation in developing countries: a systematic reveiw

The economics of ehealth and mhealth

Assessing evaluation of ehealth interventions in uganda: practices, challenges and insights

Assessing the feasibility of ehealth and mhealth: a systematic review and analysis of initiatives implemented in Kenya

Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of google scholar, pubmed, and 26 other resources

Can economic evaluation in telemedicine be trusted? A systematic review of the literature. Cost Effectiveness and Resource Allocation

Economic evaluation of telemedicine: review of the literature and research guidelines for benefit-cost analysis

Effectiveness of telemedicine: a systematic review of reviews

Conducting systematic reviews of economic evaluations

Effect of the number of response categories on the reliability and validity of rating scales

Cost effectiveness of mobile health for antenatal care and facility births in nigeria

A roadmap and cost implications of establishing comprehensive cancer care using a teleradiotherapy network in a group of sub-saharan african countries with no access to radiation therapy

Sustainable cost models for mhealth at scale: modeling program data from m4RH Tanzania

Scale matters: a cost-outcome analysis of an m-health intervention in Malawi

Cost analyses of peer health worker and mHealth support interventions for improving AIDS care in Rakai

Global diffusion of eHealth: making universal health coverage achievable

WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening

Prioritizing integrated mhealth strategies for universal health coverage

Information and Communication Technologies for Women's and Children's Health: A Planning Workbook for Multi-Stakeholder Action. Geneva: World Health Organization: The Partnership for Maternal

mHealth Assessment and Planning for Scale. Geneva: World Health Organization

Momentum. 18 Factors to Make Telemedicine a Success

Principles and framework for eHealth strategy development

Consolidated health economic evaluation reporting standards (CHEERS) statement

Grade: an emerging consensus on rating quality of evidence and strength of recommendations

Developing NICE Guidelines: The Manual. National Institute for Health and Care Excellence

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

The effectiveness and costeffectiveness of hospital-based specialist palliative care for adults with advanced illness and their caregivers

Cochrane Handbook for Systematic Reviews of Interventions

Being spontaneous: the future of telehealth implementation? Telemedicine and e-Health

ORCID iDs Sean C. Broomhead  https://orcid.org/0000-0002-1904-4383 Richard E. Scott  https://orcid.org/0000-0002-9036-7070