key: cord-0079375-wzc83s0v
authors: Donnelly, Dusty-Lee
title: First Do No Harm: Legal Principles Regulating the Future of Artificial Intelligence in Health Care in South Africa
date: 2022-01-11
journal: Potchefstroom Electron Law J
DOI: 10.17159/1727-3781/2022/v25ia11118
sha: ede7434cf20837a41230f6e2ac5c7bc06d50e5ba
doc_id: 79375
cord_uid: wzc83s0v

What sets AI systems and AI-powered medical robots apart from all other forms of advanced medical technology is their ability to operate at least to some degree autonomously from the human health care practitioner and to use machine-learning to generate new, often unforeseen, analysis and predictions. This poses challenges under the current framework of laws, regulations, and ethical guidelines applicable to health care in South Africa. The article outlines these challenges and sets out guiding principles for a normative framework to regulate the use of AI in health care. The article examines three key areas for legal reform in relation to AI in health care. First, it proposes that the regulatory framework for the oversight of software as a medical device needs to be updated to develop frameworks for adequately regulating the use of such new technologies. Secondly, it argues that the present HPCSA guidelines for health care practitioners in South Africa adopt an unduly restrictive approach centred in the outmoded semantics of telemedicine. This may discourage technological innovation that could improve access to health care for all, and as such the guidelines are inconsistent with the national digital health strategy. Thirdly, it examines the common law principles of fault-based liability for medical negligence, which could prove inadequate to provide patients and users of new technologies with redress for harm where fault cannot clearly be attributed to the healthcare practitioner. It argues that consideration should be given to developing a statutory scheme for strict liability, together with mandatory insurance, and appropriate reform of product liability pertaining to technology developers and manufacturers. These legal reforms should not be undertaken without also developing a coherent, human-rights centred policy framework for the ethical use of AI, robotics, and related technologies in health care in South Africa.

From time immemorial doctors have sworn to treat their patients to their greatest ability and to do them no harm. This spirit is retained in the revised Geneva declaration in which doctors also pledge to respect patient autonomy and dignity, eschew discrimination, and maintain patient confidentiality while sharing their medical knowledge in the interests of the patient and the advancement of medicine. 1 But how do regulators ensure that autonomous artificial intelligence (AI) systems, medical robots and related technologies are designed to obey the same laws and ethical codes? This is an urgent question as AI is set to play a growing role in all aspects of public and private health care and health research, including the making of great advancements in clinical diagnostics and decision-making and health care management. For example, during the COVID-19 pandemic AI facilitated disease surveillance and outbreak monitoring across the globe.

The capacity of AI systems to operate at least to some degree autonomously from the human health care practitioner and to use machine-learning to generate new, often unforeseen analyses and predictions is what sets AI systems and AI-powered medical robots apart from all other forms of advanced medical technology. A key priority is to develop laws and policy to support the "ethical and transparent use" of these new technologies, 2 and the transparent and secure management of health data sets on which algorithmic models can be built. 3 While a core set of general principles for the ethical development of AI has emerged, 4 those principles must still be operationalised through legal regulations, 5 and this is particularly important in a high-risk area such as health care. The enactment of comprehensive data protection laws, while important, is not sufficient to address the unique regulatory challenges posed by AI. 6 South Africa has no laws specifically regulating AI. 7 Thus existing legal principles must be adapted, or new principles developed to mitigate the risks to human well-being (comprising of both health-related and human rights-related risks) while not stifling innovation and leading (unintentionally) to non-compliance. 8 This article examines the extent to which current South African laws and policy in health care align with the normative framework of international principles for ethical AI and the values underpinning South Africa's constitution. It examines three legal issues central to the effective regulation of AI: the regulatory oversight mechanisms for the registration of new AI health technologies, the health professions ethics framework governing the use by health care practitioners of these new technologies, and the common law principles of liability for harm caused to a patient or user of the technology. It concludes with recommendations for the development of a clear AI strategy with clear ethical guidelines centred in a humanrights narrative for the implementation of AI in health care in South Africa.

an AI treaty, the work planned for 2021 remains at the stage of a study of its feasibility and scope. 29 However, guiding normative principles have been developed by several international organisations and are largely convergent, emphasising respect for human rights and freedoms 30 alongside transparency, fairness, security and, more broadly, beneficence and accountability as core components of ethical AI development. 31 These values are encapsulated in the OECD's five Principles on AI: 32

AI should benefit people and the planet by driving inclusive growth, sustainable development and well-being.

• AI systems should be designed in a way that respects the rule of law, human rights, democratic values and diversity, and they should include appropriate safeguards -for example, enabling human intervention where necessary -to ensure a fair and just society.

• There should be transparency and responsible disclosure around AI systems to ensure that people understand AI-based outcomes and can challenge them.

• AI systems must function in a robust, secure and safe way throughout their life cycles and potential risks should be continually assessed and managed.

Organisations and individuals developing, deploying or operating AI systems should be held accountable for their proper functioning in line with the above principles.

As a member of the United Nations Educational, Scientific and Cultural Organisation (UNESCO), it is to be expected that South Africa will be guided in its national legislative and policy development agenda by the Recommendation on the Ethics of Artificial Intelligence adopted by UNESCO's General Conference at its 41st session on 24 November 2021. 33 In addition, as a member of the G20 South Africa should take guidance from the G20 AI principles 34 adopted in 2019, which are in turn modelled on the OECD Principles on AI. These principles strongly overlap with the EU framework for "trustworthy AI", 35 However, differences in how these "soft" principles are interpreted and the extent to which they are applied by corporate actors 41 require the development of enforceable obligations in laws, regulatory policy and professional codes of conduct.

The artificial intelligence applications developed for or used in a health care setting must operate in full compliance with the National Health Act 61 

South Africa adopted a telemedicine strategy in 1998 but failed to achieve the targeted improvements in access to health care in under-resourced rural communities that telemedicine promised. 47 Policymakers have since set their sights even higher on a global digital health strategy led by the World Health Organisation (WHO), 48 which still includes telemedicine in the broader rubric of e-health, 49 but now also includes 4IR technologies such as AI, big data analytics and robotics. 50 At a regional level digital health is also a key pillar in the African Union (AU)'s Digital Transformation Strategy, 51 policy. But the policy itself and the existing legislative and regulatory policy environment in South Africa are lacking in substantive principles to guide such development or deployment.

The term "health technology" refers to "machinery or equipment that is used in the provision of health services", 54 excluding medicines. 55 At national and provincial level, the Health Council is to advise the Minister of Health on policy concerning any matter that will protect, promote, improve and maintain the health of the population, including-… (v) development, procurement and use of health technology. 56

The acquisition of any "prescribed health technology" by a health establishment is subject to the issue of a certificate of need by the Director-General. 57 The Minister of Health, after consultation with the National Health Council, may promulgate regulations 58 and prescribe quality requirements and standards relating to health technology, 59 and the Office of Standards Compliance and the Inspectorate for Health Establishments must monitor and enforce compliance by health establishments with such standards. 60 The framework thus exists in which the use of AI in health care could be evaluated, but it continues to face challenges in implementation. 61

The Medicines and Related Substances Act 101 of 1965, as amended, 62 defines the term "medical device" widely to include inter alia any "machine" and "software" intended by the manufacturer for use in the "diagnosis, treatment, monitoring or alleviating" of any disease or injury, and the "prevention" of any disease. Many but not all possible applications of AI in the field of health care will fall within this definition, 63 including software that can assist with diagnosis in a clinical setting, and the hardware embedded with AI software that makes robotic surgery assistants, nursing aides and nano-robots possible.

In both examples the AI software is clearly intended by the manufacturer to be used for the medical purposes defined. General software that is not specifically intended for such a purpose is not a medical device, "even if it is used in a health care setting." 64

The lines become blurred in the area of smart wearable devices and "fitness" and "health" mobile apps for smartphones, which may be considered "lifestyle" or "general wellness" products that mostly fall outside the ambit of health care regulations. 65 developed in Kenya to offer sexual and reproductive health care information (but not medical "advice") and the chatbots developed during the COVID-19 pandemic to provide symptom checking, reporting and exposure services would not prima facie be classified as medical devices as they are not being used in the diagnosis of disease (or a prescribed course of treatment). Nevertheless, there can be clear health implications if these chatbots incorrectly direct a patient, raising ethical concerns and the question of how they should be regulated to prevent the risk of harm. 66

However, the involvement of a human health care practitioner is not a requirement imposed by the definition of software as a medical device under the Medicines and Related Substances Act 101 of 1965. Thus, currently medical devices intended for self-monitoring by a patient, for example blood pressure monitors or blood glucose tests, fall within the definition. It is conceivable that in future AI-powered devices that provide an interpretative analysis of data for a diagnosis of the underlying disease or injury would fall within the definition, provided the device is objectively intended by the manufacturer to be used in this way.

Interpretative clarity on the ambit of the definition is essential to ensure that the developers of such software are directed to appropriately consider the risks posed by the software and to implement a quality management system for the software lifecycle, which is especially important when software is used outside of a clinical setting.

Medical devices that meet defined "standards of quality, safety, efficacy and performance" 67 are registered by SAHPRA after evaluation and assessment. SAHPRA may declare that a medical device (or any class, or part of any class, thereof) must be registered. 68 The sale of any medical device that has not been registered as required by such a declaration is prohibited. 69 The process by which applications for registration are reviewed by SAHPRA is governed by section 15 of the Medicines and Related Substances Act 101 of 1965, and requires SAHPRA to receive particulars and "where practicable" samples of the medical device.

This single stage model for regulatory review according to pre-defined, static specifications and standards cannot adequately address safety, quality and efficacy concerns as AI systems are "adaptive", with the software algorithms being trained from large data sets so that the machine may change its behaviour over time in response to new insights learned from real-world applications.

The United States Food and Drug Administration (FDA) have proposed a "total product lifecycle" 70 regulatory oversight mechanism for software such as medical devices in health 70 FDA 2021 https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learningsoftware-medical-device 2.

care. Pre-market certification of software would require manufacturers to provide the FDA with a "pre-determined change control plan" outlining the modifications that can be anticipated, coupled with transparent monitoring throughout the product lifecycle. 71

In the EU, Regulation 2017/745 on medical devices 72 expands the definition of medical device to include the "prediction and prognosis" of disease, which may bring certain mobile applications such as heart rate monitors on smartphones and smartwatches into the regulatory regime. 73 Further, a specific classification standard for software has been introduced. 74 To complement sectoral product safety legislation the EU has also adopted a proposal for an AI Act to regulate the conditions applicable to the development and marketing of all AI-products and services and has established post-market controls. 75

At an international level the Focus Group on AI for health (FG-AI4H), established in 2018 by the International Telecommunications Union (ITU) in partnership with the World Health Organization (WHO), provides a standardized assessment framework for the evaluation of AI-based methods for health, diagnosis, triage or treatment decisions. 76

In 2021 the WHO published a framework to guide the evaluation of clinical evidence supporting AI software development, software validation and reporting, deployment, and post-market surveillance. 77 The framework is a ground-breaking development that will assist in ensuring that safety and performance claims are supported by robust, transparent evidence. Importantly it emphasises that evidence must be free of the existing biases in healthcare on racial, ethnicity, age, socio-economic and gender lines that are perpetuated when they are encoded into the data used to train AI algorithms. 78

It is essential that consideration be given to these developments to reform the regulatory regime in South Africa. 79 Public authorities must have oversight and the ability to intervene at all stages of the AI product lifecycle. The development of technical standards, robust ethical guidelines and a certification process could be considered as means to ensure oversight before market launch, so that health care practitioners and patients have access to trustworthy AI products and services only.

In the case of high-risk use, where indicated by a risk assessment, there would be a general obligation upon developers to deposit the documentation on the use, design and safety instructions with public authorities, and where "strictly necessary" this might include 71 FDA 2021 https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learningsoftware-medical-device 2. information on the "source code, development tools, and data used by the system". 80 Allowing authorities access to the data, software and computer systems of developers and deployers of AI technologies is necessary to verifying not only the intended purpose but also the actual uses to which AI is put. 81 Such access must of course take place with safeguards to protect data, privacy, intellectual property rights and trade secrets. 82 In this regard, without duplicating duties, there needs to be co-operation between the Information Regulator and the health sector regulatory bodies to ensure that new technologies identified as "high risk" are developed and deployed in accordance with legal and ethical obligations 83 and an approved certification process. 84 Consideration also needs to be given to support for end-of-life products, and "independent trusted authorities" must have the means to provide services such as maintenance, repair and software updates and patches to the users of "vital and advanced medical appliances" where the developer or deployer of the technology ceases to do so. 85

The Health Practitioners Council of South Africa (HPCSA)'s ethical guidelines for practitioners remain rooted in the outdated era of telemedicine. 86 Telemedicine is defined in the guidelines as:

The practice of medicine using electronic communications, information technology or other electronic means between a health care practitioner in one location and a health care practitioner in another location for the purpose of facilitating, improving and enhancing clinical, educational and scientific health care and research, particularly to the under serviced areas in the Republic of South Africa. 87

Thus, telemedicine seeks to replicate traditional face-to-face practitioner-patient consultations using ICTs such as video conferencing. It could also include the exchange of information electronically (between practitioner and patient or, for example, between the primary and secondary health care practitioner for a specialist diagnosis or a second opinion) but an actual face-to-face consultation and physical examination of the patient in a clinical setting by at least one of the health care practitioners remains mandatory. 88

The guidelines are further restricted by the requirement that both the consulting practitioner and the servicing practitioner must be registered health care practitioners, either in South Africa or in the country where they are located. 89 A medical examination must be performed and documented, with a clinical history of the patient, before any course of treatment is prescribed or prescription issued. 90 may be issued on the basis of a questionnaire alone, 91 and informed consent must still be obtained when a prescription is issued electronically. 92 The guidelines have been relaxed recently, but only for the duration of the COVID-19 pandemic, and only to the extent of permitting "telehealth" 93 even where there is not "an already established practitioner-patient relationship". 94

The HPCSA ethical guidelines are thus inadequate to regulate the lawful and ethical development and deployment of AI applications. Worse, they may in fact inhibit the adoption of new technologies in health care in South Africa by virtue of the threat of sanctions against health care practitioners if they are found guilty of unprofessional conduct 95 or a breach of the professional duties imposed by common law. 96 Although machine-learning has transformed the role of the medical device from a mere tool to a powerful collaborator with the health care practitioner, 100 there is no room in the guidelines to regard an AI system as a servicing practitioner working in partnership with the consulting practitioner. 101 While South African law recognises juristic persons, it does not presently afford any legal status to "things". 102 A radical re-imagining may be necessary to address the new risks and roles of AI and there is, at least in principle, no reason why a statute cannot create a statutory right of action against an AI system (the thing) which would impeach it (without necessarily citing or requiring jurisdictional competence over the person 93 HPCSA 2020 https://www.hpcsa.co.za/Uploads/Events/Announcements/ APPLICATION_OF_TELEMEDICINE_GUIDELINES.pdf clause (a) substitutes the term "telemedicine" with "telehealth" which "includes amongst others, Telemedicine, Telepsychology, Telepsychiatry, Telerehabilitation, etc., and involves remote consultation with patients using telephonic or virtual platforms of consultation". 94 HPCSA 2020 https://www.saheart.org/cms/content/104-notice-to-amend-telemedicine-guidelines-during-covid-19-%E2%80%93dated-3-april-2020-%7C-hpcsa-e-bulletin clause (b who owns or operates the thing). 103 However, without comprehensive, insurance-backed provisions for recourse in the event of harm, such provisions may be meaningless.

As a corollary to the development of a regulatory oversight and professional ethics framework for the development and use of AI, consideration must be given to the basis upon which civil liability may be attributed when technology fails and causes harm. In this section two guiding principles are put forward to guide future regulation in this area.

Informed consent is the bedrock to the provision of any health care service. Sections 6 and 7 of the National Health Act 61 of 2003 respectively provide the way a patient is to be informed, and stipulate that a health service may not be provided to a user without that user's informed consent, save in limited exceptional circumstances. 104 In terms of section 7(2),

[a] health care provider must take all reasonable steps to obtain the user's informed consent.

The only guidance available on the use of technology in a health care setting is that in addition to obtaining the patient's informed consent to a prescription or any course of treatment, the patient must also give informed consent to the use of the technology. 105 While the technologies underlying telemedicine such as video conferencing and email are now so commonplace that one can see little difficulty in providing an understandable explanation to the patient, the same cannot be said about AI. While this may change somewhat as new technologies infiltrate all areas of daily life, it is unlikely to ever be the case that an average patient will understand the complex algorithms that power AI systems. The scholarly debates taking place around the legal requirement for "transparency" 106 (or "explainability") 107 must be tempered by pragmatism. Just as case law has held that a detailed explanation of a complex medical procedure is more likely to bamboozle than inform, 108 an unduly technical explanation of the computing processes underlying AI systems, robotics or related technologies would be counterproductive. A purposive interpretation of the consent requirement must focus on the need for the patient to understand enough about the risks of the process to make an informed decision about whether to proceed. 109

The National Health Act 61 of 2003 sets out the principle that the "user" 110 of health care services is to have "full knowledge" 111 in that the health care provider must inter alia inform the "user" of "the range of diagnostic procedures and treatment options generally available" 112 and the "benefits, risks, costs and consequences generally associated with each option", 113 as well as any implications, risks or obligations arising from the "user's" exercise of the right to refuse treatment. 114 Moreover the explanation must "where possible" be given in a language and in a manner that the user can understand. 115 This qualification is a paradox. Informed consent simply cannot take place where the patient has not understood the explanation. South African law requires that the patient have "full knowledge" and there is a statutory, 116 common law 117 and ethical duty 118 to obtain informed consent. How this requirement is to be met in practice requires careful consideration. Besides the obvious difficulties of explaining complex technologies in understandable terms, we must also explain what is presently unknown. Providing the patient with full knowledge may paradoxically require explaining that even the developers of the software and the treating doctors do not always fully understand the inner algorithmic workings of the AI. 119 Further, we must put in place mechanisms to provide patients with additional information when it becomes available, and to obtain informed consent for sharing clinical data for research and development. 120 Electronic patient consent and record management systems make this feasible. 121

As illustrated above, the assumption underlying the existing legislation and ethical guidelines in health care in South Africa is that all instances of patient diagnosis and treatment are mediated through a human health care practitioner registered with the HPCSA in terms of the Health Professions Act 56 of 1974. In many instances this will continue to be the case and therefore, no matter how complex the AI system may be, "the last call" 122 rests with the human health care practitioner. 109 Castell v De Greef 1994 4 SA 408 (C) 425H-I/J, in which it is held that informed consent requires knowledge and appreciation of the nature and extent of the harm or risk. 110 The patient, as the "user" of a health care service as defined in s 1 of the NHA, is also the "data subject", being the person to whom the personal health information relates, under the Protection of Personal Information Act 4 of 2013. The latter Act also imposes additional stipulations for the processing of health data and other "special" personal information. 111 NHA s 6. 112 NHA s 6(b). 113 NHA s 6(c). 114 NHA s 6(d 119 Gerke, Minssen and Cohen "Ethical and Legal Challenges" 310 outlines three aspects on which guidance is needed: when it must be disclosed that AI is being used, to what extent the clinician has a responsibility to explain the complexities of the AI to the patient, and if the limits of the doctor's own understanding of the AI must be disclosed. These questions also need to be addressed in healthcare settings that are not mediated through a traditional doctor-patient relationship, such as the use of health apps and chatbots. See McPake 2020 https://medium.com/frontier-technologies-hub/pilot-story-will-access-to-sex-positive-andreproductive-health-information-through-a-chatbot-d41738947d0c. 120 The requirement to obtain informed consent for the collection of any personal data (even if it will be shared only in anonymised form) must be adhered to in clinical and research settings. At common law a health care practitioner's liability when a treatment or diagnosis causes harm to a patient is based on the Aquilian action and involves applying a test for negligence based on an interrogation of what a reasonable medical professional ought to have done in the same situation. 123

There is no reason to relax the ordinary standard of professional conduct because of the limitations of the technology or medium of communication used. A doctor could be found liable for harm on common law fault-based principles for failing to apply his or her own mind to the diagnosis or recommendations generated by the AI-software. The HPCSA guidelines state that professional discretion in relation to the course and scope of treatment "should not be limited by nonclinical considerations" 124 such as the constraints of any technology. The consulting health care practitioner is also responsible for ensuring that the patient's well-being comes first, and the patient's rights to privacy, dignity, information about their condition and confidentiality are respected by servicing health care practitioners. 125 They must ensure that adequate measures are in place to ensure the quality of service, as well as the confidentiality and security of the patient's information, both in respect of their own employees as well as of non-health care personnel providing auxiliary or technical services, 126 the optimal functioning of the technology, 127 unauthorised access to patient information, 128 and damage to or the loss or alteration of patient information. 129

Thus, when a servicing health care practitioner is consulted the primary health care practitioner remains responsible. The primary health care practitioner must interpret and apply his or her own mind to results in advising a patient on treatment options, risk, and likely outcomes. By analogy, when AI systems are used the health care practitioner remains liable for errors and omissions in a diagnosis or treatment that were reasonably foreseeable 130 or would not have been made by a reasonable practitioner in the same branch of the profession. 131 Likewise the practitioner remains liable for a failure to obtain informed consent from the patient. 132 To the extent that a greater degree of skill and care is required in the use of new and complex AI technologies, the practitioner would be expected to meet this higher standard, 133 and could face civil or even criminal liability for the consequences of acting without the required knowledge and skill in the use of new technologies. 134

There is, however, no guidance in case law on how to apply the principles of fault-based liability in a scenario where the outcome is primarily attributable to an unknown flaw or failing in the AI system that could not reasonably have been anticipated. One could theorise that if there is no causative fault on the part of the doctor, 135 he or she would escape liability altogether, with the unfavourable outcome that the injured patient is left without recourse. 136 Even if one turned to the legal doctrine of vicarious liability, there would be great difficulty in establishing, firstly, that the AI system "acted negligently" and, secondly, that the medical practitioner exerted a sufficient degree of control over the AI system to be held responsible. 137 Moreover, one may well see an increase in the use of contractual exemption clauses to exclude all liability, save where the harm was intentionally caused, 138 which all points to the need for clear legislative and policy guidelines to be developed in this area.

The principle of "explainability" requires that AI developers give clear, understandable explanations of how the algorithms function and present results to data protection and consumer protection authorities and the end user. 139 This is the bedrock of consumer trust in new technologies, "even if the degree of [explicability] is relative to the complexity of the technologies". 140 Nevertheless, it is impossible in some cases even for the developer of the technology to explain how an algorithm arrived at a particular result, 141 and this has given rise to the term the "black box algorithm". 142

When the machine makes a mistake that cannot be anticipated or explained, this raises difficulties about how to apply the common law of fault-based liability to the human health care practitioner. In simple terms, the doctor cannot be held liable on any standard of reasonableness. Moreover, the existing statutory and ethical framework does not impose any duty of care on the developers of AI applications in health care to prevent harm or obtain informed consent from the users of those technologies. At common law there is no general duty to prevent harm to others; and liability can be imputed for conduct only that is found to be wrongful when tested against the legal convictions of the community and the values 136 The principle that the loss lies where it falls applicable at common law holds that a person must bear any injury suffered unless there was both a duty on another person to prevent the injury, and failure by that person to act reasonably in the discharge of the duty of care caused the injury. 137 By analogy in the operating theatre a surgeon may be held vicariously liable for the negligence of his or her theatre nurse, but not for the negligence of the anaesthetist, unless the doctor could have acted to prevent the harm. S v Kramer 1987 1 SA 887 (W). 138 As to the validity of such clauses, see Afrox Healthcare Bpk v Strydom 2002 6 SA 21 (SCA). The judgment was, and remains, controversial. This only strengthens arguments for sui generis AI legislation to address the necessary balance between public benefit from technological innovation and patient safety and privacy concerns. 139 EU Framework Resolution paras 17-18. 140 EU Framework Resolution para 23. 141 EU Framework Resolution para 23. 142 The term is a reference to the fact that the inputs (data) and outputs (diagnosis) of the machine are known, but the inner logic by which it reached that conclusion is inscrutable. Watson et al. 2019 BMJ 365. embodied in the Constitution. 143 In addition, causative fault in the form of negligence or intentional wrongdoing must be proved.

While there is a basis for imposing strict liability for high-risk activities under South African common law, 144 legislation developed for the health care sector would be preferable in that it would provide a clear and certain framework to facilitate widespread adoption of and trust in such new technologies by health care practitioners and patients.

The latest EU legislative proposal on civil liability generally proposes joint and several fault-based liability on the operator(s) of AI systems. 145 Health is classed as a "high risk" use case based on the sensitivity of health data and the potential for harm and the infringement of human rights, alongside consideration of the specific purpose or proposed use of the technology in any particular case, as well as the severity of possible harm. 146 For this reason, strict liability (and mandatory insurance schemes) for health care practitioners are under consideration. 147

7.2.1 Product liability-At common law, when a product fails liability is attributed either under the terms of the supply contract, using contractual warranties and service level agreements, or through the imposition of fault-based product liability for manufacturers and so-called expert retailers. This presented an "often insurmountable challenge". 148 For the non-lawyer, the term fault-based liability refers to the requirement that in addition to providing that the product was defective and caused harm, the claimant must prove that the supplier was negligent by failing to act in a reasonable manner and that the harm was caused by this negligence. Fault-based liability must therefore be distinguished from strict-liability, in terms of which a supplier is liable even if there was no fault.

One solution being considered in Europe is the application of the existing provisions of statutory product liability regimes, subject to appropriate amendments to incorporate digital goods and services within the ambit of the legislation. 149

Product liability is governed in South Africa by the Consumer Protection Act 68 of 2008. 150

Section 61 of the Act attempts to impose strict liability for product defects upon all parties in the supply chain, which would in theory include manufacturers, doctors, and hospitals. However, the Act provides for several defences that considerably vitiate its effectiveness. 151 is "defective". 150 Product liability, which is concerned with harm resulting from defects in goods such as the AI-software or medical robot, must in turn be distinguished from liability for harm arising from services. The CPA does also apply to services, and although it does not impose strict liability for harm arising from the provision of a service per se, s 54(1)(c) of the Act provides that, when those services involve the use or supply of goods, the goods must be free of defects.

The Act also provides for a statutory warranty of quality and safety enforceable jointly and severally against "the producer or importer, the distributor and the retailer" but only for six months after purchase. 152 Leaving aside the limited scope and duration of the warranty, the first problem is that the provision of goods and services to the State falls outside the ambit of the Act. 153 There are also problems with the statute's scope of application to private sector health care. Patients are unlikely to be parties to any transaction supplying AI software as a medical device (save in relation to mobile apps and wearable health monitors), although they may be able to claim protection under the Act as the term "consumer" is defined widely to include the end-user of the product or the recipient or beneficiary of the service, 154 and would in those instances most likely seek to claim against the health care practitioner. 155 When the health care practitioner uses AI technology in the course of performing a health care service or at any health care facility, the provisions of section 58(1) require that "any risk of an unusual character or nature" be disclosed, potentially widening the ambit of the informed consent obligations. 156 The health care practitioner or facility that has purchased or used the AI technology will ordinarily be unable to rely on the Act for recourse against the developer. The Act's protections apply to a consumer, and its provisions do not apply to a juristic person (which includes partnerships) with an annual turnover above R2 million. 157 The application of the Consumer Protection Act 68 of 2008 to AI is thus an area requiring further research and possible reform.

The first obvious problem with any proposal to impose liability on developers is that most AI applications will be developed outside South Africa. The solution in the Telemedicine guidelines is that the practice of medicine takes place where the patient is located at the time the telemedicine technologies are used. 158

This simple solution remains fit for purpose in relation to the liability of the health care practitioners treating the patient if it is extended to include all AI, robotics, and related technologies. However, for the purposes of establishing jurisdiction over the developer or deployer of such technology, or service-providers processing or storing the data on their behalf, it is inadequate. The elegant solution in article 3 of the EU Framework proposal could be considered as a model for a similar South African regulation:

This regulation applies to artificial intelligence, robotics and related technologies, where any part thereof is developed, deployed or used in the Union, regardless of 151 CPA s 61(4). Notably s 61(4)(c) muddies the water by providing that it is a defence if the person could not reasonably have known of the defect. It is also open to argue that when AI software is approved by SAHPRA (as it must be), then s 61(4)(a) provides a complete defence to damages claims on the grounds that the product defect is "wholly attributable to compliance with any public regulation", and likewise s 61(4)(b)(ii), which applies when the product was operated in accordance with the supplier's instructions. whether the software, data or algorithms used or produced by such technologies are located outside of the Union or do not have a specific geographical location.

The provision overcomes the difficulties associated with the fact that technology components may be developed, manufactured, deployed, and operated by multiple parties in multiple jurisdictions. Pinning down the place where the cause of action arose and establishing personal jurisdiction over the responsible parties by the application of ordinary common law principles of jurisdiction may be cumbersome, if not impossible in some cases. While jurisdiction is commonly settled by agreement and recorded in the terms of the contact between the parties, this may also be an inadequate solution if it limits South Africans who have suffered harm to rights to action in a foreign court, where the cost and difficulty of enforcing their rights may render the rights nugatory.

Competing policy considerations must be carefully weighed up, which in the field of health care include not only the protection of the individual but the broader policy goals of innovation and the widespread, cost-effective availability of new technologies. 159 On the one hand, onerous strict liability regimes that leave health care practitioners with no recourse to claim an indemnity from the developers or manufacturers of AI products are unduly burdensome. 160 Doctors and health facilities must rely on contractual service level agreements, software and hardware warranties and indemnity clauses to seek recourse against the supplier of AI products, or compulsory insurance schemes must be in operation which may in themselves be prohibitively costly. On the other hand, to impose direct liability on manufacturers and developers or to overregulate the field may stifle innovation, investment and SMME participation. 161

South Africa presently has no overarching national AI strategy, which contrasts poorly with the approach in countries such as Canada 162 and China, 163 that are moving forward swiftly with a 4IR policy agenda. The reports for the 4IR commission and the work of C4IR and ASSAf are moving in this direction. However, it is imperative that technical frameworks be developed in tandem with the guiding ethical principles and the review of the legal frameworks.

At their core, ethical AI principles seek to defend human autonomy, which is the very essence of the rights to dignity and privacy, 164 against machine profiling and the practices it enables, which range from the somewhat innocuous (even helpful) functions of behaviourally targeted advertising and content suggestions to the subtle and insidious reenforcement of hidden bias and discrimination. The cornerstone of a human rights-centred regulatory framework is the recognition that AI is made by people for people. It should therefore be designed "to serve people and not to replace or decide for them." 165

The regulation of AI in health care must therefore take due cognisance of the constitutional rights of dignity 166 and privacy, 167 alongside equality, 168 life, 169 bodily and psychological integrity, 170 access to health care services, including reproductive health care, 171 and access to information, 172 as well as the rights in the Patient's Rights Charter, 173 including the right to the confidentiality of one's information required by the National Health Act 61 of 2003.

There is a strong alignment between the international normative framework of principles for ethical AI development and the rights in the Bill of Rights under the Constitution of South Africa.

There is a robust body of constitutional case law recognising that there is a "strong privacy interest" in maintaining the confidentiality of health information, 174 and that

[t]he more intimate that information, the more important it is in fostering privacy, dignity and autonomy that an individual makes the primary decision whether to release the information. That decision should not be made by others. 175

However, the conceptualisation of privacy purely in terms of the right to decide whether to disclose data at all, for example, must make way to permit the free flow of data for research and innovation but still respect the individual's human rights. In doing so the central challenge to the ethical development of AI is to ensure that we do not reduce the human being to an object "to be sifted, sorted, scored, herded, conditioned or manipulated." 176 A human rights-centred narrative in any AI strategy is thus essential.

South Africa's digital health strategy places a "person-centred focus" as the first of five key principles underpinning the strategy 177 and highlights the need for digital health solutions to respect "patient privacy". 178 The report by the Presidential Fourth Industrial Revolution Commission 179 recognises that AI could herald great advances in health care but that "the reputation, dignitas and privacy in giving effect to the value of human dignity in our Constitution." Also see National Coalition for Gay and Lesbian Equality v Minister of Justice 1999 1 SA 6 (CC) para 30.

165 See EU Framework Resolution para 2. Also see paras 10-11 identifying human well-being, individual freedom and international peace and security as the guiding objectives for the development and deployment of AI, and the need for mechanisms to ensure human agency, oversight and resumption of control. 166 Section 10 of the Constitution. 167 Section 14 of the Constitution. 168 Section 9 of the Constitution. 169 Section 11 of the Constitution. 170 Section 12 (2) data ecosystem also brings about the critical need for policy and legislation relating to the use of data, including ethics and security." 180 Referring to the "central productive force of data" 181 in the 4IR, the report recognises perhaps more importantly, that fundamental human rights are now intertwined with the protection of data. 182

The danger I point out is that trite references in passing to "patient privacy" are insufficient, and a clear commitment to and detailed treatment of human rights issues such as that contained in the EU "trustworthy AI" approach 183 is required.

South Africa has neither an overarching AI strategy nor any specific laws governing AI. Although there may be some temptation to adopt a "wait and see" approach, 184 early and proactive engagement in the regulatory endeavour is important to ensure that laws are not Western "imports" but are fashioned to be appropriate to the South African context. 185

The development of a national policy framework of guiding ethical principles would in no way undermine the existing legislation and ethical guidelines governing health care practitioners, which must be read alongside AI guidelines, and implemented to their full effect. 186

This article has examined three key areas for legal reform in relation to AI in health care. The first is that the regulatory framework for the oversight of software as a medical device needs to be updated to develop frameworks for adequately regulating the use of such new technologies. In this regard the WHO framework 187 provides a solid starting point for the planning of clinical and research studies and the reform of South Africa's regulatory system to accommodate AI software as a medical device.

Secondly, the present HPCSA guidelines for health care practitioners in South Africa adopt an unduly restrictive approach centred in the outmoded semantics of telemedicine. This may discourage technological innovation that could improve access to health care for all, and as such the guidelines are inconsistent with the national digital health strategy. As a first step, such guidelines should be amended to expressly permit the use of AI and to provide additional guidance on informed consent in such contexts.

Thirdly, the common law principles of fault-based liability for medical negligence could prove inadequate to providing patients and users of new technologies with redress for harm. Consideration should be given to developing a statutory scheme for strict liability, together with mandatory insurance, and the appropriate reform of product liability pertaining to 180 technology developers and manufacturers. It is suggested that the EU model should be considered as a starting point for developing an AI Act for South Africa.

These legal reforms should not be undertaken without also developing a coherent, human rights-centred policy framework for the ethical use of AI, robotics, and related technologies in health care in South Africa. 

Artificial Intelligence in Healthcare: Past, Present and Future

The Global Landscape of AI Ethics Guidelines

Heart Rate Monitoring Apps: Information for Engineers and Researchers About the New European Medical Devices Regulation

Ethical Implications of Conversational Agents in Global Public Health

COVID-19: The Role of Artificial Intelligence in Empowering the Healthcare Sector and Enhancing Social Distancing Measures During a Pandemic

What Constitutes Medical Negligence?

From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices

Addressing the Challenges of Implementing a Health Technology Assessment Policy Framework in South Africa

Medicine and the Law: The Consumer Protection Act: No-fault Liability of Health Care Providers

The Application of the Consumer Protection Act in the South African Health Care Context: Concerns and Recommendations

The Ghost in the Machine: The Ethical Risks of AI

The Chinese Approach to Artificial Intelligence: An Analysis of Policy, Ethics, and Regulation

Artificial Intelligence in Healthcare: A Critical Analysis of the Legal and Ethical Implications

The Global Approach to Regulation of Medical Devices and IVDs

Software as Medical Devices (SaMDs): Critical Rights Issues Regarding AI Softwarebased Health Technologies in South Africa

Navigating Uncharted Waters: Biobanks and Informational Privacy in South Africa

Clinical Applications of Machine Learning Algorithms: Beyond the Black Box

World Health Organization Generating Evidence for Artificial Intelligence-based Medical Devices: A Framework for Training, Evaluation and Validation

World Health Organization Global Strategy on Digital Health 2020-2025 (WHO Geneva 2019) World Health Organization Recommendations on Digital Interventions for Health System Strengthening

GN 591 in GG 43834 of 23

Internet sources

African Union

South Africa Leading the Way in the

Council of Europe's Ad Hoc Committee on Artificial Intelligence 2020 Towards Regulation of AI Systems

Special investigation: Claims of 90-90-90 Success in KZN Districts were Premature

Council Directive 85/374/EEC of 25 July 1985 on the Approximation of the Laws, Regulations and Administrative Provisions of the Member States Concerning Liability for Defective Products

Communication from the Commission: Artificial Intelligence for

European Commission, Directorate-General for Communications Networks, Content and Technology

Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts

/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA Relevance)

on Medical Devices, Amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Reguation (EC) No 1223/2009 and Repealing Council Directives 90/385/EEC and 93/42/EEC (Text with EEA Relevance)

European Parliament 2020 Resolution of 20 October 2020 with Recommendations to the Commission on a Framework of Ethical Aspects of Artificial Intelligence, Robotics and Related Technologies 2020/2012(INL)

European Parliament 2020 Resolution of 20 October 2020 with Recommendations to the Commission on a Civil Liability Regime for Artificial Intelligence 2020/2014(INL)

and-machine-learning-software-medical-device United States Food and Drug Administration 2021 Artificial Intelligence and Machine Learning (AI/ML) Software as a Medical Device Action Plan

Principled Artificial Intelligence: A Map of Ethical and Rights-based Approaches

Google AI date unknown Artificial Intelligence at Google: Our Principles

Health Professions Council of South Africa date unknown Booklet 1: General Ethical Guidelines for Health Professions

HPCSA date unknown Booklet 3: National Patient's Rights Charter

Health Professions Council of South Africa date unknown Booklet 4: Seeking Patients' Informed Consent: The Ethical Considerations

Confidentiality: Protecting and Providing Information

Health Professions Council of South Africa date unknown Booklet 10: Guidelines for the

Health Professions Council of South Africa 2020 Guidance on the Application of Telemedicine Guidelines During rhe Covid19 Pandemic

Health Professions Council of South Africa 2020 Notice to Amend Telemedicine Guidelines During COVID-19

Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems https:// standards.ieee.org/industry-connections/ec/autonomous-systems

International Telecommunication Union 2020 Guidance on AI and Digital Technologies for COVID Health Emergency

International Telecommunication Union and World Health Organization date unknown Focus Group on

Amend or Create Policy and Legislation Enabling the 4IR

Pilot Story: Will Access to Sex-positive and Reproductive Health Information Through a Chatbot Lead to Increased Contraceptive Use Amongst Kenyan Youth?

Microsoft date unknown Responsible AI Principles

Microsoft 2019 Artificial Intelligence in Middle East and Africa: South Africa Outlook for

The Debate on the Ethics of AI in Health Care: A Reconstruction and Critical Review

Oxford Insights

The Future of Health in South Africa

RIA Technical Assistance Provider for AU's Digital Health Strategy

Singh V 2020 AI and Data in South Africa's Health Sector

UNAIDS date unknown 90-90-90: Treatment for All

Scientific and Cultural Organization 2019 Steering AI and Advanced ICTs for Knowledge Societies: A Rights, Openness, Access, and Multi-stakeholder Perspective

United Nations Educational, Scientific and Cultural Organization

Eighteen Years After Doha: An Analysis of the Use of Public Health TRIPS Flexibilities in Africa

Walch K 2020 AI Laws are Coming

Declaration of Geneva Adopted by the 2 nd General Assembly of the World Medical Association

The support of the HSRC/Facebook Ethics & Human Rights and AI in Africa grant is gratefully acknowledged. I also acknowledge the support by the US National Institute of Mental