key: cord-0714414-ghn2u2kh authors: Zidaru, Teodor; Morrow, Elizabeth M.; Stockley, Rich title: Ensuring patient and public involvement in the transition to AI‐assisted mental health care: A systematic scoping review and agenda for design justice date: 2021-06-12 journal: Health Expect DOI: 10.1111/hex.13299 sha: 6f97e94a17fe0daae213a7f436d317a96ad3f8b3 doc_id: 714414 cord_uid: ghn2u2kh BACKGROUND: Machine‐learning algorithms and big data analytics, popularly known as ‘artificial intelligence’ (AI), are being developed and taken up globally. Patient and public involvement (PPI) in the transition to AI‐assisted health care is essential for design justice based on diverse patient needs. OBJECTIVE: To inform the future development of PPI in AI‐assisted health care by exploring public engagement in the conceptualization, design, development, testing, implementation, use and evaluation of AI technologies for mental health. METHODS: Systematic scoping review drawing on design justice principles, and (i) structured searches of Web of Science (all databases) and Ovid (MEDLINE, PsycINFO, Global Health and Embase); (ii) handsearching (reference and citation tracking); (iii) grey literature; and (iv) inductive thematic analysis, tested at a workshop with health researchers. RESULTS: The review identified 144 articles that met inclusion criteria. Three main themes reflect the challenges and opportunities associated with PPI in AI‐assisted mental health care: (a) applications of AI technologies in mental health care; (b) ethics of public engagement in AI‐assisted care; and (c) public engagement in the planning, development, implementation, evaluation and diffusion of AI technologies. CONCLUSION: The new data‐rich health landscape creates multiple ethical issues and opportunities for the development of PPI in relation to AI technologies. Further research is needed to understand effective modes of public engagement in the context of AI technologies, to examine pressing ethical and safety issues and to develop new methods of PPI at every stage, from concept design to the final review of technology in practice. Principles of design justice can guide this agenda. Machine-learning algorithms and big data analytics will revolutionize contemporary health care. Popularly described as 'artificial intelligence' (AI), these technologies are being used in health-care systems across the globe, for example to process population data and identify at-risk groups, or to determine the best treatment options for individual patients, and to develop precision medicine. 1, 2 Despite the advantages of efficiency of scale and depth of computational power, [3] [4] [5] concerns have been expressed by scientists, practitioners and broader publics about the systematic datafication of people's lives and their lived experiences of health and illness. [6] [7] [8] [9] [10] It is unclear whether AI-assisted health care always leads to better patient outcomes, whether it empowers and enables patients/service users, carers and their families, and whether patients or the public have a meaningful say over AI-assisted processes of care or design of such systems. [11] [12] [13] This paper explores the issues from the perspective of ensuring that patient and public involvement is not overlooked in imaging and transitioning to AI-assisted health care. There are implications here for the values of equality, diversity and inclusion in a human/digital intelligent world, which there is only limited space to touch upon. In medicine and health, patient and public involvement (PPI) has become a principle for health-care providers and a field of practice and research. In different countries, alternative terms include personal and public involvement (P&PI) or patient and public en- Previous research on PPI in health has drawn attention to issues of equality, diversity and inclusion (EDI) and the professional dominance of the PPI agenda. [15] [16] [17] [18] [19] However, this body of work has yet to contend with the urgent issues 20 -22 of how PPI might be developed in an AI-assisted health and research system where 'unexplainable' decisions are being made by computers and technology designers. [22] [23] [24] [25] New forms of interdisciplinary collaboration 26 between patients, designers, data scientists, clinicians, researchers, computer scientists, developers and entrepreneurs are emerging, but very slowly and disproportionately to the scale and speed of technological change. They aim to create innovative, user-validated and socially responsible products and e-services with the people who stand to benefit from them, not only with specialists, or health professionals whose views are known to differ from patient perspectives. 27, 28 Advances have been made in participant co-design theory and methods, [29] [30] [31] such as the Design Council's (UK) Double Diamond methodology, and in participative medical device design. 32, 33 New 'social licences' for digital technologies, 34 new guidance such as the World Health Organization's mHealth Evidence and Assessment (mERA) checklist and a push for 'explainable AI' (XAI) highlight the need to improve the quality and consistency of user-centred and more inclusive technology conception and design processes. 35 This paper draws together evidence about public engagement in the context of newly emerging AI technologies for health to inform new strategies for PPI in health care. The concept of design justice provides a useful perspective that promotes engagement 16, 36 and aims to explicitly challenge exclusion and inequalities by valuing inclusion and diversity in design. 17, 19, [37] [38] [39] We chose to focus our exploration on mental health as this is an area of care where the take-up of big data and machine-learning software has already gathered significant pace, and often without public debate on what desirable safeguards should be put in place. 1 Machine-learning applications in clinical psychology and psychiatry are appealing as cost-cutting sources of scientific knowledge and evidence-based policy in public mental health care. 40, 41 This trend is very likely to accelerate as the COVID-19 pandemic worsens the mental health crisis and digital solutions become available to triage patients, address the 'backlog' to care and to expediate treatment or interventions. 42, 43 Depending on the direction of development, AI could also have major benefits for smoothing patient pathways, enhancing workflow in health systems, regulating quality and enabling quality improvement of care. As this is a new and rapidly evolving context for PPI, we did not want to be too narrow in our view of what PPI is or could become. In the UK context, PPI is at present defined in government policy as a requirement for all publicly funded research and health care. We drew upon existing definitions (to develop our search terms) while also exploring a wider notion and the broader landscape of 'public engagement' to allow possibilities for new modes and opportunities for PPI to be identified in the literature. The approach was therefore to look broadly at evidence on patient participation, patient perspectives, approaches to coproduction and user-led projects, as well as patient engagement in clinical care, care delivery and service design. We included patient engagement with AI technologies in health-care contexts and in selfmanagement of mental health conditions and personal well-being as these are important ways that patients are engaging with AI technologies and mental health care. The review explored the following questions: a. What are the main issues and challenges associated with datadriven AI-assisted care that public engagement might help to address? b. How and in what contexts have patients and the public been involved in the design of AI technologies in mental health? Our approach was informed by a conceptual framework, illustrated by Figure 1 , which draws on design justice perspectives (described below), the sociology of digital health interventions, 44, 45 the anthropology of scientific expertise 46 and advances in transdisciplinary knowledge mobilization (KM), all of which can inform policyoriented research and public engagement through co-production or co-design methodologies. 47 These areas of practice and expertise share a concern with attending to the uses of knowledge 48 and its conditions of possibility: how it is produced, for what purpose, the type of knowledge produced, about what or whom and on whose terms. 49 The framework enabled us to draw on these perspectives to inform the methods described below. We used a systematic scoping review as this would allow us to focus on identifying issues and themes across an emerging topic field and draw together evidence from relevant published literature, including, but not limited to, evidence about potential avenues for PPI. 50, 51 We sought information about how machine-learning and dataintensive technologies might enable meaningful and effective public engagement, as well as clarification on how patients and broader publics can contribute to the design of such technologies through a formal PPI process. We sought evidence from research studies and insights from professional or patient expertise about public engagement in AIassisted mental health. Included articles were those that mentioned or reflected on: (a) any type of patient and public groups involved in one capacity or another, for example as users of technologies, research subjects, public reviewers, patient representatives or co-researchers; (b) contexts of involvement in the design process; (c) approaches to involvement, ranging from reports on user engagement, to user testing, interviews, consultations, participatory design concepts and shared decision-making practices; (d) evidence of influence or impact of involvement on design decisions, practices or outcomes, such as published case studies of impact or evaluation reports that explain how a PPI element influenced a research study or the design of a health intervention; and (e) patient and public views on data-driven approaches to mental health care and research. We sought journal articles, conference F I G U R E 1 Exploring patient and public involvement in AI-assisted mental health care proceedings and grey literature. We excluded articles published before 2010, articles published in other languages and articles that did not relate to mental health. We used Web-based searches (carried out in December 2020) Searches used terms associated with 'public engagement' used by Brett and colleagues 18 in their systematic review of the impact of PPI and terms used by PCORI. We developed composite terms for AI technologies derived from the literature (see Table 1 ). Databasespecific MeSH terms for mental health (and other terms where available) were used to expand and consolidate the searches. The full searches can be made available upon request. The titles/abstracts of 182 identified articles were read, and if deemed to be relevant to the aim of the review, they were retrieved in full for analysis. The screening process is illustrated by an inclusion flow diagram (Figure 2 ). We explored and identified themes in the data using a series of thematic analysis techniques 52 supported by the use of summaries of articles (literature tables) and analytic tools within NVivo. First, we iteratively developed a thematic code framework to draw out 53 emerging themes and subthemes, while retaining links to the original sources to aid retrieval. 52 Next, seven principles of design justice (drawing on the international Design Justice Network's ten principles) (see Figure 1 ) for AI technologies for mental health were applied as a lens through which to consider how AI technologies are currently being used, and as a perspective to identify issues for future research to address. We considered these principles against the themes in the literature, particularly in relation to PPI in the development and use of AI technologies. The review method was developed and revised by all team members, including identification of databases to be searched and key search terms. Strategies for minimizing biases in the search strategy were as follows: (a) two team members independently cross-checked more than half of the returned papers against included/exclusion criteria; (b) members of the team discussed and reached agreement on the thematic code framework; and (c) inclusion and use of grey literature extended the searches beyond peer-reviewed articles. Exclusion criteria Articles that did not relate to mental health or mental health research (n=41) Articles that did not relate to AItechnologies or data-driven approaches (n=26) Articles that did not include evidence of public engagement (patient perspectives/ participation, patient and public involvement, or co-production) (n=22) 6 Lack of critical and reflective public debate on the broader socio-political context and digital technologies 40, 41 New awareness of legal regulations, data risk assessment and PPI 158 Tools that are culturally relevant 153,154 and culturally safe 155 Maintaining public services in the context of growth of commercial/private providers of digital technologies 26, 151 Misperceptions that digital technologies replace professional/medical advice or make it unnecessary 152 Looking at the broader contexts/environments of people's lives, for example urban design 156 Designing new technologies in a context of patient/professional trust, 187 mental health stigma and fears of self-disclosure 59,95 Data-intensive technologies as a way to support disclosure of sensitive information, for example chatbots 95 Anonymous feedback tools, for example on quality of care provided or across multiple providers 77 Patients show willingness to trust and use technologies 188 Complicating or compromising patient/carer/professional trust 26, 152 The impact of initial beliefs about digital health technologies on engagement with interventions 189 Interpersonal competencies, 'bond' 190 and inspiring feelings of fondness 191 Barriers to data-intensive technologies in mental health, including the impact of inequalities 97, 111, 116, 146, 193 Building public trust through transparency about data ownership, privacy and data securityy [83] [84] [85] 160, 192 How data are or could be collected, collated and interpreted marks a common challenge for integrating digital technologies in care services 59 and interpreting data. 21 However, the toughest challenge for digitally enabled integrated care is in the complexities as- Using AI technologies for the early detection of mental health concerns and improving access to evidence-based interventions have great potential to lead to improved health outcomes, 145 particularly for underserved, minority or indigenous populations. 146,147 However, the literature is full of evidence that datadriven approaches to mental health care can further entrench gendered, ethnic, racial, age-based, class-based and geopolitical inequalities. 97, 111, 116, 146 The primary concern is that if the ground truth data are limited to social media data or self-reported medical diagnoses of mental health status, the data will produce biased results: for example, insights will only be generalizable to digitally Public engagement on AI-assisted mental health care will need to include critical and reflective debates on the broader socio-political context and its influence in shaping professional practice and treatment of mental disorders. 40, 41 There are concerns that maintaining public service provision at the forefront of policymaking and technological development will be an uphill struggle, but this remains an open question. 26 26 and how to discuss privacy and data protection with patients when the clinically ideal data set is also the most intrusive, and contend with the issue of why the most insightful algorithms are often the ones whose reasoning cannot be accounted for. 85, 116, 161 Given the importance of the ethical and moral alignment issues at stake, raising public awareness and understanding of the pros and cons of AI -technologies is essential. Failing to involve patients and the public could lead to innovations or applications that are considered unacceptable, are publicly criticized and finally withdrawn. This rejection of AI is exemplified in the public response to the Samaritans Radar, a Twitter app for suicide prevention that failed to engage with the community that it was designed for. 79 In the case of mental health apps, leaders in mHealth research, industry and health-care systems from around the globe are seeking to promote consensus on implementing standards and principles for their evaluation. 56 Emerging guidelines indicate two promising strategies for safety and acceptability. One is the rethinking of informed consent in the context of AI technologies as a dynamic, on-going and relational process, instead of a one-off event 114,161-163 The second strategy is a push for redefining the roles of researcher, clinician and developer alike as morally responsible not only for ensuring that adequate protection and safeguards are in place but also for conveying their importance to the public in accessible ways 105 to maintain motivation and adherence to treatment, 97,168 or to facilitate engagement with clinicians, other patients, or friends and family. 59,72,106 The development of creative digital approaches includes the following: online communities, digitization and redesign of psychology interventions, biometrics and data-driven approaches, creative sharing of stories, symbolic engagements, and creative offerings of comfort and encouragements of self-care. 169 As such, data-intensive technologies offer new possibilities for relationship-based involvement (eg based on a patient's support network), especially for people with severe psychiatric disorders where good outcomes often entail the involvement of informal caregivers. 75, 118 However, in the clinical context, carers may feel separated and distanced from the technical aspects of data gathering and analysis. Data-intensive technologies could break new ground in PPI for service design and quality improvement, for example as media for relaying real-time feedback on service quality 77 and patient-reported outcomes (PRO), 170 or for engaging underserved populations who are less likely to engage with specific services for sustained periods of time. 73, 93, 146 Co-design of virtual reality (VR) scenarios with young people shows promise, 171 but little is known about the safety of implementing immersive VR technologies in sensitive settings. 172 It is known that patient engagement in such technologies is negatively affected by poorly designed features, bugs and didactic information giving. 104, 173 Other needs depend upon the type of user, for example those with severe insomnia 101 or people who are homeless, 174 and the practical/ annoyance issues of using technologies. Wearable technologies tend to address a specific use case/health area, such as bedwetting 175 ; however, this could result in non-scalable and 'silo' solutions. 176 Despite multiple examples of co-design of mental health technologies (mostly apps or Web-based information) with children and young people, 67,177-180 future work will need to consider diversity in the user group, for example children experiencing psychosis. In one study, user design and testing of a Web-based portal for dementia showed users felt an increased sense of autonomy and found the portal to be user-friendly, helpful and efficient but felt that more information should be accessible. 182 Moreover, data can be collected passively or through active user engagement (eg automatically tracked UV exposure vs. user-inputted data on daily activities); various options can appeal for some patients and not others. 62, 183 Clinicians prefer some technologies as opposed to others, 59,72,78,103 depending on features, functionalities or approaches. This has implications for involvement choices as some patients could be more likely to get involved in design of game-based approaches, 184 realworld stories or problem-solving tasks. 73 Patient trust is a major concern throughout the literature, 187 not least because negative past experiences of care and fears of selfdisclosure and stigma have been known to fray service users' trust in mental health-care providers. 59,95 Interestingly, chatbots and anonymous digital reporting have been found to increase patients' willingness to disclose sensitive information about their mental health. 77, 95 Patients have even suggested that professional training makes greater use of virtual reality spaces to see things from the patient's perspective. 188 Other studies caution that data-intensive technologies may complicate relationships and information sharing. 26, 152, 189 The health-care provider, be they a person, avatar or computer program, requires good interpersonal competencies to build a working relationship 'bond' with the client 190 and to inspire feelings of fondness, 191 privacy, security 83-85,160,192 and responsiveness. 193 AI technologies are being used to tackle many mental health challenges, including meeting service demand, 194 supporting service improvement, improving access to clinicians, integrating care and support networks and eliciting feedback about services. 195 A data-rich AI-assisted health environment holds great promise as a means to overcome enduring points of weakness in contemporary PPI initiatives, such that PPI can become more inclusive, wider reaching and accessible. Figure 3 illustrates the interrelationships between the themes of our results to provide an overview of an agenda for the future development of PPI in AI technologies for mental health described below. 5.1.1 | Meaningful and authentic public engagement in all areas of AI technologies can be supported and guided by the core principle that such technologies should sustain, heal, connect and empower people and communities Design of AI technologies for mental health can be widened from a focus on addressing a problem of service demand, towards improving quality and safety, and protecting holistic well-being from the perspective of diverse patients and healthy people. By developing inclusion frameworks that engage patients and healthy people, innovative AI technologies for mental health can be centred on the voices of those who are directly affected by the outcomes of the design process. 17, [196] [197] [198] Practically or figuratively, putting digital intelligence into the hands of the person could enable them to take more control of their own health and well-being. There is a need to encourage cultures of technology development that actively and ambitiously engage with people who are affected by the outcomes of such work-including groups of the public who stand to experience the worst effects or miss out on the benefits of technological innovations. 5.1.2 | Substantial ethical concerns, such as inequalities, cultural and population biases, safety, acceptability and broader socio-political issues, could be better understood and moderated if design focused on the concerns of the community over the intentions of the designer The case for public engagement is more complex in relation to those design processes that do not formally lie within the publicly funded sphere, 199,200 for example social media giants and technology designers who sell directly to the public. In practical terms, developing PPI in such territory means connecting communities of practice around issues, mobilizing the knowledge and expertise that exists in both public and private sectors, and finding avenues to incentivize and value PPI through existing regulatory, governance and public accountability structures. 90 5.1.3 | AI technologies need to emerge from an accountable, accessible and collaborative process that describes how patients and the public have been involved The literature reviewed here supports a call to approach the design of data-intensive technologies as an ethical and political issue, as opposed to strictly technical. 6, 7, 10, 13, 201, 202 There is a need to examine and challenge current power asymmetries (access to funding and networking, for example) in processes of designing AI technologies that will affect patients and the public, not just in the use or application of those technologies. PPI should play a central role in addressing the relative lack of guidelines for design and best practice in F I G U R E 3 PPI in the conception and transition to AI-assisted mental health care in health and care was 'Ground AI in 'problems' as expressed by the users of the health system'. From our analysis, we argue that this message needs to now include the fact that in some cases, the problem might be the AI. The new data-rich digital era creates multiple ethical issues and op- Thanks to the London School of Economics for providing funding for open access publication. We are also grateful to NHS England and NHS Improvement for funding that made this research possible. There are no conflicts of interest. Data sharing is not applicable as no new data were generated. (3) people who are homeless. A group of 59 participants contributed to the 6 iterations of the co-creative development of the QoL-ME. In the brainstorming stage, participants stressed the importance of privacy and data security and of receiving feedback when answering questionnaires. Participants in the design stage indicated a preference for paging over scrolling, linear navigation, a clean and minimalist layout, the use of touchscreen functionality in various modes of interaction, and the use of visual analogue scales. The usability evaluation in the usability stage revealed good to excellent usability. The co-creative development of the QoL-ME resulted in an app that corresponds to the preferences of participants and has strong usability. Further research is needed to evaluate the psychometric quality of the QoL-ME and to investigate its usefulness in practice. not account for bias detection. The design of the majority of algorithms ignore the sex and gender dimension and its contribution to health and disease differences among individuals. Failure in accounting for these differences will generate sub-optimal results and produce mistakes as well as discriminatory outcomes. Cohen 2020 A mixed-methods descriptive study using the adapted authenticity scale to rate four exemplar scenarios along with thematic analysis of workshop discussions. Digital audio compared less well to visual media in authenticity scales. Still photobook-style images were also perceived as less authentic than dramatic film employing professional actors. Digital media must be selected carefully not just in relation to the education needs of the student but in relation to their social, cultural norms and digital skills. Creating digital scenarios co-productively could contribute to a teaching resource that holds authenticity and real-world relevance. Research: Qualitative exploration of the suitability of digital stories as pedagogical tools Digital storytelling as an approach designed to apply the theory of authentic learning in a co-productive context A participant group (n = 7) comprising family carers, people with lived experience and mental health nursing students were invited to join two facilitated workshops. The group reviewed four contrasting forms of digital stories with the aim of eliciting and sharing their perspectives. Digital audio compared less well to visual media in authenticity scales. Still photobook-style images were also perceived to be less authentic than dramatic film employing professional actors. Furthermore, it was found that the essence of authenticity became richer as the process and activities of co-productive engagement developed. It is proposed that creating digital scenarios co-productively provides a relational environment in which the essence of authenticity can be felt and expressed. we would alter the power balance by building trust and avoiding a top-down approach. Giving the students responsibility for much of the interaction with the end users had risks. The seriousness of the project and its potential for real impact -positive or negative inspired the students to perform at an even higher level. They identified with the issue of mental health, a high-profile subject on campuses right now, and they wanted to use their design training to help others. Their dedication to the project grew stronger as they became accountable to real 'clients' who were even more invested in the outcome. Husebo 2020 for behaviour, such as sleep disturbances, agitation and wandering. Up to half of the studies represented proof of concept, acceptability and/or feasibility testing. Overall, the technology was regarded as non-intrusive and well accepted. The authors highlight awareness of legal regulations, data risk assessment, and patient and public involvement, with a necessary framework for sustainable ethical innovation in health-care technology. The success of this field will depend on interdisciplinary cooperation and the advance in sustainable ethic innovation. from mental health services, school counsellors and nurses, and participants from a previous study. 82% completed questionnaires before and after the programme related to the feasibility and acceptability of the programme and evaluation process, and changes in mood, knowledge, attitudes, and behaviour, and their Web usage was monitored. A subsample of 19 were interviewed Key themes from the interviews and groups related to the design features, sections and content, and integration and context of the programme in the young person's life. Overall, the participants found the intervention engaging, clear, user-friendly, and comprehensive, and stated that it could be integrated into existing services. Young people found the 'Self-help' section and 'Mood monitor' particularly helpful. The findings provided initial support for the intervention programme theory, for example, depression literacy improved after using the intervention. Findings from this early stage evaluation suggest that MoodHwb and the assessment process were feasible and acceptable, and that the intervention has the potential to be helpful for young people, families and carers as an early intervention programme in health, education, social, and youth services and charities. The authors suggest a randomized controlled trial is needed to further evaluate the digital programme. Review: Literature and practice in the co-design of digital mental health technologies with children and young people Co-design of digital mental health technologies with children and young people Review of 30 studies using co-design (no direct public engagement) The review found 30 digital mental health technologies that were designed/developed with children and young people. The themes identified were as follows: principles of co-design (including potential stakeholders and stages of involvement), methods of involving and engaging the range of users, codesigning the prototype and the challenges of co-design. Co-design involves all relevant stakeholders throughout the life and research cycle of the programme. The authors suggest future work in this field will need to consider the changing face of technology, methods of engaging with the diversity in the user group, and the evaluation of the co-design process and its impact on the technology. Kidd 2019 Research: feasibility and outcomes study Multi-function mobile health for schizophrenia App4Independence use metrics were assessed as was qualitative feedback through semi-structured interview. Among the 38 individuals with a primary psychosis who participated, there was no research attrition and classic retention on the app was 52.5%. This study contributes to the small but emergent body of work investigating digital health approaches in severe mental illness populations. Kipping 2016 Research: Quantitative evaluation Web-based portal for service users Mental health service users (n = 461) accessed personalized health information via a web-based portal over a year Users felt an increased sense of autonomy and found the portal to be user-friendly, helpful, and efficient but felt that more information should be accessible. Klein 2014 Computerized recovery support programme for addiction Patients accessed individually tailored clinical content in a multimedia format over 18 mo following residential treatment. Low engagement with computerized health programmes is a widespread problem. Several factors were found to predict programme engagement, including several demographic variables, the number of recovery coach contacts, motivation to be in recovery, and attendance at 12-step groups. Research: Environments that support adolescent health and well-being An urban health model and conceptual framework for researching environments that support health and well-being in 10-19-y olds Based on a review of the evidence from urban planning and environmental psychology literature, this article emphasizes the need for a more adolescent-responsive urban design process, the need for more research into age-specific urban affordances; integration of new technologies to forge mobility in and engagement with in the co-design of cities allowing stakeholders to make better-informed planning decisions. the channels in which the technology can be leveraged while keeping the patients' rights front and centre. The potential barriers that an 'e-ready' MHP can expect and directions for moving ahead are discussed, keeping a critical eye on the lacunae in using technology. Kuru 2020 Design research: Product development and testing Intelligent autonomous treatment of bedwetting Children and parents involved at multiple stages of product development and testing -from 'proof of concept', device development, intelligent software development, a user-friendly smartphone application, bedside alarm box, and a dedicated undergarment, and self-adhesive gel pad Involving children and parents at multiple stages of the design process has helped to develop a useful and usable design. An enhanced device will be tested with children with NE at their homes for 14 wk, to gain feedback relating to wearability and data collection involving the cloud platform UK Larsen 2019 Research: Review of apps Mental health self-help apps Google Play and iTunes were searched for apps related to depression, self-harm, substance use, anxiety, and schizophrenia. (no direct public engagement) Seventy-three apps were coded, and the majority (64%) claimed effectiveness at diagnosing a mental health condition, or improving symptoms, mood or self-management. Scientific language was most frequently used to support these effectiveness claims (44%), although this was not backed up by citations to research evidence. Lawn 2019 Research: Smoking cessation for smokers with severe mental Illness (SMI) Using co-design principles, the researchers will adapt the Kick.it smartphone App in collaboration with a small sample of current and ex-smokers with SMI. This pilot work will inform a larger definitive trial. Dependent on recruitment success, the project may extend to also include smokers with SMI who are aged 30 y or more. Lea 2018 Reuse of health data Review of issues associated with the reuse of health data Findings revealed four key challenges: (1) uncertain reliability of consent as a cornerstone of trust due to the limits to understanding and awareness of data sharing; (2) ethical challenges around equity and autonomy; (3) ambitious overly theoretical governance frameworks lacking practical validity; and (4) a clear desire for further public and individual engagement to achieve clearer and more nuanced knowledge dissemination around data sharing practice and governance frameworks. Lee 2014 Discussion/opinion Using social media to detect people with mental health issues Discusses the withdrawal of the Samaritan's Radar, a twitter-based app to identify people at risk of suicide (no direct public engagement) Argues that had the app been developed with the twitter community it would have been better received and more appropriate to user needs. Research: Co-design eHealth technologies for people with dementia 2 experts with dementia were invited to lead sessions for early career researchers at The Connected Health Summer School Early-stage researchers developed 6 app mock-ups based on their discussions and co-creation activities with the two experts with dementia. The reflections on this experience highlight positive learning experiences for researchers in eHealth and mHealth. Research: Algorithm development Suicide prevention 3035 participants from US National Epidemiologic Survey on Alcohol and Related Conditions with suicidal ideation at their lowest mood at baseline were included to develop a risk algorithm. The developed risk algorithm for predicting the recurrence of suicidal ideation has good discrimination and excellent calibration. Clinicians can use this algorithm to stratify the risk of recurrence in patients and thus improve personalized treatment approaches, make advice and further intensive monitoring. Research: quantitative research Virtual humans in clinical interviews 239 participants interacted with a rapport-building VH, some with a human interviewer, some with a computer. Overall, this paper provides the first empirical evidence that VHs can increase willingness to disclose in a clinical interview context. Participants who believed they were interacting with a computer reported lower fear of self-disclosure, lower impression management, displayed their sadness more intensely, and were rated by observers as more willing to disclose. These results suggest that automated VHs can help overcome a significant barrier to obtaining truthful patient information. Research: Systematic review Apps for young people's mental health Review of the literature on the effectiveness of mobile apps designed to support adolescents' management of their physical chronic or longterm conditions. (no direct public engagement) A key finding of the review is the paucity of evidence-based apps (n = 4) that exist, in contrast to the thousands of apps available on the app market that are not evidence-based or user or professional informed. Only 3 apps reported some form of public involvement. Research: Design development e-Screening for mental health issues in young people A bicultural mixed-methods co-design approach involving Māori youth. 3 phases over a 3-y period will provide an iterative evaluation of the utility, feasibility, and acceptability of YouthCHAT, aiming to create a framework for wider-scale roll-out and implementation. YouthCHAT has potential as a user-friendly, time efficient, and culturally safe screening tool for early detection of mental health and risk behaviour issues in NZ youth. Involving input from community providers, users, and stakeholders will ensure that modifiable elements of YouthCHAT are tailored to meet the health needs specific to each context and will have a positive influence on future mental, physical, and social outcomes for NZ youth. Community attitudes towards the appropriation of mobile phones for the monitoring and self-management of depression, anxiety, and stress appear to be positive as long as privacy and security provisions are assured, the programme is intuitive and easy to use, and the feedback is clear. Research: Description of a codesign project young people with psychosis Co-designed with service users, the researchers adapted existing manualized social cognition intervention for people with a first episode of psychosis to a virtual world environment. A group of young people who have used mental health services co-designed a virtual environment to deliver an accessible social cognition intervention to a hard to engage service user group. An iterative process with young service users and the design team that included developing initial ideas, creating a prototype and testing the virtual world. The co-design process led to the development of a specific design, approach and protocol to be tested in a proof-ofconcept trial. Young service users were integral to an agile and iterative design. The authors argue that technological innovations should be routinely co-designed and co-produced if they are to realize their potential to deliver acceptable and affordable mental health interventions. Research: Review of apps key stakeholders and carers as end users to co-design and co-produce the eHealth intervention, using an agile build process. Further public involvement activities are integral to the on-going project oversight and management of the evaluative study Co-production work helped optimize the intervention design. The researchers conducted a usability study on the prototype involving carers to test the delta-build. These have led to the co-production of an eHealth intervention (COPesupport) providing information and psychosocial support for carers through the internet, promoting flexible access and individualized choices. The authors suggest the co-production work has optimized the intervention design and usability fitting the end users' needs and usage pattern in the real world. Research: Co-produced design and build study eHealth intervention for family carers for people affected by psychosis Participatory research methodologies were used to integrate public, patients, and carers perspectives into the eHealth intervention design and build process to improve the product's usability and acceptability. The participatory research work led to the co-production of an eHealth intervention (COPe-support). The study methodology, results, and output have optimized the intervention design and usability, fitting the end users' needs and usage pattern. Study protocol for a randomized controlled trial (RCT) and participants are not as diverse as they could be. The issue of stigma around mood disorders needs to be placed centre-stage. People with dementia (PWD) residing in long-term care Conference presentation discussing the use of virtual reality (VR) as a tool to provide 360°-video based experiences for individuals with moderate to severe dementia residing in a locked psychiatric hospital. The authors discuss at depth the appeal of using VR for PWD, and the observed impact of such interaction. They present the design opportunities, pitfalls, and recommendations for future deployment in health-care services. This paper demonstrates the potential of VR as a virtual alternative to experiences that may be difficult to reach for PWD residing within locked setting. Research: Test of wearable device Wearable health devices Forty-five depressed patients and 41 healthy controls participated, creating a combined 5250 days' worth of data. The results indicated that utilizing wearable devices and machine learning may be useful in identifying depression as well as assessing severity. be recruited to participate in an open pilot trial to evaluate its acceptability, usability, and preliminary efficacy. If acceptable and useful, this game-based eHealth intervention may offer a cost-effective and clinically useful intervention for addressing the psychological needs of over 16 000 young people with long-term health conditions in New Zealand. Thompson 2020 Research: Proof-of-concept trial Early psychosis Five groups of three to five individuals per group were recruited over 6 mo. Eight sessions of SCIT-VR therapy were delivered through the virtual world platform. The SCIT-VR therapy delivered was feasible (36% consent rate and 73.3% intervention completion rate), acceptable (high overall post-session satisfaction scores) and safe (no serious adverse events), and had high levels of participant satisfaction. Users found the environment immersive. Research: Analysis of social media posts Detection of mental illness Language samples were collected from the social media website Reddit, drawing on posts to discussion groups focusing on different kinds of mental illness (clinical subreddits), as well as on posts to discussion groups focusing on non-mental health topics (non-clinical subreddits). Words drawn from the clinical subreddits could be used to distinguish several kinds of mental illness (ADHD, anxiety, bipolar disorder, and depression). Interestingly, words drawn from the non-clinical subreddits (eg travel, cooking, cars) could also be used to distinguish different categories of mental illness, implying that the impact of mental illness spills over into topics unrelated to mental illness. Most importantly, words derived from the non-clinical subreddits predicted future postings to clinical subreddits, implying that everyday language contains signal about the likelihood of future mental illness, possibly before people are aware of their mental health condition. Finally, whereas models trained on clinical subreddits learned to focus on words indicating disorder-specific symptoms, models trained to predict future mental illness learned to focus on words indicating life stress, suggesting that kinds of features that are predictive of mental illness may change over time. Torenholt 2020 Research: Qualitative research Patient data work Ethnographic fieldwork carried out among cancer patients receiving PRO (patient-reported outcome) based follow-up care PRO patient data work as conceptualized as two simultaneous processes: the process of data filtering -patients filter information to fit the envisaged recipient and purpose; and the process of data sensing -patients evaluate their embodied experiences. Patients' data work has implications beyond simply providing data that represent their experiences. Torous 2017 Discussion and recommendations Predictive models in suicide prevention Discusses ethical considerations regarding the use of predictive models in suicide prevention clinical care. Recommendations for navigating the ethical issues are provided as an initial framework for others who are considering the implementation of a predictive model to trigger suicide prevention initiatives. Electronic health records Review of the evidence base on electronic health records (no direct public engagement) Electronic health records combined with tethered patient portals now support a range of functions including electronic data capture of patient-reported outcomes, trend reporting on clinical targets, secure messaging, and patient-mediated health information exchange. The applications of these features require special consideration in psychiatric and behavioural health settings. Nonetheless, their potential to engage patients suffering from disorders in which passivity and withdrawal are endemic to their mental health condition, is great. Digital well-being interventions for people with intellectual disabilities Digital intervention, developed with and for people with intellectual disabilities, to improve their subjective well-being. Using a single-group pre-post design, 12 participants with intellectual disabilities and their caregivers completed the 4week intervention. Participant acceptability of the intervention was high, and feedback covered multiple aspects of the intervention, including (1) programme concept and design, (2) programme content, and (3) intervention usage. The study shows people with intellectual disabilities and their caregivers are receptive to using digital well-being interventions and such interventions are feasible in routine practice. Villani 2017 Internet use by people with schizophrenia Review of the literature on people with schizophrenia's use of the internet for mental health information (no direct public engagement) People experiencing schizophrenia spectrum disorders or other psychotic disorders wish to find on the Internet trustful, non-stigmatizing information about their disease, flexibility, security standards, and positive peer-to-peer exchanges. E-mental health also appears to be desired by a substantial proportion of them. For example, VR experiences are used for diversional therapy in aged care and as therapy for people living with conditions such as phobias and post-traumatic stress. While these uses of VR offer great promise, they also present significant challenges. Given the novelty of VR, its immersive nature, and its impact on the user's sense of reality, it can be particularly challenging to engage participants in co-design and predict what might go wrong when implementing these technologies in sensitive settings. (Continued) The Creative Destruction of Medicine: How the Digital Revolution Will Create Better Health Care The rise of artificial intelligence in healthcare applications Predicting the future -big data, machine learning, and clinical medicine An integrated big data analytics-enabled transformation model: application to health care Artificial intelligence for diabetic retinopathy screening: a review Datafication and accountability in public health: Introduction to a special issue The datafication of health Understanding and applying practitioner and patient views on the implementation of a novel automated Computer-Aided Risk Score (CARS) predicting the risk of death following emergency medical admission to hospital: qualitative study Ambivalence in digital health: Co-designing an mHealth platform for HIV care Digital health now and in the future: Findings from a participatory design stakeholder workshop Personalized Medicine: Empowered Patients in the 21st Century From data fetishism to quantifying selves: Self-tracking practices and the other values of data Quantifying the body: monitoring and measuring health in the age of mHealth technologies MoodNetwork: an innovative approach to patient-centered research Public partnerships, governance and user involvement: a service user perspective: service users and partnership From 'other' to involved: user involvement in research: an emerging paradigm From tokenism to empowerment: progressing patient and public involvement in healthcare improvement Mapping the impact of patient and public involvement on health and social care research: a systematic review The importance of power, context and agency in improving patient experience through a patient and family centred care approach Patienthood and participation in the digital era Data work: meaning-making in the era of data-rich medicine The legal and ethical concerns that arise from using complex predictive analytics in health care A critical take on the policy recommendations of the EU high-level expert group on artificial intelligence Decentering technology in discourse on discrimination Ethical and legal challenges of artificial intelligence-driven healthcare The digital revolution and its impact on mental health care How to improve eRehabilitation programs in stroke care? A focus group study to identify requirements of end-users E-co-innovation for making e-services living labs as a human-centered digital ecosystem for education with ICT Value co-creation in third-party managed virtual communities and brand equity Value co-creation through patient engagement in health care: a micro-level approach and research agenda Designers as Brokers of Meaningful Innovation and Socio-Cultural Significance Developing medical device technologies from users' perspectives: a theoretical framework for involving users in the development process The role of the user within the medical device design and development process: medical device manufacturers' perspectives Establishing a social licence for Financial Technology: Reflections on the role of the private sector in pursuing ethical data practices Designing theory-driven usercentric explainable Ai Patient engagement as an emerging challenge for healthcare services: mapping the literature Patient engagement in research related to dementia: a scoping review Talking the talk or walking the walk?' A bibliometric review of the literature on public Embedding patient and public involvement: managing tacit and explicit expectations Access, accountability, and the proliferation of psychological therapy: on the introduction of the IAPT initiative and the transformation of mental healthcare Digitising psychiatry? Sociotechnical expectations, performative nominalism and biomedical virtue in (digital) psychiatric praxis Therapists and teachers warn of looming mental health crisis. The Guardian The judgement process in evidence-based medicine and health technology assessment Understanding participation: the "Citizen Science" of genetics Scientific expertise How should we think about clinical data ownership Brain Imaging Identifies Different Types of Depression: Biological markers could enable tailored therapies that target individual differences in symptoms Digital interventions for screening and treating common mental disorders or symptoms of common mental illness in adults: systematic review and metaanalysis A Digital Intervention for Adolescent Depression (MoodHwb): Mixed Methods Feasibility Evaluation The global gig economy: toward a planetary labour market Ensuring patient and public involvement in the transition to AI-assisted mental health care: A systematic scoping review and agenda for design justice Research: qualitative eRehabilitation programmes in stroke care End users were involved in six focus groups with patients/informal caregivers to identify user requirements Requirements between stroke patients/informal caregivers and health professionals differed on several aspects. Therefore, involving the perspectives of all end users in the design process of stroke eRehabilitation programmes is needed to achieve a user-centred design.The Netherlands Research: participatory research study Development of an app for substance use disorders A user-oriented design approach, used three co-design workshops 10 health care professionals and 10 patients to develop an app for substance use disorders Patients critiqued the existing app, common issues identified were those of the design, visual probe task, and the included images. Outpatients were concerned with the safety of administration of the intervention. Inpatient participants recommended the addition of functionalities, such as information on the harms associated with the substance use, and for there to be enhancements in the design, images, and task. There were differences in opinion on the inclusion of gaming features, as only health-care professionals endorsed their inclusion. The results from this research will guide the development of an app that meets the specific needs of patients and is still based on a pre-existing validated task paradigm. Zhu 2020 We used the affiliation of the first author to classify the country of the article; however, several studies had international teams or were reviews of the global literature.Additional inclusions (title/abstract screening): Trial protocols that demonstrate models of PPI in the concept, design, testing or implementation of AI technologies.Reviews that synthesize findings relating to PPI but do not have any direct public engagement.Discussion of theory/building or conceptual papers that seek to engage end users/patients but do not have direct public engagement.Additional exclusions (title/abstract screening): Video or digital therapeutic or educational interventions or information giving that is not interactive.Telemedicine (by telephone or video call).Algorithms/ development that are not computerized or digital data.Online patient support groups that are used for online social support.