key: cord-0026306-orltbqxu
authors: McNeil, Heather; McMurray, Josephine; Byrne, Kerry; Grindrod, Kelly; Stolee, Paul
title: Engagement of older adults in regional health innovation: The ECOTECH concept mapping project
date: 2022-01-21
journal: SAGE Open Med
DOI: 10.1177/20503121211073333
sha: 1435f9907a587566dcaf90b96744755998304159
doc_id: 26306
cord_uid: orltbqxu

OBJECTIVES: Regional health innovation ecosystems can activate collaboration and support planning, self-management and development and commercialization of innovations. We sought to understand how older adults and their caregivers can be meaningfully engaged in regional health innovation ecosystems focused on health and aging–related technology innovation. METHODS: A six-phase concept mapping technique gathered data over six time points across Canada. Brainstorming conducted online and in person identified engagement ideas. Statements were sorted by similarity and rated by participants on importance and feasibility. Qualitative approaches and multidimensional scaling, hierarchical cluster analysis, descriptive statistics and t tests were used for analysis. RESULTS: Sixty-two unique ideas were assembled into a seven-cluster framework of priorities for engagement in regional health innovation ecosystems including public forums, co-production and partnerships, engagement, linkage and exchange, developing cultural capacity, advocacy and investment in the ecosystem. CONCLUSIONS: This study identified a framework of priorities for directions and strategies for older adult and caregiver engagement in regional health innovation ecosystems. Next steps include collaborations to develop regional health innovation ecosystems that actively engage older adults and their caregivers in health and aging–related technology innovation.

Social, cultural and technological innovation designed to address issues in health and aging must eventually improve the well-being of older adults and their caregivers. No matter how transformative, all innovations face challenges moving from ideation to adoption or commercialization. The foundational understanding that disruptive and incremental innovations must solve 'real-world' problems has driven the adoption of approaches such as 'design thinking' processes 1 that are fuelled by understanding people's wants and needs through direct engagement with them. 2 Regional Innovation Ecosystems (RIEs) 3 are complex networks where a 'triple helix' of industry, government and academic representatives 4 develops localized infrastructure, institutional knowledge and skills capacity. Over time, this builds capacity for innovation from ideation through to commercialization by mobilizing networks of collaborators.

There is a societal view of older adults as a vulnerable population 5 but more often they are invisible, overlooked as a market opportunity 6, 7 and only recently actively engaged in managing their own health service needs. 8 Similarly, older adults are often overlooked as active participants in regional innovation, where a mostly youth-focused culture has supported ageist stereotypes. 9 In order to reflect a growing understanding of the importance of the 'creative class' in regional innovation systems, Carayannis and Campbell 10 identified a fourth 'strand' to the triple helix. This quadruple helix model acknowledges the importance of a pluralistic approach to regional innovation, and the 'public' as an important participant working with triple helix models, to set and achieve innovation system goals. This approach has been used to explain healthcare technology development and the improved performance of translational research -where active involvement of a diverse user group helps move an innovation from 'bench to bedside'. 11 The benefits of including end-users in co-production of healthcare innovations 12 suggest that increased citizen engagement might have similar impact in larger systems of innovation. The related concept of integrated knowledge translation 13 which emphasizes a collaborative approach between researchers and knowledge users to co-create knowledge across the entire research process, optimizing relevance and impact, situates this work within the broader policy discourse of research translation. Application of a quadruple helix approach to the development of regional ecosystems of health innovation, and the exploration of enduser's engagement, willingness and ability to participate in that process, is the focus of this article.

In the health and aging sectors, the development of innovation capacity has trailed other industries; however, it is attracting increasing attention. 14,15 Regional Health Innovation Ecosystems (RHIEs) are innovation systems that focus developmental and entrepreneurial activities on addressing complex issues such as the rising costs of care delivery and an ageing population. 16 The involvement of civil society is particularly relevant in health as it aligns with democratization trends in many healthcare systems, where the user community increasingly provides direction and agency to the other helices (i.e. government, industry and academic).

Active public participation in the collaborative development of health services is central to contemporary healthcare systems that are 'responsive to a patient's needs and values'. 17 Benefits include improved health outcomes, patient and provider experiences, financial performance and community capacity. 18 These improvements most often affect those with more complex needs who are generally also more frequent users. In Canada, this includes older adults, who coincidentally make up the fastest growing segment of the population. 19 Furthermore, we know that the care of older adults outside of hospitals is most often provided by a supportive network of family, friends and other informal caregivers; 20 while not recipients of care, this group is affected by and often impacts decisions related to the care of these older adults. There is a wealth of literature supporting not only the essential role caregivers play in the healthcare system, but also how they are impacted by it. [21] [22] [23] The 'ECOTECH' (Engaging Canada's Older adults in TECHnology ECOsystems) project aims to identify strategies that encourage the meaningful involvement of the community, specifically older adults and their caregivers, in RHIE development.

This article reports on a group concept mapping project following the methodology of Kane and Trochim. 24 This participatory mixed methods study was undertaken during an 18-month period as part of a programme of research funded by the AGE-WELL Networks of Centres of Excellence. AGE-WELL brought together research, government, industry and non-profit partners from across Canada, with aims of improving the quality of life of older adults, and generating socioeconomic benefits through innovation in aging-related technology. 25 As recognized by Sutherland and Katz, 26 group concept mapping offers a way to understand complex systems and relationships. The methodology inherently values community engagement building in participation by experiential users throughout to generate a prioritized list of approaches to engage older adults and their caregivers in the development of RHIEs.

A six-step group concept mapping methodology approach was undertaken: (1) planning, (2) idea generation, (3) structuring, (4) analysis, (5) interpretation and (6) utilization. 24 For the planning step, a core group (n = 10) of community dwelling older adults and caregivers over the age of 55 was recruited from the Seniors Helping as Research Partners (SHARP) Group. SHARP is a group of older adults in Southern Ontario who collaborate with our research group to advance the development of research priorities and collaborate on research projects, with an aim of improving the healthcare system for older adults. 27 For the other steps in the concept mapping process, participants from across Canada were recruited through purposive sampling 28 to include older adults and their caregivers (n = 22), as well as other key stakeholders (n = 9) who are typically involved in RIEs, with the latter group including policymakers, industry representatives and researchers. Up to 31 (67.7% female; 32.3% male; mean ± SD age: 62.6 ± 16.4 years; range in ages: 30-91 years) participants took part in the concept mapping activities.

Snowball sampling and email recruitment were used for all of the participant groups. For the older adult and caregiver participants, in-person recruitment through posted recruitment flyers and 'gatekeepers' was also used to aid in recruiting a diverse sample. Depending on location, different individuals (such as community workers and healthcare providers in targeted long-term care homes) acted as 'gatekeepers', approaching potential participants on behalf of the research team. Diversity was mainly achieved in living situation (e.g. congregate versus community living), age and health status.

The lead author met with the core group of older adults and caregivers at the outset of the project to collaborate on (1) the focus prompt (which was used in the next phase of idea generation to guide the brainstorming), (2) the foci for the rating activity completed during phase 3 and (3) participant recruitment for subsequent phases.

The goal of the idea generation step was to generate ideas/ concepts that would be used throughout the remaining stages of the project. Once participants consented to be involved in the project, they were asked about their desired platform for participation: either direct use of the Concept Systems 4.0 29 online software application or in-person participation. The Concept Systems software was chosen for use in this project as a fully integrated tool to support all stages of the project. The lead author received training in this methodology and software prior to the commencement of this project. 29 Software was used under the rights provided by a purchased licence.

For those choosing to participate in person, a focus group (n = 8) interview was conducted by the lead author (a female, Master's prepared student in our research group with experience in qualitative methodologies and interest in innovations for health and aging). Six females and two males over the age of 55 participated in a 1-h meeting held at a local community centre. Participants were asked to brainstorm ideas in response to the focus prompt (a question generated in the planning phase). Data were audio recorded and transcribed verbatim following the methodology of Krueger and Casey. 30 Older adults who could not travel to attend the focus group and did not want to participate through the online software were individually interviewed. Ideas generated by the focus group and interviews were manually entered by the lead author into the Concept System software. Following Kane and Trochim, 24 specific themes and ideas unearthed in the lead author's review of the literature on how citizens have been engaged in RIEs including initiatives outside of health that currently incorporate end-user engagement and previous consultations were also manually entered into the software system until information sources were exhausted.

Within a pragmatist paradigm 31 which underpinned the study, an 'idea analysis' was then conducted following the closure of the brainstorming activities. In addition to analysis, this process was useful to 'slim down' the concepts generated to no more than 100 ideas, as suggested in the Concept Systems Facilitator Training. 29 Due to the needs of the populations engaged in this project, it was determined in consultation with the research team that this project should have fewer than the maximum 100 statements. The brainstormed ideas were organized using Excel to remove duplicates and amend wording for clarity of ideas. Statements were then coded in NVivo by the lead author and reviewed by a member of the research team for key themes to ensure that the final list of statements was true to original participant content, thematically saturated and concise.

Once the final list of idea statements was generated by participants representing all groups, participants contributed to two structuring activities: sorting (n = 28 participants) and rating (n = 24 participants for the feasibility rating activity; n = 31 participants for the importance rating activity). An appropriate minimum sample size for this phase is between 20 and 30 participants. 32 This process was completed in the Concept System software for most participants by first sorting (organizing the idea statements into categories and labelling each category with a theme they found appropriately described the grouping) then rating each statement. 24 The rating foci generated in the planning phase were used to ask participants to make judgements about each statement with an x-point ordinal response scale. 'Importance' was defined as having great significance to the engagement of older adults and their caregivers in communities innovating for health and aging. 'Feasibility' was defined as being easily or conveniently accomplished in achieving this goal, including factors such as social acceptance and financial/ economic appropriateness.

Older adult participants who were unable to participate using the software were supported to participate in person by performing the same tasks as those using the online software application, sorting identical idea statements using a set of cards with individual statements printed one per card. Older adult participants who chose an in-person format, but were unable to travel, were either sent rating sheets to participate remotely in the rating activity or (when feasible) were met with in person at a location of their convenience to facilitate participation in the study.

Once the structuring phase was complete, analysis was conducted by the author using the Concept Systems software. Multidimensional scaling was applied, and bridging analysis was conducted to depict results of the sorting activity in map form. 24 Hierarchical cluster analysis, as described by Kane and Trochim, 24 was then conducted to group individual statements into clusters. Quality of this analysis was assessed by a stress index. Stress here was measured by the discrepancy between the distances of points on the map and their original value in the similarity matrix. This value was used to determine the degree to which the map represents the grouping data; meta-analysis has revealed that an ideal stress value should range between 0.205 and 0.365. 24 In the relational map produced, each point represents an individual statement that was sorted by participants, with similar ideas located closer to each other on the map and less similar ideas located further apart. To determine the number of clusters appropriate for this concept map, an agglomerative method 24 was applied by merging statements together at each stage of cluster analysis and reviewing the merging at each stage to observe how the statements were clustered.

Mathematical bridging values assist in understanding the relationships between the statements in a concept map; those that are closer to one indicate that a statement 'bridges' areas of the map, meaning that it is a link between more distant areas of the framework; bridging values closer to zero indicate an anchor in the map, or a statement that was sorted by many people along with others that are more immediately adjacent to it.

A cluster map was generated using the Concept Systems software, and 'pattern matches' and 'go-zones' were generated from the rating data. Pattern matching as described by Kane and Trochim 24 commonly uses a ladder graph to show correlations between a chosen pair of rating values. Go-zones refer to statements with the highest relative importance and feasibility ratings. 24 To identify go-zones, bivariate X-Y graphs were created to display quadrants created using the means for importance and feasibility, with the go-zone falling into the upper right quadrant where statements with the highest relative importance and feasibility ratings were found. 24 Utilization Engagement throughout the project and a final meeting with the core group helped researchers understand best approaches for knowledge translation and usability of the results.

This study received ethics clearance from the Office of Research Ethics at the University of Waterloo (#21329) and all participants provided written informed consent to participate in the project.

Through group consensus, participants modified the researchergenerated focus prompt agreeing on the final statement: 'A specific way that older adults and their caregivers can help our communities develop innovative technologies to support health and aging is . . .' to be used to guide the idea generation phase. A list of appropriate stakeholders to recruit was co-generated and the foci of 'importance' and 'feasibility' were chosen by the group for the rating activities in phase 3.

In response to the focus prompt, a total of 83 statements were generated. The statements were analysed using the method of qualitative thematic analysis as outlined by Creswell. 33 The analysis revealed key themes including systemic changes, individual approaches, collaborations, improvements to current practice, novel ideas and methods, and processes central to older adult engagement in RHIEs. After duplication removal, 62 individual statements were kept for use in the structuring phase.

Our goal was to recruit a purposive sample with representation from each of the quadruple helix groups as well as representation from jurisdictions across Canada. While we achieved these goals, we observed that none of the healthcare providers who participated in the study was self-identified as such. Instead, all participating healthcare providers self-identified with other roles, such as researcher or caregiver. The distribution of participants remained consistent across this phase with over half of the participants in each activity choosing to self-identify as an 'older adult'.

Appropriate sample size, as described by Rosas and Kane, 32 was achieved for all activities. The rating of feasibility activity achieved the lowest participation rate. Lower participation in this activity can be attributed to (1) drop off (as this was the third of three activities) and (2) the relative complexity of the idea of feasibility.

The final product of the multivariate statistical techniques was a seven-cluster map representing a framework of priorities for the engagement of older adults and their caregivers in RHIEs ( Figure 1 ). This map yielded an acceptable final stress value of 0.350 23, 29 falling within the high range of normal, consistent with the known complexity of the research topic.

The concept map demonstrates the major concepts and relationships between ideas, expressed by distance between items. Each cluster of items was named by participants; qualitative analysis of the suggested cluster names was performed (Table 1 ). Each cluster is described as follows.

The first cluster, public forums, includes 11 ideas/statements. The bridging value of this cluster is 0.34 with individual statement values ranging from 0.23 to 0.49. The ideas sorted in this cluster are related to the idea of places and spaces for in-person and virtual interactions within the innovation ecosystem. In these places, events and activities can occur that increase interactions between and the knowledge capacity of different groups in the ecosystem. These activities could include opportunities such as conferences and workshops for collaborators to gain knowledge about health and aging, and pitch events that facilitate shared experiences in understanding and building solutions to improve health and aging.

The co-production and partnerships cluster is made up of 11 statements about roles and perspectives of those interacting in the ecosystem. The bridging value of this cluster is 0.29 and statement bridging values range between 0.00 and 0.47. The statement with the 0.00 bridging value, the lowest bridging value in this cluster that sorted primarily with other statements close by, has a theme of supporting older adults' decision-making abilities. In terms of roles, some older adults and their caregivers would like to partner with innovators (e.g. from planning stages) and others would like to volunteer within an innovating organization. In terms of perspective, the diversity in technology adoption and use among older adults and their caregivers suggests they should be involved in developing and testing technologies created to address their needs.

With a cluster bridging value of 0.06, the engagement concept is the anchor for this map. Through multiple iterations, the sorting of statements within this cluster stayed central to the framework. The low statement bridging values ranging from 0.00 to 0.14 indicate that the six statements that make up this cluster were frequently sorted with each other. This cluster label 'engagement' was identified by the software as frequently chosen by participants' sort labels. Statements in this cluster included ideas of interactions between collaborators, opportunities to understand and learn from older adults and their caregivers, brainstorming and dialogues between all those involved in innovation. 

In contrast, the linkage and exchange cluster is found to be a bridging cluster. This cluster is about how connections are made between those involved in innovation. Statements in this cluster include ideas about feedback and generating connections between collaborators so that they can have the opportunity to reach out to each other. With a bridging value of 0.51, ideas here help to link to others in the map. Statement bridging values range from 0.06 to 1.0 among nine statements. The statement with the bridging value of 1.0, 'encourage communication between those interested in innovation', highlights the importance of communication to engagement in innovation ecosystems. There is a need for improvements in how knowledge is exchanged between those traditionally involved in innovation (researchers, industry, government representatives) and experts by experience (older adults and their caregivers) who want to be more meaningfully involved in these ecosystems.

The developing cultural capacity cluster is made up of nine statements. With a cluster bridging value of 0.39 and statement bridging values ranging from 0.12 to 0.81, this cluster incorporates a range of ideas about the use of media to create awareness, coaching and mentoring opportunities in the ecosystem and other ways to bridge cultural divides. Ideas of 

Advocacy comprised eight statements which include ideas about advocating for flow of information, changing the status quo and getting government and decision-makers involved in innovation ecosystems for health and aging. With a bridging value of 0.32 and individual statement bridging values ranging from 0.24 to 0.50, this cluster can be interpreted as another anchor cluster; ideas here were consistently sorted together. Older adults and their caregivers need to have access to information and be able to share their experiences with government, industry and university representatives in the ecosystem to be meaningfully engaged.

As another bridging cluster, the investment in the ecosystem cluster incorporates ideas of creating buy-in in the ecosystem, economic development and incentives for older adults, their caregivers and companies involved in innovation for health and aging. These ideas link to other clusters, represented with the bridging value of 0.56. The eight ideas in this cluster have statement bridging values ranging from 0.25 to 0.94. This connection demonstrates the support necessary for many of the ideas in this cluster (e.g. financial reimbursement and incentives).

As displayed in Figure 2 , engagement was rated on average as the most feasible cluster. Co-production and partnerships, and investment in the ecosystem clusters were rated as relatively less feasible. These cluster rating maps provide information to guide interpretation and utilization. For example, although all the statements in the framework are important, cluster ratings reveal a smaller number of themes which could be understood as priorities.

The pattern matching graph (Figure 3 ) reveals the average ratings for each cluster according to participant's ratings of importance compared to feasibility. Overall, there is a moderately positive relationship (r = 0.49), and the relationship is linear. Engagement is considered relatively more important than the other clusters, followed by the co-production and partnerships, linkage and exchange, and advocacy clusters. Participants rated the clusters higher for importance (n = 31) than for feasibility (n = 24). Significant differences highlight potential areas for further investigation and utilization of the findings. Compared to other clusters, 'co-production and partnerships' has the largest difference, as it was rated the second most important (3.98/5) but also one of the least feasible (3.59/5); the difference is statistically significant (p < 0.002). Similarly, participants rated 'advocacy' with high importance but low feasibility. By comparison, participants rated 'developing cultural capacity' as having lower importance but higher feasibility. The t test revealed that there were no significant differences for engagement, linkage and exchange, investment in the ecosystem and developing cultural capacity. There is a significant difference between feasibility ratings of the engagement and co-production and partnership clusters; the engagement cluster was rated as the most feasible cluster (3.89/5), and the co-production and partnerships cluster was rated as the least feasible cluster (3.59/5). The significant differences may be helpful when implementing this framework of priorities as key starting points for implementation. The lack of statistically different importance ratings between clusters indicates that all of the clusters fall within a range considered to be important by participants. Further comparison of ratings data is presented through gozone graphs. Figure 4 presents a go-zone graph of the framework of priorities visually displaying the statements with the highest relative importance and feasibility ratings; a moderately positive relationship (r = 0.65) was found.

The go-zone quadrant (those perceived to be both highly feasible and important by participants) is comprised of a total of 23 statements, representing six of the seven clusters. The linkage and exchange cluster was most highly represented here with six of the cluster's statements. As a proportion of statements in the cluster, however, there was a tie between this cluster and the engagement cluster with both represented by 67% of their statements. No statements from the developing cultural capacity cluster were rated highly enough to fall within the go-zone. Twenty statements were rated to be relatively low both in importance and in feasibility. Every cluster had representation in this quadrant with the public forums cluster as the most highly represented, and the engagement cluster as the least represented.

Collaboration and relationship building throughout this project allowed for meaningful contributions of the core group. In the final meeting, knowledge translation was explored with older adults and their caregivers where they shared what approaches would be meaningful to them including (1) opportunities for knowledge mobilization and communicating the results, (2) how best to position the research going forward (i.e. in living labs and developing RHIEs) and (3) opportunities for actioning go-zone items. Themes from this discussion as well as issues of usability and implementation of the framework of priorities are elaborated in the 'Discussion' section.

Regional systems of innovation build capacity for transformative activities that have the potential to improve the lives of citizens in surrounding communities. Explanation of the success of these systems has evolved from literature on regional science combined with evolutionary economics, 34,35 the economics of innovation, 36,37 theories of interactive learning 38 and institutional economics. 39 In this article, we examine how older adults, a historically excluded population from the world of technology and innovation, might participate in a manner that is both empowering for them and instructional for those developers and decision-makers that are the generative engine of the ecosystems. The evolution of the triple helix model, 4 to include a complementary helical thread that accounts for the importance of civic engagement in RIE development, 10 aligns with the evolution of healthcare's current focus on patientcentred care [40] [41] [42] and patient engagement. [43] [44] [45] [46] [47] The clearest take home message from this research study is that partners representing each helix in innovation for health and age tech value engagement of older adults and their caregivers in RHIEs. According to participants, 'engagement' refers to interactions and opportunities to understand and learn from older adults and their caregivers, and meaningful dialogues between diverse groups involved in innovation. By virtue of including older adults and their caregivers in the process of this project, we have demonstrated that older adult and caregiver citizens want to and can be included in regional innovation system development. Their contributions have provided important insights into the mechanisms which might ensure innovations in the health and aging-related technology sectors are useful, relevant and valuable to target populations, a crucial component of successful adoption of novel products and processes. These findings align with a recent systematic literature review of older adult involvement in gerontechnology design by Merkel and Kucharski. 7 This review found that a focus on user characteristics, needs and preferences by designers and developers will increase market penetration and encourage older people to adopt, implement and use technology. The authors suggest the integration of older adults into the innovation process following a participatory approach, and advocate for partnerships among researchers, designers, developers and older people, throughout the whole innovation process.

Through multidimensional scaling and hierarchical cluster analysis, individual ideas generated by participants to identify ways that older adults and their caregivers might engage with RHIEs were arranged in a seven-cluster framework. This framework and the final list of 62 statements represent a wide range of ideas related to the engagement of older adults and their caregivers in innovation for health and aging-related technology.

For engagement to be realized, the ECOTECH framework must be actionable; this study provides direction on how meaningful engagement of older adults and their caregivers in RHIEs can be achieved by examining perceptions of importance and feasibility. Cluster analysis identified that the clear priority for stakeholders is engagement, with the highest feasibility and importance. Participants identified the ideas in this cluster as closely related, and thus important to understanding the framework of priorities. The importance of engagement is consistent with extant literature and more formal efforts to engage patients, caregivers and citizens in healthcare systems from clinical decision-making 48 to research and planning efforts. 49 In contrast to the engagement cluster, the co-production and partnerships cluster was understood as a highly important but unfeasible cluster. This cluster incorporates ideas of varying levels of involvement, from including older adults and their caregivers in brainstorming sessions to fostering relationships between innovators and older adults and their caregivers. The idea of partnerships offers an opportunity to provide this meaningful engagement, but also creates complexity 50 when engaging older adults in health innovation. Partnerships have been defined in numerous ways but typically include aspects of trust and interdependence of participants. 51 The importance of the relationship between those involved in engagement activities has been highlighted in the literature on older adult engagement in healthcare research and planning. 49 The extension of this principle to the context of innovation ecosystems is a novel contribution of this study. The co-production and partnerships theme has links to best practice guidelines and strategies of current leaders in innovation for health and aging-related technology. 52 As we seek to establish meaningful engagement of older adults and their caregivers in RHIEs, the perceived challenges identified in this study through the 'no go-zone' must be addressed. For example, statement 36, 'have residents of long-term care be involved in their facility's ethics committees to make decisions about projects taking place related to innovation' could invite important conversations to minimize the barriers to participation reported by older adults and their caregivers. Recent work by Gauvin et al. 53 has supported this illumination of the challenges associated with adopting health and aging-related technologies in long-term care. The potential solutions that emerged from their study support our work, including their suggestion for engagement of long-term care home operators, staff, residents and their caregivers in developing and adopting technologies. 53 One recognized benefit of working towards practices of co-production is the link to social capital. Needham et al. 54 suggest that co-production increases social capital through the resulting creation of supportive relationships and an increase in personal self-confidence. Being meaningfully involved in the direction of projects can positively affect the health and well-being of not only the participants, but also of society more broadly. Despite this positive impact, the contrast between perceived importance and feasibility of this cluster raises questions about the status quo of innovation in health and aging. Themes within this cluster of older adults and their caregivers getting involved in research and innovation from early stages of projects were perceived to be important, however were rated as less feasible. Previous work on engagement of older adults and their caregivers in health research and planning has identified environmental or contextual aspects (McNeil, Elliott and Huson, 2016) that might be relevant in understanding this disconnect. The importance of establishing an understanding of organizational support for engagement efforts, for example, will impact time commitments of those conducting research, recognizing that partnership approaches to knowledge generation often take more time. 49 Participants' perspectives reflected in misaligned ratings (i.e. differing rates of feasibility and importance) suggest a discouraging status quo for innovation in health and aging, where co-production and partnership efforts are not thought to be feasible by the very people for whom the innovations are being developed. Contemporary work by Merkel and Kucharski 7 supports encouraging meaningful involvement of older adults and their caregivers throughout the innovation process, noting that most studies currently have only paid attention to single phases of the innovation process, but an opportunity remains to focus on engagement across the span of the innovation process. Future research in this area should investigate how we can ensure that feasibility is not a barrier to engaging older adults and their caregivers who may require unique considerations to participate meaningfully. Adopting a co-production mindset could help to minimize tokenistic attempts at engaging older adults by enabling a deeper level/ complexity of engagement.

Transdisciplinary approaches to knowledge generation and exchange may address barriers and encourage engagement in RHIEs. 49 Greenhalgh et al. 55 discuss this in the context of culture change, highlighting the importance of interpersonal networks to overcome barriers in linkage and exchange efforts. In their framework for health system change, Goodyear-Smith et al. 56 highlight the need for 'cultural humility' among stakeholders in order to generate a willingness to engage across traditional boundaries for coordinated action. Developing humility might evolve from mutual understanding.

Results and next steps for this study include the development of evaluation tools to determine the level of civic engagement in RHIEs, based on the framework of priorities. It has been established in recent literature that a limitation of research involving engagement or co-design methods is the lack of evaluation. 57 Our study provides an example of a feasible method for implementing engaged research in an RHIE from conceptualization of a project to evaluation. Concept mapping has been recognized as a tool for such implementation because manageable subtopics and tasks emerge from the results. 24 The rating activity results, displayed through the pattern matches and gozones, can be used to guide the operationalization of the concepts in an ecosystem wishing to build-up its civic involvement. The breadth of topics in the individual statements suggests that jurisdictions wishing to move ahead with the concepts will need to clearly understand which ecosystem entities are responsible for action on the statements. Furthermore, the importance of historical and local regional context will also be an important consideration as ECOTECH moves through the knowledge to action cycle in those developing RHIEs.

A strength of this project is the standardized group concept mapping research approach which relies on meaningful participation of stakeholders from different backgrounds with diverse disciplinary perspectives and life experiences. 24 The flexible methods used to incorporate the perspectives of older adults and their caregivers who were not able to participate using the online software were important in achieving this diversity of perspectives. To the research team's knowledge, the use of in-person brainstorming, card sorting and rating with older adults and their caregivers is unique to studies adopting this approach. 58 The work of Hanson et al., 58 while a novel and important entry of these methods into gerontological studies, was limited in that they relied solely on online brainstorming, sorting and rating methods. Their recruitment through email may have limited the contributors participating in the project. In contrast, our project incorporates in-person opportunities to contribute throughout all phases of the concept mapping, providing greater diversity of age and life experience with technology and innovation.

The use of the online software enabled participants from many geographical locations to participate in this study. Although, as expected due to the location of the research team, the majority of participants were recruited from Ontario; online tools enabled participation from across Canada. A few challenges arose with the use of this method with older adult participants, leading to some possible limitations of this study. The first challenge arose in the sorting phase. Some participants experienced difficulty with the online software used for participation -even those older adults who considered themselves computer-literate.

Although a desirable option by some of the participants, the amount of time associated with in-person sorting proved to be another issue for some older participants. The in-person sorting activity was scheduled for a 60-min session. Though there were some 'fast finishers' in the group, who began the rating activity while others remained working on their sorting, many participants spent approximately 90-min working on the sorting activity. In debriefing after the session, participants remarked that while they enjoyed the activity, they found it 'draining'. At the end of the sorting session, one participant handed in his completed piles with a note on the top of the cards stating, 'and now my mind is blank'. This mental and physical fatigue experienced could have contributed a limitation to the quality of the completed sorting. To mitigate this, the author reviewed the submitted sorting data for completeness and connection to the themes of the project to ensure quality. In future applications of this method with older adults, the authors suggest changes to the pacing of the activities. For example, it might be appropriate to break up the sessions in each phase, perhaps into two shorter meetings or to provide more opportunities for health or nutrition breaks.

Notwithstanding these challenges, the group aggregate map and identification of opportunities and next steps for implementation of this framework of priorities demonstrates the success of the collaboration efforts, and the usefulness of concept mapping for research related to older adults and their caregivers.

This study identified a framework of priorities for directions and strategies on older adult and caregiver engagement in RHIEs. Implementation of this framework could help to advance the development of theory and evaluation in the area of older adult and caregiver engagement in innovations for health and aging. It is hoped that future planning of interventions and ecosystem development efforts will be improved by the results of this study, specifically through the implementation of the framework of priorities.

By examining how older adult and caregiver engagement can be realized in RHIEs, this project demonstrated the acceptability of a modified concept mapping technique for gerontological research. The next steps of this study involve continuing to collaborate to develop engagement in Canada's emerging RHIEs that support the health and well-being of older adults and their caregivers.

Design thinking for social innovation

The importance of user involvement: a systematic review of involving older users in technology design

A system theoretical approach to the characteristics of a successful future innovation ecosystem

The triple helix -universityindustry-government relations: a laboratory for knowledge based economic development

Vulnerable people, groups, and populations: societal view

The silver market phenomenon: marketing and innovation in the aging society

Participatory design in gerontechnology: a systematic literature review

The meaning and significance of self-management among socioeconomically vulnerable older adults

Ageism and technology

Mode 3' and 'Quadruple Helix': toward a 21st century fractal innovation ecosystem

From translational research to open technology innovation systems

Userinnovated eHealth solutions for service delivery to older persons with hearing impairment

Defining integrated knowledge translation and moving forward: a response to recent commentaries

Improving Canada's position in healthcare innovation: learning from the Swiss experience

Ontario Health Innovation Council. The catalyst: towards an Ontario health innovation strategy: a report to the government of Ontario

Regional innovation ecosystems: a decade of literature, conceptual, and theory development

WikiBuild: a new application to support patient and health care professional involvement in the development of patient support tools

Patient and citizen participation in health: the need for improved ethical support

Canada's population estimates: age and sex

The state of seniors health care in Canada

Care transition experiences of spousal caregivers: from a geriatric rehabilitation unit to home

Information sharing with rural family caregivers during care transitions of hip fracture patients

An examination of family caregiver experiences during care transitions of older adults

Applied social research methods: concept mapping for planning and evaluation

Aging gracefully across environments using technology to support wellness, engagement and long life -AGE-WELL

Concept mapping methodology: a catalyst for organizational learning

A SHARP response: developing COVID-19 research aims in partnership with the seniors helping as research partners (SHARP) group

Qualitative research and evaluation methods

Concept Systems Incorporated. The concept system®. Ithaca, NY: Concept Systems Incorporated

Focus groups: a practical guide for applied research

Mixed methods research: a research paradigm whose time has come

Quality and rigor of the concept mapping methodology: a pooled study analysis

Research design qualitative, quantitative, and mixed methods approaches

The economic foundations of technology policy: equilibrium and evolutionary perspectives

An evolutionary theory of economic change

Sources, procedures, and microeconomic effects of innovation

An overview of innovation

National systems of innovation: towards a theory of innovation and interactive learning

Systems of innovation: technologies, institutions, and organization

Transforming care teams to provide the best possible patient-centered, collaborative care

Patient-centred care is a way of doing things: how healthcare employees conceptualize patient-centred care

Patient-centered care: achieving higher quality by designing care through the patient's eyes

Patient engagement: what is patient engagement?

Patient and staff engagement in health system improvement: a qualitative evaluation of the experience-based co-design approach in Canada

Re-orienting health systems through a commissioning approach: finding solutions for improved consumer engagement

UK Standards for Public Involvement. UK standards for public involvement

Seven lessons from the field: research on transformation of health systems for older adults

Engaging older adults in health care decision-making: a realist synthesis

Engaging older adults in healthcare research and planning: a realist synthesis

Partnerships: panacea or pretence?

Trust the process: community-based researcher partnerships

Evidence brief: identifying and harnessing the potential of technology in long-term care settings in Canada

Co-production: an emerging evidence base for adult social care transformation

Diffusion of innovations in service organizations: systematic review and recommendations

Co-design and implementation research: challenges and solutions for ethics committees

Research co-design in health: a rapid overview of reviews

Concept mapping applied to the intersection between older adults' outdoor walking and the built and social environments

We wish to thank all the participants in our study, especially the members of the SHARP Group who partnered in ECOTECH as members of the core group.

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the AGE-WELL (Aging Gracefully across Environments using Technology to Support Wellness, Engagement and Long Life Networks of Centres of Excellence) which is funded by the Government of Canada's Networks of Centres of Excellence Programme.

Ethical approval for this study was obtained from University of Waterloo Office of Research Ethics (approval number 21329).

Written informed consent was obtained from all subjects before the study.

Paul Stolee https://orcid.org/0000-0002-5685-0843