key: cord-0870520-gmc7hba3
authors: Strudwick, Gillian; Risling, Tracie; Kassam, Iman; Shin, Hwayeon Danielle; Moss, Tyler; Carlberg, Courtney; Zhou, Wenjia
title: Lessons learned developing and deploying a provincial virtual mental health support during the COVID-19 pandemic
date: 2022-05-21
journal: Healthc Manage Forum
DOI: 10.1177/08404704221092691
sha: ea759fc58332e3613fad27381bb439acb0d2738a
doc_id: 870520
cord_uid: gmc7hba3

Significant efforts have been put into implementing virtual forms of healthcare and supports since the beginning of the pandemic. However, limited information has been shared with health leaders about how this has taken place, and what can be learned from this to move forward into the future. The purpose of this paper is to describe lessons learned co-designing and developing a virtual health support during the COVID-19 pandemic in the province of Saskatchewan. In this paper, we anchor these lessons learned on a specific virtual health service support, “SaskWell,” which offers a digital service, and aims to connect residents of the province to digital mental health supports and resources.

Since the onset of the COVID-19 pandemic, there has been a significant increase in the use of virtual forms of health services, including in the area of mental health. [1] [2] [3] This has been especially true as increasing numbers of Canadians have reported mental health challenges since the onset of the pandemic. 4 These virtual forms of health services were catalyzed, and in numerous cases, initially developed, out of the need to maintain physical distancing between individuals to prevent viral spread. 5 Virtually enabled healthcare thus became coined as a digital form of "Personal Protective Equipment (PPE)," to ensure the safety of both the healthcare provider and patient. 5 Examples of virtual health service delivery in Canada are numerous. For example: the BC COVID-19 Self-Assessment Tool 6 and SaskWell 7 (a text-based service to support all residents of Saskatchewan who are looking for mental health and wellness supports and resources during and beyond the COVID-19 pandemic). These are two examples among thousands of others. While some of these services and supports have gone back to "in-person," many remain virtually delivered today and may continue to do so in the foreseeable future. There is also the potential that many of these virtual service offerings are here to stay long term; however, scientific evidence will need to catch up with regards to what kinds of care and support are appropriately delivered virtually, how frequently, for whom, who this kind of delivery leaves out, and other important equity and clinical considerations. [8] [9] [10] [11] Purpose While significant efforts have been put into implementing virtual forms of healthcare and supports, limited information has been shared about the process in which this took place and what was learned from these endeavours that could be useful in the future. Rightfully so, organizations that switched to "virtual" or developed a digital health service or support to address a pandemic need, did so quickly. There was little time to document, reflect, and communicate this experience. Thus, the purpose of this paper is to describe lessons learned codesigning and developing a virtual health support during the COVID-19 pandemic. In this paper, we refer to the virtual health support as "SaskWell." A formal research study is currently underway with SaskWell to understand usage patterns and preliminary outcomes of using the service. The results of this study, which has a different purpose than what the current paper discusses, will be reported in an upcoming research paper.

SaskWell, a 10-week text-based service was developed with the purpose of delivering mental health tips and resources to residents across the province of Saskatchewan. The service was co-designed in partnership with a Patient and Community Advisory Committee (PCAC), incorporating community voices into the design of the service. An implementation science framework called "RE-AIM" 12 was used to support the iterative development of the service, such that four consecutive 10-week iterations of the SaskWell service were deployed (see Figure 1 for further details). In each iteration, new features were incorporated into the service based on the feedback and suggestions of the PCAC. The The service connects individuals across the province to mental health resources and digital mental health tools derived from a curated library of web and app-based digital health interventions. 13 The use of text messages allowed those with the most commonly accessible technology, mobile devices, to receive digital mental health supports in regions of the province where access to supports is limited. Users can enrol into the SaskWell service by clicking the enrollment link: https://mtxt.io/besaskwell, texting "JOIN" to 759355, or by calling a toll-free number: 1-855-237-5934. Prior to enrolling into the service, individuals fill out a brief enrolment survey, consisting of three sections: (1) Demographic Questions;

(2) Technology and Internet Access; and (3) Mental Health Self-check Questionnaire. Upon enrolling into the service, users are paired with a digital mental health tool (ie, mobile app or web-based resource) that reflects their access to technology and the internet. Additionally, users receive two wellness tips and one polling question each week. Further details of the SaskWell service are described in the study protocol. 7

The following section outlines the lessons learned as they relate to: (1) people; (2) process and technology; and (3) partnerships.

Working with knowledge users improves the value, relevancy, quality, 14, 15 and uptake 16 of research. At the outset of the study, we worked closely with knowledge users consisting of a patient research partner, provincial and federal organizations, nurses, digital mental health researchers, and a digital patient engagement company. Specifically, our research team is comprised of a diverse group of digital health researchers, population mental health researchers and nurses, with experience in implementation science and patient engagement research. The intersecting professional backgrounds and research interests equipped our team with the competencies to plan and execute the SaskWell service.

Our patient research partner has experience in leading patient advisory committees in addition to considerable knowledge of Saskatchewan based research organizations and initiatives (eg, Saskatchewan Centre for Patient Oriented Research and eHealth Saskatchewan). In collaboration with our patient partner, we formed our PCAC, comprised of five residents of Saskatchewan with a range of lived experiences. The PCAC members were each located in different communities across Saskatchewan and worked within a variety of sectors. Due to this, the PCAC helped greatly in extending the reach of SaskWell across the province through their diverse community networks. Each month we held virtual onehour committee meetings with the PCAC members, where we discussed the project progress and sought their advice and insights on each project stage (eg, design, development, and deployment). In these meetings, we reminded the PCAC of the research objectives and project scope, as prior studies have highlighted the importance of agenda-setting and clear communication when collaborating with research partners. 17 Moreover, we presented how their insights and feedback were incorporated into the service, demonstrating how their knowledge was implemented in practice. During these meetings, feedback was obtained about the SaskWell service from the PCAC, such as:

Even though I don't always look at the message right away, the fact that it comes through text is helpful because it will always be there. So if I miss something two days ago, I can go back and access it easily. And because the texts are short, it's easy to scroll through the messages -PCAC Member Doing the mental health assessment questionnaire was really helpful to have to do. My mental health was not as bad as I thought it was. This made me feel like I was just like everybody else, normal.

This was helpful in making further decisions regarding design iterations, confirming content length and other important considerations. Outside of the committee meetings, the PCAC members were given a tracking document for each iteration to make note of any technical bugs or glitches, suggestions to improve the usability of the service, and text message content updates. Additionally, to learn more about the PCAC members' individual experiences as both partners and users of SaskWell, we conducted semi-structured interviews with each member. One challenge in building the PCAC was recruiting diverse voices from across the province. The pandemic measures limited our reach to northern and remote regions of the province, as such we lacked representation from these regions.

In consideration of the technical development of the SaskWell service, we utilized the expertise of our collaborators: MEMOTEXT, a digital patient engagement company, and Canada Health Infoway, a federal organization tasked with supporting the adoption of digital health technologies across Canada. MEMOTEXT has prior experience in developing population health text-based services and provided extensive support in devising the blueprint and back-end logic of SaskWell. A representative from MEMOTEXT greatly supported the research team and served as a point of reference and expert for technical components of the service. Canada Health Infoway provided support in developing our user engagement and recruitment strategies, given their experience in implementing provincial and national digital health technologies. Both entities frequently attended meetings with the internal research team and were engaged throughout each iteration of SaskWell.

Reflecting on our experience working closely with a variety of knowledge users, we believe that ongoing learning and engagement with each stakeholder was a key facilitator in ensuring SaskWell was relevant and useful for Saskatchewan residents during the COVID-19 pandemic. As echoed by researchers who work closely with public partners, continued communication and engagement is a necessary component of building partnerships and meaningful outputs. 18 Furthermore, each partner provided different levels of engagement with the SaskWell service. The PCAC was most engaged in decision making processes throughout the duration of the project and can be considered at the level of "empowerment" on the International Association for Public Participation spectrum. 19 

Designing, developing, and maintaining a text-based service through four consecutive 10-week iterations was a new undertaking for most of the research team. In the design phase, we leveraged the expertise of our collaborating developer, MEMOTEXT, in addition to insights from our PCAC members. SaskWell was conceived through a series of planning meetings in which the services structure, content and methods of enrolment were determined. In designing the structure and content of the service, we leveraged the findings from a knowledge synthesis study which curated a list of digital mental health interventions that could be used to support the mental health of Canadians during the COVID-19 pandemic. 5 Despite the plethora of digital mental health tools available, it was challenging to find resources specific to the Saskatchewan context and freely available to residents of Saskatchewan. Furthermore, to ensure the digital tools we included within the service were accessible to the user, only digital tools which were free of cost, available across Canada, and had supporting literature were included. This measure of accessibility limited the digital tools we were able to include in the service. To fill this gap, we relied on the lived/living experiences of the PCAC members to identify digital mental tools or resources available in Saskatchewan that they have used previously. Further to this, the PCAC was frequently consulted to assess whether existing content, in particular the wellness tips, was helpful or relevant. The adaptive nature of text messaging allowed for frequent updates to the service's content. Thus, as the COVID-19 landscape evolved in the province, the wellness tips and resources were updated to reflect these changes. These updates were well received by users of the service, whereby relevancy of the messages to the pandemic climate, increased user engagement. Although this was largely beneficial, in some instances, changes to public health guidance and measures outpaced our ability to update or remove messages, resulting in some users receiving outdated messages.

When developing the SaskWell service and preparing for the initial launch, a major milestone achieved was ensuring users were able to enrol in the text service through differing modalities. The most used enrolment method was texting the short code (759355) registered to the SaskWell service, followed by clicking on the web-based enrolment link, and lastly calling the toll-free phone number linked to the project. Providing users with numerous enrolment options furthered the driving purpose of SaskWell in bridging technology and access divides. Those who may not have reliable internet access or digital literacy to use a short code or complete a web-based enrolment survey had the option to call our toll-free number to enrol in SaskWell. Although the enrolment modalities improved the accessibility of the service, one lesson learned was accounting for administrative delays in obtaining Canadian Wireless Technology Associations' approval for the short code and institutional approval for the toll-free phone number.

The iterative nature of the SaskWell service enabled the team to implement and test new changes, updates and features for each iterative cycle. This provided the opportunity to build off each cycle, incorporating the lessons learned from the PCAC and end-users of the service to improve the relevancy and usability of the service. The PCAC, in addition to the research team, served as user testers of the SaskWell service. Prior to the public launch of each iteration, both the PCAC and research team would extensively test the service to identify and resolve technical issues. The PCAC would also share how the service integrated into their daily lives and ideate potential features to incorporate into the service. One such feature, which was implemented in the third iteration of SaskWell, involved users being able to select the days and times they wish to receive messages. This suggestion was brought forward and advocated by the PCAC with the notion that it would allow users to further personalize or tailor the service to their needs. While the agile and iterative approach to designing and improving the service supported user engagement, unforeseen challenges arose with regards to deploying the iterations in a timely manner. Incorporating new features and updating components of the service required complex technical changes to the back end of the service. Consequently, these changes delayed the start date of new iterations and in some cases resulted in the appearance of technical bugs and glitches. Upon reflection, allocating more time in-between iterations to implement and test new features would have reduced some of the challenges we faced.

Recruiting participants for the SaskWell service was challenging given the limitations of recruiting and advertising virtually. To mitigate these challenges, we built partnerships with individuals, organizations and community networks to bring awareness to SaskWell. Our partnership with the PCAC was instrumental in enhancing recruitment capacity in our project. The PCAC members aided our research team in strategizing innovative recruitment methods to ensure SaskWell could recruit a diverse range of participants. Through a combination of traditional and innovate methods both the research team and the PCAC: (1) distributed posters and flyers within community settings across the province (eg, community centres, public libraries, grocery stores, and gas stations), (2) advertised the SaskWell service on social media (eg, Facebook, Instagram, Twitter, and Reddit), (3) posted digital posters on discussion forums and on-line public boards (eg, University of Saskatchewan's' Announcement board and SCPOR's research board), (4) contacted community and provincial organizations, not-for-profits, unions and professional associations to share our digital recruitment information with their patrons and members, and (5) supplemented recruitment through word of mouth. In addition to partnering with the PCAC, we also established partnerships with reputable organizations in Saskatchewan, specifically SCPOR, 211 Saskatchewan, the Saskatoon Public Library and the Saskatoon Engineering Students Association. Through these partnerships we gained public attention, resulting in interest from print and on-line news media (eg, newspaper articles, radio interviews, video interviews, and magazine articles).

A lesson learned in recruiting users for SaskWell was highlighting the credibility and trustworthiness of the textbased service at the outset of recruitment. 20 Initially, we developed recruitment posters that did not include the logos of our affiliate academic and healthcare organizations. The PCAC members expressed that we should include these logos in all our advertisements to make it explicitly known to the public that we are a resource they can trust, rather than a commercial digital health platform. One PCAC member shared the following: I had some issues when putting up the posters [in the community] there was previously little to no branding on them. It is important to identify who exactly is running this program [SaskWell] as some people can be quite wary." -PCAC Member Moreover, the importance of peer recommendations and word of mouth was underscored throughout our recruitment process. In discussions with the PCAC members it was suggested that individuals would be more likely to try a new resource if an individual or entity they trust (eg, university) recommended or promoted the resource. In consideration of this, we leveraged our partnerships with Saskatchewan based organizations and the PCAC to spread the word of SaskWell within their community networks.

In conclusion, numerous organizations and health systems across Canada have rapidly deployed virtual health technologies especially given considerations related to the COVID-19 pandemic. That said, despite these rapid activities being done, little information has been shared with healthcare leaders about lessons learned engaging in this work that could be helpful for future endeavours. This paper outlined several of the lessons learned from deploying a particular technology called SaskWell to support population mental health by considering implications related to people, process and technology, and partnerships. Significant engagement of service users in the development, launch, use, and evaluation of similar technologies is essential for its success.

Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario

Virtual care use before and during the COVID-19 pandemic: a repeated cross-sectional study. C Open

Utilization of physicianbased mental health care services among Children and adolescents before and during the COVID-19 pandemic in Ontario

Centre for Addiction and Mental Health

Digital interventions to support population mental health in Canada during the COVID-19 pandemic: a rapid review (Preprint)

Government of British Columbia. BC COVID-19 self-assessment tool. Victoria, BC: Government of British Columbia

Supporting population mental health and wellness during the COVID-19 pandemic in Canada: protocol for a sequential mixed-method study

The efficacy of remote virtual care in comparison to traditional clinical visits for elective orthopaedic patients: a metaanalysis of prospective randomised controlled trials. Surgeon; 2021

A systematic review of the efficacy of telemedicine for making diagnostic and management decisions

The Evidence Base for Telehealth: Reassurance in the Face of Rapid Expansion During the COVID-19 Pandemic

Virtual Care in Canada: Discussion Paper

RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Heal

Digital interventions to support population mental health during COVID-19: a knowledge synthesis

The dark side of coproduction: do the costs outweigh the benefits for health research?

Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge

Increasing research impact through partnerships: evidence from outside health care

Trainee-led research using an integrated knowledge translation or other research partnership approaches: a scoping review

Engaging indigenous patient partners in patient-oriented research: lessons from a one-year initiative

International Association for Public Participation. Core values, ethics, spectrum: the 3 pillars of public participation

A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects

The authors would like to thank the insights shared and contributions of the Patient and Community Advisory Committee. We would also like to thank MEMOTEXT, for their support in developing the SaskWell service.

Funding for the service was obtained from the Canadian Institutes of Health Research and the Saskatchewan Health Research Foundation.

Gillian Strudwick  https://orcid.org/0000-0002-1080-7372 Hwayeon Danielle Shin  https://orcid.org/0000-0003-4037-4464