key: cord-0935398-6swipe4m authors: Hawke, Lisa D.; Sheikhan, Natasha Y.; MacCon, Karen; Henderson, Joanna title: Going virtual: youth attitudes toward and experiences of virtual mental health and substance use services during the COVID-19 pandemic date: 2021-04-14 journal: BMC Health Serv Res DOI: 10.1186/s12913-021-06321-7 sha: 43bb7b681f1cf19bc1921f9d12f7d3d131628ab8 doc_id: 935398 cord_uid: 6swipe4m BACKGROUND: During the COVID-19 pandemic, youth mental health and substance use services rapidly moved to virtual modalities to meet social distancing requirements. It is important to understand youth attitudes toward and experience of virtual services. OBJECTIVE: This study examined the attitudes toward and experiences of virtual mental health and substance use services among youth drawn from clinical and non-clinical samples. METHOD: Four hundred nine youth completed a survey including questions about their attitudes toward and experience of virtual services. The survey included quantitative and open-ended questions on virtual care, as well as a mental health and substance use screener. RESULTS: The majority of youth with mental health or substance use challenges would be willing to consider individual virtual services, but fewer would consider group virtual services. However, many have not received virtual services. Youth are interested in accessing a wide variety of virtual services and other supportive wellness services. Advantages and disadvantages of virtual services are discussed, including accessibility benefits and technological barriers. DISCUSSION: As youth mental health and substance use services have rapidly gone virtual during the COVID-19 pandemic, it is essential that we hear the perspectives of youth to promote service utilization among those in need. Diverse, accessible, technologically stable virtual services are required to meet the needs of different youth, possibly with in-person options for some youth. Future research, engaging youth in the research process, is needed to evaluate the efficacy of virtual services to plan for the sustainability of some virtual service gains beyond the pandemic period. The COVID-19 pandemic has transformed the way mental health and substance use (MHSU) services are delivered around the world. Services moved rapidly into virtual formats, with little opportunity for thorough planning, to comply with social distancing requirements [1] . Individuals who were previously connected with in-person care have had to rapidly shift in the way they access services, while those who are newly accessing care have had to navigate a new virtual service system to access services for the first time. Adolescence and emerging adulthood is a key developmental period for the emergence of MHSU challenges [2] [3] [4] . In Canada, about one in five youth had MHSU disorders prior to the COVID-19 pandemic [5] . This is a key developmental period, marked by developmental milestones such as progress and transitions in education, labor force integration, occupational development, social relationships, and autonomy [6] . However, public health pandemic guidelines have interrupted many milestones for youth. For example, educational progress and experiences, employment, and social interaction have been disrupted, which may constitute risk factors for emerging or worsening MHSU challenges. Indeed, youth are experiencing mental health challenges during the pandemic [7] [8] [9] [10] [11] [12] , including heighted symptoms of internalizing mental health challenges [7, 13] , posttraumatic stress disorder [10, 13] , and general stress and distress [10] [11] [12] . Behaviors associated with the maintenance of positive mental and physical health, such as physical activity and sleep, have also been disrupted for many youth [12, 14, 15] . Although the rates of MHSU service use have been increasing in recent years [16] , many youth in need do not access services [17, 18] . Research conducted prior to COVID-19 has highlighted many service access barriers, such as long waitlists, uncoordinated services, age-based transition challenges, and stigma [19] [20] [21] [22] [23] . System transformation initiatives have been underway to address many of these barriers by creating integrated, youth-friendly service delivery models [24, 25] . However, none of those initiatives foresaw the rapid and systemic shift toward virtual services that occurred with the onset of the COVID-19 pandemic. Delivering MHSU services virtually is not a new idea. A wide variety of virtual modalities of service delivery have been developed and tested over the past decades [26] [27] [28] . Some efficacy has been found, with benefits such as easier accessibility, greater flexibility, lower cost, and reduced stigma [26] [27] [28] . While these findings set the stage for transitioning to virtual services with optimism, the urgent nature of COVID-19 pandemic did not provide time to conduct reviews of evidence-based virtual service delivery models and paced implementation according to the implementation science literature [29] . This rapid shift in service modality provides a timesensitive opportunity to generate new evidence to better understand virtual service delivery [30] [31] [32] . The Margaret and Wallace McCain Centre for Child, Youth and Family Mental Health at the Centre for Addiction and Mental Health works from a stakeholderengaged standpoint to advance research with real-world implications for youth, caregivers, and the service delivery system [33] . Our community mental health partners identified virtual service acceptability among youth during COVID-19 as a priority area of inquiry. Specifically, youth-serving organizations wanted to hear how willing youth are to consider virtual services, what kinds of services they want, what their experience of virtual services has been, and why some youth are not willing to access virtual services. When developing new services, it is essential to understand service users' experiences of and perspectives on these services [34, 35] . To promote access, services must be designed and delivered in manners that are responsive to service users' preferences [30, 32] . Given the rapidity with which the transition to virtual models of service delivery occurred due to the physical distancing requirements of the COVID-19 pandemic, without the time required to rigorously follow the evidence base with regard to virtual service design and implementation, it is essential that we examine the success of virtual service delivery from diverse standpoints. An emerging literature has begun describing the transition to virtual care during the pandemic and discussing the successes and challenges [36] [37] [38] [39] [40] . However, there is a paucity of information on patient experiences in general, and particularly on the experiences of youth receiving virtual MHSU care during the pandemic. This study builds on pre-pandemic research on virtual service delivery and emerging literature on virtual services during the pandemic by examining youth preferences for and experiences of MHSU services delivered virtually during COVID-19 pandemic. This paper examines youth preferences for and experiences of virtual MHSU services during the COVID-19 pandemic. As part of a larger longitudinal study examining youth MHSU during the COVID-19 pandemic in clinical and non-clinical samples, this paper analyzes data from the August 2020 wave of data collection. Youth were defined as young people aged 14 to 29 [41] . Four hundred nine participants completed the survey, including 164 drawn from a clinical sample and 245 from a non-clinical sample, representing 66% of the original sample. Details on the participant pools are described in Hawke et al. [7] . Participants received an email message with a web link taking them to an online survey on REDCap software [42] , specifying that the study was about COVID-19 and mental health. After electronic written informed consent, youth completed a 20-30 min survey about MHSU experiences during the COVID-19 pandemic. Repeat survey questions were not carried out. As per our Research Ethics Board approval, consent was sought from all participants regardless of age, and parental consent was not sought. Data collection took place between August 7 and August 30, 2020, i.e., approximately 5 months after COVID-19 was declared a pandemic by the World Health Organization [43] and a state of emergency was declared provincially [44] , and therefore about 5 months after youth MHSU services began moving to virtual modalities. Centre for Addiction and Mental Health (CAMH) Research Ethics Board approval was obtained. Youth engagement is a core component of this study. As part of the CAMH McCain Centre Youth Engagement Initiative [45] , youth co-researchers supported the study. In addition to regular consultations with youth during survey development, results were brought to youth coresearchers for discussion and interpretation purposes. The feedback of the youth co-researchers is presented in the Discussion section. Questions were developed by the research team, in consultation with local service providing agencies, to understand youth virtual MHSU service experiences. These were embedded into the larger survey. Youth coresearchers also collaborated on survey development, in line with the McCain youth engagement model [45] . Participants were asked about their service experiences, including whether they would consider and had experienced virtual MHSU services individually and, separately, in group format, and if so, what type of technology they had used. Based on their responses, they provided qualitative feedback on the types of services they would be interested in, why they might not consider such services, what the optimal features would be, and what the advantages and disadvantages of virtual services were. The GAIN-Short Screener (GAIN-SS) is a self-report screener that identifies the likelihood of meeting diagnostic criteria for internalizing, externalizing and substance use disorders, as well as crime/violence issues [46] . Items are rated on a time-based scale for each symptom, ranging from zero (never) to four (past month). Three or more items endorsed at the pastmonth level suggest a high likelihood of meeting diagnostic criteria or requiring services in that area [46, 47] . The GAIN-SS is validated among youth [46] . Participants were included in the sample if they completed up to the end of the section querying about virtual services. Missing data was minimal, and frequencies are reported. The GAIN-SS Crime/Violence scale was not analyzed with service use preferences, given very low endorsement. Frequencies and chi-square tests were conducted to describe the sample and their virtual service experiences. After examining the demographics by clinical and non-clinical sample of origin, the two samples were merged to understand willingness to consider using virtual services and experience based on the presence of a MHSU concern. Willingness to consider and experience were then further examined across sample by age group (< 20, N = 134; vs. 20+, N = 271, 4 missing age) and pre-COVID-19 connection with services as reported by the participants at our first wave of data collection. The quantitative analysis was performed using the IBM SPSS statistics software (Version 25) [48] . A 'conventional' content analysis was conducted to analyze the qualitative data [49] . The limited research on youth perspectives of virtual care services during a pandemic made this approach most appropriate in analyzing the views of participants. The content analysis was performed by one of the authors (NYS), a female research analyst with a master's degree in public health and training in health promotion. During the open coding process, a series of codes were identified, defined, and derived from data drawn from responses to each question [49] . In order to meaningfully group the codes, they were organized into categories based on similarities and differences. Confirmability of the results was established through team debriefings with NYS and the first author (LDH), a female scientist with a doctoral degree in psychology, to discuss the emergent categories. This process was repeated for each category until a team consensus was met. The qualitative analysis was performed on Microsoft Office Excel. Table 1 consisted of marginally more girls/young women than boys/young men, χ 2 (1) = 3.987, p = .046, φ = .101), and participants who were less likely to be employed prior to COVID-19 χ 2 (1) = 12.776, p < .001, φ = .183). The clinical sample was more likely to screen positive for internalizing, externalizing, or substance use disorders (ps < .001). There were no significant differences between samples regarding ethnicity, or Canadian-born, English speaking, or student status (ps > .10). While the univariate majority of the sample was a girl/young woman, Caucasian, student, and born in Canada, with English as a first language, this multivariate profile accounted for 50 (12.2%) participants. The majority of participants screening positive for internalizing disorders (72.1%), externalizing disorders (62.7%) or substance use disorders (78.9%) would be willing to consider individual virtual services, but a minority would consider group virtual services (< 40% in Among 144 participants who reported that MHSU services were in place for them prior to the pandemic, 67 (46.5%) reported that their services have changed for the worse during the pandemic. Another 25 (17.4%) considered their services to have not changed either for the better or for the worse. For 25 (17.4%), services are perceived to have changed for the better. Qualitative findings are presented in Table 2 . Participants expressed a desire for various forms of counselling or therapy and mindfulness/meditation, online supportive activities such as fitness, nutrition, and educational activities, as well as awareness initiatives. Some further pointed to the wish for recreational and fun activities. For individual virtual care, video calls received the largest proportion of responses, followed by phone or voice calls and text or chat-based support. Some youth were accepting of any format. Technical features included smooth audio and video, good internet connection and lighting, good camera quality, a mute button and the ability to turn off the camera and share files. Youth wanted clear communication in a non-judgemental environment. Many youth valued anonymity and privacy. They wanted services to be free, continuous, convenient and accessible in terms of booking and availability. In terms of group virtual services, the largest proportion of participants wanted video calls, followed by phone and chat/text. Technical features such as a mute button, good camera quality, the ability to turn off the camera and share files were also appreciated. Some youth specifically reported wanting to access group services, while others reported a preference for one-on-one services. Online activities Online workshops that provide information and advice on varying areas of concern related to mental health/ wellness. Online chats with peers who are struggling with similar issues. Activities such as fitness activities, educational activities, fun activities, interactive activities, surveys Awareness Bringing awareness and spreading information about resources. Participating in studies like this one. Therapy I would like one-on-one therapy to help me cope when I am in distress. Counselling or therapy, skills-based, one-on-one therapy, mindfulness or meditation, family therapy Participants who were not willing to consider individual virtual services expressed a preference for in-person services, mostly for human connectedness and more personal care. Others expressed anxiety or discomfort with using virtual platforms, and security, privacy (e.g., living with at home), and technical concerns (e.g., poor internet connection). Many would not consider virtual services because they did not need them or already had supports in place. These findings were consistent across group virtual services, with one added finding for group virtual services: youth expressed that they preferred individual services. Participants who had received individual virtual services qualitatively reported service accessibility advantages including convenience, easy scheduling, and the lack of travel time. They appreciated being able to connect with a professional safely in a comfortable, socially distanced manner during COVID-19, and appreciated seeing the service provider's face. Some participants further reported that virtual individual care was good for communication, an uncomplicated process, and helped their mental health. Expressed disadvantages of individual virtual care included discomfort and problematic interaction factors: it was less personal and awkward, and it was difficult to read body language, establish rapport, and discuss sensitive issues. Participants mentioned privacy concerns and noted greater distractions and interruptions, with more challenges focusing. They expressed technological issues, including a poor internet connection and security concerns. Many did not like that it was not in person. Some further pointed to administrative challenges around scheduling. Participants who had received group virtual services qualitatively mentioned accessibility as an advantage, including convenience and simplicity of use. They appreciated the human connection. Some further appreciated the lack of travel time, comfort of remote services, and flexible scheduling. They appreciated the ability to receive care during the pandemic. Disadvantages were less reported on, although participants did state that there was less human connection, it was less personal, and it was harder to focus. They further noted privacy concerns and technological limitations, such as a poor internet connection. Some mentioned the fact that it is not in person as a disadvantage. This study examined youth attitudes toward and experiences of virtual services for MHSU during the COVID-19 pandemic, providing important youth perspectives on virtual MHSU services during an unprecedented rapid move toward virtual service delivery. The majority of youth screening positive for MHSU challenges would consider using individual virtual MHSU services, but fewer are willing to use group services. While some have experienced such services, using a variety of modalities, many youth with MHSU challenges have not. Overall, virtual service delivery has a number of advantages, but also multiple disadvantages. Previous studies have shown that youth are willing to use virtual MHSU services and that some prefer virtual services [50] . Similar advantages and disadvantages of virtual care have been reported by youth and clinicians alike [26, [50] [51] [52] [53] [54] [55] . For example, accessibility benefits have been identified [26, 50, 55] , as has the value for interpersonal connectedness in virtual services [54] . Confidentiality concerns have been previously expressed [56] [57] [58] [59] , particularly when accessing services in the home [58] . Texting or chatting modalities might reduce this challenge for some youth [58] . Some youth expressed discomfort regarding virtual services as a whole; it may be that in-person services are required for a subset of youth, or it may be possible to work with these youth to understand how to increase comfort with virtual services. Importantly, different youth have different preferences, and diversity of service options should therefore be available. The digital determinants of health are more important than ever as they appear to be exacerbating health problems during COVID-19 [60, 61] . Concerns have been raised around inequitable access to digital health services [32] . Although remote service provides physically distanced care during a pandemic, future research should consider strategies that may address the health inequities exacerbated by the shift to virtual MHSU services, for those without adequate technology and a safe, private space to use it. From a stakeholder-engaged perspective, we worked directly with youth members of the McCain Centre Youth Engagement Initiative (YEI) [45] to develop the study and interpret the findings. Youth staff agreed that the findings resonated with them and the perspectives they are hearing from other Canadian youth. They highlighted the willingness to consider virtual services among youth who are already connected to MHSU services, but less so among those who are not already connected to services. They emphasized accessibility benefits and hoped that these benefits would persist after the pandemic. They also noted concerns around in-home confidentiality and technological barriers, especially for youth without easy access to technology. They added that cameras make some youth selfconscious and can provide a very personal look into a youth's private space. They further indicated that virtual care does not provide a needed psychological separation between the therapeutic environment and the home environment. The youth importantly noted that youth are a heterogeneous group, emphasizing the importance of offering a variety of services and modalities to meet the needs of different youth. While the move to virtual services was conducted as an urgent response to a public health emergency, it is likely that some of the changes will be sustained [53] . As a new paradigm of virtual MHSU service delivery emerges, it will be essential to rigorously evaluate virtual interventions for efficacy and effectiveness, to understand for whom virtual care has benefits, and examine virtual interventions that can be implemented effectively into existing clinical service systems. Adapting virtual services requires a clear understanding of youth experiences and preferences [62] . Sustained efforts to ensure the diverse needs of youth are met are thus essential in the era of digital health. The survey was limited to Ontario, Canada; a more diverse, representative sample collected over a broader geographical area would increase generalizability. Qualitative findings are derived from short survey answers rather than semi-structured interviews and are therefore limited in breadth. Since the survey was conducted online, it missed youth who do not use the internet regularly. Response rates may be differentially associated with level of MHSU challenges. Future research should connect with youth who are not regular internet users or who have limited technology access to understand their perspectives on virtual services, perhaps including telephone alternatives, and on possible means of mitigating technological barriers. As the system continues expands on virtual models of MHSU service delivery, it is essential that researchers, service providers, and interventionists engage directly with youth, caregivers, and service providers [33, 34, 45, 63, 64] to understand their perspectives and experiences. While the majority of youth are willing to consider virtual services, many youth have not received virtual services despite willingness and clinical need. Diverse, identifiable, accessible, technologically stable virtual services are required to meet the needs of different youth, possibly with in-person options for some youth. Future research, engaging youth in the research process, is needed to evaluate the efficacy of the virtual service modalities. Community-based measures to mitigate the spread of coronavirus disease (COVID-19) in Canada Age of onset of mental disorders: a review of recent literature. Curr Opinion in Psychiatry Drug use among Ontario students, 1977-2019: detailed findings from the Ontario student drug use and health survey (OSDUHS). In. Toronto: Centre for Addiction and Mental Health Substance use, abuse and dependence in adolescence: prevalence, symptom profiles and correlates Disorders of childhood and adolescence. In: Mental disorders in Canada: An epidemiological perspective Emerging adulthood: the winding road from the late teens through the twenties Impacts of COVID-19 on youth mental health, substance use, and wellbeing: a rapid survey of clinical and community samples Mental health among transgender and gender diverse youth: an exploration of effects during the COVID-19 pandemic. Psychology of Sexual Orientation and Gender Diversity Impacts of the COVID-19 pandemic on youth mental health among youth with physical health challenges The effect of COVID-19 on youth mental health Emotional distress in young adults during the COVID-19 pandemic: evidence of risk and resilience from a longitudinal cohort study Effects of COVID-19 on college students' mental health in the United States: interview survey study Factors associated with depression, anxiety, and PTSD symptomatology during the COVID-19 pandemic: Clinical implications for U.S. young adult mental health Impact of the COVID-19 virus outbreak on movement and play behaviours of Canadian children and youth: a national survey Covid-19 pandemic brings a sedentary lifestyle in young adults: a cross-sectional and longitudinal study Mental health service use among children and youth in Ontario: population-based trends over time Trends in psychological distress, depressive episodes and mental health treatment-seeking in the United States Disorder-specific mental health service use for mood and anxiety disorders: associations with age, sex, and psychiatric comorbidity Mental health services in Canada: barriers and cost-effective solutions to increase access Systematic review of barriers and facilitators to accessing and engaging with mental health care among at-risk young people What do parents perceive are the barriers and facilitators to accessing psychological treatment for mental health problems in children and adolescents? A systematic review of qualitative and quantitative studies What makes mental health and substance use services youth friendly? A scoping review of literature Transition to adult services for young people with mental health needs: a systematic review Key attributes of integrated community-based youth service hubs for mental health: a scoping review Integrated (one-stop shop) youth health care: best available evidence and future directions Do online mental health services improve help-seeking for young people? A systematic review E-mental health: a rapid review of the literature Technology at the Service of pediatric mental health: review and assessment Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science Online group psychotherapy: challenges and possibilities during COVID-19-a practice review COVID-19 and the future with digital mental health: need for attention to complexities Digital approaches to remote pediatric health care delivery during the COVID-19 pandemic: existing evidence and a call for further research Integrated knowledge translation and grant development: addressing the research practice gap through stakeholder-informed research Youth services system review: moving from knowledge gathering to implementation through collaboration, youth engagement, and exploring local community needs Patient involvement: a vital principle for patient-centred health care Remote consultations in the era of COVID-19 pandemic: preliminary experience in a regional Australian public acute mental health care setting Mental health Services in Lombardy during COVID-19 outbreak Remote treatment delivery in response to the COVID-19 pandemic The COVID-19 pandemic and treating suicidal risk: the telepsychotherapy use of CAMS Implementation lessons from the transition to telehealth during COVID-19: a survey of clinicians and young people from youth mental health services Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support WHO Timeline -COVID-19 Ontario enacts declaration of emergency to protect the public Implementation of a youth-adult partnership model in youth mental health systems research: challenges and successes Development and validation of the GAIN short screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults Global appraisal of individual needs-short screener (GAIN-SS):administration and scoring manual version 2.0.3. In. Bloomington: Chestnut Health Systems IBM SPSS statistics for windows, version 25.0 Three approaches to qualitative content analysis Logging into therapy: adolescent perceptions of online therapies for mental health problems COVID-19 and telepsychiatry: early outpatient experiences and implications for the future Mental health care goes online: practitioners' experiences of providing mental health care during the covid-19 pandemic Integration of online treatment into the "new Normal" in mental health Care in Post-COVID-19 times: exploratory qualitative study Engaging children and Young people in digital mental health interventions: systematic review of modes of delivery, facilitators, and barriers Exploring young people's perceptions of the effectiveness of text-based online counseling: mixed methods pilot study Review of key telepsychiatry outcomes Telepsychiatry with incarcerated youth I'm Kinda stuck at home with unsupportive parents right now": LGBTQ Youths' experiences with COVID-19 and the importance of online support A qualitative study of young people's perspectives on receiving psychiatric services via televideo Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health Virtual inequity: do virtual visits risk leaving some people behind? In: Can Fam Physician Multidisciplinary research priorities for the COVID-19 pandemic Evidence-based practice in real-world services for young people with complex needs: new opportunities suggested by recent implementation science Patient involvement: a vital principle for patient-centred health care Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations We would like to thank the Mental Health and Addictions Youth Network (MAYN) for their insights into the pressing need to understand youth perspectives on virtual service during the COVID-19 pandemic. We further thank the members of the Youth Engagement Initiative at the Centre for Addiction and Mental Health for their insights into the findings. Lastly, we thank the youth participants in the study. Authors' contributions LDH: Secured funding; conceptualized, designed, and operationalized study; analyzed and interpreted data; drafted manuscript. NYS: Analyzed and interpreted data, supported drafting of the manuscript. KM: Interpreted data, edited manuscript. JH: Secured funding; provided project leadership; conceptualized and designed study; interpreted data; edited manuscript. All authors have read and approved the manuscript. This study was funded by the Canadian Institutes of Health Research (CIHR). The authors have no conflicts of interest to declare. Data are available upon reasonable request to the corresponding author (Joanna.Henderson@camh.ca), with Research Ethics Board approval. The authors have no competing interests to declare.