key: cord-0694883-9gwcxw1x
authors: Rodríguez-Rivas, Matías E.; Cangas, Adolfo J.; Fuentes-Olavarría, Daniela
title: Controlled Study of the Impact of a Virtual Program to Reduce Stigma Among University Students Toward People With Mental Disorders
date: 2021-02-09
journal: Front Psychiatry
DOI: 10.3389/fpsyt.2021.632252
sha: dddd7f4ac3dd8030cdeed6c8b5b3631fdedc1342
doc_id: 694883
cord_uid: 9gwcxw1x

Stigma toward mental disorders is one of today's most pressing global issues. The Covid-19 pandemic has exacerbated the barriers to social inclusion faced by individuals with mental disorders. Concurrently, stigma reduction interventions, especially those aimed at university students, have been more difficult to implement given social distancing and campus closures. As a result, alternative delivery for programs contributing to stigma reduction is required, such as online implementation. This paper reports the results of a controlled study focused on an online multi-component program on reducing stigma toward mental illness that included project-based learning, clinical simulations with standardized patients and E-Contact with real patients. A total of 40 undergraduate students from the Universidad del Desarrollo in Santiago, Chile, participated in the study. They were randomly divided between an intervention and control group. The intervention group participated in the online multi-component program, while the control group participated in an online educational program on cardiovascular health. We assessed the impact of the program by using the validated Spanish-language versions of the Attribution Questionnaire AQ-27 and the Questionnaire on Student Attitudes toward Schizophrenia with both groups, before and after the intervention. In addition, an ad hoc Likert scale ranging from 0 to 5 was used with the intervention group in order to assess the learning strategies implemented. Following the intervention, the participants belonging to the intervention group displayed significantly lower levels of stereotypes, perception of dangerousness, and global score toward people with schizophrenia (p < 0.001). In addition, participants presented lower levels of dangerousness-fear, avoidance, coercion, lack of solidarity, and global score (p < 0.001). The control group displayed no statistically significant differences in the level of stigma before and after the evaluation, for all of the items assessed. Finally, the overall assessment of each of the components of the program was highly positive. In conclusion, the study shows that online programs can contribute to reducing stigma toward mental disorders. The program assessed in this study had a positive impact on all the dimensions of stigma and all of the components of the program itself were positively evaluated by the participants.

Stigma toward mental disorders is one of today's most pressing global issues. (1) , with important implications such as high levels of social exclusion (2) , poor quality of life (3) , and low social support (4) , for those affected by them. Furthermore, there are barriers to access and continuity of education and employment (5, 6) , and higher risks of dying by suicide (6) among this specific population.

Mental health professionals and medical students have been found to stigmatize psychiatric patients (7, 8) , and 40% of people with a mental disorder report unfair treatment by health professionals such as medical doctors, psychiatrists, psychologists, and nurses (9, 10) . Experiencing stigma in medical settings is a stressor that contributes, among others, to reducing patients' quality of life and to exacerbating inequities in health outcomes and access to healthcare (11) (12) (13) . Furthermore, stigma is considered a major obstacle against processes of mental health recovery (14) (15) (16) and is a main determinant of the quality of care delivered by health professionals and medical students (17, 18) .

At the international level, several strategies have been implemented within the general population, as well as medical students, to reduce stigma toward mental disorders (19) , and the most successful ones have been those involving direct contact with patients (20, 21) as well as programs with strong educational components (16, (22) (23) (24) . It has been shown that direct, personto-person contact and activities where the participants and patients share thoughts and experiences around mental health are key to the success of these programs (25) (26) (27) .

While these strategies have been implemented in different countries (28) (29) (30) and have led to decreasing stigma (31, 32) and promoting positive perceptions of psychiatric patients among students and professionals (33, 34) , their implementation in virtual learning spaces is recent, particularly for university students (35) . Several authors highlight the advantages of using multimedia resources, for instance audiovisual educational resources with standardized patients (36, 37) , E-contact with psychiatric patients (38) , and even the implementation of educational videogames (39, 40) .

In the context of the SARS-CoV−2 pandemic, lockdowns and physical distancing recommendations, stigmatization and instances of discrimination against individuals with mental disorders have increased and have negatively impacted their wellbeing and quality of life (41) . Additionally, the implementation of measures aimed at reducing the transmission of the virus have hampered the implementation of on-site interventions focused on the reduction of stigma toward psychiatric patients (42) , and in that context, developing and implementing innovative and online alternatives is paramount.

Among such alternatives, for university students, active and interdisciplinary project-based learning, where students lead and develop a final project, has been proven useful (43, 44) , as it promotes critical thinking, practical application of knowledge (45) , and problem-solving skills (46) . Its implementation in the training of health professionals on mental disorders is recent, however it has proven to effectively foster professional skills (47) and a comprehensive and inclusive understanding of the different social processes involved in mental health outcomes and care (48) (49) (50) . As it allows students to reflect on their knowledge to implement their skills in a creative way (51) , it is considered an innovative educational tool for the reduction of stigma toward mental health disorders and psychiatric patients.

In addition to project-based learning, the use of standardized patients has shown different benefits for skill training and elearning in the field of education in psychiatry and mental health (52) . Including it as part of educational intervention strategies has had a positive impact (53) , particularly on skill and knowledge development, and for the promotion of a holistic understanding of mental disorders (54) . However, working online rather than on-site with standardized patients in order to reduce stigma toward mental disorders is a recent, underexplored development in the field, which requires complementary educational interventions (37) , along with the creation of original scripts focused on the story around experiencing the illness and the recovery process (55, 56) .

Finally, E-contact, defined as "computer-mediated real-time interactions where members of different groups interact online" (57), has been used as a strategy for raising awareness and reducing prejudice among ideologically different groups (58, 59) . However, its implementation in the field of stigma-reduction is new and innovative, and the existing evidence shows that it can reduce stigma toward the transgender population (60) . With regards to the use of E-contact to reduce stigma toward mental disorders, to date, only one experimental study has been conducted, and has demonstrated that E-contact reduces anxiety, rage and stereotypes toward individuals with schizophrenia (38) . In that sense, this confirms the relevance of including E-contact in interventions aimed at reducing stigma toward people with mental disorders, for instance through the use of synchronous videoconferencing (61) .

The objective of this study is to demonstrate the effectiveness of a multi-component online intervention incorporating Econtact with mental healthcare patients, standardized patients, and a project-based learning program.

The participants were recruited within the Universidad del Desarrollo in Santiago, Chile, during "Innovation Week, " an event organized by the university, where students belonging to the first 3 years of undergraduate studies in different disciplines work on different innovative responses to local social issues.

The sample consisted of 40 university students in their first, second or third year, 32 of which studied health science degrees and the remaining 8 studied social science degrees. Eighty percent of the participants were women and 20% were men, all aged between 18 and 23 years old (X = 20.6; SD = 1.3). None of the participants had been trained in psychiatry.

The intervention group and control group were each made of 20 randomly allocated participants, with equal proportion of men and women, area of study, and no statistically significant difference with regards to age (p > 0.05).

This instrument (62) aimed at evaluating the items of stereotyping toward schizophrenia and social distancing was developed for, and implemented with, high school students in Germany, during the World Program Against Stigma and Discrimination of the World Association of Psychiatry. The current study used the validated Spanish-language version developed by Navarro et al. (63) , which possesses the appropriate psychometric properties, with a Cronbach's alpha of 0.95 for both evaluated factors.

This instrument (64) is used to quantify stigma toward people with mental disorders among the general population. It presents the participants with the case of a person diagnosed with schizophrenia and evaluates stereotyping and prejudice through a 9-point Likert scale. The current study used the Spanishlanguage, shorter version with 14 items, validated by Saavedra et al. (65) , which possesses the appropriate psychometric properties for its four factors. (dangerousness-fear = 0.88, lack of solidarity = 0.837, coercion = 0.864, and avoidance = 0.758).

This instrument was designed specifically for the current study. Participants assessed the integration of the different intervention strategies on a scale of 0 to 5, focusing on the perceived impact on empathy, understanding of recovery and social inclusion of people affected by severe mental disorders. In addition, the degree of recommendation of the program was assessed. The scale is available as Supplementary Material.

Prior to their participation in either group, the participants gave their written informed consent in accordance with the Declaration of Helsinki (66) and the Singapore statement on research integrity (67) . The study was approved by the Ethics Committee of Universidad del Desarrollo, Santiago, Chile (protocol number: 2020-142). Once the participants were randomly allocated to the intervention and control groups, they all completed the AQ−27 and QSAS questionnaires described previously. These questionnaires were sent to the participants 15 min before the beginning of the study and were completed online. The experiment began once the participants had completed both questionnaires.

With respect to the intervention, the intervention group participated in the multi-component online program, which lasted 14 h equally distributed across two days. The program consisted of the three consecutive interventions described below: a. Two sessions including a simulation with standardized patients with emotional, substance use and anxiety disorders and an education workshop. These common mental disorders are psychiatric comorbidities and are highly prevalent among patients with schizophrenia (68, 69) . An online presentation was carried out by two trained actors focused on the main characteristics of these disorders. Previous to the presentation, the research team had designed original scripts based on the recovery process of the patient and their family environment. During every simulation, dynamic interactions took place between the trained actors and the students, who asked them, for instance, about their life-experience, how they were feeling, etc. Additionally, an educational workshop was carried out for each simulation, in order to discuss the essential aspects of the mental disorders presented during the simulation. Each session lasted 2.5 h. b. E-contact activity with an adult diagnosed with schizophrenia, who discussed and analyzed their recovery process with the students, including the role of the healthcare system, the community and their family. This activity lasted 3.5 h. c. Project-Based Learning. During the two days of the program, activities aimed at developing an intervention around the reduction of stigma and the promotion of social inclusion for people with mental disorders, designed and led by the students were implemented. At the end of the second day, the students had the opportunity to present, in a webinar, the interventions they had developed throughout the course of the program, to the other participants and to the members of the evaluation jury, which included the patient with whom the E-contact activity was carried out. This allowed the participants to receive feedback from an individual who had experienced a severe mental disorder. The presentations were carried out in groups, and the interventions introduced were related to the promotion of social inclusion and the rights of people affected by mental disorders in the academic and local community, with emphasis on raising awareness around stigma and education on mental health. This activity had a duration of 5.5 h, distributed between the development of the initiative, tutoring from the teaching team and the final presentations. An example of one of the projects carried out by the students is available as Supplementary Material.

The control group participated in an online educational program on cardiovascular health of the same duration. Finally, at the end of both days, each group answered the AQ-27 and QSAS questionnaires. Additionally, the intervention group completed the Learning Strategies Assessment Scale. Figure 1 summarizes the whole procedure of the study.

Student's t-test for independent samples was used to assess whether there were statistically significant differences between the pre-and post-intervention measures in both groups. Additionally, Cohen's d was used to determine the effect size of the intervention. The pre-and post-measures of each group were subsequently analyzed with the use of Student's t for related samples, which was complemented with the measure of the effect size of the intervention through Cohen's d. Finally, descriptive statistics were used for the analysis of the global assessment of each one of the intervention strategies. We used SPSS 22.0 software to carry out the statistical analysis. 

As Table 1 shows, there is no statistical difference between the intervention and control groups prior to the intervention for any of the items of either instruments. However, statistically significant differences are observed for the group means in the post-intervention measures for both groups for all of the analyzed variables. Using Cohen's d effect size allowed us to determine a great effect size for all of the dimensions evaluated for both instruments, except for the stereotypes indicator, which displays a more conservative effect size. Table 2 shows the average means and standard deviations for each variable for both instruments corresponding to the intervention and control groups for each phase of the study. The analysis of the post-intervention scores of the control group shows that there is no statistically significant difference for any of the variables studied in either instruments. However, statistically significant differences are observed in the same analysis of the scores for the intervention group, for all of the scores of the questionnaires in relation to stigma in every proportion. Regarding the effect size, the program had a positive impact on stigma reduction as reflected on each one of the analyzed items and the global score of both questionnaires.

The assessment given by the participants on the intervention was measured with a Likert scale ranging from 0 to 5 points. As Table 3 shows, 95% (n = 19) of the students assessed at the top level the usefulness of the program in understanding the recovery process and promoting social inclusion of patients with severe mental disorder, while 100% (n = 20) of them rated at the top level the promotion of empathy. Additionally, 95% (n = 19) stated they would recommend this program to a peer. There was no statistically significant differences between the three interventions and their integration. Finally, positive and uniform comments were presented, also in conjunction with the following representative verbatim quotes: "The activities we carried out, helped me to have a better view of mental illness, and the use of simulated patients was useful in the experience and learning"; "I really liked the simulations and the E-Contact, it makes it much more didactic to learn and understand about mental illness" and "was a wonderful experience, it would be a huge challenge to do it on-site."

Stigma toward mental disorders is a pressing issue, considering, on the one hand, its impact on the wellbeing and quality of life of individuals with mental disorders (14) (15) (16) (17) (18) (19) , and on the other hand, that it is pervasive among the general population, health professionals and university students (7) (8) (9) (10) (70) (71) (72) .

At present, and to the best of our knowledge, there is no multi-component program focused on stigma toward mental disorders entirely carried out online. For this reason, as well as the lockdowns and social distancing measures implemented as a result of the SARS-CoV−2 pandemic, this study focuses on the implementation of an online program.

The results show that the intervention had a positive impact on the reduction of stigma among the intervention group. These results may be explained by the intensity of the program, the integration of multiple components, the focus given to the recovery process and the implementation of direct E-contact between mental health patients and students, all of which have been demonstrated to be key points for the implementation of a successful stigma-reduction program (54) (55) (56) (57) (58) (73) (74) (75) (76) (77) . Additionally, further studies comparing the effectiveness of implementing the multicomponent program on-site rather than online, are necessary. Other studies have shown that comparable online interventions have an impact similar to those implemented on-site (27, 78) .

Considering that the intervention was intensive, taking into account the number of activities carried out in a short amount of time, further qualitative studies will explore the experience of the participants and the lessons learned from the implementation of the key components, in order to adapt and adjust the program and make it easier to replicate.

It is important to emphasize that the assessment that the students made of the program was positive with respect to online implementation for each of its components, and the components focused on the promotion of empathy, social inclusion and understanding of the recovery process of severe mental disorders scored the highest, especially when integrated together. This suggests that the participants enjoyed the experience, would recommend it to peers, bringing evidence on the advantages of interventions that take into account the motivation of university students toward learning and goal-achievement, which is in turn one of the main challenges of digital education (79, 80) . In that sense, the educational intervention methods used in the current study are supported by robust theoretical and empirical evidence, and are especially adequate for online implementation, which is key in the context of the SARS-CoV-2 pandemic and the restriction of on-site interventions (36) (37) (38) (39) (40) (53) (54) (55) (56) . This also represents an opportunity for future integration of similar programs in university curricula, as they offer the possibility to reach remote areas, to establish contact between people with different experiences around mental disorders and between patient communities through E-contact and to generate student-led projects. Furthermore, the online modality of these programs promotes collaborative work between individuals and institutions across different countries, contributing, on the one hand, to foster cultural diversity, and on the other hand, to develop international education networks (81, 82) .

In conclusion, the design and implementation of this type of interventions and educational spaces focused on diminishing stigma toward psychiatric patients contribute to normalizing these experiences among the population, by openly sharing challenges around mental health and promoting effective help-seeking and the delivery of adequate treatment to those who need it (83, 84) . The study shows that online programs can promote new types of interventions aimed at reducing stigma not only in a context of mandatory social-distancing and lockdowns due to the SARS-CoV-2 pandemic, but also complementing existing on-site programs.

However, the study presents several limitations. First, the sample is relatively small, comes from one specific institution and is not representative of the general population, which may undermine the generalization of the results and limit the interpretation of the effect size. Second, the phases of the intervention were not experimentally evaluated in an independent way and the measure of stigma was only evaluated for schizophrenia, making it necessary to include other prevalent mental disorders in future research. Third, specific characteristics of the participants and possible confounding factors, which may influence the results, such as their socio-cultural background and their level of knowledge, desire to work in psychiatry and closeness to mental disorders were not evaluated. Finally, the long duration of the program may limit its replication, presenting several logistical challenges in its implementation through virtual platforms, specifically when considering factors such as student motivation and the exhaustion experienced in this regard.

The datasets presented in this study can be found in online repositories. This data can be found here: https://data.mendeley. com/datasets/p5d39f3kp6/1.

The studies involving human participants were reviewed and approved by Ethics Committee of Universidad del Desarrollo, Santiago de Chile (protocol number: 2020-142). The patients/participants provided their written informed consent to participate in this study.

MR-R, AJC, and DF-O contributed to the conception, coordination, and design of the work. MR-R and DF-O contributed to the implementation of the program. MR-R and AJC performed the statistical analysis. All authors contributed to the article and approved the submitted version. 

Informe Anual Sobre Derechos Humanos En Chile

Stigma: Ignorance, prejudice or discrimination?

Measuring the impact of programs that challenge the public stigma of mental illness

Estigma hacia los trastornos mentales: características e intervenciones

Work and employment for people with severe mental disorders. Some proposals based on the available information

A theoretical and empirical framework for constructing culturespecific stigma instruments for Chile

Prevalencias y brechas hoy; salud mental mañana

Estigma en los profesionales de la Salud Mental: una revisión sistemática

Stigmatization of patients with schizophrenia: the influence of university courses on the attitudes of prospective psychologists and doctors

Mental health, poverty and development

Una aproximación al maltrato a los pacientes con trastornos mentales

Evaluation of a pilot intervention to reduce mental health and addiction stigma in primary care settings

Stigmatization of severe mental illness in India: against the simple industrialization hypothesis

Positive mental health and mental health stigma: A qualitative study assessing student attitudes

Mental health stigma and mental health knowledge in Chinese population: a cross-sectional study

Changes in attitudes toward mental illness in healthcare professionals and students

Stigma and mental health professionals: a review of the evidence on an intricate relationship

Evidence for effective interventions to reduce mentalhealth-related stigma and discrimination

Understanding and influencing the stigma of mental illness

Attitudes to mental illness among mental health professionals in Singapore and comparisons with the general population

The global fight against the stigma of schizophrenia

Key ingredients of anti-stigma programs for health care providers: a data synthesis of evaluative studies

Effects of education and social contact on mental health-related stigma among high-school students

Key ingredients to contact-based stigma change: A cross-validation

The California schedule of key ingredients for contact-based antistigma programs

Interventions to reduce stigma towards people with severe mental illness: Systematic review and meta-analysis

Ministry of Health and Health Promotion Agency. Like Minds, Like Mine National Plan 2014-2019: Programme to Increase Social Inclusion and Reduce Stigma and Discrimination for People with Experience of Mental Illness

Opening minds in canada: targeting change

Changing stigmatizing perceptions and recollections about mental illness: the effects of NAMI's in our own voice

Attitude of nurses towards mental illness in Bhutan

Lessons learned from unintended consequences about erasing the stigma of mental illness

Mental health first aid training for the Chinese community in Melbourne, Australia: Effects on knowledge about and attitudes toward people with mental illness

Effectiveness and application of an online leadership intervention to promote mental health and reduce depression-related stigma in organizations

Reducing stigma toward individuals with schizophrenia using a brief video: a randomized controlled trial of young adults

Single-day simulation-based training improves communication and psychiatric skills of medical students. Front Psychiatry

Modern avenues for intergroup contact: using E-contact and intergroup emotions to reduce stereotyping and social distancing against people with schizophrenia

Analysis of the usefulness of a serious game to raise awareness about mental health problems in a sample of high school and university students: relationship with familiarity and time spent playing video games

Effect of a Serious Game (Stigma-Stop) on reducing stigma among psychology students: a controlled study

Impact of SARS-CoV-2 (coronavirus) pandemic on public mental health

Patients with mental health disorders in the COVID-19 epidemic

Using project-based learning to develop teachers for leadership

Scaffolding interdisciplinary projectbased learning: a case study

Perceptions of the benefits of the A3 planner in facilitating project-based learning in accounting education

Project-Based Learning for the 21st Century: Skills for the Future

Educating our future colleagues: creating an MSW palliative care practicum

The impact of Project Based Learning (PBL) on students from low socio economic statuses: a review

Revisiting the effects of project-based learning on students' academic achievement: a meta-analysis investigating moderators

Lifelong learning for professional development in psychiatry: pedagogy, innovations, and maintenance of certification

Instructiveness of real patients and simulated patients in undergraduate medical education: a randomized experiment

A new method of integrating project-based and work-integrated learning in postgraduate engineering study

An online intervention using information on the mental healthmental illness continuum to reduce stigma

The mental status exam through video clips of simulated psychiatric patients: an online educational resource

Efficacy of a Photovoice-based video as an online mental illness anti-stigma intervention and the role of empathy in audience response: A randomized controlled trial

Attitudes to psychiatry and to mental illness among nursing students: adaptation and use of two validated instruments in preclinical education

A dual identity-electronic contact (DIEC) experiment promoting short-and long-term inter-group harmony

Do ideologically intolerant people benefit from intergroup contact?

Improving intergroup relations between catholics and protestants in Northern

Reducing transgender stigma via an Econtact intervention

Shared living experiences by physicians have a positive impact on mental health attitudes and stigma among medical students: a mixed-methods study

Crazy? So what! Effects of a school project on students' attitudes toward people with schizophrenia

Propiedades psicométricas de la versión en castellano del cuestionario de las actitudes de los estudiantes hacia la esquizofrenia

An attribution model of public discrimination towards persons with mental illness

Assessing the factorial structure of the mental illness public stigma in Spain

Available online at

The Singapore statement on research integrity

Current and potential pharmacological and psychosocial interventions for anxiety symptoms and disorders in patients with schizophrenia: structured review

The link between schizophrenia and substance use disorder: a unifying hypothesis

Attitudes of mental health professionals about mental illness: a review of the recent literature

Medical student belief and attitudes toward mental illness across five nations

Education students' stigma toward mental health problems: a cross-cultural comparison. Front Psychiatry

Implementation science and stigma reduction interventions in low-and middleincome countries: a systematic review

Intergroup contact and mental health stigma: A comparative effectiveness meta-analysis

Electroconvulsive therapy: a video-based educational resource using standardized patients

Tell Your Story': A randomized controlled trial of an online intervention to reduce mental health stigma and increase help-seeking in refugee men with posttraumatic stress

The impact of mental health nursing module, clinical practice and an anti-stigma program on nursing students' attitudes toward mental illness: a quasi-experimental study

A randomised controlled trial of repeated filmed social contact on reducing mental illness-related stigma in young adults

Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review

Using technology to maintain the education of residents during the COVID-19 pandemic

COVID-19 impact and virtual medical education

How we do it: modified residency programming and adoption of remote didactic curriculum during the COVID-19 pandemic

Reducing stigma toward psychiatry among medical students: a multicenter controlled trial

La lucha contra el estigma y la discriminación en salud mental. Una estrategia compleja basada en la información disponible