key: cord-0964081-qv9ox08b
authors: Favaretti, Caterina; Adam, Maya; Greuel, Merlin; Hachaturyan, Violetta; Gates, Jennifer; Bärnighausen, Till; Vandormael, Alain
title: Participant engagement with a short, wordless, animated video on COVID-19 prevention: a multi-site randomized trial
date: 2021-11-22
journal: Health Promot Int
DOI: 10.1093/heapro/daab179
sha: 9dbec1df95763f1fa463adcfa34c0763b2446ea6
doc_id: 964081
cord_uid: qv9ox08b

COVID-19 misinformation has spread rapidly across social media. To counter misinformation, we designed a short, wordless and animated video (called the CoVideo) to deliver scientifically informed and emotionally compelling information about preventive COVID-19 behaviours. After 15 163 online participants were recruited from Germany, Mexico, Spain, the UK and the USA, we offered participants in the attention placebo control (APC) and do-nothing arms the option to watch the CoVideo (without additional compensation) as post-trial access to treatment. The objective of our study was to evaluate participant engagement by quantifying (i) the proportion of participants opting to watch the CoVideo and (ii) the duration of time spent watching the CoVideo. We quantified the CoVideo opt-in and view time by experimental arm, age, gender, educational status, country of residence and COVID-19 prevention knowledge. Overall engagement with the CoVideo was high: 72% of the participants [CI: 71.1%; 73.0%] opted to watch the CoVideo with an average view time of 138.9 out of 144.0 s [CI: 138.4; 139.4], with no statistically significant differences by arm. Older participants (35–59 years) and participants with higher COVID-19 prevention knowledge had higher view times than their counterparts. Spanish participants had the highest opt-in percentage whereas Germans exhibited the shortest view times of the five countries. Short, wordless and animated storytelling videos, optimized for ‘viral spread’ on social media, can enhance global engagement with COVID-19 prevention messages by transcending cultural, language and literary barriers.

The novel coronavirus pandemic constitutes an unprecedented challenge to global health in which epidemic control is highly dependent on individual reactions and the quality of scientific information. Unfortunately, misinformation about COVID-19 has proliferated through various communication channels, including social media (Barua et al., 2020; Cinelli et al., 2020; Pennycook et al., 2020) . According to Li et al. (Li et al., 2020) , 27.5% of the most watched COVID-19 videos on YouTube (totalling 62 million views) contained misleading or inaccurate information. The proliferation of COVID-19 misinformation has been labelled by the World Health Organization (WHO) as an 'infodemic' (Pennycook et al., 2020) . At its worst, COVID-19 misinformation can undermine trust in health authorities, intensify fears and lead to harmful behaviours (OECD, 2020) .

The dissemination of short and animated storybased (SAS) videos is a promising strategy for countering COVID-19 misinformation Vandormael et al., 2020) . SAS videos draw from entertainment-education media, communication theory, and the animated entertainment industry to promote compelling, evidence-based health messages that are optimized for "viral spread" across social media channels (Singhal and Rogers, 1999; Adam et al., 2019; Lutkenhaus et al., 2020; Vandormael et al., 2021a,b) . To counter COVID-19 misinformation, we produced a SAS video (called the CoVideo) that explains the spread of the novel coronavirus and promotes evidence-based information on preventive behaviours such as hand-washing, wearing masks, social distancing and the sanitation of surfaces, among others. Importantly, the CoVideo leverages (i) a soundtrack that was designed to be emotionally compelling, (ii) wordless storytelling that transcends language and literacy barriers and (iii) animated characters that lack distinguishable facial features and are culturally neutral (Guadagno et al., 2013) . The CoVideo was released on Stanford Medicine's YouTube Channel on 21 March 2020 and within 9 months reached 1.9 million views on YouTube, 24 800 views on Instagram, 442 000 views on Facebook, 4 million views on Twitter, and 18 500 views on LinkedIn, with a cumulative count of over 6.3 million . The video has been adopted for public health education purposes by 17 international health authorities, media outlets and private and public organizations in more than 14 countries.

In a recent study, we evaluated if the CoVideo improved knowledge and behavioural intent towards COVID-19 prevention (Vandormael et al., 2021a) . Using a web-based experiment platform, we randomized 15 163 participants from Germany, Mexico, Spain, the UK and the USA to the CoVideo, an attention placebo control (APC) video or a do-nothing arm. We then offered participants who were assigned to the APC video or do-nothing arms in the main trial (n ¼ 9685) the option to watch the CoVideo as post-trial access to treatment (Vandormael et al., 2020) . In this study, we evaluate participant engagement with the CoVideo, as measured by the participants' choice to watch the CoVideo and the duration of time spent watching it. We quantify the differences in participant engagement by experimental arm (APC and do-nothing), age, gender, educational status, country of residence and COVID-19 prevention knowledge. Results from this study could inform the future design and delivery of accurate health information messages to contain the spread of COVID-19 and other infectious disease epidemics.

This is a randomized controlled trial with a post-trial access to treatment stage (Vandormael et al., 2020) . In the main trial, participants were randomly assigned (1:1:1) to the CoVideo arm, an attention placebo control (APC) arm or a do-nothing arm. At the end of the trial, participants randomized to the APC and do-nothing arms were given the chance to voluntarily watch the CoVideo as post-trial access to treatment.

Both trials (main and post) were hosted and run on the Gorilla platform and participants were recruited through Prolific. The sample comprised participants aged 18-59 years (male, female or other), having residence in one of the five countries, and having proficiency in English, German or Spanish. The study and its outcomes were registered at the German Clinical Trials Register (German Clinical Trials Register, 2021) on 12 May 2020: #DRKS00021582. Ethical approval was obtained from the Stanford University IRB on 12 April 2020, #55820.

We provide some basic details of the main trial to give context to our post-trial study (Figure 1 ).

At the beginning of the main trial, participants were asked to answer demographic questions about their age, sex, primary language, country of residence and highest education completed. Participants randomized to the intervention arm watched the CoVideo, which explains how COVID-19 spreads and how people can protect themselves and others with simple actions, such as staying at home, not congregating in public spaces, and sanitizing hands and surfaces (Stanford Medicine, 2020) . The CoVideo is animated with sound effects but does not include any words or text. Participants randomized to the APC arm watched a wordless, animated video about how the daily choices we make affect our lives (Project Better Self, 2018) . Participants randomized to the do-nothing arm did not watch any video. Afterwards, all participants from the three arms answered 18 True/False questions on preventive COVID-19 behaviours. To prevent online searching for answers, participants had a 30 s countdown to respond to each item. Further details of the main trial and its procedures can be found elsewhere (Vandormael et al., 2020) .

For our current study, participants in the APC and do-nothing arms were given a choice to watch the CoVideo (post-trial access to treatment) or end the study without watching the CoVideo. If the participants selected to watch the CoVideo, they were asked on the next page to click the 'Play' button or click the 'Finish' button at any time to end the survey. The participants were informed that they would not be compensated for the additional time taken to watch the CoVideo.

The Gorilla algorithm randomly assigned participants to one of the three arms in the main trial. Neither the study participants nor the investigators were aware of the randomization or allocation mechanism (apart from the investigators specifying a 1:1:1 ratio). The investigators could not use the data to identify the study participants. Participants self-responded to the survey questions and submitted their responses anonymously on the Gorilla platform.

We used two separate measures to assess participant engagement: (i) the decision to view the CoVideo and (ii) the duration of time spent watching the CoVideo. To measure the willingness to watch the CoVideo, we created a dummy variable which equals 1 when the participant clicked on the 'Play' button to watch the CoVideo, otherwise equals 0 if the participant skipped the CoVideo and clicked on 'Finish' the study. Because of this dichotomous response (to watch the CoVideo or not), we used a logistic regression model to quantify the effects of age, gender, country of residence, educational status and COVID-19 knowledge score. The knowledge score was computed by assigning one point to each correct question and no points for missing or incorrect answers. We then computed a total COVID-19-related knowledge score by summing the correct answers ranging from 0 to 18. In order to detect possible differences in engagement between participants in the do-nothing and APC arms, we also included the study arms as controls in our model.

Among the participants who chose to watch the CoVideo, we quantified the length of time that they spent watching the CoVideo. The Gorilla platform has a tool that records a timestamp whenever the participant interacts with Gorilla (e.g. a mouse click). We were therefore able to calculate the time from when the participant clicked the 'Play' button (the first timestamp) to when he/she clicked 'Finish' (the second timestamp). We defined the difference between the two timestamps as 'view time', which is measured in seconds and takes on values between 0 (i.e. the respondent watched zero seconds of the CoVideo) and 144 (i.e. the respondent watched the entire CoVideo). We used five ordinary least squares (OLS) regression models to quantify the associations of the socio-demographic factors with the view time. Model 1 included age only and Models 2-5, respectively, additionally included gender, education completed, country of residence and COVID-19 knowledge score.

We performed all statistical analyses with Stata software version 14.2.

Between 13 May 2020 and 23 June 2020, 15 163 participants were recruited for the main trial. The main trial design is shown in Figure 1 and described in detail elsewhere (Vandormael et al., 2020) . Of the 15 163 participants, 9685 were assigned either to the APC arm or the do-nothing arm. We excluded 343 participants that had missing data. Of the final sample of 9342 participants, 54.1% were female and more than a third (36.5%) of the participants were aged between 25 and 34 years. The majority of the respondents (59.3%) reported the UK as their residence, followed by the USA (25.6%). The most commonly reported primary language among the participants was English (84.9%) and over 81.0% of participants had obtained at least a bachelor's degree. Table 1 shows the summary statistics of the socio-demographic variables by trial arm.

A total of 6731 (72.0%, [CI: 71.1%; 73.0%]) out of 9342 opted to watch the CoVideo. Table 2 reports the logistic regression results for the willingness to watch the CoVideo for Models 1-6. Regardless of the model specification, we did not detect any difference in the willingness to play an animated video about COVID-19 prevention between participants in the do-nothing and APC arms (Table 2 ). In all models, older respondents (ages 25 and older) were more likely to watch the CoVideo. In particular, respondents between 55 and 59 years old were 73.5% more likely to watch the video than younger participants (18-24 years) (p < 0.001, Table 2 , Model 6). After adjusting for our set of controls, respondents having a master's degree or a PhD were 12.2% more likely to start the video than participants who only completed primary school (p ¼ 0.085, Table 2 ). Models 3 and 4 show that men were more likely to watch the CoVideo (Table 2 ). However, after controlling for educational status and the result of the questionnaire about COVID-19 knowledge, this effect was no longer significant (Table 2, Model 6). Figure 2 shows the proportion of participants who chose to start the video by our set of controls.

We also analysed the amount of time participants spent watching the CoVideo (144 s) and the factors associated with this view time. For this analysis, we considered only those participants who chose to watch the video (n ¼ 6731). Among these participants, the view duration was on average 138.9 s out of a total of 144 s [CI: 138.4; 139.4 ]. In Table 3 , we report the OLS estimates for the predictors of the CoVideo view time, which is a continuous variable measured in seconds (min ¼ 0, max ¼ 144 s). We use the same five model specifications as before. We found no significant difference in the view time between the do-nothing (mean-¼ 139.0) and the APC (mean ¼ 138.8) arms (Figure 3 , Table 3 ). In line with the previous findings, older people exhibited longer view times (Table 3) . Specifically, after adjusting for our controls (Model 6), participants aged 35-44 years watched 2.958 s more than younger participants (reference category), participants aged 45-55 watched 3.201 s more, and participants aged 55-59 watched 4.911 s more (p < 0.001, Table 3 , Model 6). After entering the covariates, male participants tended to watch the video almost 3 s less than females (p < 0.001, Table 3 , Model 6). After including controls, educational status was not a strong predictor of the view time. Mexican, Spanish, British, and American respondents watched the video significantly longer than German participants (p < 0.001, Table 3 , Model 6). Results show that participants who were more knowledgeable about preventive COVID-19 behaviours, as measured by our questionnaire, were likely to watch the CoVideo for a longer period of time. A one unit increase in COVID-19 knowledge was associated with a 0.598 increase in CoVideo view time (p ¼ 0.025, Table 3 , Model 6). Figure 3 displays the average view time by our set of controls. 

In this web-based, randomized controlled trial (RCT), we observed high voluntary participant engagement with a short, animated video about COVID-19 prevention. Seventy-two percent of participants voluntarily started and subsequently watched almost the entire CoVideo, with an average view time of 138.9 out of 144 s. Since participants' decision to watch was voluntary, this post-access to treatment stage of our RCT mimicked the real-world scenario in which individuals choose whether or not to engage with health information on social media. Engaging hard-to-reach audiences is a major challenge for health communication specialists. These audiences include marginalized groups who, for a variety of reasons, mistrust medical recommendations and the science underlying it (Jaiswal and Halkitis, 2019) . This mistrust can lead to reduced exposure to critical health messages and, ultimately, disparities in health outcomes. Our findings suggest that the CoVideo was able to capture and sustain participants' attention, emphasizing the potential for this health communication modality to reach and convey information to a diverse audience, spanning five global regions. In our trial, the observed absence of difference in engagement between the APC and do-nothing arms underscores the potential of this approach to overcome the novelty effect seen in some video and gamified learning contexts (Tsay et al., 2020) . One might expect that participants who had recently watched a short, animated video would be less likely to voluntarily engage with a second, but we saw no evidence of this in our study. The equally high engagement of the APC group suggests that target audiences may be willing to watch more than one short animated video in a rapid succession, if the content is compelling enough.

Within our participant pool, we also initially expected that more educated participants would be more likely to watch the CoVideo. Surprisingly, our results showed that educational status was mildly associated with the choice to watch the CoVideo and not at all with length of viewing. Instead, we observed high engagement across the different educational levels, suggesting that compelling narratives, presented in a short, animated video format, could traverse the educational spectrum-engaging people from diverse educational backgrounds through our universal human affinity for effective storytelling (Shen et al., 2015) .

Related to education levels, we also explored the effect of existing COVID-19 knowledge on participants' willingness to watch the CoVideo and found that participants with higher COVID-19 knowledge scores were somewhat more likely to watch the CoVideo when compared with participants with lower knowledge scores. This could be the result of a reciprocal relationship between knowledge and participation in educational activities. By wanting to learn more, people become more knowledgeable, and this knowledge reinforces the motivation to watch more educational content (Murayama et al., 2019) . However, the average view time was consistently high across all participants who chose to watch, including those who exhibited poor knowledge scores. This result suggests that even though the CoVideo might have been more appealing initially for participants with higher COVID-19 knowledge, it was able to sustain engagement, even in those with lower COVID-19 knowledge scores who may have been less inclined to begin watching.

We also examined gender as a predictor of willingness to engage and we detected no significant gender differences in the choice to watch the CoVideo. We did observe that the view time was slightly higher among female participants and this finding aligns with the literature suggesting that women show greater interest in health-related issues and more actively seek out health information (Rice, 2006; Iverson et al., 2008; Beck et al., 2014; Bidmon and Terlutter, 2015; Baumann et al., 2017; Escoffery, 2018; Montesi, 2021) . The similar engagement across genders observed in our trial suggests that the CoVideo managed to partially overcome the documented gender differences in health information-seeking behaviours.

Since the CoVideo was designed for rapid distribution on social media channels, we expected higher participation from younger participants in our study. Surprisingly, we found that older participants (aged 35-59 years) were even more likely than younger participants (aged 18-25 years) to watch the CoVideo and, on average, older participants had longer view times. This result could be explained by a higher perceived vulnerability among older adults who may believe they are more vulnerable to poor health outcomes associated with COVID-19 (Pew Research Center, 2020) . It also underscores the potential for short animated informational videos to reach and engage older audiences, especially if they can be distributed through social media channels, like Facebook, that are more commonly used by older audiences. Given the well-documented differences in how various age groups use the web (Barnhart, 2021) , videos specifically addressing older age groups could be more precisely targeted towards them in ways that will engage slightly older and potentially harder-toreach groups.

Finally, we explored the role of the country of residence on participant engagement. Although overall engagement was high across all countries, we observed significant differences between countries, with participants in the USA being the least likely to watch and those in Spain being the most likely to watch the CoVideo. These differences could be explained by a number of factors, including different governmental responses towards COVID-19, which research suggests may have affected public engagement with COVID-19 prevention measures (Chanley et al., 2000; OECD, 2017; Lau et al., 2020) . When examining length of view times, we found that German residents exhibited shorter but still considerably high view time when compared with British, USA, Spanish and Mexican residents. The differential engagement between countries observed in our study might partially mirror the public's trust in their government's handling of COVID-19 over the course of the pandemic.

A limitation of our study was that the majority of participants reported speaking English as their first language. This meant that our participant population was not as representative of a truly global target audience as we would like to achieve in future studies. The demographics of participants in our trail reflect those of the platform on which the trial was delivered. However, the decision to use an integrated, fully web-based platform, Gorilla, to both deliver the intervention and collect data, was both a necessity (given the challenges of face-to-face data collection during a pandemic) as well as a relative innovation in study design. The high engagement observed across the sub-groups of participants in our study yields findings that can help to inform the future development of this emerging genre of health media Vandormael et al., 2021b) . A further strength of our study is the use of an RCT design to evaluate participant engagement with the CoVideo intervention. Gorilla also enabled us to rigorously track each step of the experiment, including the time spent by each participant on each page.

In conclusion, our findings demonstrate high engagement with the CoVideo, across different audiences. We observed that the CoVideo achieved higher engagement among participants who already had knowledge on COVID-19 preventive behaviours but still managed to reach those who exhibited poor knowledge scores. We found high participant engagement across education subgroups and we detected only mild gender differences in engagement. These findings suggest that the CoVideodesigned to be inclusive and accessible-managed to reach different audiences. Age turned out to be an important factor to consider when designing informational videos. Future videos could be customized for different categories to optimize engagement. Across countries, we also observed differences in willingness to view the CoVideo. These findings could be traced back to the level of public trust in the government but should be validated by future studies. Taken together, the evidence of this study suggests that animated, wordless health media, designed to be emotionally arousing and globally accessible via social media, can be an effective method of delivering critical health messages.

C.F. and A.V. wrote the paper. C.F., A.V. and V.H. undertook the statistical analysis. M.A. designed, produced and created the CoVideo. T.B., A.V. and M.A. designed the trial. A.V., T.B., M.A. and M.G. contributed to the questionnaire development. All authors provided comments and feedback.

T.B. is funded by the Alexander von Humboldt University Professor Prize.

The funder of the study had no role in study design, data collection, data analysis, data interpretation or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

The data that support the findings of this study are available from the corresponding author upon request.

Design for extreme scalability: a wordless, globally scalable COVID-19 prevention animation for rapid public health communication

Human-centered design of video-based health education: an iterative, collaborative, community-based approach

Social media demographics to inform your brand's strategy in 2021

Effects of misinformation on COVID-19 individual responses and recommendations for resilience of disastrous consequences of misinformation

Gender-specific determinants and patterns of online health information seeking: results from a representative German health survey

Use of the Internet as a health information resource among French young adults: results from a nationally representative survey

Gender differences in searching for health information on the Internet and the virtual patient-physician relationship in Germany: exploratory results on how men and women differ and why

The origins and consequences of public trust in government

The COVID-19 social media infodemic

Gender similarities and differences for e-health behaviors among U.S. adults

German Clinical Trials Register

What makes a video go viral? An analysis of emotional contagion and Internet memes

Impact of Internet use on health-related behaviors and the patient-physician relationship: a survey-based study and review

Towards a more inclusive and dynamic understanding of medical mistrust informed by science

COVID-19 in humanitarian settings and lessons learned from past epidemics

Toward spreadable entertainment-education: leveraging social influence in online networks

Gender differences in information behavior during the Covid-19 health crisis in Spain

Process account of curiosity and interest: a reward-learning perspective

Trust and public policy

view/?ref¼135_135220-cvba4lq3ru& title¼Transparency-communication-and-trust-The-role-ofpublic-communication-in-responding-to-the-wave-of-disin formation-about-the-new-coronavirus

2020) Fighting COVID-19 misinformation on social media: experimental evidence for a scalable accuracy-nudge intervention

Worries about Coronavirus surge, as most Americans expect a recession -or worse

The choice (short animated movie

Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys

Impact of narratives on persuasion in health communication: a meta-analysis

Entertainment-Education: A Communication Strategy for Social Change

Global COVID-19 prevention

Overcoming the novelty effect in online gamified learning systems: an empirical evaluation of student engagement and performance

An entertainment-education approach to prevent COVID-19 spread: study protocol for a multi-site randomized controlled trial

The effect of a wordless, animated, social media video intervention on COVID-19 prevention: online randomized controlled trial

Effect of a story-based, animated video to reduce added sugar consumption: a web-based randomized controlled trial