key: cord-0943281-d10t23lw authors: Chen, Simiao; Forster, Sebastian; Yang, Juntao; Yu, Fengyun; Jiao, Lirui; Gates, Jennifer; Wang, Zhuoran; Liu, Haitao; Chen, Qiushi; Geldsetzer, Pascal; Wu, Peixin; Wang, Chen; McMahon, Shannon; Bärnighausen, Till; Adam, Maya title: Animated, video entertainment-education to improve vaccine confidence globally during the COVID-19 pandemic: an online randomized controlled experiment with 24,000 participants date: 2022-02-19 journal: Trials DOI: 10.1186/s13063-022-06067-5 sha: 9703ea8801e05c5d0ccc3b6848c9409b18c433b5 doc_id: 943281 cord_uid: d10t23lw BACKGROUND: Science-driven storytelling and entertainment-education (E-E) media demonstrate potential for promoting improved attitudes and behavioral intent towards health-related practices. Months after the outbreak of coronavirus disease 2019 (COVID-19), emerging research highlights the essential role of interventions to improve public confidence in the COVID-19 vaccine. To improve vaccine confidence, we designed three short, animated videos employing three research-informed pedagogical strategies. These can be distributed globally through social media platforms, because of their wordless and culturally accessible design. However, the effectiveness of short, animated storytelling videos, deploying various pedagogic strategies, needs to be explored across different global regions. METHODS/DESIGN: The present study is a multi-site, parallel group, randomized controlled trial (RCT) comparing the effectiveness of (i) a storytelling-instructional-humor approach, (ii) a storytelling-analogy approach, (iii) a storytelling-emotion-focused approach, and (iv) no video. For our primary outcomes, we will measure vaccine hesitancy, and for secondary outcomes, we will measure behavioral intent to seek vaccination and hope. Using online platforms, we will recruit 12,000 participants (aged 18–59 years) from the USA and China, respectively, yielding a total sample size of 24,000. DISCUSSION: This trial uses innovative online technology, reliable randomization algorithms, validated survey instruments, and list experiments to establish the effectiveness of three short, animated videos employing various research-informed pedagogical strategies. Results will be used to scientifically support the broader distribution of these short, animated video as well as informing the design of future videos for rapid, global public health communication. TRIAL REGISTRATION: German Clinical Trials Register DRKS #00023650. Date of registration: 2021/02/09. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13063-022-06067-5. Discussion: This trial uses innovative online technology, reliable randomization algorithms, validated survey instruments, and list experiments to establish the effectiveness of three short, animated videos employing various research-informed pedagogical strategies. Results will be used to scientifically support the broader distribution of these short, animated video as well as informing the design of future videos for rapid, global public health communication. The funders will have no role in the collection, management, analysis, and interpretation of data; writing of the report or submission decisions. In the midst of the COVID-19 pandemic, research highlights the critical role of interventions to increase vaccine confidence, even before vaccines for COVID-19 become universally available [1, 2] . The WHO has dubbed vaccine hesitancy one of the greatest threats to global health [3] , yet we are lacking interventions that effectively promote vaccine confidence in ways that are accessible and globally scalable. Many existing interventions have focused on providing information in the form of scientific data, using a didactic approach. This strategy has proven ineffective for key target audiences [4] . Designing effective interventions for a broad range of target audiences has become even more important during the COVID-19 pandemic as misinformation has spread rapidly around the world. Negative claims about vaccines often appeal to the emotions of the target audiences, eliciting vaccine doubt and hesitancy. This observation-that emotion-focused messages resonate and affect health behaviors [4, 5] -can also potentially be used to the advantage of health communicators. Research suggests that activating positive emotions, like hope and altruism, can actually bolster vaccine education interventions [5] . Other promising pedagogical strategies, including the use of instructional humor [6] , analogies [7, 8] , and storytelling, could also be leveraged to promote vaccine confidence, and researchers have advocated for a transdisciplinary approach to successful health communication on vaccines [4] . By integrating different fields of expertise, including those outside of academialike entertainment and marketing-we may be able to design more effective vaccine promotion interventions. Animated E-E videos, developed using transdisciplinary approaches, could be an especially effective method for distributing evidence-based health messages globally through social media platforms [9, 10] . To improve vaccine confidence, we designed three videos, employing three research-informed pedagogical strategies: (i) a storytelling-instructional-humor approach (ii) a storytelling-analogy approach (iii) a storytelling-emotion-focused approach. The first prototype video was released on Stanford Medicine's YouTube channel (https://youtu.be/Ut_6 GInouYg) on October 19th, 2020, and was viewed 27,290 times within the first 3 weeks. This video belongs to a collection of science-driven, storytelling COVID-19 animations that have already reached several million viewers globally. Because all of the videos contain no spoken words, these interventions can be rapidly distributed to global audiences without translation. Video interventions could play an important role in broadly disseminating health information during the COVID-19 pandemic, but there is a need to evaluate the efficacy of such interventions for improving vaccine confidence. A secondary aim of the intervention videos is to convey hope, a measurable parameter that research suggests is related to improved health, psychosocial, and academic outcomes [11] . Recent research even suggests that leveraging positive emotions, including hope, may be leveraged as part of COVID-19 vaccine education interventions [5] . Hope has been defined as the perceived capacity to build pathways towards our goals and motivate ourselves to use those pathways. Hope theory proposed by Snyder [11] has been likened to the theories of optimism, self-efficacy, and self-esteem. Especially during the COVID-19 pandemic, a period characterized by widespread emotional distress [12] , bolstering hope could have meaningful positive effects on the mental health of the global public. The aim of this study is to systematically evaluate the efficacy of animated, short video interventions, with different pedagogical approaches, for (a) reducing vaccine hesitancy, (b) increasing behavioral intent to get vaccinated, and (c) increasing hope. Here, we propose an online experiment in which the video intervention and a survey will be randomly ordered and assigned to 24,000 participants between the age of 18 and 59, living in China or the USA. These two countries represent a considerable population in the world, which are culturally different from the other and have also managed the pandemic in significantly different ways [13] . Results will be used to scientifically support the ongoing distribution of these interventions as well as optimizing the design of future animated, E-E videos for public health communication. Our study aims to achieve the following objectives. To: This trial will be conducted online, using the SPIRIT reporting guidelines [14] . For the United States, we will use the research platform created and managed by Prolific Academic Ltd (ProA: https://www.prolific.co/) to recruit participants and an online web platform Gorilla (www.gorilla.sc) to host and deploy our study; for China, we will use Kurundata, which recruits members in a variety of ways, including through its own platform (https://www.kurundata.com/), partnerships with other websites, and encouraging registered members to recruit new members through the popular mobile application WeChat (Appendix S1). Our study is a multi-site, parallel group, randomized controlled trial (RCT) comparing the effectiveness of each of the short intervention videos with each other and with no video (the control condition). Via the online research platforms, participants will be randomly assigned to four intervention arms (Video A: storytelling-instructional-humor approach, Video B: storytelling-analogy approach, Video C: storytellingemotion-driven approach) and a control arm (no video). Participants will be randomized at a 1:1:1:1 ratio (Fig. 1) . In each trial arm, there is a questionnaire survey, arranged in the following order. Intervention arm a: participants will receive the storytelling-instructional-humor video, followed by the survey. Intervention arm b: participants will receive the storytelling-analogy video, followed by the survey. Intervention arm c: participants will receive the storytelling-emotion-focused video, followed by the survey. Control arm: participants will first receive the survey. After survey responses are submitted, participants in the control arm will be given access to the video interventions arranged in a single loop (to ensure post-trial access to treatment). To be eligible, participants must be between the age of 18 and 59, living in China or the USA. Since this video is wordless and culturally inclusive, therefore accessible to participants of all language and cultural groups. The English questionnaire will be translated into Chinese, so participants from the USA must have reading competency in English, and participants from China must have reading competency in Chinese to complete the trial questionnaire. Those who do not sign the consent form or drop out will be excluded from the data analysis. Ethics approval was obtained from the Stanford University IRB on January 12th, 2021, protocol #59503, and from the Chinese Academy of Medical Sciences & Peking Union Medical College IRB on March 10th, 2021. Who will obtain informed consent? Participants must preview an information and consent form before they can begin the survey. The form explains the purpose of the study, the risks and benefits of the research, and how to contact the study investigators (or the Stanford University ethics review board). By clicking the link, participants consent to participate in our study and will be redirected to the Gorilla or Kurundata platform, where additional information is given. Participants can exercise their freedom to participate (or decline participation) at recruitment or at any point during the study. Since this is a minimal risk study of an online educational video intervention, we do not anticipate needing to discontinue or modify the allocated interventions during the course of the study. Participants can withdraw from the study at any time and participants will not be compensated for incomplete surveys. Those participants who choose not to sign the consent form or fail to complete the experiment in its entirety will be excluded from the data analysis. The video interventions are a short (2-3-min) animated E-E videos about vaccines. All videos use a storytelling approach, but each video differs slightly in its pedagogical approach. Intervention arm a will view a video that uses a storytelling-instructional-humor approach. Intervention arm b will view a video that uses a storytelling-analogy approach and intervention arm c will view a video that uses a storytelling-emotion-focused approach. Explanation for the choice of comparators The comparators are similar-length E-E videos, all animated in the same styles by the same animator. They all convey the same message (i.e., vaccines work) using a storytelling approach, without the use of spoken or written language. Each intervention arm video uses a slightly different pedagogical approach. The control arm receives no video intervention. Comparing the intervention arms a, b and c with the control arm will allow us to quantify the effect of each intervention on the primary and secondary outcomes. Comparing intervention arms a, b and c with each other will allow us to explore the differential effect of these pedagogical approaches on the primary and secondary outcomes. We will use list experiments, also referred to as the unmatched count technique [15] , to eliminate social desirability bias regarding vaccine-seeking behavioral intent. For the list experiments, we will use the control list as the comparator. Our primary outcome is vaccine hesitancy (including COVID-19 vaccine hesitancy). We will ask participants how much they agree or disagree with statements related to perceived vaccine safety as well as their attitudes, preferences, beliefs, and hesitancies regarding regular vaccines and the COVID-19 vaccine, respectively [16] . In the statistical analysis process, we will use nine questions to measure the degree of vaccine hesitancy in general and seven questions to measure the COVID-19 vaccine hesitancy specifically. Then we will normalize the final score to a range of 1-5 for both measures for comparison, which higher scores indicate higher degrees of vaccine hesitancy. These questions are shown in Table 1 and the data elicited from this survey will enable us to achieve objective 1. Our secondary outcomes include participants' behavioral intent to get vaccinated as well as participants' level of hope. In order to reduce the social desirability bias often associated with direct questioning about sensitive items [15] , we will use the unmatched count or list randomization approach to devise a series of list experiments [17] . Four list experiments will be used, as shown in Table 2 . For each experiment, the control group will receive a list of three items. Participants are asked how many items they are likely to do in the coming months without stating which ones they chose. The treatment group will get the identical item lists but with one additional "sensitive" item relating to their behavioral intent to get vaccinated or encourage their loved ones to do so. For example, imagining that the control group select 1 out of the 3 items on average while the treatment group select 1.3 out of the 4 items, with the assumption that the average acceptance of these two cohorts is the same, we can conclude that the prevalence of participants who would get vaccinated against COVID-19, is 20%. We have designed the list of experiments in accordance with published best practices [17] , and these data will be used to assess objective 2. Second, we will assess participants' level of hope using the Adult Hope Scale [18] , a 12-item scale measuring participants' level of hope ( Table 3 ). The Adult Hope Scale is comprised of two subscales relating to Snyder's cognitive model of hope: (1) Agency (goal-directed energy) and (2) Pathways (planning to accomplish goals) [11] . The 12 items include 4 Agency items, 4 Pathways items, and 4 distractors. Participants are asked to respond using an 8-point Likert-type scale ranging from Definitely False to Definitely True. As recommended in the literature, we will refer to the scale as "The Future Scale" within the survey experiment. We also aim to measure the difference of the outcomes above between the three intervention arms a (storytelling-instructional-humor), b (storytellinganalogy), and c (storytelling emotion-focused). These data will be used to assess objective 3. L14. It is better to develop immunity by getting sick than to get a shot *Note: the survey tool was designed based on [16] , which requires the level of agreement from "strongly disagree" to "strongly agree" To calculate the sample size needed for pairwise comparison between three groups, we used a one-way analysis of variance (ANOVA). The formula is as follows. where κ, the matching ratio, is equal to 1 in our study; μ A and μ B are the means of group A and group B; σ A and σ B are the standard deviations; α = 0.05 and β = 0.2 are the type-I and type-II error, respectively; Z is the quantile function; and τ = 2 is the number of comparisons to be made. For vaccine hesitancy (and COVID-19 vaccine hesitancy), we assume that the means of two arms are 2.00 and 2.01, which represents the level of vaccine acceptance of this arm (1 means total acceptance and 5 means complete refusal), the standard deviations are the same, 0.10. To detect a difference of 0.01 between the vaccine hesitancy between arms, we can calculate the total minimal sample size is n A = n B = n C = n D = 1570, so the N = 6280. To test the sensitivity of the result, we can change our assumption and increase the standard deviation to 0.12, i.e., σ A = σ B = 0.12. Then we have n A = n B = n C = n D = 2260 and N = 9040. To achieve a higher level of accuracy, we will recruit 12,000 participants in each country for this study. We will recruit participants from the ProA and Kurundata platforms. Study participants will be compensated $1.30 for completing the survey at ProA and 5 RMB at Kurundata. ProA or Kurundata will make the reward payment to study participants. To complete Table 2 List experiments in the trial Note: In each trial arm, both groups will receive four lists. For each list, the control group will get the first three items only; the treatment group will receive the three items and the fourth sensitive item, indicated by an asterisk (*). Each list experiment will be preceded by the question: "In the coming 3 months, how many of the following things are you likely to do? I do not need to know which of these things you are likely to do, just how many." I12. I meet the goals that I set for myself. Note: Items 2, 9, 10, and 12 make up the agency subscale. Items 1, 4, 6, and 8 make up the pathway subscale. Researchers can either examine results at the subscale level or combine the two subscales to create a total hope score the survey, a potential study participant must open an account on ProA or Kurundata and provide his or her personal information. Participants must agree to ProA's or Kurundata's data privacy terms and conditions. ProA or Kurundata will assign each participant a unique, anonymized ID. Because ProA and Kurundata handle the interaction between the study investigators and participants, the participants will be anonymous to the study investigators. ProA and Kurundata are both research platforms with access to large numbers of study participants, enabling us to achieve the desired sample size for this study. All participants who give consent for participation and who fulfill the inclusion criteria will be randomized. After they complete the consent form, Gorilla and Kurundata will use a web-based randomization algorithm to randomly allocate participants to the intervention arm a, intervention arm b, intervention arm c, or control arm (sequence generation) at a 1:1:1:1 ratio. The randomization algorithm involves a web-based algorithm using random permutation, which is independent from the researchers who will recruit patients and implement the protocol. Once an arm of participants is fully recruited, subsequent participants will be randomly assigned to the remaining groups until all arms are fully recruited. The study investigators and those involved in the data analyses and statistics will be blinded to the group allocation. Plans for assessment and collection of outcomes The study investigators will be responsible for data collection. Data will be collected on either the Gorilla or Kurundata platforms. The data retrieved from the platforms will be anonymous. Data downloaded will be stored on an encrypted and secure server. The data will be deleted two years after the study has been completed. Participants will be automatically timed out from the online platform if they take more than 45 min to complete the study. Though participants can withdraw the study at any time, they will not be compensated for incomplete survey participation. Incomplete data will be excluded from our analyses. Since the participants are anonymous to us, there is no way to initiate follow-up in the time limit. Data will be collected on the Gorilla or Kurundata platforms. Third parties except for ProA or Kurundata will not have access to the data. The data will be downloaded and safely stored for statistical analysis on a computing system maintained by the University of Heidelberg in Germany. Because of the anonymized participant IDs, the study investigators can never meet or know the identity of the study participants. The study investigators will only have access to the participants' anonymized ID and no other personal or confidential information, and the data will be deleted 2 years after the study has been completed. The study investigators will keep this information confidential. We will use descriptive statistics to obtain summaries of the demographic data (age, sex, education status, country of residence, etc.). For each participant, we will calculate their (COVID- 19) vaccine hesitancy score based on their survey responses. Let K k denote the mean vaccine hesitancy for each trial arm, where k ∈ {a, b, c} such that a represents the intervention arm a (video with storytelling-instructionalhumor approach), b represents the intervention arm b (video with storytelling-analogy approach), c represents the intervention arm c (video with storytelling-emotionfocused approach), and d represents the control arm. To evaluate the effectiveness of the video, we will use ANOVA for continuous endpoints and χ 2 test for binary endpoints to assess the statistical significance of the difference between the control arm and the treatment arms. To investigate the factors that affect the endpoints, we will use generalized linear mixed models (GLMM) for the analysis of our endpoints. We use ordinary least squares (OLS) regression for our continuous endpoints; we will use (modified) Poisson regression for our binary endpoints; we will use negative binomial regression for our count endpoints. The reason for the choice of modified Poisson regression for our binary endpoints is that this analysis has good statistical properties and generates risk ratios, which are far easier and safer to interpret than the effect size measures generated by alternative methods (such as odds ratios or marginal effects) [19] [20] [21] [22] . For the vaccination plan in the list experiment, we will calculate the mean score for the control list and treatment list, denoted by C i and T i respectively, where i is the index of the list. Then we can calculate the mean difference between the control list and treatment list within each trial arm, which is considered as the participants' behavioral intent to get vaccinated in this trial arm, denoted as D ik ¼ T ik −C ik . Analogous to difference-in-difference analyses, we can identify the effect of each treatment. For other secondary outcomes, we can use the same statistical procedure described above to assess the effectiveness of our E-E video. We will use R statistical software to undertake the analysis. No interim analyses are planned. Methods for additional analyses (e.g., subgroup analyses) Since we will conduct the trial in the USA and China, we will conduct both country-specific and cross-country pooled analyses for further comparison. Methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data Participants who decide to withdraw from the trial will not be replaced Plans to give access to the full protocol, participant level data, and statistical code This document is the full protocol. Anyone interested in other data or documentation should contact the corresponding author. Oversight and monitoring Composition of the coordinating center and trial steering committee The trial will be overseen by a trial steering committee (TSC). The TSC will have an independent chairperson and members but also includes the trial collaborators. Two TSC meetings are planned. As this is an online survey, there is very minimal risk for study-related injury or harm. We will present nonthreatening video interventions and questions about vaccine attitudes. There are no foreseeable risks to participating in the online study. No ancillary and post-trial care is planned. The results of this study will be disseminated through presentations at international conferences and publications in peer-reviewed journals. Results will be used by the study collaborators and their institutions (Stanford School of Medicine, Heidelberg University, Chinese Academy of Medical Sciences, and Peking Union Medical College) to improve the design and universal appeal of future educational and health promotion videos. All investigators who meet authorship criteria will be included as co-authors and anyone who contributed, but does not meet the criteria for authorship, will be acknowledged. No professional author services will be used. Global government and health authorities have actively engaged in efforts to combat the COVID-19 pandemic, including imposing a variety of strict lockdown policies on regions and countries. Also, since the World Health Organization (WHO) is currently orchestrating the global campaign to fight against the spread of the SARS-CoV-2 virus, the progress of vaccination has dramatically accelerated and it is foreseeable that numerous safe and effective vaccines will come to market by 2022 [23] . However, previous research has indicated that vaccine availability does not guarantee vaccine acceptance, especially given global variations in cultural and educational backgrounds [24] . In this study, we propose using animated, E-E videos to decrease general vaccine hesitancy globally and to evaluate their effectiveness using three innovative approaches. First, we will use the latest online technology to conduct our multi-site, parallel, randomized controlled trial. ProA, the online platform, enables researchers to connect with individuals around the world who are interested in participating in online studies [25] . Kurundata fills a similar function in China. Second, we will host and deploy our study on the Gorilla platform, which is an experiment builder that provides users with the tools for undertaking online behavioral research. The Gorilla randomization algorithm, which demonstrates the platform's capability to implement innovative trial designs, will guide us in randomly assigning participants at two levels: (1) . Participants will be randomized to the storytelling-instructional-humor video arm, the storytelling-analogy video arm, the storytelling-emotionfocused video arm or no video arm (2) . Within each arm, participants will be randomized to control list or treatment list within the list experiment portion of the survey. Third, we will deploy list experiments to reduce the social desirability bias associated with sensitive questions such as the intention to seek vaccination. Prior research highlights the documented discrepancies between publicly declared vaccine intentions and privately held reservations, including safety and efficacy concerns [24] . Therefore, we have designed the list experiment to minimize social desirability bias toward the behavioral intention questions. This study is expected to establish the effectiveness of short, animated, E-E videos, using different pedagogical approaches, for improving vaccine confidence. We hope to document a reduction in vaccine hesitancy and guide future E-E video development strategies to support education and health communication campaigns globally. This protocol was submitted on March 15th, 2021, and resubmitted on Jan 5, 2022. Recruitment for the trial and all data collection will have been completed by February 2022. Mapping global trends in vaccine confidence and investigating barriers to vaccine uptake: a large-scale retrospective temporal modelling study It is time to get serious about vaccine confidence World Health Organization. Ten threats to global health in 2019 How Storytelling Can Combat Vaccine Hesitancy: a Transdisciplinary Approach Considering Emotion in COVID-19 Vaccine Communication: Addressing Vaccine Hesitancy and Fostering Vaccine Confidence An explanation of the relationship between instructor humor and student learning: Instructional humor processing theory Health Education through Analogies: Preparation of a Community for Clinical Trials of a Vaccine against Hookworm in an Endemic Area of Brazil The Place of Analogies in Science Education Design for extreme scalability: A wordless, globally scalable COVID-19 prevention animation for rapid public health communication An entertainmenteducation approach to prevent COVID-19 spread: study protocol for a multi-site randomized controlled trial Hope theory: Rainbows in the mind Mental Health and the Covid-19 Pandemic The impact of the COVID-19 pandemic on physical and mental health in the two largest economies in the world: a comparison between the United States and SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials Determinants of social desirability bias in sensitive surveys: a literature review Measuring vaccine hesitancy: The development of a survey tool What Can We Learn with Statistical Truth Serum? The will and the ways: development and validation of an individualdifferences measure of hope A modified poisson regression approach to prospective studies with binary data Extension of the modified Poisson regression model to prospective studies with correlated binary data Direct provision versus facility collection of HIV self-tests among female sex workers in Uganda: a cluster-randomized controlled health systems trial The impact of continuous quality improvement on coverage of antenatal HIV care tests in rural South Africa: Results of a stepped-wedge clusterrandomised controlled implementation trial The race for coronavirus vaccines: a graphical guide Vaccine hesitancy: the next challenge in the fight against COVID-19 Prolific Academic Research Platform Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Authors' contributions SC, MA, TB, and SF conceived the trial. SC, MA, SF, and FY wrote the first draft of the manuscript. SC, MA, JG, SF, and TB designed the survey. SC and FY conducted the quantitative data analysis. SC, JY, CW, and TB obtained funding. All authors provided critical revisions to the manuscript. This study is funded by the Alexander von Humboldt Foundation and the Sino-German Center for Research Promotion (Project C-0048), which is funded by the German Research Foundation (DFG) and the National Natural Science Foundation of China (NSFC). This study is also funded by an Alexander von Humboldt University Professor Prize awarded to Dr. Till Bärnighausen. Open Access funding enabled and organized by Projekt DEAL.Availability of data and materials Data will be collected and stored on the Gorilla and Kurundata platform. The study investigators own and have complete control of the research data, which can be accessed at any time. For statistical analysis, the data will be downloaded and safely stored on a computing system maintained by the University of Heidelberg. Ethics approval and consent to participate Ethics approval was obtained from Stanford University in the USA (ID#59503) and from the Chinese Academy of Medical Sciences and Peking Union Medical College in China (ID#0622021). Participants must preview an information and consent form before they can begin the survey. The form explains the purpose of the study, the risks and benefits of the research, and how to contact the study investigators (or Stanford University ethics review board). By clicking the link, participants consent to participate in our study and will be redirected to the Gorilla or Kurundata platform, where additional information is given. Participants can exercise their freedom to participate (or not) at recruitment or at any point during the study. Protocol amendments will be promptly communicated with the relevant ethics committees and the TSC. The authors declare that they have no competing interests. The online version contains supplementary material available at https://doi. org/10.1186/s13063-022-06067-5.Additional file 1:. Appendix S1