key: cord-0726021-zn6kfzmx authors: Shah, Harita S; Dolwick Grieb, Suzanne; Flores-Miller, Alejandra; Phillips, Katherine H; Page, Kathleen R; Cervantes, Ana; Yang, Cui title: A Crowdsourcing Open Contest to Design a Latino-Specific COVID-19 Campaign: Mixed Methods Analysis date: 2022-05-12 journal: JMIR Form Res DOI: 10.2196/35764 sha: 9ae6ab41235c0f27cbdd27497dc17205d586364e doc_id: 726021 cord_uid: zn6kfzmx BACKGROUND: Latino communities are among the most heavily impacted populations by the COVID-19 pandemic in the United States due to intersectional barriers to care. Crowdsourcing open contests can be an effective means of community engagement but have not been well studied in Latino populations nor in addressing the COVID-19 pandemic. OBJECTIVE: The aims of this study are to (1) implement and evaluate a crowdsourcing open contest to solicit a name for a COVID-19 social marketing campaign for Latino populations in Maryland and (2) conduct a thematic analysis of submitted entries to guide campaign messaging. METHODS: To assess the level of community engagement in this crowdsourcing open contest, we used descriptive statistics to analyze data on entries, votes, and demographic characteristics of participants. The submitted text was analyzed through inductive thematic analysis. RESULTS: We received 74 entries within a 2-week period. The top 10 entries were chosen by community judges and the winner was decided by popular vote. We received 383 votes within 1 week. The most common themes were collective efficacy, self-efficacy, and perceived benefits of COVID-19 testing. We used these themes to directly inform our social marketing intervention and found that advertisements based on these themes became the highest performing. CONCLUSIONS: Crowdsourcing open contests are an effective means of community engagement and an agile tool for guiding interventions to address COVID-19, including in populations impacted by health care disparities, such as Latino communities. The thematic analysis of contest entries can be a valuable strategy to inform the development of social marketing campaign materials. Latino communities are among the most heavily impacted populations by the COVID-19 pandemic in the United States, with 1.9 times the infection rate of non-Hispanic White individuals and 2.3 times the age-adjusted mortality rate [1] . Latino individuals account for 18% of the US population but 27.1% of all COVID-19 cases. Among individuals 18-50 years of age, the disparity widens, with Latino adults comprising 42% of COVID-19 cases and 40% of COVID-19 deaths, with mortality rates up to 8 times higher than among non-Hispanic White individuals [2] . Certain subgroups within the diverse Latino populations have been shown to be disproportionately impacted by COVID-19. Latino immigrant populations have experienced a disproportionate burden of COVID-19 infections given occupational exposures, higher household occupancy, and barriers to accessing health care (eg, lack of insurance, fear of deportation, and limited English proficiency) [3] [4] [5] . Similar to national trends, the impact of systemic inequities has been painfully evident in our community in Baltimore, Maryland. Baltimore is an emerging immigrant-receiving city that has seen a marked influx of Latino immigrants from Central America over the last 2 decades [6] . Data from Johns Hopkins Health Systems showed that as the pandemic unfolded between March 11, 2020 , to May 25, 2020, the overall SARS-CoV-2 positivity rate was 16.3%, but the rate was significantly higher for Latino patients (42.6%) compared to Black (17.6%) and White (8.8%) patients [7] . As the pandemic unfolded, our team partnered with community-based organizations (CBOs) to develop and implement a comprehensive Latino COVID-19 response strategy that addressed systemic barriers to care for Latino populations. The team focused on expanding access to free COVID-19 testing and, later, vaccination through trusted community sites, a bilingual hotline, and a team of bilingual and bicultural peer navigators (or promotoras). Although these components were positively received and used by community members, positivity rates remained disproportionately high among Latino populations in Baltimore [8] . Given this disparity and the need to dynamically respond to the evolving COVID-19 pandemic, our team developed and implemented a social marketing intervention as part of the Rapid Acceleration of Diagnostics for Underserved Populations (RADx-UP) initiative [9] . The intervention aim was to use crowdsourcing and a web-based platform to develop culturally congruent messaging to facilitate access to testing as well as expedite linkage to care services and/or promotora support. Crowdsourcing is an effective means of community engagement, based on the principles of collective wisdom and open access [10, 11] . Crowdsourcing is defined as "an approach to problem solving which involves an organization having a large group attempt to solve a problem or part of a problem, then sharing solutions" [12] . Open contests are a form of crowdsourcing that allow community members to provide solutions to a problem through a contest; the solutions are then consolidated into a unified product [13, 14] . Open contests incorporate local knowledge and culture by directly involving a large number of community members in developing and vetting ideas [15] . The crowdsourcing approach avoids the less effective alternative of top-down interventions and can incorporate community-based participatory research (CBPR) principles by involving community stakeholders throughout the process [16] . In fact, the impetus for this crowdsourcing open contest arose when the name originally proposed in the research grant, Juntos ("Together"), was poorly received by community health workers (CHWs) on our team. The CHWs found it reminiscent of political propaganda and military juntas; if CHWs did not have such integral roles within our team, we may not have received this important feedback. Thus, we sought community input through this crowdsourcing open contest to develop more representative messaging for the COVID-19 social marketing campaign. Crowdsourcing methods have been shown to be feasible among Latino populations regardless of English-or Spanish-language preference and thus are suitable to guide our approach to the COVID-19 pandemic [17] . Latino individuals have been shown to have high rates of internet and smartphone use, making web-based outreach a reasonable approach; in 2016, 84% of Latino respondents reported internet and social media use and this use has increased during the COVID-19 pandemic [18, 19] . To our knowledge, no studies have been published evaluating crowdsourcing open contests for Latino audiences. The ability to crowdsource virtually makes it an apt tool for efforts to combat COVID-19 given social distancing practices. However, few studies have been published detailing the use or results of crowdsourcing for COVID-19-focused interventions. To understand and incorporate community members' attitudes, beliefs, and norms toward the COVID-19 pandemic, we implemented a crowdsourcing open contest to solicit the central messaging for a social marketing campaign promoting equitable COVID-19 testing for Latino populations in Maryland. In this paper, we describe the process and outcomes of a crowdsourcing open contest and thematic analysis of contest entries, which we subsequently used to guide campaign messaging. Prior studies have used the thematic analysis of contest entries to understand community attitudes [20] [21] [22] [23] . To our knowledge, this is the first study to use the thematic analysis of contest entries to directly inform intervention development. In November 2020, our team implemented a crowdsourcing open contest to name a COVID-19 social marketing campaign for Latino populations in Maryland. The focus of the contest was on COVID-19 testing and general prevention given that vaccines were not publicly available at that time. The contest was guided by CBPR principles in that our team consisted of diverse stakeholders involved in each aspect of the project (eg, CHWs, physicians, researchers, and designers) in addition to the community advisory board (CAB). Participants were invited to submit a name, hashtag, and an optional call to action (1-2 sentences) for what was referred to as the COVID-19 public health campaign in Maryland. The announcements were written primarily in Spanish with an English translation provided as the target audience for the campaign was Spanish-speaking Latino individuals who may not be reached by city-wide public health campaigns with English messaging (Figure 1 ). The implementation and evaluation of the open contest consisted of the following steps: (1) organize a CAB, (2) establish contest rules and incentives, (3) solicit crowd input via community partner social media profiles, (4) evaluate contest entries using both community judge assessments and popular voting, (5) disseminate contest results, and (6) use thematic analysis to understand community values and perspectives so that they may be emphasized in the campaign [20] [21] [22] [23] [24] [25] . A CAB was established to represent various CBOs, religious organizations, advocacy groups, and government sectors (ie, the Mayor's Office on Immigrant Affairs) within Baltimore's Latino community. Members of the CAB were involved in every stage of this study and the subsequent social marketing intervention. CAB members from nongovernment organizations were compensated with US $50 per meeting. The CAB met monthly to provide guidance on how to promote the open contest to relevant communities, recruit contest participants, translate the winning entry into social marketing messages, and implement the social marketing campaign. The contest was open for entries for 2 weeks (November 13, 2020-November 26, 2020), using a brief timeline given the urgency of the COVID-19 pandemic. Participants submitted a name, hashtag, and an optional call to action (1-2 sentences) via a brief Qualtrics form. Participants were given the option to enter a raffle for 1 of 5 US $25 Visa gift cards by including an email address or phone number. The top 3 entries received US $200, $100, and $50 respectively. At the end of the contest period, 2 study team members independently verified entry eligibility based on the completion of all required fields and relevance to the contest focus of COVID-19. To tailor contest recruitment to Latino immigrants, we announced the contest with Spanish-language posts on community partner social media profiles with established followings, including Somos Baltimore Latino, Centro SOL (Center for Salud/Health and Opportunities for Latinos), Comité Latino, and Sólo Se Vive Una Vez. We primarily focused on CBO Facebook profiles for recruiting given our prior experience with social media reach among Latino immigrants in Baltimore [26] . Our experience and previous studies also showed that complementing web-based activities with in-person recruitment is important for contest inclusivity [23] [24] [25] [26] [27] . The contest was advertised in person at our community-based COVID-19 testing site, with a goal to expand the pool to participants who are less engaged in social media. Entries were judged based on 3 criteria: creativity, appeal to the local community, and potential for increasing COVID-19 testing and prevention; each criterion was scored on a scale of 1-10. The criteria for judging were shared with all participants at the top of the Qualtrics form. No examples were provided to avoid cognitive fixation, a known obstacle to developing innovative ideas [28] [29] [30] . A total of 17 individuals were invited to serve as judges, including 14 Latino community members or leaders, some of whom were also CAB members, and 3 non-Latino individuals who worked in social or health services for the Latino community. A total of 6 judges participated. Judges were provided with a deidentified list of all entries, including the proposed name, hashtag, call to action, and any optional comments submitted. Judges were given 5 days to score all entries in the contest and were compensated US $30 for their time. We totaled the judges' scores for each entry and used the cumulative score to identify the top 10 entries. The top entries were shared across the same community partner profiles for a public vote via a Qualtrics form, which was open for 1 week. We notified the participants who submitted the top entries to congratulate them and encourage them to mobilize their social networks to vote. Voters were offered the option to enter a raffle for 1 of 5 US $25 Visa gift cards. At the end of the popular vote, the top 3 finalists were reviewed by the study team to ensure the feasibility of their ideas, meaning they were relevant to the COVID-19 pandemic and there was no overlap with existing campaigns or brands. The finalists were personally notified and were announced to the public via community partner social media pages, including a weekly live video segment on a popular CBO Facebook page. The winner was also recognized on the campaign website with a photo and a quote sharing their inspiration for participating. To assess the level of community engagement with the crowdsourcing open contest, we measured descriptive statistics for the number of entries and votes and demographic characteristics of the contestants (eg, gender). The text (campaign name, hashtag, and call to action) was analyzed through inductive thematic analysis by 3 team members. The analysis followed 4 steps: (1) data familiarization, (2) initial coding generation and reduction of codes, (3) development of themes, and (4) development of the hierarchical thematic structure. One team member independently reviewed the text from each entry and developed a formative coding scheme. Then, 2 additional team members reviewed the text and coding scheme, which was revised through discussion among the 3 team members. The codes were then applied to the text, and the resulting coded text was reviewed and discussed by the 3 team members to develop the hierarchical thematic structure [31] . We partnered with Altavista Studios, a company with expertise in Spanish-language social marketing campaigns, to design and implement the campaign advertisements. In addition to providing the name of the campaign from the open contest, we also shared the key themes that emerged to guide the content of campaign advertisements. Altavista Studios incorporated these themes in a range of advertisements and provided regular evaluations of advertisement performance through web-based metrics, such as reach and engagement [26, 32] . This project was deemed exempt by the Johns Hopkins University Institutional Review Board (CIR00066868). We received 75 submissions in 2 weeks, of which 1 entry was deemed ineligible as it was an advertisement for a business unrelated to the COVID-19 pandemic. Of the 74 eligible entries, 72 (97%) were in Spanish. Over one-quarter (20/74, 27%) of participants were men and 73% (54/74) were women. The 74 eligible entries were then scored by 6 community judges. With a total possible score range of 18-180, the contest entries ranged from the lowest score of 59 to the highest score of 161. The highest scoring entry was Descoronando el COVID-19 ("Dethroning COVID-19," a play on corona or "crown"). There was a tie for the 10th best score and thus the top 11 entries were included in the popular vote (see Multimedia Appendix 1). During the popular vote period for the top entries, we received 383 unique votes within a week. The entry with the most votes was ¡Yo te cuido y tú me cuidas! ("I take care of you and you take care of me!"). Regrettably, this popular entry was already a name for an established COVID-19 social marketing campaign. A concurrent COVID-19 campaign for a Latino audience with the same name and similar content would confound the evaluation of our intervention, not to mention the potential trademark infringement. We discussed the options with the CAB and collectively decided to announce a tie for first place, with both winners receiving US $200. We also discussed the rationale for selecting the campaign name with the winners for full transparency. The next most popular entry, which was selected to be the name for the campaign, was Vive Sin Duda ("Live Without Doubt"). A total of 3 themes emerged from the crowdsourcing entries. In Table 1 , the themes are presented in order of most to least relevant entries. Although the majority of campaign names related to only 1 theme, many entries touched on more than 1 of the 3 themes through their suggested campaign name, hashtag, and call to action. Juntos contra COVID-19 ("Together against COVID-19"). Todos unidos ("Everyone united"). Nos cuidamos entre todos ("We take care of each other"). ¡Yo te cuido y tú me cuidas! ("I take care of you and you take care of me!"). 90.6 (59-111) 7 Taking care of each other Por un mañana sin COVID-19 ("For a tomorrow without COVID-19"). Hazte la prueba. Vivamos sin COVID ("Take the test. Let's live without COVID"). Hagamos hoy por nuestra familia ("Let's do it today for our family"). Tu salud es primero. Cuidate y cuida a los demás. Haste la prueba ("Your health is first. Take care of yourself and others. Take the test"). Salvando vidas ("Saving lives"). Ningún contagiado más y menos funerals por COVID-19 ("No more infected and fewer funerals due to COVID-19"). Vive sin duda ("Live without doubt"). ¡Siéntete como nueva!¡Haste la prueba! ("Feel like new! Take the test!"). Knowing your status Ama, confía y protégete ("Love, trust and protect yourself"). 95.0 (72-115) 16 N/A c a Some entries are counted in more than 1 category. A total of 6 entries were not included under the themes in this table, including the following: El COVID-19 no es un juego ("COVID-19 is not a game"), Pruebas gratis de COVID-19 ("Free COVID-19 tests"). b Total scores for each entry could range from 18 to 180. c N/A: not applicable. There is no subtheme for this category. We shared the 3 themes with our social media partner to develop advertisements that would resonate with the local community. These themes were integrated into the Vive Sin Duda ("Live without a doubt") campaign, hereafter referred to as the Sin Duda campaign. Campaign messaging was initially focused on COVID-19 testing but was then adapted to include advertisements on vaccines and COVID-19 variants; we continued to incorporate the 3 themes as messaging topics evolved with the pandemic. Figure 2 provides an example of an advertisement based on the theme of collective efficacy, created in response to the emergence of the Delta variant in fall 2021. An example of messaging based on the theme of perceived benefits with a focus on protecting oneself and one's family is displayed in Figure 3 . In fact, we found that these advertisements were among the highest performing in terms of reach and engagement in comparison to advertisements created independently by the design team. Overall, we successfully crowdsourced to guide the messaging for a COVID-19 social marketing campaign for Latinos in Maryland. The winning entry, Vive Sin Duda, has since been used for the development of a public health campaign and website [33] . The Sin Duda campaign initially focused on COVID-19 testing and, after the approval of the COVID-19 vaccine, subsequently expanded to include vaccination information. This study fills an important gap in the literature as we are the first team to document how to tailor crowdsourcing open contests to Latino populations, especially in response to a public health emergency. First, in a 2-week period limited by the urgency of the COVID-19 pandemic, we received 74 eligible entries, which were scored by a panel of community judges and a popular vote. The brief duration and specific audience make this project difficult to compare to the existing literature as most published studies were conducted over longer durations of time, with larger populations, and did not focus specifically on Latino populations [16, 21] . Compared to contests of similar scope, however, we achieved a similar number of entries [27, 34] . We succeeded in soliciting relevant Spanish-language entries from local immigrant community members through targeted CBO recruitment. Given the diversity within Latino populations, it is important to tailor recruitment strategies to reach a specific community through the careful selection of partnerships, platforms, and language. For example, although platforms like Instagram and Tiktok have surpassed Facebook in use among younger populations, our experience has taught us that for Latino immigrant adults in Baltimore, Facebook, and specifically CBO Facebook pages, are most effective [26] . Second, we identified themes that may be useful for future public health campaigns or initiatives that seek to address COVID-19 testing and prevention for Latino populations. The most popular theme, collective efficacy, has been shown to facilitate addressing other health conditions, such as obesity, in Latino populations and has been associated with the potential to reduce health care disparities [35, 36] . Another theme that emerged was that of the perceived benefits of COVID-19 testing, mainly the ability to protect oneself and others. The importance of protecting one's family has emerged as a common driving force among Latino populations, including among young Latino fathers [37] . The final theme that emerged was self-efficacy, defined as an individual's belief in their ability to plan and execute a course of action [38] . Self-efficacy is a well-known predictor of positive health behaviors across races and ethnicities, including behaviors related to nutrition, physical activity, medication adherence, and HIV testing [39] [40] [41] . Recent media coverage of the impact of COVID-19 on Latino populations has often focused on vaccine hesitancy. Our team has found that in addition to addressing intersectional factors that drive vaccine hesitancy, it is important to include strength-based messaging along the theme of self-efficacy, as opposed to deficit-based messaging, while concurrently removing barriers to care [42] [43] [44] . Our novel approach of using thematic analysis to directly inform campaign materials resulted in advertisements with high performance and reach. Crowdsourcing open contests offer a low-cost and prompt way to solicit community perspectives and facilitate community-engaged interventions. The findings from this study provide a timely opportunity to document community-driven strategies to make COVID-19 services more accessible and acceptable to marginalized communities. Given the lack of publications on crowdsourcing open contests focusing on Latino populations, particularly for COVID-19 initiatives, this manuscript provides an important addition to the literature and may guide future crowdsourcing initiatives in similar populations. By building the Sin Duda campaign on the foundation of community input, we not only arrived at a name but also elicited key perspectives within the community that helped guide the creation of successful messaging throughout the campaign. This study has noteworthy limitations. First, although the web-based form facilitated prompt participation and voting, this modality inherently excluded community members who lacked internet access. We selected the web-based modality based on high internet and smartphone use rates among Latino individuals as this is the target audience for the social media campaign [18, 19] . However, this contest may have excluded people from more disadvantaged backgrounds who lack internet access or are not engaged with CBOs and are disproportionately impacted by COVID-19. Second, given the short duration of the contest, the number of entries and therefore viewpoints were limited. We tried to optimize community input by using a public vote to choose the winning entries, and we ensured that all campaign messaging incorporated community viewpoints based on input from the CAB and community members obtaining COVID-19 testing. A further limitation is that the entry with the most popular votes was not a feasible choice as it was already in use for another COVID-19 campaign for a Latino audience. We navigated this issue with the guidance of the CAB and thus announced a tie for first place after openly discussing the rationale with the winners. In future contests, all entries should be checked upon submission such that any options that are not original may be disqualified early on to avoid confusion. Lastly, although our findings support the use of crowdsourcing, more rigorous evaluations through randomized controlled trials are needed to compare the effectiveness of crowdsourcing approaches to noncrowdsourcing approaches in campaign development [45, 46] . Crowdsourcing is an effective means of community engagement and an agile tool for guiding interventions to address COVID-19, including in populations affected by health care disparities, such as Latino communities. In considering the role of crowdsourcing in community-engaged research, the thematic analysis of contest entries is a useful and timely way of understanding community values such that they may be prioritized in the content of the intervention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity Demographic trends of COVID-19 cases and deaths in the US reported to CDC : cases by race/ethnicity; deaths by race/ethnicity; cases by age group; deaths by age group; cases by sex; deaths by sex Risk for COVID-19 infection and death among Latinos in the United States: examining heterogeneity in transmission dynamics Undocumented U.S. immigrants and Covid-19 Black and Latinx community perspectives on COVID-19 mitigation behaviors, testing, and vaccines Hispanics in Maryland: health data and resources. Maryland Department of Health and Mental Hygiene SARS-CoV-2 positivity rate for Latinos in the Baltimore-Washington, DC region Community testing and SARS-CoV-2 rates for Latinxs in Baltimore Agency for Toxic Substances and Disease Registry Crowdsourcing methods to enhance HIV and sexual health services: a scoping review and qualitative synthesis Crowdsourcing in medical research: concepts and applications The Wisdom Of Crowds Methods in Community-Based Participatory Research for Health Crowdsourcing applications for public health Crowdsourcing to improve HIV and sexual health outcomes: a scoping review Using crowdsourcing technology for testing multilingual public health promotion materials Spanish-language consumer health information technology interventions: a systematic review Understanding perceptions of genital herpes disclosure through analysis of an online video contest The 4 youth by youth HIV self-testing crowdsourcing contest: a qualitative evaluation What would an HIV cure mean to you? Qualitative analysis from a crowdsourcing contest in Guangzhou, China HIV cure research community engagement in North Carolina: a mixed-methods evaluation of a crowdsourcing contest A crowdsourcing open contest to design pre-exposure prophylaxis promotion messages: protocol for an exploratory mixed methods study Crowdsourcing in health and health research: a practical guide. World Health Organization Sólo se vive una vez: the implementation and reach of an HIV screening campaign for Latinx immigrants Creative contributory contests to spur innovation in sexual health: 2 cases and a guide for implementation Structured imagination: the role of category structure in exemplar generation Constraining effects of examples in a creative generation task Design fixation. Des Stud Theme development in qualitative content analysis and thematic analysis Using a media campaign to increase engagement with a mobile-based youth smoking cessation program ¡Para trabajar con tranquilidad te deberías de vacunar!: mejor vive sin duda, vacúnate. Mejor vive sin duda Assessment of a crowdsourcing open call for approaches to university community engagement and strategic planning during COVID-19 Advocacy, efficacy, and engagement in an online network for Latino childhood obesity prevention The role of collective efficacy in reducing health disparities: a systematic review Young Hispanic fathers during COVID-19: balancing parenthood, finding strength, and maintaining hope Self-efficacy: toward a unifying theory of behavioral change Self-efficacy, health literacy, and nutrition and exercise behaviors in a low-income, Hispanic population Self-efficacy and HIV testing among Latino emerging adults: examining the moderating effects of distress tolerance and sexual risk behaviors Self-efficacy and health: behavioral and stress-physiological mediation No populations left behind: vaccine hesitancy and equitable diffusion of effective COVID-19 vaccines Lessons we've learned -Covid-19 and the undocumented Latinx community KFF COVID-19 vaccine monitor: COVID-19 vaccine access, information, and experiences among Hispanic adults in the U.S. Kaiser Family Foundation Crowdsourcing to expand HIV testing among men who have sex with men in China: a closed cohort stepped wedge cluster randomized controlled trial Reimagining health communication: a noninferiority randomized controlled trial of crowdsourced intervention in China Abbreviations CAB: community advisory board JMIR Form Res 2022 | vol. 6 | iss. 5 | e35764 | p CBO: community-based organization CBPR: community-based participatory research CHW: community health worker NIH: National Institutes of Health RADx-UP: Rapid Acceleration of Diagnostics for Underserved Populations Edited by A Mavragani ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information We thank our community partners, including Pedro Palomino at Somos Baltimore Latino, Johns Hopkins Centro SOL, Catalina Rodriguez at the Mayor's Office of Immigrant Affairs, Comité Latino, the Esperanza Center, the Latino Providers Network, the Baltimore City Health Department, and CASA de Maryland. This project was funded by a National Institutes of Health and National Institute on Drug Abuse RADx-UP grant (grant R01DA045556-04S1) None declared. Top contest entries in order of votes.[DOCX File , 15 KB-Multimedia Appendix 1]