key: cord-0783556-rf7udfqd authors: Fues Wahl, H.; Wikman Erlandson, B.; Sahlin, C; Nyaku, M.; Benĉina, G. title: Analysis of vaccine messages on social media (Twitter) in Scandinavia date: 2022-02-01 journal: Human vaccines & immunotherapeutics DOI: 10.1080/21645515.2022.2026711 sha: c3c452e40f8ac73a5bdf9bdb341ed9a0039e7e3c doc_id: 783556 cord_uid: rf7udfqd Vaccine hesitancy is listed as one of the top 10 global health threats by the WHO. Existing studies investigating the relationship between vaccine hesitancy and social media have found that misinformation and vaccine concerns on social media can cause significant declines in vaccine coverage rates. The objective of this study was to provide insight into the dynamics of vaccine messages on Twitter in Scandinavia (Denmark, Norway, Sweden), by analyzing tweets in local languages during 2019. A validated measure, the 5C scale, was used to map relevant predictors of vaccination behavior, capturing the factors confidence (in vaccines and the system that delivers them), complacency (not perceiving diseases as high risk), constraints (structural and psychological barriers), calculation (engagement in extensive information searching) and collective responsibility (willingness to protect others). A total of 1794 tweets met the inclusion criteria (DK: 48%, NO: 15%, SE: 37%), predominantly tweeted by private users (86%). The HPV vaccine was mentioned in 81% of tweets. Tweets were classified as expressing confidence (61%), complacency (18%), constraints (15%), calculation (15%), and collective responsibility (4%). Confidence in vaccines and the system that delivers them was expressed in 57%. A lack of confidence was expressed in 4% of all tweets, in combination with calculation in 39%. Analyzing public sentiment toward vaccination on Twitter is a useful tool to leverage for better understanding of the dynamics behind vaccine hesitancy. This analysis could provide actionable information for healthcare professionals and public health authorities to mitigate online misinformation and public vaccine concerns. In 2019, the World Health Organization (WHO) listed vaccine hesitancy as one of the top 10 threats to global health, together with other threats such as drug-resistant pathogens, growing rates of obesity and physical inactivity, environmental pollution, climate change and multiple humanitarian crises. 1 Vaccine hesitancy is defined by the Strategic Advisory Group of Experts on Immunization (SAGE), a working group of WHO, as the "delay in acceptance or refusal of vaccines despite availability of vaccination services." 2 In addition to access to vaccines, addressing vaccine hesitancy is key to reaching achieving and maintaining high vaccination coverage rates (VCR). The increase in vaccine hesitancy has recently been facilitated through the spread of unsubstantiated negative information about vaccination on social media platforms and has been shown to negatively impact VCRs. 3 Most recently, the Covid-19 pandemic fueled an online anti-vaccination movement, spreading misinformation on vaccine safety and attempting to undermine public health. 4 Understanding the causes of vaccine hesitancy and developing strategies to fight misinformation and regain vaccine confidence are vital. [5] [6] [7] From 2013 to 2015, the VCR for human papillomavirus (HPV) vaccine rapidly declined in Ireland, 8 Denmark 9 and Japan 10 following negative media coverage coinciding with increasing suspected adverse-event reporting. 9 These concerns have been dispelled in Ireland and Denmark but negative sentiments around the HPV vaccine still persist in Japan. 11 Denmark rebuilt public trust in HPV vaccination by launching the campaign "Stop HPV, Stop Cervical Cancer." This awareness campaign was developed based on results from surveys and focus groups to better understand parental concerns with the HPV vaccine. Through this campaign, articles about cervical cancer prevention were highlighted to newspapers and lifestyle magazines throughout Denmark. In addition, parents were engaged via YouTube and Facebook -two social media platforms that have previously been identified as key information sources, by addressing their questions and personal stories shared by women with cervical cancer. 12, 13 The VCR of vaccines included in the Scandinavian pediatric National Immunization Programs (NIPs) are generally high, at >97%. [14] [15] [16] The VCR of pediatric vaccines not covered by the NIP, therefore 100% financed out-of-pocket (e.g., varicella), are generally not estimated in these countries but expected to be lower. HPV vaccination has been included in the NIPs of Norway (NO), Denmark (DK), and Sweden (SE) for girls aged ≥12 years, since 2009/2010. HPV vaccination for boys aged 12 years was first introduced in Norway in August 2018, followed by Denmark in September 2019, and in Sweden in August 2020. As of the end of 2020, the VCR for at least one dose of HPV vaccine for girls born in 2008 was 91% in Denmark, 17 94% in Norway, 18 and 89% in Sweden, 19 although larger differences between countries have been observed over time. The VCR for boys has so far been slightly lower (DK: 89%, 17 NO: 93%, 18 SE: 78% 19 ) but is expected to reach coverage rates in girls. Adult vaccination rates, such as for seasonal influenza, pneumococcal disease, etc. are not tracked with the same granularity and are below 75% across the Scandinavian countries. 20 Despite generally high VCRs in the Scandinavian NIPs, it is important to understand the level and nature of exposure the populations of these countries have to social media messages on vaccines and vaccination in their native languages. Twitter is one of the largest social media platforms worldwide, with more than 320 million users. 21 The platform is suitable for sentiment analysis because it is open-access, and used by 19% of Nordic (Denmark, Finland, Norway, Sweden) inhabitants. 22 While a number of sentiment analyses to capture vaccine conversations globally, or in relation to a specific vaccine and/or country, have been performed using Twitter, [23] [24] [25] [26] [27] no study has categorized vaccine sentiments on Twitter in local Scandinavian languages, to the best of our knowledge. A study on how Scandinavian populations are exposed to vaccinerelated Twitter messages in their native languages may help healthcare professionals and public health officials understand the undercurrents of vaccine conversation sentiment and concerns. It may also help health authorities proactively develop communication campaigns promoting vaccination or addressing misinformation on social media platforms. To monitor and assess the psychological antecedents of vaccination and vaccine hesitancy, Betsch et al. 28 developed and validated a tool known as the 5C scale, an extension of the 3C model that SAGE proposed for explaining vaccine hesitancy. 2, 28 The 5C scale includes the following factors: confidence (in vaccines and the system that delivers them), complacency (not perceiving the disease as high risk), constraints (structural and psychological barriers), calculation (engagement in extensive information searching) and collective responsibility (willingness to protect others). The objective of this study is to analyze vaccine-related Twitter messages in native languages in Denmark, Norway, and Sweden, and describe the predictors of vaccination behavior, using the 5C framework scale. Twitter is one of the largest and most commonly-used social media platforms, with more than 320 million users worldwide. 21 It allows the publishing of any kind of information in messages, or "tweets," within a 280-character limit. We used Twitter as a data source for this study because all user accounts are public, meaning that information on the users (e.g., number of followers) and their impact (likes and retweets) are publicly available and retrievable. We conducted a search on Twitter covering posts that were published between 1 January 2019 and 31 December 2019. The one-year time span was chosen to capture any seasonal trends in vaccine conversations, such as school-based vaccinations in the spring and fall. The searches were conducted using a combination of keywords, medical subject headings and free text terms, in Danish, Norwegian and Swedish. Search strings combined keywords for 'vaccine' and 'vaccination' with names for local public health authorities, childhood NIP, or specific vaccines: HPV, pneumococcal and varicella vaccines (see Supplementary Material for further details). Vaccinespecific keywords were included to capture conversations regarding well-known vaccines. These vaccines are pneumococcal coNjugate vaccine for children -part of NIPs, HPV for girls and boys -partially part of NIPs, and vaccines not part of NIPs (varicella, pneumococcal conjugate, or polysaccharide for adults). The Social Studio platform 29 was used to search for tweets during the defined time period and based on predefined search terms. The search was performed as a one-time action. Data gathered was structured into categories, (e.g., tweet, user, date, etc.), to generate a Microsoft Excel report. Data uploaded in Social Studio was thereafter deleted and conversations were not tracked further. The report was uploaded to the listeningupload.com tool 30 to prevent outside access and data manipulation. The Twitter developer policy 31 was adhered to during all stages of the search, analysis and reporting processes. The following inclusion and exclusion criteria were applied to identified Twitter messages: ○ Messages in a language other than Danish, Norwegian, or Swedish ○ Messages not related to human vaccines or human vaccination After application of the inclusion and exclusion criteria, a breakdown of each tweet by its source and vaccine-related messaging was conducted (Table 1) . Tweets were specifically stratified by a set of vaccines: pneumococcal vaccines for children versus adults, HPV for girls versus boys, and varicella. Reference to other vaccines not included in the stratification and the terms "vaccines" or "vaccination" mentioned in general terms (i.e., with no reference to a specific vaccine), were classified separately into their own categories. Potential country-specific temporal trends were investigated by plotting the number of tweets per month. Each included tweet was mapped according to the 15-item 5C scale. 28 This scale is intended to be administered as a questionnaire, with respondents rating their agreements or disagreements with scale items. The use of a questionnaire was outside the scope of the current study which rather focused on categorizing authors' vaccine-related sentiments (agree, disagree, neutral) conveyed in tweets based on the 5C framework ( Table 2 ). Each tweet was interpreted individually and categorized manually into the 5C's by one researcher, then reviewed by a second researcher and disputes resolved by consensus. Within the year of 2019, a total of 3 012 potentially relevant tweets in Danish, Norwegian, and Swedish were identified and retrieved. After application of the inclusion and exclusion criteria, 60% (1 794) of the tweets were included for further analysis, as visualized in Figure 1 . Amongst the 40% (1 218) that were excluded, 39% (469) contained the keywords pneumonia, 13% (155) chickenpox/varicella, 12% (152) cervical cancer and 12% (145) HPV, without including vaccine-specific keywords. Figure 2 illustrates a breakdown of included tweets by language, vaccine, and type of author. Approximately half of the included tweets were in the Danish language, 48% (869), followed by 37% (654) in Swedish and 15% (271) in Norwegian. Most tweets, 81% (1 459), were related to the HPV vaccine, with 13% (190) and 37% (545) of these referring specifically to HPV vaccination for girls and boys, respectively. Vaccination or vaccines in general, i.e., without a reference to any specific vaccine, were mentioned in 11% (190) of tweets. Varicella vaccine was only mentioned in 4% (68) of tweets, and pneumococcal vaccines in 1% (18, of which 2 referred to pneumococcal vaccines in children), respectively. Additionally, 3% (59) of tweets mentioned other vaccines such as rota, measles, and influenza. Eighty six percent (1 536) of tweets were posted by private users, 8% (150) by news agencies, 3% (62) by authorities and 3% (46) by other organizations. The vaccine most mentioned in the identified tweets was the HPV vaccine, present in 81% of messages (84% Danish, 24% Norwegian and 60% Swedish), and thus any temporal trends for HPV-specific tweets follow the overall time trends closely. When looking at all HPV-related tweets over time and by language, peaks can be seen in all three countries between February to April. A peak in Danish tweets can be seen in June, with all countries thereafter having peaks between September-December. Seventy three percent (1 315) of the tweets were classified into at least one of the 5C's, with 39% (695) being classified into only one C, 31% (548) into two C's, and 4% (67) and 0.3% (5) being classified into 3 and 4C's, respectively. We were unable to classify 27% (479) of tweets because they did not include any statements that could be matched against the 15-item scale. The majority, 57% (1 024) of all tweets agreed with the confidence-statement, i.e., they were interpreted to expressed trust in vaccines, their safety and the system that delivers them ( Figure 3 ). Only 4% (75) of tweets disagreed with the confidence-statement, while 39% (695) of tweets did not include any wording related to confidence. Among Danish tweets, 56% (487) expressed confidence (agree) in vaccines. The corresponding numbers for Norwegian and Swedish tweets were • Vaccinations are effective (agree) • Regarding vaccines, I am confident that public authorities decide in the best interest of the community (agree) Vaccination is unnecessary because vaccine-preventable diseases are not common anymore (agree) • My immune system is so strong, it also protects me against diseases (agree) • Vaccine-preventable diseases are not so severe that I should get vaccinated (agree) Everyday stress prevents me from getting vaccinated (agree) • For me, it is inconvenient to receive vaccinations (agree) • Visiting the doctor's makes me feel uncomfortable; this keeps me from getting vaccinated (agree) • Willingness to pay/affordability prevents me from getting vaccinated* (agree) • Physical constraints prevent me from getting vaccinated* (agree) When I think about getting vaccinated, I weigh benefits and risks to make the best decision possible (agree) • For each and every vaccination, I closely consider whether it is useful for me (agree) • It is important for me to fully understand the topic of vaccination before I get vaccinated (agree) Collective responsibility When everyone is vaccinated, I don't have to get vaccinated (R) (agree) • I get vaccinated because I can also protect people with a weaker immune system (disagree) • Vaccination is a collective action to prevent the spread of diseases (disagree) *Sub-items related to affordability/willingness-to-pay and physical constraints were not part of the original scale but were added to capture these constraints. (R): Item with (R) is reverse coded in relation to sub-items. 62% (169) and 56% (368), respectively. Of all tweets that were categorized under confidence, 76% (838 tweets) were related to HPV vaccine, with 92% (775) expressing confidence: agree and 8% (63) expressing a lack of confidence (disagree). Eighteen percent (320) of all tweets disagreed with the complacency-statement, i.e., they were interpreted to express that vaccine-preventable diseases are severe or common and that 1 794 (60%) tweets were included for further analysis (of these 124 were quote tweets and 896 retweets) 1 218 (40%) tweets were excluded as they matched the exclusion criteria 3 012 (100%) tweets were extracted with the keyword search vaccination is necessary. Only one tweet expressed that vaccination is unnecessary as vaccine-preventable diseases are not severe or common (complacency: agree). Two tweets expressed that vaccine-preventable diseases are common but did not link this with an opinion on whether vaccination is necessary or unnecessary (complacency: neutral). Among the Swedish tweets, 22% (146) disagreed with the complacency-statement, compared to 15% for both Danish and Norwegian tweets (133 and 41 tweets respectively). Of all tweets that were categorized under complacency, 74% (238) were related to the HPV vaccine, with 0.4% (1) expressing complacency (agree), 99% (236) expressing no complacency (disagree), and 0.4% (1) were neutral. Thirteen percent (239) of all tweets expressed a constraint but did not state whether that would prevent them from getting vaccinated (neutral). Only one tweet clearly expressed that the constraint would not prevent them from getting vaccinated (constraints: disagree), while 1% (23) Four percent (64) of the tweets expressed that vaccination is a collective action to help prevent the spread of diseases and to protect other people (collective responsibility: disagree), while no tweet expressed that vaccination is not a collective action (collective responsibility: agree). Collective responsibility was most mentioned among Swedish tweets, 7% (46), compared to 3% (7) among Norwegian tweets and 1% (11) among Danish tweets. Of those tweets that expressed collective responsibility, 84% (54) did so in relation to the HPV vaccine, of which all disagreed with the statement. Confidence was the C under which most tweets, 61% (1 099), were categorized. A lack of trust in vaccines, their safety and the system that delivers them (confidence: disagree), was expressed in 4% (75) of all tweets. Among these, 39% (29) also expressed calculation, with only one other tweet respectively, also expressing complacency or constraints. This combination of confidence (disagree) and calculation was most pronounced among Danish tweets, where 79% (26) of the 33 Danish tweets that expressed a lack of confidence also expressed an engagement in information searching ( Figure 4) . Confidence (agree) in vaccines was expressed in 57% (1024) of tweets. Among these, 31% (321) furthermore expressed complacency, 20% (203) expressed constraints, 7% (76) expressed calculation, and 6% (64) collective responsibility. Of the tweets expressing both confidence (agree) and constraints, only one tweet expressed that the perceived constraint would not prevent them from getting vaccinated (constraint: disagree) while 22 tweets explicitly expressed constraints that prevented them from getting vaccinated (constraints: agree). The high price of the HPV vaccine was the constraint mentioned in 17 of these latter messages. The remaining 180 tweets that expressed constraints did not specify if they were preventing the author from getting vaccinated (constraints: neutral). In this study we analyzed vaccine-related Twitter messages in local languages in Denmark, Norway and Sweden during 2019, in relation to the components of the 5C scale, a model developed for measuring the psychological antecedents of vaccination. 28 We found that confidence, in vaccines, their safety and the system that delivers them, was the most expressed antecedent when taking a stance for or against vaccines, across all countries. Confidence (agree) was expressed in 57% of all tweets. A combination of confidence (agree) with complacency (disagree, i.e., expressing that vaccine-preventable diseases are severe and common) was found in 31% of tweets, while confidence (agree) combined with constraints was found in 20% of tweets. The high price of the HPV vaccine was the most mentioned constraint for not getting vaccinated or vaccinating one's child, despite expressing confidence (agree) in the vaccine. Only 4% of all the tweets explicitly expressed a lack of confidence (disagree) in vaccines. Among the tweets expressing a lack of confidence, 39% also expressed a tendency to engage in information searching (calculation), indicating that individuals that have no confidence in vaccines also tend to take information they have come across into account. This study shows that statements aligning with confidence and complacency are equally mentioned in all three languages. Of the three remaining C's, collective responsibility was the most mentioned among Swedish tweets. Danish tweets most expressed engagement in information searching (calculation), and constraints were highlighted most among Danish and Norwegian tweets. These findings might suggest that there are different motivators for vaccination and areas of importance between the three countries, and that monitoring Twitter could provide further understanding of the general sentiment toward vaccination expressed through social media. Conversations around vaccines on Twitter in 2019 were found to focus on the HPV vaccine (81% of messages) regardless of 5C categorization, with approximately half of these being Danish tweets. Peaks in the volume of Danish HPV-related messages can be linked to three events; 1) newly published numbers by Statens Serum Institut showing an increasing VCR for girls during 2018 32 (February), 2) Statens Serum Institut publishing a study examining the association between media coverage and the decline in HPV VCR in Denmark 9 (March), and 3) a published meta-analysis on 14 high-income countries, including Denmark, showcasing the substantial impact of HPV vaccination programs on the incidence of HPV-related disease 33 (June-August). Furthermore, peaks in HPV-related messages coincided with media coverage of the inclusion of boys in the Danish HPV NIP in February, June and September 2019. 34 In both Norway and Sweden, peaks in HPV-related messages coincided with the timing of the school-based HPV vaccination programs (February-March, August-September). Thus, the amount of HPV-related Twitter activity in all three countries correlated with the release of published public health authority reports, scientific studies, media coverage and timing of vaccination programs. This activity, together with most tweeters being private users, indicates that Twitter could be an effective platform for Scandinavian public health authorities to leverage for reaching and sharing accurate vaccine-related information with the general public. Vaccines and vaccination in general terms were mentioned in 11% (190) of tweets, of which 126 were in Danish. Of these Danish tweets, 88% were in response to a Twitter campaign that called on doctors to take to Twitter to showcase that they and their families get vaccinated, under the hashtags #VaxFactsFebruary, #VaccinateYourKids and #vaccinervirker. Only a small proportion of tweets, 8% (127), were related to other type of vaccines such as pneumococcal, varicella, rotavirus, measles, and influenza vaccination, potentially indicating that these vaccines are not as widely known or prone to incite debate on Twitter. Wiyeh et al. 35 used the 5C scale to identify the determinants of vaccine hesitancy amongst responses provided by social media users to a post on Facebook by the Department of Health of the Western Cape Province of South Africa, announcing the implementation of a school-based HPV vaccination campaign. The authors found that issues related to confidence, complacency, calculation, and constraints were determinants of vaccine hesitancy. The determinants of HPV vaccine acceptance in the relevant South African population were found to be confidence and calculation, with no comments expressing complacency, constraints, and collective responsibility. In line with our findings, confidence was a main contributor to messages expressing both vaccine hesitance and vaccine acceptance. However, in contrast to Wiyeh et al.'s 35 findings, complacency-related statements were exclusively (except for one tweet) mentioned among confident (agree) tweets in Scandinavia. As with the present study, the patterns of the C-scale categorization seem to reflect ongoing discussions in South Africa and thus are not directly comparable to our setting. Several studies have used the 5C scale as a survey administered directly to the general population or to target specific groups in society, e.g., healthcare workers, to assess participants' vaccine acceptance and intentions for e.g., influenza and Covid-19. [36] [37] [38] [39] [40] Even though these studies used the same framework for assessing attitude toward vaccines, analyzing sentiments on social media may not, in contrast to a survey, represent the views of the general public, but rather the views of those that are robustly either for or against vaccines and vaccination, and might not be linked to actual vaccination behavior. However, it is important to capture these "loudest voices" because they are the ones likely to be influential. The European Center for Disease Prevention and Control (ECDC) recently conducted a systematic literature review of social media monitoring methods and interventions related to vaccine hesitancy. 41 They identified 86 published studies between 2006-2019 that analyzed social media messages related to vaccination. Sixty of these studies performed some type of sentiment analysis, coding data into e.g., positive/provaccination versus negative/anti-vaccination sentiments, and 42 of these monitored Twitter. A large proportion of studies (41) did not restrict monitoring to one specific country, instead containing global results. None of these studies had been performed in Scandinavia, using native language social media messages. This study was subject to some limitations. All tweets are subject to a 280-character limit and were interpreted individually and manually. Even though the 5C scale provided a solid basis for categorization, the manual interpretation of the message content was limited by the brevity and in certain instances the full context of the tweet could not be deciphered. In addition, the use of sarcasm, slang, or hyperbole obscured the true intent of a tweet. For instance, 27% of tweets meeting inclusion criteria were impossible to categorize using the 5C scale, in large part due to a lack of context for interpreting the tweet. The limitations of manual sentiment coding might have influenced the interpretation of individual tweets, however, we believe the general sentiments of vaccine conversations on Twitter during 2019 were adequately captured in our study. If this study were to be conducted using 2020 Twitter data, we hypothesize that the results will be different, considering the implications of the Covid-19 pandemic on individuals. The 5C scale 28 was used as a basis for organizing tweets, as well as the basis for comparing findings between Scandinavian countries. Even though it was not possible to administer the scale as a survey as intended by Betsch et al. 28 and therefore not possible to follow their proposed approach 42 for adapting the 5C scale to the Scandinavian country context, we believe that the framework provided a good basis for organizing findings since it was developed in WEIRD (Western, Educated, Industrialized, Rich Democratic) societies. Furthermore, Scandinavian countries have many commonalities in culture and language, making comparisons feasible. Twitter was the social media platform used in this study due to tweets being publicly available and retrievable. Twitter usage rates are at around 20% in the Scandinavian countries and are relatively low in comparison to other social media such as Facebook or Instagram. Due to this and differences in Twitter versus country demographic profiles, the results are not generalizable to the total Scandinavian populations, or other social media platforms. This study provides insights into the predictors of vaccination behavior in local languages in Denmark, Norway, and Sweden on Twitter during 2019, using the 5C scale framework. It has illustrated that confidence in vaccines, their safety and the system that delivers them is the most common reason for expressing a stance for or against vaccines on Twitter in all three Scandinavian countries. The occurrence of the other four antecedents for vaccination differed between the Scandinavian languages, indicating that there might be different areas of importance and various ongoing discussions in the three countries. While additional research is needed to investigate the complex issue of vaccine hesitancy, noting that a single scale will most likely be unable to capture every single cause, this framework may be appropriate for capturing the main predictors of vaccine acceptance and hesitancy in Scandinavian Twitter messages. Monitoring social media conversations and public sentiment toward vaccination on Twitter is a potentially useful tool to leverage for better understanding of the dynamics behind vaccine hesitancy and could provide actionable information for healthcare professionals and public health authorities to mitigate misinformation or vaccine concerns. Future sentiment analysis with longer time horizons is needed to study potential shifts in conversation surrounding vaccines in Scandinavia. Furthermore, setting up a framework that automatically codes social media messages into the 5C's based on keywords would enable a refinement of the current study, analyzing the tweets on a larger and more systematic scale. World Health Organization. Ten threats to global health in 2019 Report of the SAGE working group on vaccine hesitancy Social media and vaccine hesitancy The anti-vaccination infodemic on social media: a behavioral analysis Leveraging media and health communication strategies to overcome the COVID-19 infodemic Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases Strategies for addressing vaccine hesitancy -a systematic review Rapid response to HPV vaccination crisis in Ireland Decline in HPV-vaccination uptake in Denmark -the association between HPV-related media coverage and HPV-vaccination Impact of HPV vaccine hesitancy on cervical cancer in Japan: a modelling study Access to HPV vaccination in Japan: increasing social trust to regain vaccine confidence World Health Organization Regional Office for Europe. Danish health literacy campaign restores confidence in HPV vaccination Denmark campaign rebuilds confidence in HPV The Public Health Agency of Sweden (Folkhälsomyndigheten) The Danish childhood immunization programme The Norwegian Institute of Public Health. Statistikk for barnevaksinasjon Overvågning i tal, grafer og kort The Norwegian Public Health Institute (Folkehelseinstituttet) The Public Health Agency of Sweden (Folkhälsomyndigheten) The Public Health Agency of Sweden (Folkhälsomyndigheten) Digitala medier på föräldriska -Twitter DEVICE STUDY 2016: Social media across the Nordics Addawood A Usage of scientific references in MMR vaccination debates on Twitter Evaluation of a multinational, multilingual vaccine debate on Twitter Sentiment, contents, and retweets: a study of two vaccine-related Twitter datasets Observation of public sentiment toward human papillomavirus vaccination on Twitter Monitoring negative opinion about vaccines from tweets analysis Beyond confidence: development of a measure assessing the 5C psychological antecedents of vaccination ATCS Fortsat stigning i HPV-vaccination i Population-level impact and herd effects following the introduction of human papillomavirus vaccination programmes: updated systematic review and meta-analysis Ny sundhedsminister: Gratis hpv-vaccine er på vej til drenge, in Berlingske Social media and HPV vaccination: unsolicited public comments on a Facebook post by the Western Cape Department of Health provide insights into determinants of vaccine hesitancy in South Africa Looking inside the 'black box' of vaccine hesitancy: unlocking the effect of psychological attitudes and beliefs on COVID-19 vaccine acceptance and implications for public health communication Covid-19 vaccine acceptance, hesitancy, and refusal among Canadian healthcare workers: a multicenter survey Editor's choice: influenza vaccine uptake, COVID-19 vaccination intention and vaccine hesitancy among nurses: a survey Acceptance of a COVID-19 vaccine in Japan during the COVID-19 pandemic. Vaccines (Basel) Will they, or won't they? Examining patients' vaccine intention for flu and COVID-19 using the Health Belief Model European Centre for Disease Prevention and Control. Systematic scoping review on social media monitoring methods and interventions relating to vaccine hesitancy Sample study protocol for adapting and translating the 5C scale to assess the psychological antecedents of vaccination