key: cord-0950104-qi7taj8y
authors: Chan, Man-pui Sally; Jamieson, Kathleen Hall; Albarracin, Dolores
title: Prospective associations of regional social media messages with attitudes and actual vaccination: A big data and survey study of the influenza vaccine in the United States
date: 2020-08-10
journal: Vaccine
DOI: 10.1016/j.vaccine.2020.07.054
sha: 7bc9a8b5a17c409dd76284ee5f11b999887c128c
doc_id: 950104
cord_uid: qi7taj8y

OBJECTIVE: Using longitudinal methods to assess regional associations between social media posts about vaccines and attitudes and actual vaccination against influenza in the US. METHODS: Geolocated tweets from U.S. counties (N = 115,330) were analyzed using MALLET LDA (Latent Dirichlet allocation) topic modeling techniques to correlate with prospective individual survey data (N = 3005) about vaccine attitudes, actual vaccination, and real-life discussions about vaccines with family and friends during the 2018–2019 influenza season. RESULTS: Ten topics were common across U.S. counties during the 2018–2019 influenza season. In the overall analyses, two of these topics (i.e., Vaccine Science Matters and Big Pharma) were associated with attitudes and behaviors. The topic concerning vaccine science in November-February was positively correlated with attitudes in February-March, r = 0.09, BF(10) = 3. Moreover, among respondents who did not discuss the influenza vaccine with family and friends, the topic about vaccine fraud and children in November-February was negatively correlated with attitudes in February-March and with vaccination in February-March, and April-May (rs = −0.18 to −0.25, BF(10) = 4–146). However, this was absent when participants had discussions about the influenza vaccine with family and friends. DISCUSSION: Regional vaccine content correlated with prospective measures of vaccine attitudes and actual vaccination. CONCLUSIONS: Social media have demonstrated strong associations with vaccination patterns. When the associations are negative, discussions with family and friends appear to eliminate them. Programs to promote vaccination should encourage real-life conversations about vaccines.

Although the influenza vaccine is one of the greatest achievements of public health and each year prevents millions of illnesses and thousands of deaths [1] , vaccine messages expressing skepticism about vaccine safety and describing fake patient stories abound on social media [2, 3] . Important research has advanced our understanding of social media contents, including how Russian linked-accounts have discussed vaccines in an attempt to influence the American public [4] . However, the ultimate associations between these contents and vaccination behavior remains an underexplored problem that we investigated in this study. To address this question, we analyzed regional variability in social media messages in the US in combination with longitudinal survey measures from a representative sample of the U.S. adult population.

1.1. Regional associations between social media messages and attitudes and health behavior

As an example, even though more than a hundred thousand people had died of SAR-CoV-2 by the time this paper was written, and a vaccine may not be available until 2021, tweets opposing an eventual vaccine against the SARS-CoV-2 virus already populate the cyberworld. An example of these appears below.

''To each their own. I believe in medical freedom, not medical tyranny. If you want to get the flu shot, go ahead. But it shouldn't be mandatory. Same with any medicine or vaccine. I https://doi.org/10.1016/j.vaccine.2020.07.054 0264-410X/Ó 2020 Elsevier Ltd. All rights reserved.

personally would never take a Coronavirus vaccine. I know these sneaky snakes want it mandatory!" [5] .

Other tweets, of course, support vaccination, such as the following example:

''The problem is when you make that choice [vaccine vs. chip] for yourself, you're also potentially making it for other people who are unable to get vaccinated. Polio, measles, diphtheria, etc are not things anyone should suffer through. Especially not when we have means of preventing them." [6] A growing number of important studies have analyzed vaccine content on Facebook, Twitter, and YouTube. Although some studies have involved manual coding of posts [7] [8] [9] , others have used computational techniques such as co-occurrence network analysis [10] , and yet others have combined manual coding with computational methods, such as a support vector machine [11] . A recent study of anti-and pro-vaccine clips on YouTube identified words such as ''chemical," ''mercury," and ''toxic" in anti-vaccine content and ''hospital," ''chronic," and ''unvaccinated" in pro-vaccine material [9] . Similarly, an analysis of tweets showed factors containing words such as ''cdcwhistleblower" and ''coverup" posted by antivaccine users and topics containing ''risk," ''health," ''chronic," and ''science" posted by pro-vaccine users [11] . These studies have demonstrated that linguistic analyses can identify alternative types of vaccine content on social media. Yet, most of these studies covered only content obtained by means of pre-determined keywords. More exploratory methods of unsupervised learning (e.g., topic modeling) may provide further insights into vaccine social media contents [12] .

The psychological theory has shed light on the nature of the relations between health messages and attitudes, as well as correspondence between attitudes and behaviors themselves [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] . Specifically, health contents can include information for people to act on, and include both persuasive posts by health officials as well as the personal opinions of lay people [18, [23] [24] [25] . Health messages can then influence attitudes, which in turn guide people's decisions about what to do, including whether or not to vaccinate against the disease [14, 25, 26] . That is, people holding favorable attitudes toward the influenza vaccine are more likely to get vaccinated than those with unfavorable attitudes [27] . Recently, the relations between social media posts and vaccine attitudes and actual vaccination have also begun to be explored.

Prior studies have obtained cross-sectional evidence of relations between the use of or exposure to social media and vaccination status [28, 29] . According to research on individuals, people who use either Twitter or Facebook, or both, as sources of health information are more likely to have received the influenza vaccine than are people who do not [28] . Moreover, across the US states, how common HPV vaccine topics are on Twitter accounts for 68% of the variance in the proportion of females receiving at least one dose of the HPV vaccine [29] . Although each of these studies offers useful evidence, both were cross-sectional, and the regional study [29] lacked individual-level data, which limits its conclusions and introduces the possibility of the ecological fallacy [30, 31] . Thus, longitudinal data and a combination of individual-and regionallevel data are necessary to understand the relations between social media messages posted within a region and individual attitudes and behaviors in that region.

Social media posts questioning the need for vaccines, for instance, may lead people to form negative attitudes toward vaccination and not vaccinate [32] [33] [34] . However, social media do not exist in a vacuum. Tweets posted in a region can be highly relevant to the local context [35] , and their impact may also depend on discussions of the influenza vaccine with family and friends in real life (for similar work in other areas, see [36] ). In this regard, although the CDC encourages people to talk to friends and family about the influenza vaccine [37] , the effectiveness of discussing vaccines with them is unclear. On the one hand, discussions with family and friends may further vaccine misinformation and hesitancy [2, 38] . For example, people seek and circulate health information among family and friends [39] , and this creates an information bubble (i.e., reading news or information that supports what people already believe in) that can pose challenges for public health announcements and physician recommendations to get through [40] .

On the other hand, discussing vaccines with family and friends may correct inaccurate information found on social media. For example, discussing vaccination with family and friends predicts college students' positive dispositions toward vaccination [41] , and norms can have positive influences on actual vaccination as well [42] . Moreover, discussions within one's network have been shown to reduce bias [43] . For example, Guilbeault and Centola [43] examined the idea of ''networked collective intelligence" by assessing the impact of tobacco-prevention messages exchanged within the network. Participants who did and did not smoke were randomly assigned to either an experimental condition that allowed participants to have discussions with their networks or a control condition in which participants evaluated the health message by themselves. The results showed that smokers who discussed the message with others evaluated the health message more favorably than did smokers who evaluated the message alone. In short, introducing more diverse contents through discussions with family friends can be useful in correcting misinformation [44, 45] .

This study attempted to contribute to (a) provide a longitudinal assessment of the regional effects of social media messages on vaccine attitudes and vaccination, (b) incorporate individual-level data to minimize the threat of the ecological fallacy, and (c) examine the role of discussions with others as a factor that may strengthen or weaken the influences of social media. The guiding research questions included: What are the regional associations between social media messages and vaccine attitudes and vaccination? And how does discussing vaccines with family and friends influence the relation of social media content and vaccination? We used Twitter and the survey data corresponding to the 2018-2019 influenza season to examine these questions.

The proposed study combined individual-level survey data with county-level tweets about vaccines, which were geolocated. The archival survey data involved a probability-based, nationally representative sample of American adults and assessed attitudes concerning the influenza vaccine, actual vaccination, and real-life discussions of the influenza vaccine in a five-wave panel (between This social media analysis platform offers full access to all public tweets (i.e., Twitter Firehose). The survey data and the Twitter data are described next.

2.1.1. Survey data 2.1.1.1. Design and methods. We used five waves of a large panel survey assessing attitudes, knowledge, and health behaviors concerning infectious diseases and vaccines. The archival data included a probability-based, nationally representative sample of American adults who were randomly selected from AmeriSpeak panels of NORC at the University of Chicago in September-October and followed in October-November, November-February, February-March, and April-May (i.e., the 2018-2019 influenza season). This sampling frame covered 97% of U.S. households via a two-stage stratified sampling [46] . Respondents primarily completed the survey online, although a small percentage answered the survey over the phone (landlines and cell phones) to avoid a bias favoring people who use the Internet. This panel is the only probability panel that is available for hybrid online/phone administration in the US. After the data collection in September-October, weighted distributions of age, sex, race, and income showed only trivial differences compared to unweighted distributions. However, there was a discrepancy in education, with more of the survey sample having a college degree than the 2018 U.S. Census estimates. To mitigate this problem, respondents with a lower level of educational attainment were successfully targeted for retention in following waves with more incentives and reminders. We reverse-scored all items except item (b), so a higher score indicates more favorable vaccine attitudes. The Cronbach's alphas of the responses for all four waves were about 0.81-0.85, suggesting good internal consistencies. Therefore, we calculated a mean of these questions to represent respondents' attitudes regarding the influenza vaccine in each wave.

Using a 4-point scale, participants also answered the question: ''In the past month, how frequently have you discussed the flu vaccine with friends and family? (1: None, 2: rarely, 3: sometimes, 4: often). We rescored the responses of this question into a binary variable (i.e., 1: rarely, sometimes, or often as with discussions, and 0: none as without discussions of the influenza vaccine with family and friends) for the moderator analyses.

Twitter continues to be one of the most popular social media platforms [47] and, unlike Facebook, provides open data access and has more precise and frequent location information. In this study, we collected Twitter data using Crimson Hexagon API. We adopted a top-down approach to collect relevant tweets by searching keywords related to vaccines posted between September 2017 and August 2018 to be the training data. We used additional keywords concerning influenza and the influenza vaccine with the keywords used for the training data to collect tweets between September 2018 and March 2019 to be the test data. In total, we collected 628,597 tweets and geotagged about 18% of the collected tweets for the analyses (see Table S1 ). This percentage is similar to the geotagging rates reported in previous studies [48, 49] .

We developed lists of keywords about influenza and flu by reviewing relevant literature [50] [51] [52] and consulted a team of health communication researchers to identify all keywords and hashtags relevant to the study period. Keywords used to collect

We began with the geotagging procedure of tweets to US counties and applied topic modeling techniques to identify vaccinerelated words that co-occurred (i.e., ''topics"). We followed Maier et al.'s [53] recommended approach to select a final topic model and then computed each topic's probabilities for U.S. counties during the 2018-2019 influenza season (see Fig. 1 ). Next, we combined topic probabilities with the survey data and performed correlations to examine the regional association of social media messages with vaccine attitudes and actual vaccination. Finally, we conducted the moderator analyses to assess if real-life discussions with friends and family affected the associations, and if so, what that influence was. Technical details about each step along with the topic modeling implementation follow.

Tweets, including retweets, are informative about popular topics and conversations within a community. Prior studies have shown that county-level tweets can provide signals for predicting HIV/STI infections [49, 54] and are associated with attitudes, knowledge, and behaviors of infectious diseases such as Zika across U.S. counties [55] . Therefore, we decided to use county as the unit of analysis. We used the timestamp metadata to exclude tweets that do not originate from U.S. time zones. For each tweet, the user profile location and each tweet's precise latitude and longitude coordinates were used to map tweets to U.S. counties using Geographic Information Systems (GIS) database operations (see [49] for details of the geotagging process). Tweets from the same county were organized into a single document, which constituted the unit of analysis. Thus, the tweets could not be used to reidentify individual users, and no personal information was included in the analyses. Details about the training and test data appear in Table S1 .

We adapted Maier et al.'s [53] recommended approach to select a valid and reliable topic model, including (a) appropriate preprocessing of the text collection, (b) adequate selection of model parameters (i.e., the optimal number of topics of the topic model [K], alpha, and beta), and (c) evaluation of the model's reliability. Before analysis, we removed unnecessary words and characters from the text corpus to retain only meaningful words and formed bi-grams (i.e., two-word phrases, see supplementary materials for details of the pre-processing step). Messages in each county were analyzed as a single document, and the words from all tweets from each county were combined into one single word-by-frequency matrix (see Fig. 1 ).

We used the Python package gensim to convert a collection of documents (i.e., each county's tweets) into a frequency matrix of token counts. The matrix of token counts was then analyzed using a well-established algorithm in computer science, namely Latent Dirichlet Allocation [56] . LDA is a Bayesian mixture model that groups words that often appear together to create topics (see Fig. 1 ). Then, we used a Python wrapper for LDA by implementing MALLET [57] , and calculated coherence and log perplexity scores to determine the optimal number of topics (i.e., K) and hyperparameters (i.e., alpha and beta, see supplementary materials for details). This process led to a topic model with ten topics (i.e., K = 10) and another with twenty topics (i.e., K = 20).

Next, we used the codes provided by Maier et al. (2018) to evaluate the reliability of two LDA topic models by conducting 33 experiments. Fig. 3 shows the reliability and coherence scores of the experiments, and a higher score indicates higher reliability and semantic coherence of the identified topics. As shown in Fig. 2 , the topic model with a K of 10 had significantly higher reliability and semantic coherence than the topic model with a K of 20. Therefore, the selected topic model had ten topics, random initialization, and 1000 iterations.

Finally, we computed the probability of each topic for each county in each wave, p(topic, county) = P p(topic | word) Â p(w ord | county), where p(word | county) represents the normalized proportion of words in a county document, and p(topic | word) represents the probability of the topic given that word, using tweets that correspond to the survey period. We also calculated the standardized topic probabilities per county (i.e., the sum of all topic probabilities per county is equal to 1) by replacing unpresentable topic probabilities with ''0.00001". The topic probabilities in each wave were then matched with the individual-level survey data by county and analyzed with Bayesian correlation analyses.

The survey data involve individual participants who live in U.S. counties, and the Twitter topics belong to counties. Such a structure requires researchers to first determine the optimal way of analyzing the data, particularly whether participants should be nested within counties or whether the variance in survey responses can be assumed to vary as a function of participants. Hence, we calculated intra-class correlations (ICCs) to assess the overall variation in vaccination and vaccine attitudes explained by U.S. counties. Values of ICC can range from 0 to 1, and a value close to zero indicates that the observations within clusters are no more similar than observations from different clusters [58] . Values of ICC > 0.20 is used as evidence of the need to use multilevel modeling, which in our case would imply modeling participants and counties as separate levels. Values above 0.20 would imply that participants must be modeled as nested within counties. We used the performance package [59] in R to calculate ICCs with vaccination and vaccine attitudes as the outcomes for all waves (see Table S3 ).

We next used an R package BayesMed to calculate the Bayes factors for correlations between topics and attitudes or vaccination using a default uniform prior (i.e., 1) in which the possible correlation between À1 and +1 is equally plausible [60] . A Bayes factor (i.e., BF 10 ) between 3.2 and 10 provides substantial evidence that the data support the alternative hypothesis of a relation between two variables at least 3.2 to 10 times more than the null hypothesis of the absence of association [61] .

Our selected topic model appears in Table 1 . Two of the authors named these ten topics: Vaccines Protect Children, Immunization Contributes to Global Health, Doctors Vaccinating Children, Fighting Table 1 shows the top ten words of each topic, and Fig. 2 shows word clouds with the top 30 words of each topic. These topics were then used in correlation analyses to determine associations with our survey measures of attitudes and vaccination for each wave.

The main goal of the study was to combine the topics in Table 1 with the survey data. This required assembling a data frame in which participants are nested within counties in each wave. Then, this data frame was combined with the topics, which are designated with probabilities for each county. We first describe the waves and demographics of the survey. We then describe the hierarchical analyses of our data frame, and last, we describe the correlational and moderator analyses.

The sample sizes of the multiple-wave survey ranged from 1591 to 3005 (N in September-October = 3005, N in October-November = 2470, N in November-February = 2091, N in February-March = 18 03, and N in April-May = 1591). The descriptive characteristics of the sample in September-October were as follows: About 63% Non-Hispanic White, 13% Black, 6% Asian, 14% Non-White Hispanic, 1% Other, and 3% Mixed. The mean age was 48.4 years old (SD = 18), with a median education level of an associate degree and a median annual household income level at $50,000-$59,999.

As explained before, when data are hierarchical, it is first necessary to determine if all levels contribute to the variability observed at the individual level. This decision is important to know if, for example, differences across counties are due to differences in the individuals who live there or to aspects of the counties, such as policies, driving attitudes, or behaviors. We computed ICCs in all waves and found very low levels of variability attributable to the county in which participants lived (ICCs < 0.05; see Table S3 ). This suggested that the associations of attitudes and vaccination with the topics could be assessed at the individual level. Therefore, we used single-level Bayesian correlation analyses in the following analyses.

Our first research question concerned the relations between the topics in Fig. 2 and vaccine attitudes and actual vaccination longitudinally (see Table 2 ). We conducted Bayesian correlations 

Next, we conducted moderator analyses to examine whether discussions with friends and family affected the correlation paths, and if so, what that influence was. We repeated the correlation analyses for respondents reporting real-life discussions of the influenza vaccine and those reporting no real-life discussions of the influenza vaccine, respectively (see Table 3 ). For respondents with no discussions with family and friends, we observed negative associations between Vaccine Fraud and Children in November-February and attitudes in February-March, r = À0.18, BF 10 = 4. Vaccine Fraud and Children in November-February was also inversely associated with vaccination in February-March and April-May, rs = À0.24 to À0.25, BF 10 = 49-146. Additionally, the results showed no evidence of any relations in respondents having reallife influenza vaccine discussions.

Combating an ''infodemic" [62] on social media is important in preventing the spread of misinformation in the community. Prior research on social media and vaccines has focused mainly on either cross-sectional correlations between the use of or exposure to social media content and vaccination status, or cross-sectional correlations between the volume of tweets and state-level vaccination rates. Thus, the prior findings have not shed light on the paths between social media and vaccination, which require individual and longitudinal data. Filling this void, our study identified whether social media messages across U.S. counties are associated with vaccine attitudes and vaccination, and how vaccine discussions moderate the relations of social media messages with vaccine attitudes and with actual vaccination. We also found three prevailing vaccine topics: Vaccine Science Matters, Big Pharma, and Vaccine Fraud and Children. Furthermore, the probabilities of Vaccine Science Matters were prospectively associated with vaccine attitudes, and the probabilities of Vaccine Fraud and Children were prospectively associated with vaccine attitudes and actual vaccination. Importantly, in line with the idea of networked collective intelligence [43] , discussing the influenza vaccine with others in real-life appeared to mitigate the negative regional relations between Vaccine Fraud and Children and vaccine attitudes and vaccination.

Even though our results showed strong to very strong evidence of associations between topics (i.e., Vaccine Science Matters, Big Pharma, and Vaccine Fraud and Children) and vaccine attitudes and behavior, these associations do not necessarily imply causation and await experimental results. However, the findings can provide important insights into social media effects on health attitudes and behaviors. For example, given the prospective associations, it is possible that posts about scientific evidence of vaccines promote the spread of factual information of vaccines, which contribute to more favorable attitudes towards the flu vaccine. Future studies may assess whether the presence of an unobserved or third variable, such as the change of regional patients' minimum age for convenient immunization at pharmacies [63] , increases discussions on Twitter while also increasing vaccine attitudes and actual vaccination. Another direction of research may be to study the spread of conspiracy theories (e.g., Big Pharma) on social media, which may raise concerns about vaccine safety. People who are high in needle sensitivity may be more prone to the influences of these posts, and a downward spiral may develop, which may result in more negative attitudes toward vaccines [64] . This study has a number of noteworthy limitations. Twitter users may not be representative of social media messaging in general or of communication in the non social media world. First, although Twitter continues to be a popular social media platform [47] , other social media should be analyzed in future research. Second, those on Twitter use that platform to connect to friends and acquaintances who live in their area, cross-regional influences also may be important and deserve future consideration [65] . Third, the present study identified a topic model based on a limited albeit large set of tweets (i.e., tweets with location metadata that allow geotagging to U.S. counties). Fourth, in addition to social media, other factors such as health insurance coverage are critical determinants of vaccination, and their influence in interaction with social media should be investigated in the future. Despite these limitations, our findings echo the notion that Twitter can have an influence on the local context [35, 66] . 

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Effects of influenza vaccination in the United States during the 2017-2018 influenza season

Establishing the truth: Vaccines, social media, and the spread of misinformation

Temporal trends in anti-vaccine discourse on Twitter

Russian Twitter accounts and the partisan polarization of vaccine discourse

But it shouldn't be mandatory. Same with any medicine or vaccine. I personally would never take a Coronavirus vaccine. I know these sneaky sna

The problem is when you make that choice for yourself, you're also potentially making it for other people who are unable to get vaccinated. Polio, measles, diphtheria, etc are not things anyone should suffer through. Especially not when we have means of p

A content analysis of HPV related posts on instagram

It's not all about autism: The emerging landscape of anti-vaccination sentiment on Facebook

Expressions of pro-and anti-vaccine sentiment on YouTube

Using text mining and sentiment analysis to analyse YouTube Italian videos concerning vaccination

Understanding anti-vaccination attitudes in social media

Social media harvesting

Prediction and change of health behavior: Applying the reasoned action approach

Theories of reasoned action and planned behavior as models of condom use: A meta-analysis

Advancing tailored health communication: A persuasion and message effects perspective

Telling stories, saving lives: Creating narrative health messages

Text4baby influenza messaging and influenza vaccination among pregnant women

A test of major assumptions about behavior change: A comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic

The cognitive impact of past behavior: influences on beliefs, attitudes, and future behavioral decisions

Persuasive communications to change actions: An analysis of behavioral and cognitive impact in HIV prevention

Action, inaction, and actionability: Definitions and implications for communications and interventions to change behaviors

The social creation of action and inaction: From concepts to goals to behaviors

Cognition in persuasion: An analysis of information processing in response to persuasive communications

Elaborative and nonelaborative processing of a behavior-related communication

Action and Inaction in a Social World: Predicting and Changing Attitudes and Behaviors

The Influence of Attitudes on Behavior

Perceived seriousness of seasonal and A (H1N1) influenzas, attitudes toward vaccination, and vaccine uptake among U. S. adults: Does the source of information matter

Social media use and influenza vaccine uptake among White and African American adults

Mapping information exposure on social media to explain differences in HPV vaccine coverage in the United States

Information in ecological inference: An introduction

The ecological fallacy in national culture research

Forming attitudes that predict future behavior: A meta-analysis of the attitude-behavior relation

Medical students' attitude towards influenza vaccination

Tweet this: A uses and gratifications perspective on how active Twitter use gratifies a need to connect with others

Event classification and location prediction from tweets during disasters

Evaluating the Impact of a National AIDS Prevention Radio Campaign in St. Vincent and the Grenadines 1

Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate

Public health working to fight misinformation through trust, relationships: Facts not enough

Attitudes to vaccination: A critical review

The role of social networks in influenza vaccine attitudes and intentions among college students in the southeastern united states

The influence of social norms on flu vaccination among African American and White adults

Networked collective intelligence improves dissemination of scientific information regarding smoking risks

Science audiences, misinformation, and fake news

Group Problem Solving

Documentation for NORC's AmeriSpeak for Institutional Review Boards

Social Media Use

Future-oriented tweets predict lower county-level HIV prevalence in the United States

An Online Risk Index for the crosssectional prediction of new HIV, chlamydia, and gonorrhea diagnoses across U. S. counties and across years

The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic

Twitter Influenza Surveillance: Quantifying Seasonal Misdiagnosis Patterns and their Impact on Surveillance Estimates

A case study of the New York City 2012-2013 influenza season with daily geocoded Twitter data from temporal and spatiotemporal perspectives

Applying LDA topic modeling in communication research: Toward a valid and reliable methodology

Multi-attribute topic feature construction for social media-based prediction

Associations of topics of discussion on Twitter with survey measures of attitudes, knowledge, and behaviors related to Zika: Probabilistic study in the United States

Latent Dirichlet Allocation

MALLET: A machine learning for language toolkit

Multilevel Analysis: Techniques and Applications

performance: Assessment of regression models performance

Bayesian benefits with JASP

Bayesian Approaches to Clinical Trials and Health-Care Evaluation

World Health Organization. Novel Coronavirus(2019-NCoV) Situation Report -13

Most states make it difficult for children to get a flu shot

Conspiracy theories and fear of needles contribute to vaccine hesitancy for many parents. The Conversation. https:// theconversation.com/conspiracy-theories-and-fear-of-needles-contribute-tovaccine-hesitancy-for-many-parents-1208

The social integration of American cities: Network measures of connectedness based on everyday mobility across neighborhoods

Social media use and participation: A meta-analysis of current research

Research reported in this publication was supported by the National Institute of Mental Health under Award Number R01MH114847, the National Institute on Drug Abuse under Award Number DP1 DA048570, and the National Institute Of Allergy And Infectious Diseases under Award Number R01AI147487. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We would like to thank the APPC ASK Group for reviewing keywords and hashtags about influenza and flu that are relevant to the study period.

Supplementary data to this article can be found online at https://doi.org/10.1016/j.vaccine.2020.07.054.